Meta-analyses exhibiting large initial effect size magnitudes that decrease over time follow a pattern known as decline effect, and are of particular concern when caused by methodological and publication biases that ultimately delay understanding of a scientific phenomenon (Clements et al., 2022). Reasons for a decline effect may include changes in ecologists’ choice of study systems or ecological variables over time and selective publication bias in favor of early rapid publication of large effect sizes in high impact journals. A recent study conducted by Costello and Fox (2022) tested for decline effects in 466 ecological meta-analyses and found that 3-5% of these meta-analyses exhibited a significant directional change in mean effect size over time. Although decline effects appear to be rare in ecology, explaining specific cases of true decline effect remains an important task. We evaluated alternative hypotheses to explain the nominally significant decline effect in five recent meta-analyses identified in Costello and Fox (2022) by regressing effect sizes on ecological variables, methodological variables, and journal impact factor. We also explored other cases and causes of decline effect that have been reported in ecology literature.
Results/Conclusions:
The main driver of decline effect in many scientific disciplines is the low power of initial exploratory studies. Often, the striking new findings of these early studies are published in higher impact journals. Preliminary results from this investigation suggest that the decline effect observed in several recent ecological meta-analyses appears to be associated with the low power of initial studies, as evidenced by the negative relationship between sampling variance and publication year. However, in these same meta-analyses, effect sizes were not positively related to journal impact factor. In fact, several of the meta-analyses revealed a significant negative relationship between effect sizes and journal impact factor. It appears that low-impact journals are willing to publish more imprecise effect sizes, while high impact journals are interested in larger studies even if effect sizes are lower. In some meta-analyses, decline effects were also associated with changes over time in variables related to study methodology. These analyses within five ecological meta-analyses indicate that decline effect is not related to selective publication bias, but can potentially be explained by variables specific to the meta-analyses. Compilation of further case studies is required before generalizations can be made about the causes of rare decline effects in ecology.
