255 Views

Organized Oral Session

Session: Meta-analysis in Ecology: Challenges, Advances, and Successful Applications

OOS 1-3 - Effect sizes in ecology: where we started, where we are, where we still need to go

Monday, August 15, 2022

2:00 PM – 2:15 PM EDT

Location: 520E

Presenting Author(s)

CO

Craig W. Osenberg

Odum School of Ecology, University of Georgia
Athens, Georgia, United States

Co-author(s)

AB

Amy A. Briggs

Odum School of Ecology, University of Georgia
Athens, Georgia, United States
CS

Chao Song

College of Ecology, Lanzhou University
Lanzhou, Gansu, China (People's Republic)
SP

Scott D. Peacor

Department of Fisheries and Wildlife, Michigan State Univesity
East Lansing, Michigan, United States
JB

James R. Bence

Department of Fisheries and Wildlife, Michigan State Univesity
East Lansing, Michigan, United States

Background/Question/Methods
Modern meta-analyses have three basic goals: 1) quantify the average effect of a factor; 2) assess if this effect varies among studies; and 3) explore what other factors (e.g., traits of the organism, the environment or the experimental protocol) potentially explain the observed heterogeneity in effects. Central to achieving these goals then, is the meaning of “an effect”. This apparently simple problem is deceptively complex, and this complexity forms the core of this presentation. We start by reflecting on the 30-year history of effect sizes in ecology, by comparing effect size metrics used in the first decade of meta-analysis’ use in ecology with the effect size metrics used in the most recent decade. We then provide concrete examples of problems that underly the use of several common effect size metrics: Hedges’ d, the correlation coefficient, and log(response ratio), and illustrate how more explicit connections between ecological processes, response variables, and effect size metrics can facilitate (vs. obscure) progress in our discipline.

Results/Conclusions
Hedges’ d (a measure of the difference between two groups expressed in standard deviation units defined by among-replicate variation), the correlation coefficient, and the log(response ratio) are the most common metrics of effect used in ecology today; indeed, the relative frequency of using these three metrics in ecological meta-analyses has changed very little over the past 30 years, despite research demonstrating their limitations. Further, although the interpretation of meta-analytic results depends strongly on the choice of an effect size metric, fewer than 20% of all meta-analyses provide conceptual justification for their chosen metric. We discuss the use of absolute vs. relative differences and standardized (by variance) vs. unstandardized effect size metrics, and put forth arguments for the conditions under which each might be best used (vs. yield misleading results). We illustrate ways effect size metrics can be more explicitly linked to a conceptual model of an ecological process to reduce the effects of extraneous variation and facilitate interpretation, with the hope that future evaluations will more often avoid some of the pitfalls highlighted in this talk.