PS 51-185 - Scientific evidence for environmental decision-making: Increasing timeliness while maintaining confidence in conclusions

Background/Question/Methods

For scientific evidence to inform environmental decision-making, it needs to meet various user requirements. Timely delivery of evidence is key, given that most environmental decisions have a defined window in which to utilize evidence inputs. However, ensuring timeliness can lead to trade-offs with other desirable attributes of an evidence assessment including comprehensiveness of the assembled evidence base and a clear understanding of the level of confidence in conclusions. To minimize the trade-offs between timeliness and other user requirements, we propose several methods of sub-sampling and analyzing a body of evidence derived from published literature that was originally designed to be comprehensive. We used a Monte Carlo approach to create 1000 iterations of different sample sizes to answer the question “In streams and rivers, what is the evidence for the relationships between total nutrients (TN and/or TP) and chlorophyll a?” Then we analyzed the samples using a mixed-effects meta-analysis model and compared the results to meta-analyses of a full census of the evidence.

Results/Conclusions

We examined relationships between two in-stream nutrient measures (total nitrogen- TN and total phosphorus- TP) and two measures of chlorophyll a (chl-a) concentration (sestonic chl-a and benthic chl-a). We found the statistical power to detect non-zero overall effect sizes was high for all four nutrient-chl-a relationships, even with small sample sizes. It was more difficult to accurately estimate the magnitude of the population-level effect size using small samples of the evidence base. Our results indicate that sub-sampling could be an effective strategy for ensuring the timeliness of evidence delivery for environmental decision-making while also maintaining confidence in the conclusions of an evidence assessment, especially for certain types of questions (e.g., presence of a non-zero effect). Our study is one of many contributing to a larger effort by environmental evidence synthesizers and users to develop, compile, and communicate methods suitable for “rapid evidence assessment” to support environmental and conservation management and policy decisions.