Skip to main content

Table 2 Study validity assessment framework for observational field studies

From: Response of chlorophyll a to total nitrogen and total phosphorus concentrations in lotic ecosystems: a systematic review

Domain of bias Characteristic Low risk of bias High risk of bias
Study design and sampling Pairing of nutrient and chl-a measurements Nutrient and chl-a measurements taken at same place and time or index period Nutrient and chl-a measurements are not paired in time and space
Study timeframe Sampling includes relevant periods over multiple years Sampling occurs only over a single season or year
Gradient definition Gradient based on a nutrient related variable or its causal antecedent Gradient based on a common causal descendant of TP, TN or chl-a
Sample size Acceptable # sites (≥ 10) across gradient Low # sites (< 10) across gradient
Event-based replicates (sample replicates) Multiple samples taken during each sampling event for nutrient and chl-a Single samples taken during each sampling event for nutrient and chl-a
Site-based temporal replicates (within-site replicates) Multiple samples taken at each site (over study period) Single samples taken at each site (over study period)
Randomization of sampling (selection bias) Some form of randomized site selection (e.g., stratified random sampling) No randomization of site selection
Confounding factors If not controlled by study design, confounding factors are measured and adjusted for in statistical analysis Confounding factors reported and not accounted for, or are likely, and are not able to be adjusted for post hoc
Data analysis and results Clarity and detail Analysis methods described in detail sufficient to permit repeating Missing information not allowing for repeatability
Uncertainty Some estimate of uncertainty in effect or relationships provided (e.g., confidence intervals, standard error, standard deviation, etc.) No estimates of uncertainty provided
Reporting bias All variables, measurements, and statistical tests mentioned in methods are reported in results or supplementary material Some variables, measurements, or statistical tests mentioned in methods are not reported
Other biases Detection bias No indication that outcomes were measured differently in high versus low exposure sites Some indication that outcomes were measured differently in high versus low exposure sites
Attrition bias No differences in loss of high versus low exposure sites Differences in loss of high versus low exposure sites