Skip to main content

Table 6 Study quality assessment framework for experimental mesocosm studies

From: Response of chlorophyll a to total nitrogen and total phosphorus concentrations in lotic ecosystems: a systematic review protocol

Bias area Characteristic Low risk of bias High risk of bias
Study design and sampling Study timeframe Study timeframe considers risk of container effects (<30 days, but can depend on size and flow-through vs. recirculating) Study timeframe is long enough to be at risk of container effects and other random processes (typically >30 days but depends on size and flow-through vs. recirculating)
  Randomization of sampling (selection bias) Some form of randomized assignment of control and treatment samples No randomization of control and treatment assignments
  Control matching and performance bias Control and treatment mesocosms are similar, or there is a clear effort to account for any differences Evidence that control and treatment mesocosms differ in aspects not related to treatments, with no effort to account for differences
  Treatment clarity and detail Method of nutrient addition clearly explained (e.g. pulse, continuous drip) Method of nutrient addition not clear
  Confounding factors No obvious confounding factors reported, or if reported are accounted for Confounding factors reported and not accounted for, unclear how accounted for, or are likely
  Replication ≥3 replicates per treatment, no pseudoreplication <3 replicates per treatment, or evidence of pseudoreplication
  Measurement clarity and detail Methods for design and sampling described in detail, including chl-a extraction and measurement, water filtering, and nutrient measurement Missing information not allowing for repeatability
Data analysis and results Clarity and detail Analysis methods described in detail sufficient to permit repeating Missing information not allowing for repeatability
  Uncertainty Some estimate of uncertainty in effect or relationships provided (e.g. confidence intervals, standard error, standard deviation, etc.) No estimates of uncertainty provided
  Treatment vs. control Differences in treatments vs. controls reported quantitatively (e.g. actual values, response ratios, effect size) Differences in treatments vs. controls reported only qualitatively (higher, lower) or otherwise unclear (e.g. only P value given)
  Reporting bias All variables, measurements, and statistical tests mentioned in methods are reported in results or additional file Some variables, measurements, or statistical tests mentioned in methods are not reported
Other biases Detection bias No indication that outcomes were measured differently in control and treatment samples Some indication that outcomes were measured differently in control and treatment samples
  Attrition bias No differences in loss of control and treatment samples Differences in loss of control and treatment samples
  Research aim consistency Questions clearly stated and answers match questions Questions not clearly stated or answers do not match stated questions