Skip to main content

Table 3 Critical appraisal criteria used to assess included studies

From: What is the impact of human wastewater biosolids (sewage sludge) application on long-term soil carbon sequestration rates? A systematic review protocol

Group

Criteriona

Low susceptibility to biasb

High Susceptibility to Bias

Study design

Study type

Before-After Control-Impact and Randomized Controlled Trials (though we do not expect to find RCTs as they are uncommon in this field)

Control-Impact study designs (no pre-impact data)

Experimental randomizationc

Suitable experimental designs for randomization (e.g., completely randomized, randomized complete block, Latin square, factorial, split-plot, strip-plot)

Purposive (selective)

Experimental replication (spatial)

Replication at level of intervention (i.e., spatial replication) and of large sample size (3 +)

Replication at level of intervention but of low sample size (< 3)

Similar starting point

Experimental treatment and control groups are similar at the start of the trial

Experimental treatment and control groups are not similar at the start of the trial

Randomization

Some degree of randomization in sample selection

No randomization (i.e., purposive sampling)

Replication of sampling

Replication of samples (3+)

Low replication (< 3)

Study measurement and data analysis methods

Time between intervention (i.e., last amendment/biosolids application and soil measurement sampling)

At least 1 year since last application

 < 1 year since last application

Soil organic carbon/matter measurement method and details

Uses dry combustion/elemental analysis or Walkley–Black titration method to measure soil organic carbon/matter. Uses and details acid treatment to remove inorganic carbonates, if necessary

Uses loss-on-ignition or other method to measure soil organic carbon/matter or missing methodological detail to determine soil organic carbon/matter

Soil measurement depth (if incorporated or injected in soil subsurface)

Soil measured to at least depth of biosolids incorporation/injection

Soil not measured to depth of biosolids incorporation/injection or does not include details to determine depth measured

Soil bulk density measured

Soil bulk density measured

Soil bulk density not measured

Incomplete/Missing outcome data

No missing data; reasons for missing data not related to outcome; missing data balanced across control and intervention groups (and reasons similar); or proportion missing/plausible effect size not enough to have a relevant effect

Reasons related to outcome, and imbalance in numbers or reasons; or proportion missing/plausible effect size enough to have a relevant effect

Account for confounding variables

Presence of confounders

No obvious confounders or adequately accounted for as a result of blocking/pairing

Confounders present and/or unaccounted for (e.g., different irrigation strategies with no blocking of added treatment effect)

  1. a Criterion to assess internal validity (risk of bias)
  2. b Unclear classification given to any study where substantial details within the methods are either unclear or missing. Not applicable (N/A) is given to any study where the variable is not applicable to the study
  3. c See Singh and Masuku [24] for appropriate experimental designs and statistical techniques