Table 3 Criteria for assessing study validity

Study design

 Comparison domain

Comparison across both space and time (e.g., BACI)

Comparison across space or time only (e.g., CI, BA)

Selection bias

 Control matching at study initiation (not applicable to BA studies, where control and intervention take place at the same site)

Well-matched control and intervention sites, close in proximity or clearly similar, but without spillover effects

Well-matching not evident, other factors likely to influence difference between treatment and control sites

Selection bias

 Effect modifiers and confounding factors during study period

No or minimal presence of co-interventions, co-exposures or trends present that are likely to differ between groups, baseline environmental conditions similar to intervention period; or if present, confounding factors are accounted for

Co-interventions, co-exposures or trends present that are likely to differ between groups, e.g., differing climate conditions, weather events, or other environmental or anthropogenic changes; and factors are not accounted for

Performance bias

Groundwater measurements

 Measurement similarity and representativeness

Similar and representative measurements across groups, unlikely to differ systematically

Systematic differences in measurements, outcomes not assessed similarly, or measurements unlikely to be representative

Detection bias

Other

 Confounding variables not included in other bias types (e.g., funding source)

No obvious confounding factors, or confounding factors present but accounted for

Confounding factors stated or obviously present but not accounted for

Other bias