Category | Bias and generic data quality features | Specific data quality features | Validity | Design of assessed study |
---|---|---|---|---|
1 | Selection and performance bias: study design | Design (i.e., well-controlled) | High | Controlled trial (randomized or not) or Gradient of intervention intensity including “zero-control” |
High | CI | |||
2 | Assessment bias: measurement of outcome | Replication (level of total fish released/surveyed) | High | Large sample size (n > 100 fish) |
Medium | Moderate sample size (n = 50–100 fish) | |||
Low | Low sample size (n < 50 fish), or unclear/not indicated | |||
Measured outcome | High | Quantitative | ||
Medium | Quantitative approximations (estimates) | |||
Low | Semi-quantitative, or no extractable results | |||
Outcome metric | High | The change in a metric related to fish mortality, injury, or productivity relative to an appropriate control | ||
Low | A metric related to risk of impingement/entrainment (i.e., number of fish entrained) and not mortality/injury/productivity per se | |||
3 | Selection and performance bias: baseline comparison (heterogeneity between intervention and comparator with respect to defined confounding factors before treatment) | Habitat type | High | Control and treatment samples homogenous |
Low | Control and treatment samples not comparable with respect to confounding factors OR insufficient information | |||
Sampling | High | Treatment and control samples homogenous with respect to sampling distance | ||
Low | Control and treatment samples not comparable with respect to confounding factors OR insufficient information | |||
Other confounding environmental factors | High | Intervention and comparator sites homogenous | ||
Low | Intervention and comparator sites not comparable with respect to confounding factors OR insufficient information | |||
4 | Selection and performance bias: Intra treatment variation [heterogeneity within both treatment and control samples (i.e., releases or surveys) with respect to confounding factors] | Intervention type | High | No heterogeneity within treatment and control samples |
Low | Samples within treatment and control arms not comparable OR insufficient information | |||
Sampling | High | No heterogeneity within treatment and control samples | ||
Low | Samples within treatment and control arms not comparable OR insufficient information |