Skip to main content

Table 3 Critical appraisal tool for study validity assessment

From: What are the relative risks of mortality and injury for fish during downstream passage at hydroelectric dams in temperate regions? A systematic review

CategoryBias and generic data quality featuresSpecific data quality featuresValidityDesign of assessed study
1Selection and performance bias: study designDesign (i.e., well-controlled)HighControlled trial (randomized or not) or Gradient of intervention intensity including “zero-control”
HighCI
2Assessment bias: measurement of outcomeReplication (level of total fish released/surveyed)HighLarge sample size (n > 100 fish)
MediumModerate sample size (n = 50–100 fish)
LowLow sample size (n < 50 fish), or unclear/not indicated
Measured outcomeHighQuantitative
MediumQuantitative approximations (estimates)
LowSemi-quantitative, or no extractable results
Outcome metricHighThe change in a metric related to fish mortality, injury, or productivity relative to an appropriate control
LowA metric related to risk of impingement/entrainment (i.e., number of fish entrained) and not mortality/injury/productivity per se
3Selection and performance bias: baseline comparison (heterogeneity between intervention and comparator with respect to defined confounding factors before treatment)Habitat typeHighControl and treatment samples homogenous
LowControl and treatment samples not comparable with respect to confounding factors OR insufficient information
SamplingHighTreatment and control samples homogenous with respect to sampling distance
LowControl and treatment samples not comparable with respect to confounding factors OR insufficient information
Other confounding environmental factorsHighIntervention and comparator sites homogenous
LowIntervention and comparator sites not comparable with respect to confounding factors OR insufficient information
4Selection and performance bias: Intra treatment variation [heterogeneity within both treatment and control samples (i.e., releases or surveys) with respect to confounding factors]Intervention typeHighNo heterogeneity within treatment and control samples
LowSamples within treatment and control arms not comparable OR insufficient information
SamplingHighNo heterogeneity within treatment and control samples
LowSamples within treatment and control arms not comparable OR insufficient information
  1. Reviewers provided a rating of high, medium, or low for each of the specific data quality features