Skip to main content

Table 3 Critical appraisal tool for study validity assessment

From: What are the relative risks of mortality and injury for fish during downstream passage at hydroelectric dams in temperate regions? A systematic review

Category

Bias and generic data quality features

Specific data quality features

Validity

Design of assessed study

1

Selection and performance bias: study design

Design (i.e., well-controlled)

High

Controlled trial (randomized or not) or Gradient of intervention intensity including “zero-control”

High

CI

2

Assessment bias: measurement of outcome

Replication (level of total fish released/surveyed)

High

Large sample size (n > 100 fish)

Medium

Moderate sample size (n = 50–100 fish)

Low

Low sample size (n < 50 fish), or unclear/not indicated

Measured outcome

High

Quantitative

Medium

Quantitative approximations (estimates)

Low

Semi-quantitative, or no extractable results

Outcome metric

High

The change in a metric related to fish mortality, injury, or productivity relative to an appropriate control

Low

A metric related to risk of impingement/entrainment (i.e., number of fish entrained) and not mortality/injury/productivity per se

3

Selection and performance bias: baseline comparison (heterogeneity between intervention and comparator with respect to defined confounding factors before treatment)

Habitat type

High

Control and treatment samples homogenous

Low

Control and treatment samples not comparable with respect to confounding factors OR insufficient information

Sampling

High

Treatment and control samples homogenous with respect to sampling distance

Low

Control and treatment samples not comparable with respect to confounding factors OR insufficient information

Other confounding environmental factors

High

Intervention and comparator sites homogenous

Low

Intervention and comparator sites not comparable with respect to confounding factors OR insufficient information

4

Selection and performance bias: Intra treatment variation [heterogeneity within both treatment and control samples (i.e., releases or surveys) with respect to confounding factors]

Intervention type

High

No heterogeneity within treatment and control samples

Low

Samples within treatment and control arms not comparable OR insufficient information

Sampling

High

No heterogeneity within treatment and control samples

Low

Samples within treatment and control arms not comparable OR insufficient information

  1. Reviewers provided a rating of high, medium, or low for each of the specific data quality features