Skip to main content

Table 2 Criteria for study validity assessment

From: How do changes in flow magnitude due to hydroelectric power production affect fish abundance and diversity in temperate regions? A systematic review protocol

Question/criterionResponse to questionType of bias addressed
YesPartiallyNoUnclear
1. Did the study consist of both temporal and spatial comparisons?BACI, RCTBA, CI, ALT-CI, CI-gradient, RCA, NRN/A as study is not eligible for inclusion based on inclusion criteriaLacking sufficient information to judgeSelection
2. Are experimental/observational units replicated?≥ 2 independent experimental/observational units (i.e., the level of replication at which the intervention was administered/the exposure experienced)There were at least two experimental/observational units but there is a lack of independence between these units (pseudoreplication)No replication (i.e., < 2 independent experimental/observational units)Lacking sufficient information to judgeSelection
3. Are intervention and comparator sites well-matched at site selection/study initiation?Intervention and comparator sites are well-matched (i.e., similar physical characteristics). N/A for BA designs (i.e., intervention and comparator at the same site)Intervention and comparator sites are moderately matched. N/A for BA designsIntervention and comparator sites are poorly matched. N/A for BA designsLacking sufficient information to judgeSelection
4. Can the intervention be clearly interpreted?It is clear that a change to flow magnitude has occurred and quantitative data on magnitude is reportedIt is clear that a change to flow magnitude has occurred but either no quantitative data on magnitude is reported, or the quantitative data is difficult to interpret (e.g., averaged across intervention and control sites within the same river)N/AThe study compares an unregulated stream (or section of a stream) to a regulated stream (i.e., regulated via a hydro dam), or reports unspecified multiple components affecting flow (i.e., study does not report effects of components separately to isolate individual impacts of components)Selection, performance, reporting
5. Was the study free of other potential confounders after sample selection/study initiation?No or minimal confounding factors present, including e.g.: (1) no or minimal differences in environmental conditions between intervention and comparator sites and/or time periods (e.g., unplanned human alterations, floods, droughts, time-related trends), (2) no additional experimental manipulations of other flow regime components (e.g., flow frequency, duration) at the same time as magnitude alternations. Or if present, are accounted for appropriately in analysisN/AConfounding factors present that could have an impact on the outcome and these are not accounted for in analysisLacking sufficient information to judgePerformance
6. Did the study use different sampling/measurement method(s) between intervention and comparators?Similar/consistent sampling/measurement methods are used between intervention and comparator sites and/or times (e.g., gear type, timing or size of sample areas)Different/inconsistent sampling/measurement methods are used between intervention and comparator sites and/or times (e.g., gear type, timing or size of sample areas)N/ALacking sufficient information to judgeDetection