Skip to main content

Advertisement

Table 2 Study validity assessment criteria

From: What is the effect of prescribed burning in temperate and boreal forest on biodiversity, beyond pyrophilous and saproxylic species? A systematic review

Question/criterionYesPartlyNoUnclear
Did the study have a temporal and/or spatial control?BACI studyBA study or CI studyN/a (not eligible according to inclusion criteria)Lacking sufficient information to judge
Degree of replication appropriate and representative? (to outcome measure)Replicated burns, OR a single large burn area, with samples considered to be representative and spatially independentSingle burn with limited sampling within burnA single/small burn area with very limited sampling within burnLacking sufficient information to judge
Does treatment allocation account for spatial heterogeneity? (e.g. blocked or randomised) and/or  intervention and comparator sites well-matched i.e. similar at baseline (e.g. aspect, soil type, forest type)Treatment allocation (multiple burn) or replication (within a single burn) accounts for spatial heterogeneity i.e. appropriately randomised, randomised and blocked or stratified OR stated/clear attempt at matching OR highly likely to be similarSpatial heterogeneity not fully accounted for i.e. partial randomisation/blocking/stratification OR Moderately matchedPurposive treatment allocation that clearly does not account for spatial heterogeneity AND/OR Do not matchLacking sufficient information to judge
No severely confounding factors present (factors likely to confound the effect of the intervention on outcome)? apart from those present at baseline (e.g. other interventions)Confounding factors likely to be minimal, were minimised or lackingSome confounders present, likely to have moderate impact on outcomeSubject to confounders with major impact on outcome (such that outcomes are not clearly the effect of the intervention)Lacking sufficient information to judge
Intervention was likely appropriately and realistically applied?Typical of a prescribed burn in terms of temperature, area, low intensity/severity of burn, or described as a prescribed burn and likely to be a typical prescribed burnPartly typical of a prescribed burn in one (or more) aspect/s but not in othersNot typical of a prescribed burn (e.g. inappropriate temperature, very high intensity/severity)Lacking any information/description of burn
Outcome measure method was appropriate?A reliable or standard measurement method for the outcome, and at a seasonal, temporal and spatial scale that is likely to capture any impact of the intervention on the outcomeOutcome measure method was partly appropriate and reliableOutcome measure method was not appropriate or reliable (due to method choice and/or poor analysis of the data)Lacking sufficient information to judge
  1. Reviewers answered the questions in the left column with ‘Yes’, ‘Partly’, ‘No’ and ‘Unclear’ based on the specifications in the table. The answer ‘n/a’ was used if the criterion was not applicable in a particular instance. Reviewers could also provide comments on each study regarding its external validity