Skip to main content

Table 2 Study validity assessment criteria

From: What is the effect of prescribed burning in temperate and boreal forest on biodiversity, beyond pyrophilous and saproxylic species? A systematic review

Question/criterion Yes Partly No Unclear
Did the study have a temporal and/or spatial control? BACI study BA study or CI study N/a (not eligible according to inclusion criteria) Lacking sufficient information to judge
Degree of replication appropriate and representative? (to outcome measure) Replicated burns, OR a single large burn area, with samples considered to be representative and spatially independent Single burn with limited sampling within burn A single/small burn area with very limited sampling within burn Lacking sufficient information to judge
Does treatment allocation account for spatial heterogeneity? (e.g. blocked or randomised) and/or  intervention and comparator sites well-matched i.e. similar at baseline (e.g. aspect, soil type, forest type) Treatment allocation (multiple burn) or replication (within a single burn) accounts for spatial heterogeneity i.e. appropriately randomised, randomised and blocked or stratified OR stated/clear attempt at matching OR highly likely to be similar Spatial heterogeneity not fully accounted for i.e. partial randomisation/blocking/stratification OR Moderately matched Purposive treatment allocation that clearly does not account for spatial heterogeneity AND/OR Do not match Lacking sufficient information to judge
No severely confounding factors present (factors likely to confound the effect of the intervention on outcome)? apart from those present at baseline (e.g. other interventions) Confounding factors likely to be minimal, were minimised or lacking Some confounders present, likely to have moderate impact on outcome Subject to confounders with major impact on outcome (such that outcomes are not clearly the effect of the intervention) Lacking sufficient information to judge
Intervention was likely appropriately and realistically applied? Typical of a prescribed burn in terms of temperature, area, low intensity/severity of burn, or described as a prescribed burn and likely to be a typical prescribed burn Partly typical of a prescribed burn in one (or more) aspect/s but not in others Not typical of a prescribed burn (e.g. inappropriate temperature, very high intensity/severity) Lacking any information/description of burn
Outcome measure method was appropriate? A reliable or standard measurement method for the outcome, and at a seasonal, temporal and spatial scale that is likely to capture any impact of the intervention on the outcome Outcome measure method was partly appropriate and reliable Outcome measure method was not appropriate or reliable (due to method choice and/or poor analysis of the data) Lacking sufficient information to judge
  1. Reviewers answered the questions in the left column with ‘Yes’, ‘Partly’, ‘No’ and ‘Unclear’ based on the specifications in the table. The answer ‘n/a’ was used if the criterion was not applicable in a particular instance. Reviewers could also provide comments on each study regarding its external validity