Table 2 Study validity assessment criteria

Did the study have a temporal and/or spatial control?

BACI study

BA study or CI study

N/a (not eligible according to inclusion criteria)

Lacking sufficient information to judge

Degree of replication appropriate and representative? (to outcome measure)

Replicated burns, OR a single large burn area, with samples considered to be representative and spatially independent

Single burn with limited sampling within burn

A single/small burn area with very limited sampling within burn

Lacking sufficient information to judge

Does treatment allocation account for spatial heterogeneity? (e.g. blocked or randomised) and/or  intervention and comparator sites well-matched i.e. similar at baseline (e.g. aspect, soil type, forest type)

Treatment allocation (multiple burn) or replication (within a single burn) accounts for spatial heterogeneity i.e. appropriately randomised, randomised and blocked or stratified OR stated/clear attempt at matching OR highly likely to be similar

Spatial heterogeneity not fully accounted for i.e. partial randomisation/blocking/stratification OR Moderately matched

Purposive treatment allocation that clearly does not account for spatial heterogeneity AND/OR Do not match

Lacking sufficient information to judge

No severely confounding factors present (factors likely to confound the effect of the intervention on outcome)? apart from those present at baseline (e.g. other interventions)

Confounding factors likely to be minimal, were minimised or lacking

Some confounders present, likely to have moderate impact on outcome

Subject to confounders with major impact on outcome (such that outcomes are not clearly the effect of the intervention)

Lacking sufficient information to judge

Intervention was likely appropriately and realistically applied?

Typical of a prescribed burn in terms of temperature, area, low intensity/severity of burn, or described as a prescribed burn and likely to be a typical prescribed burn

Partly typical of a prescribed burn in one (or more) aspect/s but not in others

Not typical of a prescribed burn (e.g. inappropriate temperature, very high intensity/severity)

Lacking any information/description of burn

Outcome measure method was appropriate?

A reliable or standard measurement method for the outcome, and at a seasonal, temporal and spatial scale that is likely to capture any impact of the intervention on the outcome

Outcome measure method was partly appropriate and reliable

Outcome measure method was not appropriate or reliable (due to method choice and/or poor analysis of the data)

Lacking sufficient information to judge