Skip to main content

Table 6 Study bias assessment categories

From: Strategies for managing spring frost risks in orchards: effectiveness and conditionality—A systematic review protocol

  Risk of bias High Low
General bias based on Bilotta et al. [32] Selection bias (Inadequate randomisation) No randomisation (site/tree selection based on availability, expert judgement or results) Randomisation (site/tree selection based on random number generator/tossing a coin)
Performance and detection bias (researcher bias) No blinding/blinding broken/insufficient information Blinding done
Reporting bias (incomplete results (variables)) Not all pre-specified outcomes reported; measurements/methods for some outcomes were not pre-specified; incomplete reporting of one or more outcomes; not reporting on an expected outcome Study protocol is available and pre-specified outcomes are reported on; study protocol not available but all expected outcomes are reported on
Attrition bias (incomplete results (observations)) Resulting data points lower than expected from the methodology; no reason given for missing data related to outcome; bad imputation Reported data points correspond to methodology; data omission or imputation well documented and justified
Adapted criteria Number of spatial or temporal replications (experimental unit) and observations [33] Spatial replication (experimental units)
2 fields (widespread application)  > 2 fields
Or 2 rows (row-based installation)  > 2 rows
Or 2 trees (targeted application)  > 2 trees
OR  
Temporal replication
1 frost night (temporal replication temperature change)  > 1 frost night
1 year (temporal replication yield/damage)  > 1 year
Observations (assessment level) per experimental unit
 < 4 trees (widespread or row-based)  ≥ 4 trees
 < 4 branches (targeted application)  ≥ 4 branches
Baseline of test and control groups compared At least one of the following differs or is not described: management (pruning), soil type, topographic position, differences in cultivar and/or rootstock, orchard age All of the following factors are comparable: management (pruning), soil type, topographic position, differences in cultivar and/or rootstock, orchard age
Location of temperature sensor given Closest meteorological station > 10 km away; no detail given Local sensors on field and beside flowers at several tree heights (lower, middle and upper position in tree)
Reporting method Mean effects per experiment only Raw data, measure of variation
External validity Cold chamber experiment Field experiment