Table 7 Recommendations for applying results of critical appraisal to inform data synthesis

1.All results of risk of bias assessments should inform the data synthesis, whether the synthesis is quantitative (i.e. meta-analysis) and/or narrative

The data synthesis must consider all risks of bias to ensure that the systematic review conclusions can be declared to have high validity, or to have known limitations. This applies whether the data synthesis consists of a quantitative meta-analysis and/or a narrative descriptive synthesis. Both types of synthesis should be clearly structured to demonstrate the impact of study validity on study results

2.Effects of risk of bias should be considered in the data synthesis using sensitivity or subgroup analyses or, if feasible, adjustments in statistical data synthesis models to account for bias

Analysis according to study risk of bias subgroups enables all studies to be included in the data synthesis (including narrative synthesis) and the impact of risk of bias on study outcomes to be explored [55]. This provides a transparent framework for justifying which of the included studies should or should not inform the final review conclusions. Adjustment for some types of confounding such as imbalances in group characteristics may be feasible (e.g. using stratification or statistical modelling such as inverse probability weighting or propensity scoring methods [84]), provided that any threats to validity that cannot be adjusted for are also captured in the data synthesis, in subgroup analyses

3.Do not use numeric summary scores

Numeric scores have several limitations (Box 3) and are not recommended for summarising risk of bias assessments