Skip to main content

Advertisement

Table 1 Key problems relating to the application of PRISMA [24] to evidence synthesis in conservation and environmental management

From: ROSES RepOrting standards for Systematic Evidence Syntheses: pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps

Problem 1: Does not strictly require a protocol, refers to registration (e.g. PRISMA [24] checklist #5 “Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address), and, if available, provide registration information including registration number”)
Problem 2: Vagueness hides necessary level of transparency to allow repeatability (e.g. no requirements to provide details of which databases are accessed via Web of Knowledge or Proquest, PRISMA [24] checklist #7)
Problem 3: Suggested requirements for review conduct are minimal, affecting overall comprehensiveness of the review (e.g. repeatable reporting of search in “at least one database” required only, PRISMA [24] checklist #8)
Problem 4: Heavy emphasis on meta-analysis (excludes narrative, qualitative and mixed synthesis methods): e.g. PRISMA [24] checklist #13 ‘Summary measures’: “State the principal summary measures (e.g., risk ratio, difference in means)”, PRISMA [24] checklist #14 ‘Synthesis of results’: “Describe the methods of handling data and combining results of studies, if done, including measures of consistency (e.g., I2) for each meta-analysis”, PRISMA [24] checklist #21 ‘Synthesis of results’: “Present results of each meta-analysis done, including confidence intervals and measures of consistency”
Problem 5: Lost nuance of necessary methodological steps: e.g. lack of consistency and comprehensiveness checking, no requirement to avoid bias caused by reviewing and appraising one’s own research
Problem 6: Focuses on risk of bias rather than limitations to validity: ignores external validity and limitations that do not affect bias (e.g. PRISMA [24] checklist #15 and #22)
Problem 7: Focuses on medicine and health topics (e.g. PRISMA [24] checklist #9 ‘Results of individual studies’: “For all outcomes considered (benefits or harms), present, for each study: (a) simple summary data for each intervention group (b) effect estimates and confidence intervals, ideally with a forest plot”)
Problem 8: Non-matching terminology: e.g. separating ‘screening’ from ‘eligibility’ (see PRISMA [24] flow-diagram)
Problem 9: Inappropriate use of the term ‘qualitative synthesis’ meaning ‘narrative synthesis’ (see PRISMA Statement [24])
Problem 10: PRISMA checklist misses vital information from the PRISMA [24] flow-diagram: e.g. exclusions during critical appraisal
Problem 11: Often misused as methodological rather than reporting guidance [34]
Problem 12: PRISMA checklist is only useful for authors and editors during submission: no further information provided within the process of completion: it is purely a checklist