Problem 1: | Does not strictly require a protocol, refers to registration (e.g. PRISMA [24] checklist #5 “Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address), and, if available, provide registration information including registration number”) |
Problem 2: | Vagueness hides necessary level of transparency to allow repeatability (e.g. no requirements to provide details of which databases are accessed via Web of Knowledge or Proquest, PRISMA [24] checklist #7) |
Problem 3: | Suggested requirements for review conduct are minimal, affecting overall comprehensiveness of the review (e.g. repeatable reporting of search in “at least one database” required only, PRISMA [24] checklist #8) |
Problem 4: | Heavy emphasis on meta-analysis (excludes narrative, qualitative and mixed synthesis methods): e.g. PRISMA [24] checklist #13 ‘Summary measures’: “State the principal summary measures (e.g., risk ratio, difference in means)”, PRISMA [24] checklist #14 ‘Synthesis of results’: “Describe the methods of handling data and combining results of studies, if done, including measures of consistency (e.g., I2) for each meta-analysis”, PRISMA [24] checklist #21 ‘Synthesis of results’: “Present results of each meta-analysis done, including confidence intervals and measures of consistency” |
Problem 5: | Lost nuance of necessary methodological steps: e.g. lack of consistency and comprehensiveness checking, no requirement to avoid bias caused by reviewing and appraising one’s own research |
Problem 6: | Focuses on risk of bias rather than limitations to validity: ignores external validity and limitations that do not affect bias (e.g. PRISMA [24] checklist #15 and #22) |
Problem 7: | Focuses on medicine and health topics (e.g. PRISMA [24] checklist #9 ‘Results of individual studies’: “For all outcomes considered (benefits or harms), present, for each study: (a) simple summary data for each intervention group (b) effect estimates and confidence intervals, ideally with a forest plot”) |
Problem 8: | Non-matching terminology: e.g. separating ‘screening’ from ‘eligibility’ (see PRISMA [24] flow-diagram) |
Problem 9: | Inappropriate use of the term ‘qualitative synthesis’ meaning ‘narrative synthesis’ (see PRISMA Statement [24]) |
Problem 10: | PRISMA checklist misses vital information from the PRISMA [24] flow-diagram: e.g. exclusions during critical appraisal |
Problem 11: | Often misused as methodological rather than reporting guidance [34] |
Problem 12: | PRISMA checklist is only useful for authors and editors during submission: no further information provided within the process of completion: it is purely a checklist |