Every ROSE has its thorns
Environmental Evidence volume 7, Article number: 20 (2018)
In their study, Haddaway et al.  present new reporting guidelines, RepOrting standards for Systematic Evidence Syntheses (ROSES) with a thoughtful breakdown of the issues when applying Preferred Reporting Items for Systematic review and Meta-Analyses (PRISMA) in the field of conservation and environmental management. We believe the expanding use of systematic reviews and systematic maps in non-medical fields illustrates the growing importance of evidence synthesis and recognizes the importance of transparency and reporting standards in research. However, we have some concerns regarding the methodology as well as the communication and dissemination plan for the new reporting guidelines.
The development of the ROSES reporting guidelines was limited to semi-structured questionnaires or in-depth interviews, which differs from the recommended guidance for developers of health research reporting guidelines that suggests the following steps: initial steps, pre-meeting activities, the face-to-face consensus meeting itself, post meeting activities and post publication activities . Despite this different approach, the authors provide a detailed point-by-point breakdown of the problems applying the reporting guideline PRISMA to their field (Table 1), and highlight the key differences between the two reporting guidelines (Table 2) . Presentation of the thorough rationale for changes for new reporting guidelines or extensions is not common and other new guidelines can use this structure. However, this breakdown does not make it clear why they chose to develop a separate but parallel tool to PRISMA.
Given that PRISMA is already known and widely used, ROSES could have been better framed as an extension (akin to PRISMA for network meta-analyses ). We recognize that the intention to include more methodological guidance than PRISMA is valuable as it recognizes the need for additional education. However, this intention is not reason enough for it to be distinctly separate from PRISMA.
While the authors provide detailed rationale for their individual changes, the relationship of the ROSES guidance within the existing domain of reporting guidelines is not stated. Although they mention that ROSES should be used in conjunction with the Collaboration for Environmental Evidence (CEE) guidelines for methodological guidance, the link to PRISMA is vague. Using multiple guidelines simultaneously can be a difficult task for authors and practicality and ease of use concerns should be more clearly addressed. We praise the authors for their innovative approach in making their guidelines user-friendly in the form of editable and downloadable formats.
Uptake of reporting guidelines is generally low across many fields, therefore we believe that further elaboration on plans for dissemination and communication is essential to the success of this initiative . For example, while the creation of the ROSES website is valuable and in accordance with Moher et al.’s guidance for reporting guideline developers, an appropriate dissemination plan is needed to ensure widespread uptake . Moreover, additional future plans to gather user experiences and feedback, including evaluating its impact on completeness of reporting, will be of great importance to further advance the checklist and flow diagram.
Haddaway NR, Macura B, Whaley P, Pullin AS. ROSES RepOrting standards for Systematic Evidence Syntheses: pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps. Environ Evid. 2018;7:7.
Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLOS Med. 2010;7:e1000217.
Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH, Cameron C, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med. 2015;162:777.
Samaan Z, Mbuagbaw L, Kosa D, Debono VB, Dillenburg R, Zhang S, et al. A systematic scoping review of adherence to reporting guidelines in health care literature. J Multidiscip Healthc. 2013;6:169–88.
MKS and LN led the drafting of the research letter. DH supervised and provided feedback. All authors read and approved the final manuscript.
We would like to acknowledge the students who participated in the discussion as a part of the Methods in Research on Research (MiRoR) project.
The authors declare that they have no competing interests.
Availability of data and materials
Consent for publication
The letter is original work of all authors. All authors made a significant contribution to this study. This work has not been submitted for publication, it has not been accepted for publication, and has not been published in any other journal. All authors have read and approved the final version of this letter. There are no potential competing interests to disclose from any authors.
Ethics approval and consent to participate
This project has received funding from the European Union’s Horizon 2020 research and innovation program under the Marie Sklodowska-Curie Grant Agreement No. 676207.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Sharp, M.K., Nyanchoka, L. & Hren, D. Every ROSE has its thorns. Environ Evid 7, 20 (2018). https://doi.org/10.1186/s13750-018-0132-4