Skip to main content

Response to “Every ROSE has its thorns”

The Original Article was published on 14 September 2018


Sharp et al. [1] raise a number of concerns about the development and communication of ROSES (RepOrting standards for Systematic Evidence Syntheses), and we welcome the opportunity to explain some of the underlying thinking behind development of the reporting standards for environmental evidence syntheses.

We acknowledge that development of ROSES diverged from the framework for developing reporting guidelines in healthcare, as put forward by Moher and colleagues [2]. This was deliberate: Moher et al.’s framework presents a resource-intensive and lengthy process, which we did not have the funding to pursue. Fortunately, while the Moher et al. framework produces very robust results, it is not the only way to approach standardisation: according to the British Standards Institution, there are a range of degrees of consensus which can be captured in standards; from codes of practice reflecting the views of a relatively small group of expert practitioners, to robust internationally-harmonised approaches which involve all stakeholders in a much lengthier, more comprehensively inclusive consensus process [3]. Reporting standards in environmental evidence syntheses are very new. ROSES was conceived as a specific set of standards for CEE and its journal Environmental Evidence (EEJ). Essentially ROSES is nothing drastically new—it represents a clarification, distillation and repackaging of the current reporting standards of CEE and EEJ made to be as user friendly as possible. As a result, the key aspects of our reporting standards could be captured via a relatively quick, simple and cheap, expert-led process. This also yields a useful document for advancing reporting standards in the broader environmental community. We believe our method for testing and refining ROSES was rigorous enough to achieve this to a sufficient degree. Future versions of ROSES can be refined in an iterative process as its use increases and its value is better understood. We would argue that to run a complex process in a broader community which has as yet limited experience of SR methods would not add enough value to ROSES to justify the delay in its implementation.

This point also explains why we did not wish ROSES to be a PRISMA extension. Standards should represent the consensus of a community of practice, representing as a shared understanding of minimum best practice given cultural context and community requirements. While environmental scientists can learn from PRISMA, as we have endeavoured to do, and may even ultimately converge, it would be incorrect to assume sufficient cultural and contextual similarity that one community could be said to speak for another. Complex environmental contexts, data types and study designs are fundamentally different from those in other fields, such as human health [4, 5]. In fact, these very differences led to the establishment of CEE (a dedicated coordinating body that coordinates conduct and guidelines for systematic reviews and maps), a detailed set of guidelines for environmental systematic reviews and maps [6] and finally, the development of a dedicated set of reporting standards—ROSES. We believe Table 1 of our manuscript demonstrates sufficient divergence between community requirements to justify an independent standard.

It may seem intuitive that PRISMA ought to be directly involved in the development of ROSES. However, since synthesis in healthcare and environmental science belongs to different communities of practice it follows that co-development could in fact threaten the integrity of ROSES: a joint standard may end up reflecting a diluted consensus of divergent interests, when what is needed is a standard specifically for environmental scientists. This may sound like needless reinvention of the wheel, but we believe ROSES represents an essential learning and consensus-building process for the environmental evidence synthesis field.

The link between ROSES and PRISMA necessarily becomes tenuous. This does present challenges, especially in crossover fields such as environmental health research. In this case, journals (such as Environment International) will give submitting authors a choice of complying with PRISMA or ROSES depending on preference.

In terms of communication and dissemination, we are currently reaching out to editors of environmental journals to discuss endorsement and (most importantly) enforcement of ROSES as a reporting standard. The authors can be assured that we are in contact with the developers of PRISMA and have been involved the update of the core PRISMA Statement. We see ROSES and PRISMA as complementary tools serving different community requirements, and together with the PRISMA developers we intend to continue evolving our respective standards to ensure they remain fit-for-purpose for their intended users.


  1. 1.

    Sharp MK, Nyanchoka L, Hren D. Every ROSE has its thorns. Environ Evid. 2018;7:20 (in press).

    Article  Google Scholar 

  2. 2.

    Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLOS Med. 2010;7:e1000217.

    Article  Google Scholar 

  3. 3.

    British Standards Institution. Different types of standards; 2016. Accessed 26 Oct 2016.

  4. 4.

    Pullin AS, Stewart G. Guidelines for systematic review in conservation and environmental management. Conserv Biol. 2006;20:1647–56.

    Article  Google Scholar 

  5. 5.

    Fazey I, Salisbury JG, Lindenmayer DB, Maindonald J, Douglas R. Can methods applied in medicine be used to summarize and disseminate conservation research? Environ Conserv. 2004;31:190–8.

    Article  Google Scholar 

  6. 6.

    CEE. Guidelines and Standards for Evidence synthesis in Environmental Management. In: AS Pullin, GK Frampton, B Livoreil & G Petrokofsky, Editors. Version 5.0; 2018. Accessed 23 July 2018.

Download references

Authors’ contributions

NH drafted initial version of the manuscript. PW, BM and AP edited and commented on the drafts. All authors read and approved the final manuscript.


Not applicable.

Competing interests

The authors declare they have no competing interests.

Availability of data and materials

Not applicable.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Not applicable.


Not applicable.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information



Corresponding author

Correspondence to Biljana Macura.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Haddaway, N.R., Macura, B., Whaley, P. et al. Response to “Every ROSE has its thorns”. Environ Evid 7, 21 (2018).

Download citation


  • Systematic Evidence Synthesis
  • Environmental Systematic Reviews
  • Broader Community Environment
  • Crossover Field
  • Environmental Health Research