Response to “Every ROSE has its thorns”

Abstract

Sharp et al. [1] raise a number of concerns about the development and communication of ROSES (RepOrting standards for Systematic Evidence Syntheses), and we welcome the opportunity to explain some of the underlying thinking behind development of the reporting standards for environmental evidence syntheses.

We acknowledge that development of ROSES diverged from the framework for developing reporting guidelines in healthcare, as put forward by Moher and colleagues [2]. This was deliberate: Moher et al.’s framework presents a resource-intensive and lengthy process, which we did not have the funding to pursue. Fortunately, while the Moher et al. framework produces very robust results, it is not the only way to approach standardisation: according to the British Standards Institution, there are a range of degrees of consensus which can be captured in standards; from codes of practice reflecting the views of a relatively small group of expert practitioners, to robust internationally-harmonised approaches which involve all stakeholders in a much lengthier, more comprehensively inclusive consensus process [3]. Reporting standards in environmental evidence syntheses are very new. ROSES was conceived as a specific set of standards for CEE and its journal Environmental Evidence (EEJ). Essentially ROSES is nothing drastically new—it represents a clarification, distillation and repackaging of the current reporting standards of CEE and EEJ made to be as user friendly as possible. As a result, the key aspects of our reporting standards could be captured via a relatively quick, simple and cheap, expert-led process. This also yields a useful document for advancing reporting standards in the broader environmental community. We believe our method for testing and refining ROSES was rigorous enough to achieve this to a sufficient degree. Future versions of ROSES can be refined in an iterative process as its use increases and its value is better understood. We would argue that to run a complex process in a broader community which has as yet limited experience of SR methods would not add enough value to ROSES to justify the delay in its implementation.

This point also explains why we did not wish ROSES to be a PRISMA extension. Standards should represent the consensus of a community of practice, representing as a shared understanding of minimum best practice given cultural context and community requirements. While environmental scientists can learn from PRISMA, as we have endeavoured to do, and may even ultimately converge, it would be incorrect to assume sufficient cultural and contextual similarity that one community could be said to speak for another. Complex environmental contexts, data types and study designs are fundamentally different from those in other fields, such as human health [4, 5]. In fact, these very differences led to the establishment of CEE (a dedicated coordinating body that coordinates conduct and guidelines for systematic reviews and maps), a detailed set of guidelines for environmental systematic reviews and maps [6] and finally, the development of a dedicated set of reporting standards—ROSES. We believe Table 1 of our manuscript demonstrates sufficient divergence between community requirements to justify an independent standard.

It may seem intuitive that PRISMA ought to be directly involved in the development of ROSES. However, since synthesis in healthcare and environmental science belongs to different communities of practice it follows that co-development could in fact threaten the integrity of ROSES: a joint standard may end up reflecting a diluted consensus of divergent interests, when what is needed is a standard specifically for environmental scientists. This may sound like needless reinvention of the wheel, but we believe ROSES represents an essential learning and consensus-building process for the environmental evidence synthesis field.

The link between ROSES and PRISMA necessarily becomes tenuous. This does present challenges, especially in crossover fields such as environmental health research. In this case, journals (such as Environment International) will give submitting authors a choice of complying with PRISMA or ROSES depending on preference.

In terms of communication and dissemination, we are currently reaching out to editors of environmental journals to discuss endorsement and (most importantly) enforcement of ROSES as a reporting standard. The authors can be assured that we are in contact with the developers of PRISMA and have been involved the update of the core PRISMA Statement. We see ROSES and PRISMA as complementary tools serving different community requirements, and together with the PRISMA developers we intend to continue evolving our respective standards to ensure they remain fit-for-purpose for their intended users.