Better evidence, better decisions, better environment: emergent themes from the first environmental evidence conference

Abstract

The first international Collaboration for Environmental Evidence (CEE) conference took place in August 2016 at the Swedish Museum of Natural History in Stockholm with nearly 100 participants from 14 countries. This conference reflected and contributed to the growth of a global network of people interested in the production and use of evidence syntheses in environmental management. The conference also provided an opportunity to identify emerging themes and reflect on those ideas and perspectives to help direct future activities of the CEE and the broader community. An increasingly engaged community of practice was evident but there is uneven distribution of experience, resources, capacity, and commitment to evidence synthesis in different sectors and regions. There is much opportunity to bring academics, practitioners, and other partners together which will help to further demonstrate impact of evidence synthesis activities and enhance relevance. As the discipline evolves there is growing interest in rapid evidence synthesis but the benefits and risks of that approach remain unclear. There was also a recognition that improvements in empirical science will enhance the likelihood that more studies can be fully exploited as part of evidence synthesis. There are opportunities for capacity building, engaging the next generation (e.g., students), and enhancing connections within and beyond the CEE community to advance evidence-based environmental management. It is our desire that this paper will serve as a template for future CEE activities (i.e., where to invest resources) but also as an invitation to those that were unable to attend to participate in CEE and the evidence-based environmental management movement in whichever ways resonate with them.

Background

The Collaboration for Environmental Evidence (CEE; http://www.environmentalevidence.org) was founded a decade ago to address the problem of lack of accessible, good quality evidence to inform decision making in environmental management. It was evident that a gap existed between the community generating and publishing scientific evidence and the community seeking evidence to inform their decisions (e.g., environmental managers and policy makers). This led to a disconnect between the evidence that was generated and the decisions that were made. Essentially, decision makers were not utilising the best available evidence and instead tended to cherry pick from available empirical studies or rely more on personal experience or input from peers [1].

These aforementioned observations were backed up by earlier experiences of other sectors, particularly health and medicine (e.g., Cochrane Reviews; see http://www.cochrane.org), where a movement had emerged to improve the evidence base for health interventions. There was both need and opportunity to follow the lead of the health sector and improve effectiveness in environmental practice [2] and implementing an evidence-based approach to environmental management [3, 4]. Key early objectives of CEE were to adapt existing methodologies of evidence synthesis (especially systematic reviews; see [5]) from other sectors to key questions in environmental management, to create standards of conduct for evidence synthesis (see [6]), and to create a global network seeking to conduct and disseminate findings of systematic reviews. Dissemination of reliable and useful evidence-based information to practitioners is a key issue for CEE, and the decision makers have a big role both in the formulation of key questions on which to evaluate evidence and in using the results.

Collaboration for Environmental Evidence has come a long way from its inception: its guidelines for evidence reviews are widely used, it has its own journal (http://www.environmentalevidence.org) and a library of CEE endorsed systematic reviews and maps. It has six centres around the globe (the United Kingdom, South Africa, Sweden, Australia, France, and Canada) and an increasing number of contributors to the movement. As a result of this platform and the increasingly engaged community, CEE is in a position to make a significant contribution to improving environmental management, reflected in this significant milestone of its first international conference.

The conference took place in August 2016 at the Swedish Museum of Natural History in Stockholm. Close to 100 participants from 14 countries gathered amongst the fascinating exhibits and beautiful architecture to share experiences about evidence synthesis in environmental management. This conference reflected and contributed to the growth of a global network of people interested in the production and use of evidence syntheses in environmental management. The conference also provided an opportunity to identify emerging themes and reflect on those ideas and perspectives to help direct future activities of CEE and the broader community. Here we provide a brief summary of key messages emanating from the conference based on input from members of the organising committee, including representatives from each of the CEE’s six centres. It is our desire that this paper will serve as a record of conference activities as well as a template for future CEE activities (i.e., where to focus time, effort, and resources). We also consider the paper to be an invitation to those that were unable to attend the conference to participate in CEE and the evidence-based environmental management movement in whichever ways resonate with them (e.g., as funders, reviewers, referees, policy-makers that use SRs, etc.).

A community of practice is evident

A community of practice is emerging with a collective interest in evidence-based environmental management. This was very apparent throughout the meeting and was a product of both organic cooperation (e.g., individuals sharing their successes and failures openly and informally during the event) as well as more deliberate and ongoing activities of the CEE to facilitate sharing (e.g., monthly conference calls among the CEE centres). The collective good will of participants and interest in sharing experiences and learning together is particularly promising and bodes well for continued development of the capacity for conducting evidence synthesis and incorporating it into decision-making processes.

Uneven distribution of knowledge, resources, capacity and commitment

Of the 14 countries represented at the conference most participants came from Europe, with relatively few from North America, Asia Pacific region, Africa, and South America. Much of the intellectual property of environmental evidence synthesis thus resides in Europe. There is enormous potential for environmental evidence-based decision making to have a positive global impact via the rapidly developing economies in the global South. Yet, important questions remain regarding the mechanism by which this will occur. This will be an important topic worthy of ongoing discussion and debate at future CEE conferences.

Bringing academia, practitioners, and partners together

The practice of undertaking systematic reviews of evidence will need to move beyond academia. The practitioners at the conference represented mainly decision-makers from different government agencies (representing regional, state/provincial, and federal agencies as well as international bodies). The lack of for example industry participation reflects a need to build relationships and advocate the benefits of evidence-based decision making also within industry groups (e.g., the energy sector). This is a role that CEE and its centres can fulfill with greater resources while also continuing to generate and share the necessary standards and practices that facilitate a broader global movement of evidence-based decision making.

Need to demonstrate impact

There was a resounding message that we need to be able to tell a convincing story of impact of evidence synthesis. It was clear that the environmental community was eagerly looking to systematic reviews as a method of increasing the level of evidence used for environmental decisions. Identifying specific examples of where systematic reviews and systematic maps have influenced decision making and ultimately impacted on wellbeing, environmental quality, society, the economy, culture, or human health will be of great use in garnering further support for such evidence synthesis activities. There is a need to develop a robust and scalable impact evaluation framework and case studies in order to increase adoption of systematic reviews or other forms of evidence synthesis.

An evolving discipline

A systematic review is characterised by its thorough a priori planning and consultation, search for scientific evidence, its rigourous process and its transparency in presenting the data. There are many ways to conduct an evidence synthesis, and an equally large array of terminology and semantics to match this diversity. This poses a great challenge in terms of convening credibility. The three Collaborations (Cochrane, Campbell [focused on supporting social and economic change; see http://www.campbellcollaboration.org/] and CEE) are playing a very important role in this respect, by developing guidelines to support the review process and by “endorsing” reviews of high quality. Promoting better engagement of these three collaborations altogether will probably be a key development in the near future, and this is already progressing well for example through the new network Evidence Synthesis International (evidencesynthesis.org).

Different kinds of reviews have different limitations

If widespread adoption of evidence-based decision making using systematic approaches to synthesis is to occur there were at least two key challenges raised and discussed at the conference that must be overcome. The first area is the use of rapid reviews to inform decision-making. Rapid reviews represent a trade-off between a scoping exercise and a full systematic review. They use the full systematic review approach but make shortcuts on the scope and scale of the evidence included in the results. The benefits and drawbacks of rapid reviews were discussed at a Roundtable discussion at the conference and will surely be the focus of additional discourse moving forward. The second area is an increasing need for interdisciplinary systematic reviews as environmental issues often are interdisciplinary. Decision-makers and other stakeholders need knowledge from several scientific disciplines as a basis for decisions. We therefore need to broaden our skills (improved cooperation between natural scientists and social scientists) and develop methods (for example mixed methods) to meet this need. There are many lessons to be learnt from social sciences in this respect in how they can mix various disciplines in some synthesis exercises. In environmental sciences, a systemic approach to problems has been encouraged for many years. Although interdisciplinary approaches are increasingly common [7], there is need for additional work in this area and evidence synthesis could be a very interesting and important opportunity to bridge gaps across communities.

Need for capacity building

An issue that already existed in the Cochrane and Campbell Collaborations is now being raised in CEE as well; there is a group of experts who are doing systematic reviews and maps full time. This group should not forget that most of the systematic reviews are conducted as “one-shot” by review teams who then disappear. We are losing the experience and feedback from these teams, and it is important to keep them involved as they have faced the challenge of conducting a systematic review as “novice” or near to novice. Using their feedback is a way to make sure CEE will target the most useful messages and tools in the future. Moreover, providing opportunities for learners to work as part of established SR teams so that reviews and maps remain as tools available for the greatest number and not reserved for an elite. There are many people interested in conducting systematic reviews and learning about it, and we must build capacity to assist them with doing so. Training, mentoring, placement and workshops need to be developed at a global level, relying on functional collaboration with Centres and the 3C (e.g., see the Global Evidence Synthesis Initiative through Cochrane; http://epoc.cochrane.org/news/gesi). Nonetheless, a number of practical challenges exist for making this a reality (e.g., funding, willingness to pay, training awareness).

Engaging the next generation

In the same line, there is a need to engage and train the next generation in evidence synthesis methodologies. The training courses offered on the day before the conference represented a first step in student engagement but there is opportunity to do much more. CEE is in a fortunate position given that it is a reasonably young organization which makes them easier to integrate students/learners from the very beginning (i.e., it lacks the bureaucratic hurdles that may exist in more established organizations). A practical exercise the day before could for example be linked to more theoretical lectures during the conference, or that the results of the training are presented as a poster during the conference. It is also worth investigating whether students can get credit for training obtained during the conference.

Better empirical studies = better evidence

A key component of systematic reviews is the critical appraisal of studies using a series of strict assessment criteria. Doing so can lead to apparently relevant studies being flagged such that they are not formally included in the final synthesis (or the findings are down-weighted) which is unfortunate and could presumably have been addressed if the empirical study had been done with more rigour (e.g., replication, use of appropriate controls, acknowledging and addressing various forms of bias). To that end, there is opportunity to engage directly with those doing mission-oriented science to emphasize the potential pitfalls that could lead to their work being excluded from systematic reviews and thus failing to contribute to evidence-based decision making. Another pathway for improvement would be to exchange with scientific publishers (e.g., editors, peer reviewers) to ensure that reporting of descriptive data (e.g. mean, median and sample size), settings and bias are well described in any accepted article. A parallel movement in health care has led to a marked shift in the conduct and reporting of primary studies (see [8]).

Making connections from within and beyond

It was evident during this conference and the following What Works Global Summit in London (http://www.wwgs2016.org/) that many efforts were made by the CEE community to reach out to colleagues in the health and medical communities, including the Cochrane Collaboration, and the social sciences, including the Campbell Collaboration, EPPI-Centre and others. This will help to foster a truly interdisciplinary approach to the development of solutions for the complex environmental problems our world faces. We anticipate that there will be more crossover moving forward as many environmental problems are directly linked to issues of human health and welfare (e.g. the Sustainable Development Goals).

Conclusion

Moving forward, there are a number of challenges and opportunities that present themselves to the CEE community. It was apparent that the conference brought together a new and growing community and enabled us to begin to feel a shared CEE identity. This of course leaves us with a key question: What next? The primary challenge in the short term is to determine how we can harness the energy and interest that was collectively shared in Stockholm to enable consolidation of the community and provide scope for growth and welcoming of new individuals, groups and themes. As noted above, CEE is very much an open community. Making that apparent to scientists and practitioners will be critical as they consider the ways in which they can contribute to or otherwise embrace evidence-based environmental management. With plans already underway for the next conference, there is much opportunity to work as a community to address these issues or use the next conference as a platform to do so. Some of the challenges and opportunities are tractable and can be addressed with few resources and in the short term (months to years). However, the majority of these themes are ones on which scholars and our community of practice will ruminate on, refine, and revisit for decades.

Evidence synthesis is not static—tools are evolving as are data standards (e.g. the open science community) and the publication landscape (e.g. predatory journals, falsification of data or other ethical breaches). The same themes identified here will likely re-emerge in some form (hopefully with resolution of some issues through working groups or other scholarship in the interim) at the next conference. In the interim, we encourage the community to work collaboratively to further explore the thematic issues that we have identified that require thinking. Evidence-based environmental management has moved from concept [2] to reality (i.e., SRs are being conducted and some environmental management agencies are actively using their findings to guide decision making). A number of challenges and opportunities exist that need to be purposefully explored and embraced if we are to realize the full potential of contemporary evidence synthesis for environmental management.