Skip to main content

Realising the potential of environmental data: a call for systematic review and evidence synthesis in environmental management

Environmental scientists are prolific data generators. Rates of data capture and reporting are increasing almost exponentially. The publication of primary research papers in environmental journals rises every year at a rate that challenges even the most avid reader. And yes, of course, the number of journals is increasing. The accumulation of environmental data represents the output of considerable investment by many scientists, funders and stakeholders, including taxpayers. Much of this investment seeks to address the rising tide of environmental problems the human race itself has caused. Increasing political concern over the state of the environment is evident from a number of national and global initiatives to assess, predict and mitigate the effects of environmental change. Globally, the Intergovernmental Panel on Climate Change and the Convention on Biological Diversity are likely to be followed by the Intergovernmental Platform on Biodiversity and Ecosystem Services. All have a remit to assess environmental change and predict future trends. The Millennium Ecosystem Assessment has already catalysed a cycle of follow-up national and regional assessments. These bodies are (or will be in the future) urging action to reduce the impact of human activities on our environment and maintain the ability of ecosystems to provide the goods and services on which we depend. To achieve effective action requires the use of the data we have generated in a framework that informs decision making in policy and practice.

The close linkages between human wellbeing and environmental quality are increasingly evident in the literature and are now clearly embedded in the policies of institutions concerned with international development and poverty alleviation. If our planet is to remain suitable for civilised human habitation, even when there are 10 billion of us, then we need to manage its health. At a range of scales, environmental management interventions are required to maintain ecosystem function whilst providing sufficient food, water, fuel, raw material and meeting our needs for clean air, green space and wild nature. Both the necessity and the challenge of achieving effective environmental management are very clear. But how well organised are we to succeed in this task? We certainly need more data to address a multitude of questions concerning the nature of environmental problems and the effectiveness of potential solutions. But we need to do more than just accumulate data, fragment it among a multitude of publications with varied levels of access, and frankly, let much of it disappear into obscurity. It is the purpose of science to push the boundaries of knowledge and challenge accepted wisdom but if science is also to inform decisions that wider society takes then we need synthesis; a kind of stocktaking of data that establishes the current evidence base with a view to predicting outcomes of alternative actions.

Of all the interventions that are employed, which ones work? Which ones are worth the money spent on them? Which ones do more harm than good and should cease? At present there is very little synthesis of information to help us with these vital questions. Increasing expenditure on environmental management and the rise in environmental organisations, both governmental and non-governmental have been described by Keene & Pullin [1]. But the structures required to evaluate the effectiveness of environmental management interventions are piecemeal at best. Lack of concerted use of environmental data to address impacts and improve management effectiveness may seem surprising when so many data are being generated, but data are not evidence unless presented in relation to a question.

So why a new journal? It may seem strange to criticise journals in an opening editorial for a new title but a consequence of the development of the large body of journals publishing primary research and the career pressure on scientists to publish significant quantities of papers is that we have a severely fragmented literature. If we think of the purpose of applied science as being the construction of an evidence base to inform decisions then we are currently manufacturing millions of bricks [2], but they just form an untidy pile unless there is a plan for using the bricks to construct an evidence base. Of course, literature reviews have been around almost as long as the primary literature, and there are a number of review journals with an environmental scope which publish literature reviews, but somehow we forgot to agree on construction standards for them and many of the products have design faults that are not always apparent to the reader [3]. This lack of methodological standards can potentially introduce real weaknesses in the evidence base enabling a wide range of vested interests to employ selection bias to present evidence in support of their case.

In contrast, the health sector has been using the more rigorous methodology of systematic review (SR) to inform decision making on interventions for more than 20 years and now has a well established global community conducting secondary, synthetic research. The Cochrane Collaboration ( has established a highly influential library of systematic reviews of the effectiveness of health interventions ( The call for the application of a similar process to the environmental sector has been around for some 10 years [4] and has enabled the establishment of the Collaboration for Environmental Evidence (CEE) that holds its own library of systematic reviews and provides guidance on methodology ( The CEE is now becoming influential, partnering both government and non-government organisations in ensuring standards of conduct when reviewing evidence and establishing their evidence base.

The main motivation for CEE to establish this new journal is to provide a more visible and accessible platform for formal systematic review and evidence synthesis on environmental issues. Since the establishment of CEE in 2007 a growing community of scientists has been conducting SRs to CEE standards and guidelines, including full peer review. To date these have been deposited in the CEE library accessible through the CEE website. The CEE library is becoming an increasingly important resource for policy formers and managers seeking reliable evidence to inform their decision making. However, from the scientists perspective there is also a need to achieve formal publication, and therefore recognition, of their work. SRs are far more detailed (as demanded by the methodology) than most traditional literature reviews and often beyond the word limits of most current journals. The process of conducting SRs involves formal development of protocols setting out the review methodology to be followed, thus the SR process focuses on quality even at the planning stage. Protocols themselves are peer reviewed and thus authors are potentially able to publish both protocol and the resulting SR. The standard journal submission process does not fit the requirements for publication of SRs, thus we have created an alternative system at CEE that will be applied to this journal. The requirement for both protocols and SRs to be peer reviewed means that the process is more rigorous, but is also more supportive of authors and once through the preliminary stages of protocol formation, rejection rates should be low.

Because of the clinical medicine origin of SR, the methodology is often considered synonymous with randomized, controlled trials and meta-analysis of effect sizes. But both the conduct and application of systematic review principles are now much broader and are inclusive of a much wider range of study designs and synthesis techniques. Crucially, the methodology is now widely used in the applied social sciences and we wish to fully embrace these developments to address the broad scope and interdisciplinarity of environmental issues.

The central purpose of this journal is to provide an incentive for scientists to fully engage in the development of evidence-based environmental management by providing a publication outlet for systematic reviews and their associated materials such as protocols and systematic maps. However, the development of evidence-based environmental management requires other contributions. There are many methodological aspects of systematic review and evidence synthesis where advances may bring new opportunities for communicating the state of the evidence base. Consequently we welcome submission of methodology papers reporting new techniques and approaches to systematic review and evidence synthesis. These could cover social aspects of question setting with policy makers and stakeholders, methods of searching for data, methods of synthesis, through to methods of communicating results, and the uncertainty surrounding them, to non-scientists. We are acutely aware of the problem of communicating outcomes of systematic reviews to decision makers and to this end we will be calling for contributions in the form of policy briefs or similar innovative short communications that summarise systematic review outcomes without recourse to technical jargon but which retain the true significance of and confidence in, the results. Lastly, we don’t pretend that a systematic review is the last word on any subject. We welcome short communications that comment on a published SR and, consistent with CEE guidelines, we will be encouraging the updating of SRs so that decision makers can judge the impact of new data and analyses on the evidence base.

The test for this journal and the CEE is whether they can demonstrate benefit to the environmental science community, the environmental policy community and, ultimately wider society, through more effective environmental management. They will not pass this test without the support from members of these communities and I would like to thank the founding Editorial Board and the many contributors to CEE for their support and for being the first to step up to the plate. So, not just another journal: a rather distinctive journal providing the focus of a new collective vision to provide the evidence base for greater effectiveness in environmental management.


  1. Keene M, Pullin AS: Realizing an Effectiveness Revolution in Environmental Management. J Environ Manage 2011, 92: 2130–2135. 10.1016/j.jenvman.2011.03.035

    Article  Google Scholar 

  2. Forscher BK: Chaos in the brickyard. Science 1963, 142: 339.

    Article  CAS  Google Scholar 

  3. Roberts PD, Stewart GB, Pullin AS: Are review articles a reliable source of evidence to support conservation and environmental management? A comparison with medicine. Biol Conserv 2006, 132: 409–423. 10.1016/j.biocon.2006.04.034

    Article  Google Scholar 

  4. Pullin AS, Knight TM: Effectiveness in Conservation Practice: Pointers from Medicine and Public Health. Conserv Biol 2001, 15: 50–54.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Andrew S Pullin.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Pullin, A.S. Realising the potential of environmental data: a call for systematic review and evidence synthesis in environmental management. Environ Evid 1, 2 (2012).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: