The number of primary research articles that potentially provide evidence to inform environmental management is increasing year on year. Identifying and making sense of the nature and findings of relevant articles when making decisions is an almost impossible task for most decision makers [6, 12]. Consequently, evidence syntheses, usually in the form of reviews, that collate and synthesise findings from primary research, are a key step to enabling evidence-informed decision making [9]. Unfortunately, as with primary research, syntheses can be misleading if they are susceptible to bias or if insufficient reporting of methods hides limitations that impact on the reliability of findings [4]. Biased findings could misinform decision makers, with potentially significant consequences for the environment and society.
To address problems of bias and reliability, systematic methods for producing rigorous evidence syntheses to inform policy and practice have become increasingly standardised across various fields over the last 30 or so years [3]. Such methods are applied both to the production of syntheses aggregating research findings (Systematic Reviews) and to the creation of descriptive maps that collate and configure (arrange in a logical way) existing research (Systematic Maps). For environmental management, the first guidance on how best to conduct Systematic Reviews was published in 2006 [11] and subsequently, the Collaboration for Environmental Evidence (CEE) has provided guidance on evidence syntheses [2]. This methodological guidance, available to both authors and editors, sets out conduct and reporting standards that can reduce bias and increase reliability, but it is not clear to what extent such guidance is followed in evidence syntheses currently being published.
Decision makers may be unaware of the potential for bias in evidence syntheses, or may not have the appropriate skills or time to critically evaluate the reliability of findings for their evidence needs. This uncertainty in the provision of scientific evidence to inform management and policy decisions has gained little attention to date in environmental evidence synthesis. O’Leary et al. [8] investigated evidence synthesis reliability in environmental research using a synthesis appraisal tool (CEE Synthesis Appraisal Tool: CEESAT) developed initially by Woodcock et al. [14], that focussed on key aspects of review conduct and reporting. Using a sample of 92 reviews published in 2015 they found very low reliability ratings for most.
The CEEDER (CEE Database of Evidence Reviews) project was initiated in 2018 to provide a database of available evidence syntheses specifically on impacts of human activities or effectiveness of interventions relevant to issues in the environmental field [7]. Through extensive searching of both commercially published and grey literature sources, and screening using standardised eligibility criteria, a comprehensive global database of evidence syntheses was compiled and is updated on a regular basis [7]. CEEDER provides an opportunity to assess the reliability of evidence syntheses as they are published.
In this paper we use the generic term ‘evidence synthesis’ to mean the process of combining scientific information from multiple studies that have investigated the same question, to come to an overall understanding of what they found. As such, evidence syntheses are published or otherwise made available under many different names. Currently CEEDER recognises two forms of evidence synthesis:
-
1.
Evidence reviews (including literature reviews, critical reviews, Systematic Reviews) are defined here as syntheses of primary studies (narratively and/or quantitatively) that address a question of cause and effect and provide (or claim to provide) an aggregate measure or estimate of effect (e.g., impact of anthropogenic activity or effectiveness of an intervention).
-
2.
Evidence overviews (including Systematic Maps, scoping reviews, Evidence Gap Maps) are similar but usually address a broader question (often involving multiple causes and effects) and collate and configure evidence but do not provide an aggregate measure or estimate of effect.
More details can be found in Konno et al. [7] and on the CEEDER website (https://environmentalevidence.org/ceeder/).
The subject coverage of evidence syntheses in CEEDER
CEEDER uses specific eligibility criteria to identify evidence syntheses within its subject scope (see https://environmentalevidence.org/ceeder/about-ceeder/). This means that some subjects at the margins of environmental science and management are currently excluded, for example, some aspects of public health, toxicology, and plant and animal science. Nevertheless, CEEDER includes a wide range of environmental science subjects and can reveal which of these have been a particular focus of attention over a period of time, and which have been somewhat neglected. An example of a popular overarching subject of syntheses currently in the CEEDER database (articles published between 2018 and 2020) is global change, in the context of both impacts and interventions (e.g., mitigation efforts). Not surprisingly, there appears to be a particular interest in the impacts of climate change (e.g., changes in temperature, precipitation) on ecosystems, crops, greenhouse gas emissions, and soil organic carbon stocks. In terms of mitigation some agricultural practices (e.g., tillage options) are common. Other popular subject elements include: biochar use in soils, pollutant removal through wastewater treatment, greening urban environments, and habitat restoration.
Examples of subjects currently not found or rarely found in the database include: impacts of blue/green hydrogen technology (0 syntheses), zoonotics (2 syntheses), blue carbon sequestration (two syntheses compared to 44 syntheses involving soil organic carbon), genetically modified organisms (4 syntheses), permafrost (2 syntheses), glacier melt (1 synthesis), poaching (1 synthesis) and wildlife trade (0 syntheses).
As an example of change in frequency of a subject over time, “ecosystem services” increased in frequency year by year in (author defined) keywords: 2018: 11 syntheses, 2019: 15 syntheses, 2020: 18 syntheses.
The geographic origin of evidence syntheses by author affiliation
Based on corresponding authors’ affiliation to countries, evidence syntheses included within CEEDER currently represent 71 different countries (Fig. 1). Of these, eight countries dominate, producing more than 30 evidence syntheses each and collectively accounting for 67.2% of the total number of evidence syntheses within CEEDER (Fig. 1; numbers for each country can be found in Additional File 1).