Skip to content

Advertisement

  • Systematic Map
  • Open Access

Absence of evidence for the conservation outcomes of systematic conservation planning around the globe: a systematic map

  • 1Email authorView ORCID ID profile,
  • 2,
  • 2,
  • 2,
  • 2,
  • 3,
  • 4,
  • 5 and
  • 1
Environmental Evidence20187:22

https://doi.org/10.1186/s13750-018-0134-2

  • Received: 10 May 2018
  • Accepted: 10 September 2018
  • Published:

Abstract

Background

Systematic conservation planning is a discipline concerned with the prioritisation of resources for biodiversity conservation and is often used in the design or assessment of terrestrial and marine protected area networks. Despite being an evidence-based discipline, to date there has been no comprehensive review of the outcomes of systematic conservation plans and assessments of the relative effectiveness of applications in different contexts. To address this fundamental gap in knowledge, our primary research question was: what is the extent, distribution and robustness of evidence on conservation outcomes of systematic conservation planning around the globe?

Methods

A systematic mapping exercise was undertaken using standardised search terms across 29 sources, including publication databases, online repositories and a wide range of grey literature sources. The review team screened articles recursively, first by title only, then abstract and finally by full-text, using inclusion criteria related to systematic conservation plans conducted at sub-global scales and reported on since 1983. We sought studies that reported outcomes relating to natural, human, social, financial or institutional outcomes and which employed robust evaluation study designs. The following information was extracted from included studies: bibliographic details, background information including location of study and broad objectives of the plan, study design, reported outcomes and context.

Results

Of the approximately 10,000 unique articles returned through our searches, 1209 were included for full-text screening and 43 studies reported outcomes of conservation planning interventions. However, only three studies involved the use of evaluation study designs which are suitably rigorous for inclusion, according to best-practice guidelines. The three included studies were undertaken in the Gulf of California (Mexico), Réunion Island, and The Nature Conservancy’s landholdings across the USA. The studies varied widely in context, purpose and outcomes. Study designs were non-experimental or qualitative, and involved use of spatial landholdings over time, stakeholder surveys and modelling of alternative planning scenarios.

Conclusion

Rigorous evaluations of systematic conservation plans are currently not published in academic journals or made publicly available elsewhere. Despite frequent claims relating to positive implications and outcomes of these planning activities, we show that evaluations are probably rarely conducted. This finding does not imply systematic conservation planning is not effective but highlights a significant gap in our understanding of how, when and why it may or may not be effective. Our results also corroborate claims that the literature on systematic conservation planning is dominated by methodological studies, rather than those that focus on implementation and outcomes, and support the case that this is a problematic imbalance in the literature. We emphasise the need for academics and practitioners to publish the outcomes of systematic conservation planning exercises and to consider employing robust evaluation methodologies when reporting project outcomes. Adequate reporting of outcomes will in turn enable transparency and accountability between institutions and funding bodies as well as improving the science and practice of conservation planning.

Keywords

  • Conservation assessment
  • Prioritisation
  • Resource allocation
  • Evidence synthesis
  • Protected areas
  • Implementation

Background

Conservation planning is “the process of making informed conservation decisions” [1]. It is particularly important given the scale and complexity of policy and institutional agendas when it comes to spatially allocating resources. An approach to conservation planning that has received widespread interest for its evidence-based approach amongst academics [2, 3], conservation organisations [4, 5] and government departments [68] alike, is the discipline of systematic conservation planning [9].

Systematic conservation planning emerged in the 1980s and 1990s as a response to the tendency of conservation decisions to be made in an ad hoc manner. The majority of terrestrial protected areas were designated in places that were steeply sloped or otherwise unsuitable for agriculture, rather than where biodiversity was highest and most in need of protection [10, 11]. Systematic conservation planning built on ranking approaches, popular before the early 1980s, by incorporating quantifiable objectives and enabling assessments of trade-offs between competing conservation and cost-effectiveness considerations [12].

Ecological principles such as representation and persistence are central to systematic conservation planning [2]. Terminology has changed over time (for political expediency; Pressey, in prep) since the CAR (comprehensiveness, adequacy, representativeness) principles that first drove the discipline. Here, representation refers to the extent to which a set of reserves samples the full biodiversity of a region (combining both the original uses of comprehensiveness and representativeness). Persistence (originally framed in terms of adequacy) means the long-term survival of species or other elements of biodiversity, including diverse natural processes at a variety of scales [13], often approached through connectivity of multiple species and habitats across landscapes and seascapes.

Systematic conservation planning proposes a structured, consultative process for choosing between, locating, configuring, and implementing conservation actions, often involving input from policy makers, land managers and resource users. Conservation objectives are specified quantitatively, in one of two ways [14]. First, and most commonly, objectives are expressed as threshold amounts of natural features relative to a baseline. An example is to cover at least 20% of each vegetation type, with no explicit added benefit for amounts over 20%. Second, objectives can be defined as continuous functions that accrue benefit up to 100% coverage of features. The outputs are optimal or near-optimal sets of spatially-bounded conservation actions [15, 16] (Fig. 1).
Fig. 1
Fig. 1

The stages of systematic conservation planning, reproduced from [15] and originally modified from [27]. Conservation planning approaches that are more systematic tend to follow these general steps. We define systematic conservation planning initiatives as those that also make use of spatial prioritisations and associated computational decision-support tools during stages 8 and 9 (boxed)

During the planning process, spatial conservation prioritisations (also called conservation assessments) are conducted, often using decision support software such as Marxan [17], Marxan-with-Zones [18], Zonation [19], and C-Plan [20] (Fig. 1) to sort through the vast number of potential spatial configurations. While often equated with systematic conservation planning, prioritisations are analytical exercises making up only a subset of the overall process [16].

Rigorous evidence is central to systematic conservation planning. This generally includes spatial data for biodiversity or environmental surrogates [13] and socio-economic cost predictors such as land acquisition costs or willingness of landowners to support proposed conservation activities [21]. Plans often also include assessments of vulnerability to future climatic regimes [22] and analyses of scheduling conservation actions over time [23].

The discipline of systematic conservation planning has had a major influence on conservation planning practices globally. It is used extensively by environmental organisations and government agencies alike [1]. Thousands of academic publications focus on the discipline, a trend that appears to be increasing [2, 24]. Marxan alone had over 6700 users from 184 countries between 2011 and 2016 [25]. Efforts are currently underway to centralise records about where systematic conservation plans have been developed [26], as the number of total plans is currently unknown.

This type of planning is resource intensive and can cost millions of dollars over several years [27]. Despite the influence of the discipline and the importance of evidence when developing plans, there is very little rigorous evidence about whether systematic conservation planning is effective at improving biodiversity conservation outcomes [28]. A compilation of such studies is therefore much needed.

This study outlines the results of a systematic mapping exercise to comprehensively assess the published literature on the effectiveness of systematic conservation planning [15]. Systematic maps are typically precursors to systematic reviews, and involve collating, describing and assessing the quality of studies assessing a particular intervention [29]. To our knowledge, this is the first systematic map of a planning intervention in the environmental sciences, and it introduces a new set of challenges in evidence availability and interpretation. Interest in this topic is not new and this study follows a preliminary protocol for a systematic review lodged with the Collaboration for Environmental Evidence in 2008 [30].

Our conceptual understanding of how systematic conservation planning interventions can lead to outcomes in terms of natural, human, social, financial and institutional capital is illustrated in Fig. 2 and expanded in Table 1. This theory of change is deliberately simple, to illustrate the potentially broad range of outcomes currently assumed to result from systematic conservation planning exercises [28]. Multiple pathways and mechanisms are likely to link systematic conservation planning to these outcomes, but these are not yet consistently defined. Our decision to report outcomes by five types of capital follows an earlier application of this framework to conservation planning [31] and related disciplines, e.g. [32]. For further details, see our published protocol [15].
Fig. 2
Fig. 2

A conceptual model for systematic conservation planning (reproduced from the protocol [15]). A range of inputs influence the planning process. Outputs and outcomes can arise throughout the planning process but, for simplicity, we have lumped the main stages during the planning process. Different types of outputs from the planning process will lead to different types of outcomes. However, given the causal chains are not yet well understood, single arrows have been used to indicate the influence of the planning process on the types of potential outputs and outcomes. Outputs are the material or legal products of inputs, such as numbers or total km2 of protected areas or numbers of boats available to patrol for illegal fishing. Outcomes are the observed or assumed effects of outputs, including reduced threat levels and improved state of biodiversity [58]. The feedback arrow from outcomes to inputs indicates the adaptive approach used to modify plans subject to observed outcomes

Table 1

Potential outcomes of systematic conservation planning arranged according to capitals.

Categories adapted from the typology developed by Bottrill and Pressey [28]

Capital

Definition

Outcome sub-category

Natural

The stock and flow of goods and services provided by ecosystems, including the diversity of species, regulating processes, and supporting services [87]

Reduction in loss or degradation of natural values

Persistence of biodiversity

Maintenance of ecosystem services

Financial

The gain or loss of cash, property or assets that represent the economic value of an individual or organization

Transparency in conservation investments

Efficiency of operations

Maximised benefit given limited budget

Leverage of additional funds or in-kind support

Social

Represents the relationships and interactions between individuals and groups [88]

Collaboration among agencies

Coordination between different actors

Trust in planning process

Sharing datasets between agencies

Shared vision

Attitudes of stakeholders

Power dynamics between stakeholders

Human

Knowledge or skills that enable people to develop strategies to achieve their objectives [89]

Raised awareness of biodiversity or conservation

New knowledge of ecological or social values

Learning applied in future plans

Institutional

Capacity, structure, or functioning of institutions through formal (e.g. laws) or informal means (e.g. local governance practices) [90]

Influence on future decision making by organisation or partners

Self-sustaining strategies

Role of implementing agency

Consideration of conservation issues in decision making by other sectors

Integration of priorities into policies, conventions or legislation

Influence on resource-use planning

Protected areas expanded

Representativeness was removed as an example of a natural capital outcome [58]

Stakeholder engagement

Subject experts (ranging from academic researchers to staff in environmental NGOs responsible for conducting systematic conservation plans) were consulted throughout the protocol development, searching and analysis stages. This occurred in several ways. Calls were put out for comment at relevant conferences and workshops, alongside presentations and posters about the project. Subject experts were also invited to share potentially relevant studies (see methods) and this usually led to email or phone discussions about the research and findings, as well as recommendations for additional contacts.

Objective of the systematic map

We sought to identify retrospective studies that measured the effects of systematic conservation planning exercises on biodiversity conservation at various scales. Our primary research question was: What is the extent and distribution of evidence on conservation outcomes of systematic conservation planning around the globe? The definitions used to focus our search are provided in Table 2.
Table 2

Subject, intervention, outcome and comparator search elements

Search element

Definition

Subject (population)

All countries and marine, freshwater and terrestrial realms. Studies published between 1983 and 2017

Intervention

Systematic conservation planning: a process for locating and implementing conservation actions where: (a) the benefits of conservation actions are specified either as threshold amounts of natural features to be represented or as continuous functions with increasing amounts of features; and (b) the outputs are one or more optimal or near optimal sets of spatially-bounded conservation actions

Comparator

Comparisons over time, and/or between control and intervention groups and/or sites without systematic conservation planning or with another form of planning

Outcome

Studies measuring changes in the condition of one or more of the following forms of capitals: natural, financial, social, human and institutional (either quantitatively or qualitatively)

Our intent was to categorise included studies using a data extraction framework. The framework was designed to explore the following secondary questions:
  • What are the characteristics of the current evidence base, including study location, scale and study design, intervention and outcome type?

  • What types of outcomes of systematic conservation planning exercises are measured, either by the original planning organisation(s) or others?

  • What types of study designs are used in evaluations of systematic conservation planning?

  • How robust is existing evidence? How many impact evaluations have been conducted, where and by whom?

Methods

Our systematic map protocol has been published in Environmental Evidence [15] and this section includes updates since that publication. Updates include the use of the software CADIMA, consistency checks prior to every screening stage and abandonment of an attempt to undertake the bulk of screening by one reviewer (up to six were involved in the most time-consuming stages). Also, no evidence matrix was produced due to the small sample size. Further details have been provided in related sections of the methods.

Search strategy

A search string, consisting of subject- intervention- and outcome-related keywords combined using Boolean logic and wildcards, was used to query publication databases, search engines and online repositories (Table 3). Searches were conducted in February and March 2017. Unless otherwise indicated, searches were conducted for studies produced between 1983 and 2017 inclusive, in English only given resource constraints. Publications with full-texts not in English were listed and recorded separately for future iterations of the map or for other interested parties to pursue (see “Results”).
Table 3

Search terms and strings used for searching online databases and websites

Publication database search terms (formatted for Web of Science)a

Subject terms

AND

Intervention terms

AND

Outcome terms

AND

Qualifier terms

TS = (aquatic OR “river basin” OR ecoregion* OR bioregion* OR terrestrial OR marine OR freshwater OR coastal OR landscape OR seascape OR catchment OR “coastal zone” OR “ecological network” OR corridor OR “conservation area” OR “reserve network” OR “protected area” OR “national park” OR “planning unit”)

AND

TS = (“conservation plan*” OR “spatial plan*” OR “conservation assessment” OR “reserve selection” OR “area selection” OR “reserve design*” OR “conservation zoning” OR “key biodiversity area” OR “important bird area” OR “spatial priorit*”OR “conservation priorit*” OR “conservation area priorit*” OR “spatial optimi*” OR “protected area network design” OR “resource allocation” OR “conservation decision making” OR marxan OR zonation OR “C-Plan” OR RobOff OR BioRap OR CLUZ OR ConsNet OR CPLEX OR CREDOS OR “Ecoseed Marzone” OR MinPatch OR MultCSync OR NatureServeVista OR ResNet OR SPEXAN OR “conservation evaluation” OR “area identification” OR “decision-support tool” OR “conservation action”)

AND

TS = (outcome* OR evaluat* OR output* OR impact* OR effect* OR ineffective OR success* OR fail* OR benefit* OR implement*)

AND

TS = (biodivers* OR wildlife OR species OR habitat)

Google Scholar search terms

 

“Conservation plan” “conservation planning” “spatial plan” “conservation assessment” “reserve design” “conservation zoning”

aFollowing sensitivity testing during initial pilots of the searches (in November 2016), the search terms were revised from the original protocol [15]. Terms were removed if they either inflated the number of search terms returned in an unspecified way (i.e. adding many irrelevant search results), or if they did not add value to searches based on similarity to existing terms

To ensure wide coverage of potential peer-reviewed academic publications and grey-literature publications [33], we searched three publication databases, one search engine, three online repositories, and 21 organisational websites (Table 4). Studies were also identified opportunistically, via calls for papers at major international conferences, backwards and forwards citation searches of included studies (in March 2018) and use of review papers to identify related primary studies. Subject experts were contacted to confirm no key references were missed. These individuals were identified primarily by their roles as conservation planning experts within global conservation NGOs, such as Conservation International, and prominent national organisations, such as the South African National Biodiversity Institute, and through snowball sampling (many recommended additional subject experts). Experts also included academics whom the authors knew had an interest in this topic. The detailed protocols employed to search each source, including tailored approaches to individual databases and suggestions from subject experts, are outlined in Additional file 1.
Table 4

Sources searched and search methods according to source type

Source type

Source

Weblink

Search methoda

Publication database

Web of Science ™ Core Collection

https://webofknowledge.com

Searched on title, abstract and keywords using the publication database search string adapted for each database

All results extracted

SCOPUS

https://www.scopus.com

CAB Abstracts

https://www.cabi.org/publishing-products/online-information-resources/cab-abstracts/

Search engine

Google Scholar

https://scholar.google.co.uk

Searched on title only using the Google Scholar search string

A second search required ‘evaluation’ to also appear in the title

For both searches, the first 1000 results were retrieved using the software Harzing’s Publish or Perish [91].

Online repositories

Open grey

http://www.opengrey.eu/

Searched on title only, using the Google Scholar search string

PAIS international (ProQuest)

http://www.proquest.com/products-services/pais-set-c.html

Proceedings First (OCLC Online Computer Library Center, Inc.)

http://www.oclc.org

Papers First (OCLC Online Computer Library Center, Inc.)

http://www.oclc.org

Websites of specialist organisations and online databases

Campbell collaboration

http://www.campbellcollaboration.org/lib/

Where possible, organisational databases and websites were searched on all text using the Google Scholar search string with Boolean operators

Where within-website searches could not be conducted on the whole Google Scholar search string using Boolean operators, individual terms were searched, and the resultant search lists combined, and duplicates removed

Center for International Forestry Research (CIFOR)—library

http://www.cifor.org/library/

Environmental evidence

http://www.environmentalevidence.org

Conservation evidence

http://www.conservationevidence.com

Coral triangle initiative

http://www.coraltriangleinitiative.org

Commonwealth Scientific and Industrial Research Organisation (CSIRO), Australia

https://publications.csiro.au/

Evidence on demand

http://www.evidenceondemand.info/library.aspx

IUCN evaluations database

https://www.iucn.org/monitoring-and-evaluation/monitoring-our-work/evaluations-database

IUCN Library System

https://portals.iucn.org/library/dir/publications-list

Natureserve

http://www.natureserve.org/biodiversity-science/publications

Poverty and Conservation Learning Group—Bibliographic database

https://www.povertyandconservation.info/en/bibliographies

Poverty and Conservation Learning Group—Biodiversity-poverty evidence database

https://www.povertyandconservation.info/biodiversity-poverty-evidence

Poverty and Conservation Learning Group—PCLG publications

https://www.povertyandconservation.info/en/pclg-publications

Protected Planet

http://www.protectedplanet.net

SANBI Planning database, South Africa

http://www.sanbi.org/information/documents

The Digital Observatory for Protected Areas (Europe)

http://dopa.jrc.ec.europa.eu/en/documentation

United Nations Environment Programme—World Conservation Monitoring Centre

https://www.unep-wcmc.org/resources-and-data

Biodiversity Heritage Library

http://www.biodiversitylibrary.org/

USAID Development Experience Clearinghouse (Evaluations)

https://dec.usaid.gov/dec/content/evaluations.aspx

USAID Biodiversity Conservation Gateway

https://rmportal.net/biodiversityconservation-gateway

 

World Wildlife Fund for Nature International

https://wwf.panda.org

 

Opportunistic

Four academic conferences

Subject experts

 

General calls were made during presentations and on Twitter

Requests were made via email

aDetailed records of the dates and criteria used for each search are included in Additional file 1

Data management

Following the searches, article citation details and abstracts were extracted in.ris form (or converted to.ris form using a freely accessible conversion template developed for our purposes by the EPPI-Centre [34]). When searching websites, the web scraping software Parsehub (https://www.parsehub.com) was used to extract records [35, 36]. Settings were tailored to each website and details are provided in Additional file 1.

Results were initially imported into the software EPPI-Reviewer (V.4.5.1.0 [37]), which was used to detect and remove duplicate articles. Prior to title screening, duplicate checked results were imported into the systematic review management software CADIMA [38].

Article screening and inclusion criteria

Articles were screened in three stages: title only, abstract, then full-text. When screening articles for relevance, a series of inclusion and exclusion decisions were consistently applied (Table 5). Regular reviewer team meetings were held throughout all screening stages to ensure criteria were being applied uniformly, and detailed decision rules were tailored for each screening stage in these meetings. During our discussions about definitions of systematic conservation planning and tightening the inclusion criteria, we revised our original positions and excluded several studies we had expected to include at the protocol development stage (from the test library, including [7, 39, 40]). If there was any doubt about the relevance of an article, it was included for evaluation in the subsequent screening round to avoid removing potentially relevant studies. Two of the authors (MCM and RLP) have published extensively in this field so were not involved in article screening to reduce the likelihood of authors reviewing their own work.
Table 5

Article inclusion and exclusion criteria, summarised from the protocol [15] with practical considerations adopted during screening to increase clarity and consistency in the application of the criteria

Screening criteria

Relevant

Irrelevant

Practical clarifications

Subject

Inclusive of all countries and marine, freshwater and terrestrial realms

Studies published between 1983 and 2017

Global-scale plans

Studies published prior to 1983

Studies conducted at continental or smaller scales were included

Intervention

Systematic conservation planning: a process for locating and implementing conservation actions where: (a) the benefits of conservation actions are specified either as threshold amounts of natural features to be represented or as continuous functions with increasing amounts of features; and (b) the outputs are one or more optimal or near optimal sets of spatially-bounded conservation actions

This means that plans will necessarily use existing (e.g. Marxan [17], C-Plan [20] and Zonation [19]) or custom-made (e.g. linear/non-linear programming, genetic algorithms) decision-support tools in the ‘spatial prioritisation’ stages

Studies relating to plans that have no explicitly stated (or quantifiable) biological conservation objectives

Studies relating to plans that were solely expert-based approaches

Studies that do not involve the use of computerised decision-support tools

Studies were included if they approximated the stages of systematic conservation planning in Fig. 1 (e.g. plans did not have to have been implemented), and involved stakeholder engagement, quantifiable conservation objectives, and a spatial prioritisation exercise

Outcome

Studies measuring changes in the condition of one or more of the following forms of capitals: natural, financial, social, human and institutional (either quantitatively or qualitatively)

Broad interpretation of outcomes to capture the breadth of intended and unintended outcomes and potential flow-on consequences for biodiversity conservationa

Outcomes that are not attributed to a systematic conservation planning process

Studies were included if they reported on changes in the condition of one or more types of capital, as a result of a systematic conservation planning.

Comparator

Comparisons over time (continuous or interrupted time seriesb), and/or between control and intervention groups and/or sites

Studies that measure at a single point in time, with no comparison to another site

Opinion-based assessments were excluded

Study design

Retrospective quantitative and qualitative experimental, quasi-experimental and non-experimental designs according to Margoluis et al. [82]

Theoretical studies, prospective models, or studies using only ex-post modelling to estimate business as usual versus future planning scenarios were excluded, as were studies based on researcher inference

Relevant study designs had to relate to the impact of a conservation action (e.g. baseline monitoring was not necessarily suitable)

To distinguish gap analyses from impact evaluations, studies using measures of representativeness in a gap analysis scenario were excluded

Opinions of the authors or unsubstantiated statements were treated as ‘researcher inference’ and excluded on study design

aA detailed discussion of what constitutes an outcome versus an impact is provided in [16]

bIn an interrupted time series, data are collected at several time points before and after an intervention [92]

Prior to commencing each screening stage, inter-reviewer consistency checks were conducted with the reviewers involved at that stage. A subset of the same titles/abstracts/or full-texts were screened by all and results were compared using Cohen’s Kappa (k) [41] (more details below). This exercise was repeated, and inclusion rules clarified, until the desired level of consistency was met (threshold k value of 0.5 at title screening and 0.7 for abstract and full-text screening). These k values exceed recommended guidelines [42]. After this, screening formally started (the articles used in consistency checks were included again in the overall screening). Articles were randomly divided between the reviewers and were not duplicate screened (except those marked as unsure, and at full-text extraction stage). Articles provided by subject experts were automatically included for screening at full-text.

Title screening

For title screening, four rounds of consistency checks were required, involving 200 randomly selected articles each time (800 titles total). Rounds were iteratively undertaken using CADIMA until screening decisions were above the k = 0.5 threshold. Publication titles were reviewed in CADIMA by two reviewers in accordance with the inclusion and exclusion criteria.

Abstract screening

Due to limitations of CADIMA (not allowing changes to the size of a reviewer team at different stages [38]), EPPI-Reviewer was used for abstract and full-text screening. Missing abstracts were entered for included articles. Six reviewers participated in abstract screening and five rounds of consistency checks were conducted, each involving 25 randomly selected articles, until results exceeded the k = 0.7 threshold (125 abstracts total).

Full-text screening

Full-text files were downloaded or accessed in hard copy through the Bodleian Libraries at the University of Oxford. Full-text articles were divided between and screened by three reviewers following six rounds of consistency checks (where 20 articles were screened by all three reviewers each round), until results exceeded the k = 0.7 threshold (120 full-texts). Reviewers included articles focusing on those with relevant subject, intervention, outcomes, comparator and study designs (Table 5). Articles were included only when all five criteria were relevant. Where insufficient information was provided to determine if the intervention met our definition of systematic conservation planning, further information was sought before reaching a decision, such as by following up cited references about the original plan. Any articles the reviewer was unsure about were flagged as ‘unsure’ and were discussed with the other reviewers prior to reaching a decision. Given inconsistencies in the use of the term “systematic conservation planning” [16], those studies for which the “intervention was similar to, but not systematic conservation planning” (according to our definition, Table 5) were marked as such. This provided estimates of the number of spatial conservation prioritisations and other related exercises detected. Review papers were also set aside, and the studies therein were assessed for relevance.

Data coding and mapping

Relevant studies were extracted from the included articles (where multiple studies were included in a single article). For example, a single paper often reported on multiple planning instances. The included studies were categorised by two reviewers according to a data coding template (Additional file 2) and any differences in coding were discussed and a third reviewer involved if necessary (undertaking kappa analysis was unnecessary at this stage due to the small sample size). Coding involved outlining bibliographic details, background information including location of study and broad objectives of the plan, study design, reported outcomes and context [15]. Information about the study design of relevant articles was recorded to determine study robustness, in place of critical appraisal. Where possible, the categories of the data coding template were updated to match those underlying a new database of marine spatial prioritisations [26] to make the two databases cross-compatible. Where necessary, we contacted authors of included studies for additional information (via email or phone).

Our finding of fewer than expected evaluations in the published literature meant the development of an evidence matrix and geographic map of included studies (as proposed in the protocol) was not possible. Instead we provided a narrative assessment of the available literature, gaps in our current state of knowledge and suggestions for how to fill these.

Procedural independence was managed by excluding co-authors with publication records in systematic conservation planning (MCM and RLP) from the screening process.

Results

Primary findings

In total, 15,054 results were retrieved from the 29 sources searched, including 5228 duplicates (Fig. 3; details in Additional file 1). A further 232 previously unscreened articles were screened based on forwards and backwards citation searches of the included studies (Additional file 3). After duplicates were removed, a total of 10,058 articles were screened by title, 5221 by abstract and 1209 by full-text. Of those included for full-text screening, 236 were either not in English or not accessible (Additional file 4). Where neither hard copy nor digital copies were found through Oxford University’s Bodleian Library or Google searches, digital copies were requested from authors via ResearchGate.net. Most of the remaining search results unavailable by full-text had incomplete citation details or were conference abstracts without associated full-text publications.
Fig. 3
Fig. 3

PRISMA diagram [93] outlining outcomes of articles at searching and screening stages. Articles excluded by full-text were often excluded based on multiple inclusion/exclusion criteria, hence values exceed the total of 970 excluded full-text articles. SCP systematic conservation planning

Reasons for the exclusion of the remaining 970 articles were recorded (Additional file 5). Seven articles were excluded outright because they were marked as reviews of systematic conservation planning evaluations (Additional file 6). If an article was excluded on one or more inclusion/exclusion criteria (Table 5) it was not included. All criteria were assessed, and most articles were excluded based on multiple criteria (Fig. 3). 893 articles were excluded on intervention, 186 of which were deemed “intervention is similar to, but not systematic conservation planning”, meaning they were either solely spatial conservation prioritisations (a computational component of systematic conservation planning), or approximated the stages involved in systematic conservation planning but did not involve the spatial conservation prioritisation stage [15].

Of the 69 studies included on subject and intervention (representing a systematic conservation planning intervention at a relevant scale, Table 5), 23 were excluded on outcome because they reported on the design and implementation of a systematic conservation plan but did not provide any details about consequences. In some cases, for example, at the time of publication it was too early to report whether the plan was implemented. This left 43 articles included on subject, intervention and outcome but excluded on study design and comparator. Only three articles were included based on all criteria (Additional file 7). One of these had been provided by a subject expert and the other two had been retrieved through publication database searches. One of the included articles reported on multiple different types of planning instances, but we extracted three relevant studies across the three articles.

Additionally, 142 articles were included on subject, study design and comparator but excluded on intervention and outcome, all of which related to formal evaluations of environmental management interventions, but not systematic conservation planning specifically.

Robustness of existing evidence

Coded data extracted from the three included studies is presented in Table 6 according to a standardised data coding template (Additional file 2). These impact evaluations were undertaken by NGOs and universities. In two studies, the same organisations undertook the evaluation as had developed the plan(s) in question.
Table 6

Details of included studies according to a standardised data extraction form (Additional file 2)

Category

Included study

  

General information

 Publication ID

32732910

27951807

27939613

 Source retrieved from

Subject expert

Google Scholar; Web of Science; Scopus; CAB Abstracts

CAB Abstracts

Bibliographic information

 Publication type

Journal article

Journal article

Journal article

 Author (s)

Fisher, Jonathan R. B.; Dills, Benjamin

Álvarez‐Romero, Jorge G.; Pressey, Robert L.; Ban, Natalie C.; Torre-Cosío, Jorge; Aburto-Oropeza, Octavio

Lagabrielle, Erwann; Botta, Aurélie; Daré, Williams; David, Daniel; Aubert, Sigrid; Fabricius, Christo

 Title

Do private conservation activities match science-based conservation priorities?

Marine conservation planning in practice: lessons learned from the Gulf of California

Modelling with stakeholders to integrate biodiversity into land-use planning - Lessons learned in Réunion Island (Western Indian Ocean)

 Journal or Publication title

PLoS ONE

Aquatic Conservation: Marine and Freshwater Ecosystems

Environmental Modelling & Software

 Publication year

2012

2013

2010

 Volume/edition

7 (9)

23

25

 Publisher

PLoS ONE

Wiley Online Library

Elsevier

 Page numbers

e46429

483–505

1413–1427

Basic information about the conservation plan

 Primary region of assessment

National

Sub-national

Sub-national

 Country (s) of assessment

United States of America

Mexico

France

 Location of study (region)

United States of America

Gulf of California

Réunion Island

 Location of study (GPS coordinates)

37.0902400, − 95.7128910

27.4803504, − 112.0303160

− 21.1203276, 55.5483399

 Name of resultant protected area network or similar (where relevant)

NA

NA

NA

 Name of the planning process

Ecoregional Assessments

Ecological Regional Assessment (ERA)

No name, parallel to the Schéma d’Aménagement Régional (SAR), a regional land use planning process

 Type of organisation leading the planning process

NGO

NGO

NA

 Planning domain area (km2)

Not provided

361,375

2512

 Type of biome(s)

Terrestrial

Marine

Terrestrial

 Start of planning process (years)

1990s

Unclear

Unclear

 Duration of planning process (years)

20

Unclear

Unclear

 Type of plan (intervention category)

Identify priority conservation actions

Identify priority conservation actions

Identify priority conservation actions

 Primary conservation status of area (IUCN category)

Not provided

Not reported/not applicable

Not reported/not applicable

 Vision statement

“The priority areas are developed with the intent of representing all relevant biodiversity features in the ecoregion by identifying many individual species, communities, and ecological systems to serve as the targets of planning efforts… The intent is that if protected, the priority areas should represent functional landscapes that ensure the persistence of the conservation targets…”

From Álvarez‐Romero et al. [46] Appendix 1: “Biodiversity conservation and natural resource management: Promote a regional focus in marine coastal conservation and management; provide a detailed portfolio of priority areas that represent the diversity and distribution of species, natural communities, and ecological systems of the ecoregion. Also, contribute to the knowledge of biodiversity of marine and coastal environments, and facilitate the definition and implementation of conservation strategies”

“In line with the current and future development challenges in Réunion Island, the operational objectives of this study were (i) to identify priority areas for conservation (ii) to provide guidelines for implementing conservation actions outside existing reserves while dealing with increasing pressuring factors in the lowlands; (iii) to “accompany” the conservation sector to negotiate land-use planning and decision-making, more particularly in relation to the new regional land-use plan and the management plan of the National Park, and (iv) to explore alternative scenarios for land-use and conservation planning”

 Broad objective(s) of the planning process

Biodiversity; ecological processes; species persistence

Biodiversity; ecological processes; fishing; species persistence

Agriculture, aquaculture; biodiversity; ecological processes; economic sustainability; forestry; restoration priorities; species persistence; urban development

 Level of stakeholder participation in planning

Not provided

Consulted

Consulted; negotiation

 Academic goals

No

Prioritizing/comparing actions; zoning/marine spatial planning/land/water use planning; scheduling; implementation

Incorporating socioeconomic costs/objectives; incorporating social/cultural values; incorporating ecological processes; incorporating ecological connectivity; incorporating threats; prioritizing/comparing actions; zoning/marine spatial planning/land/water use planning; stakeholder identification/engagement

 Type of process/actions considered in planning

Land/water protection

Land/water protection; external capacity building

Land/water protection; livelihood, economic & other incentives

 Cost of the planning process (prior to implementation)

Not provided

Not provided

Not provided

 Tool name

Not provided

Marxan

Marxan; CLUZ

Information on study design (evaluation)

 Methodology type (study design)

Non-experimental

Non-experimental

Qualitative

 Method of attribution

Correlational

Correlational

Researcher inference

 Overview of the methodology

“The lands acquired by The Nature Conservancy (TNC) were analysed using GIS to determine to what extent they were in areas defined as priorities for conservation”

Seven plans conducted in the Gulf of California were compared and experts were asked to assess their outcomes based on a standardised questionnaire. “…The similarities and differences between planning exercises were examined in terms of data, methods and outputs, how identified priorities match the existing MPA system, and whether plans have guided conservation and management actions”

The evaluation approach was largely reflexive, comparing planning sequences 2 (involving Marxan) and 3 (involving model co-creation with stakeholders), and based on “observations made by the participatory modelling investigators during and 12 months after the process”. The authors considered the “researcher’s posture in the participatory modelling process” and therefore attempted to recognise potential biases

Outcomes

 Reported outputs

Policy or plan

Policy or plan; academic paper(s)

Policy or plan; academic paper(s)

 Types of outcomes by capital

Institutional

Social; human; institutional

Financial; social; human; institutional

 Reported outcomes of planning process

Influence on future decision making by organisation or partners; integration of priorities into policies, conventions or legislation; protected areas expanded

Coordination between different actors; raised awareness of biodiversity or conservation; new knowledge of ecological or social values; learning applied in future plans; influence on future decision making by organisation or partners; role of implementing agency; protected areas expanded

Transparency in conservation investments; coordination between different actors; trust in the planning process; sharing datasets between agencies; attitudes of stakeholders; raised awareness of biodiversity or conservation; new knowledge of ecological or social values; learning applied in future plans; influence on future decision making by organisation or partners; consideration of conservation issues in decision making by other sectors

 Direction of change of outcome

Unclear

Positive

Positive

 Did the project outcomes reflect achievement of the original plan vision statement?

Not provided

Yes

Yes

 Context of study (evaluation)

 Location of lead author’s organisation (country)

United States of America

Australia

France

 Type of organisation leading the evaluation

NGO

University

University

 Is the lead organisation the same as that which originally conducted the planning process?

Yes

No

Yes

 Purpose/rationale for the study (stated reasons for undertaking an evaluation)

“The Nature Conservancy (TNC) and other large conservation organizations have invested substantial resources in developing conservation plans intended to guide their decisions about which land areas and bodies of water to conserve. However, despite the investment in developing a scientific method for prioritizing areas for conservation, the degree to which land acquisition actually follows these scientific priorities has not been investigated before now”

“While theory in conservation planning is developing quickly, there has been no assessment of the influence of new ideas on applications of marine conservation”

“…The overall goal was to test different approaches to bridge the scientific and operational communities by bringing multidisciplinary scientists and stakeholders to collaborate around the participatory development of spatial models for land-use and conservation planning”

 Hypotheses of evaluators

“Our first hypothesis was that overall the acquisition of lands should be well aligned with priority areas on the assumption that TNC chapters base their acquisition decisions on the best available conservation science. We did not expect perfect alignment for several reasons noted in the discussion section. Second, we hypothesized that there would be improvement over time in the match between science-based priorities and land protected by TNC as assessments and planning methods were increasingly formalized and improved. Our third hypothesis was that outright fee simple acquisition of land would show greater alignment with the priority areas than procuring conservation easements”

Not provided

Not provided

Outcome pathways

 Theory of change or conceptual model (for how the plan was expected to lead to intended outcomes) included in the study?

No

No

No

Fisher and Dills [43] reported on ecoregional assessments conducted across the USA, which were based on systematic conservation planning principles [44]. The authors explored the relationship between terrestrial areas prioritized for biodiversity conservation by an environmental NGO (The Nature Conservancy, TNC), and those acquired for protection by the NGO over several decades.

Lagabrielle et al. [45] outlined a terrestrial planning process for the island of Réunion which was conducted in parallel with the revision of a regional development plan. The evaluation approach was largely reflexive, comparing a planning attempt involving systematic conservation planning (which they referred to as sequence 2) and another where agent-based modelling and companion modelling were used to explore future land-use change scenarios (sequence 3).

Álvarez-Romero et al. [46] compared seven marine conservation planning exercises undertaken over a 15-year period in the Gulf of California, Mexico. One of these plans met our definition of systematic conservation planning (the Ecoregional Assessment [47]) and it alone is the relevant study discussed here. Experts on regional marine conservation issues were surveyed and asked to identify planning goals, the extent to which these were achieved and how planning outputs influenced implemented conservation actions.

Characteristics of the current evidence base

Two studies were conducted at subnational scales and one nationally. The areas of interest ranged from 2500 km2 to over 300,000 km2. Two studies concerned the terrestrial realm and one marine. We classified all three intervention types as aiming to ‘identify priority conservation actions’. None were intended for direct application. The objectives of all studies included biodiversity, ecological processes and species persistence and two also included other considerations, such as fisheries, agriculture and urban planning. Stakeholder engagement most commonly involved consultation, and in one instance also negotiation. The duration of the planning processes and associated costs were unclear in all three studies.

Types of outcomes of systematic conservation planning exercises

All included studies reported institutional outcomes, two also reported on social and human outcomes, and only one reported any financial outcomes. None reported natural capital outcomes. Examples of outcomes included sharing of knowledge between stakeholder groups and a greater awareness of the complexity of urban planning amongst participants [45] and influence of the planning process on future decision making by the organisation or partners [43, 46]. Two of the three studies reported on whether the project outcomes reflected achievement of the original plan vision statement, and both reported the vision had been achieved.

Types of study designs used in evaluations of systematic conservation planning

None of the studies provided theories of change or a discussion of how they expected the plans to lead to potential conservation (or other) outcomes. Only one stated a hypothesis for the evaluation. Two studies involved non-experimental study designs where the method of attribution was correlative. The other was qualitative, and attribution was based on researcher inference.

Review of our search methodology

One of the included studies [43] originated from the call to subject experts, but unlike the other included studies, had not been returned in the searches of publication databases (all included studies had been published in indexed journals). To explore the reasons for this, a review of the original search string was conducted in Web of Science Core Collections on 24/01/18 (Additional file 8). This confirmed the intervention and outcome search terms were appropriate, but the subject terms were not sufficiently diverse. By adding the term ‘protection’ to the subject terms, the missed article [43] was returned in the new search, along with 342 additional articles compared to the original search string (excluding a further 563 articles added to Web of Science in the 11 months since the original search). This finding can be used to help better design future searches (see “Discussion”). However, the 342 studies were not screened for relevance and none were included in the study since we would for parity necessarily have to have to repeated searches across all 29 sources beyond the original March 2017 cut off.

Discussion

Faced with prioritizing limited resources for biodiversity conservation, conservation planners are increasingly turning to systematic conservation planning tools and techniques. Their aims are to explore financially and socially acceptable trade-offs, whilst seeking to optimise representation and, usually implicitly, the persistence of species and habitats [2]. In this study, we collated articles on the application of systematic conservation planning and the outcomes of related plans. The aim was to assess the evidence base rather than produce meta-analyses or detailed syntheses. Despite retrieving over 10,000 articles from traditional academic and grey literature sources, only three studies were found to contain robust evaluations of this extensively applied intervention type and none that reported evaluation of natural capital outcomes. This highlights an important evidence gap, particularly given the amount of interest in systematic conservation planning and the significant cost of undertaking plans [28]. This is a null result, rather than a negative finding and was not completely unexpected, given the barriers to undertaking evaluations of complex interventions [48]. As stated in our original protocol, “a finding that no or few impact evaluations have been undertaken on systematic conservation plans would highlight an important gap in evaluations of the technique to date” [15: 4].

Rigorous Collaboration for Environmental Evidence guidelines [42] were used to identify three relevant studies, that provide valuable insights into how plans are conducted and how outcomes can be interpreted. However, the studies did not conform perfectly with our inclusion criteria, particularly in relation to comparators and study designs. The below considerations highlight the difficulties of interpreting studies of planning interventions, as well as the prevalence of incomplete records and challenges when attributing outcomes to complex interventions.

Contrasting evaluation methodologies

The three included study designs differed greatly and demonstrated the importance of understanding regional contexts and of interpreting results with care.

Fisher and Dills [43] undertook a meta-analysis, the assessment of a planning campaign over several decades, rather than assessments of individual plans and the causal processes that led to the outcomes of those plans. The authors were not able to provide conclusive evidence that systematic conservation planning influenced land acquisition decisions, but this finding masked the complexity underlying the value of the formal ecoregional assessment processes. While they did find that the land acquisition patterns by TNC were positively correlated with priority areas in all states across the USA, no difference was observed in land acquisition patterns before and after systematic ecoregional assessments were implemented.

Rather than a before/after analysis, Álvarez-Romero et al. [46] compared seven different approaches to planning in a single region, five of which approximated systematic principles although only one met our definition of systematic conservation planning. The plans overlapped spatially and temporally to some degree, and the authors employed both quantitative and qualitative data collection methods. The six plans that did not meet our definition of systematic conservation planning are not ideal counterfactuals for the relevant plan given that datasets and experiences from earlier plans often influenced subsequent planning processes. Furthermore, these plans were not explicitly contrasted to the relevant plan in the results, as data were aggregated across all seven plans. Use of a survey methodology facilitated an assessment of how goals, outcomes and spatial areas recommended for protection differed between the seven plans. Surveying people one step removed from the plans in question provides an example of how to limit bias when reporting on planning outcomes.

The third included study by Lagabrielle et al. [45] was considered borderline in terms of constituting a qualitative study design in part because it directly reported on the opinions of the authors, with little explanation of causal links or justifications for reported outcomes. We included it because the authors considered the “researcher’s posture in the participatory modelling process” [45:1425] and therefore attempted to recognise potential biases. They provided thoughtful reflections on the scientists’ role in the process, and comparisons between the experience of conducting a systematic conservation planning study using Marxan, with a process where stakeholders co-created models and planning outputs with scientists.

Another article we screened, by Carter et al. [49], provides a valuable example of how to conduct an evaluation of the outcomes of conservation plans, even though it was not included. The authors quantified land management actions (i.e., changes in the amount, location and land cover type of protected areas) in relation to individual state land management plans in Wisconsin, USA. They found that land protection activity increased in prioritized regions after plans were released, an effect that was high for local land protection projects but weak for broader, state-wide plans (a finding consistent with that of Fisher and Dills [43]). They also noted that most actions occurred within the first 5 years after a plan was released and decreased over time. Two plans discussed in this article were similar to systematic conservation plans but were excluded on intervention and outcomes. It remains an insightful article, exploring the causal pathways by which plans may influence conservation actions.

Overall, we identified 43 studies with relevant subjects, interventions and outcomes, but which we were unable to include due to unsuitable study designs and comparators (Additional file 2). In general, the evaluation strategies were not sufficiently rigorous for inclusion, or to independently verify claims. This is not uncommon in studies of environmental interventions [50, 51]. Despite suggestions that conservation experts are unaware of how to conduct high quality evaluations [51], recent research suggests the main barriers to undertaking such studies relates to a lack of funding and time constraints, as well as availability of baseline data, lack of forward planning, and availability of a suitable control group [48].

Four studies were excluded by comparator only [7, 31, 52, 53]. Authors of similar systematic maps have occasionally been flexible on screening by comparator [54], noting it is particularly unusual and challenging to employ use of comparators in multidimensional interventions such as planning. Instead we have been clear about the fact that our included studies do not perfectly match the inclusion criteria and list our reasons for exclusion (Additional file 5).

Lack of clarity around intervention definitions

The plans underlying all three included studies varied in the degree to which they met our definition of systematic conservation planning. For example, in Fisher and Dills’ study [43] the methods underpinning TNC’s ecoregional assessments [44] draw heavily from the systematic conservation planning literature. The use of spatial conservation prioritisations and computational decision-support tools is promoted by TNC, but is not mandatory, in contrast to our definition of this intervention (Table 5). We were not able to determine what proportion of ecoregional assessments in the included study involved the use of computational tools, and the authors acknowledged this type of information was not always available.

Very few full-text articles met our specific definition of systematic conservation planning (n = 80). Our definition is heavily drawn from the academic literature and may be better interpreted as a set of guidelines, rather than a distinct intervention. Groves and Game [1] recommend that conservation planners decide on which tools to include within a planning process depending on their specific needs, available funds and teams’ skill sets. It appears that this is a better representation of how planning processes are conducted in practice than the 11-step framework we used to help define the intervention (Fig. 1). The latter may be better interpreted as a list of components from which planners pick and mix. However, expanding a study of this nature to include all conservation planning studies that are systematic to some degree would introduce high levels of variation, making comparisons extremely challenging.

For these reasons, it may be appropriate to review strict definitions of systematic conservation planning, including our own. Alternative conceptualisations of conservation planning as an intervention may be required. In addition, research is warranted on the relative benefits and limitations of following some, rather than all, planning stages of outlined in Fig. 1.

Predominance of publications on prioritisations rather than planning

This systematic map illustrates how rarely evaluations of systematic conservation plans are published. Yet, despite their novelty, none of the three included studies was published in the most common journals associated with the discipline. According to a review conducted in 2012 [2], the three journals with the greatest number of articles on systematic conservation planning are Biological Conservation, Conservation Biology and Biodiversity and Conservation. Citation searches in Google Scholar revealed our three included studies have been cited between 18 and 65 times over the 5–8 years they have been in publication. This is low in comparison with a seminal paper on systematic conservation planning, cited over 4500 times since 2000 [9].

Our results corroborate claims that the literature on systematic conservation planning is dominated by methodological studies, rather than those that focus on implementation and outcomes, and support the case that this is a problematic imbalance in the literature [55]. Spatial conservation prioritisations made up the majority of the 186 studies we marked as “intervention is similar to, but not systematic conservation planning”. These were generally undertaken as hypothetical exercises or to propose a methodological innovation, rather than being linked to a broader planning and implementation strategy. Given estimates of the number of published prioritisations (e.g. 160 in the marine realm alone [26]), our finding of 186 related studies is lower than expected. This may be because we included evaluation terms in the search strings and excluded articles that were clearly only spatial prioritisations at the abstract screening stage.

Several authors acknowledged that they specifically decided not to conduct spatial conservation prioritisations as part of their (otherwise systematic) planning processes [56, 57]. Kirlin et al. explained “this technique [use of Marxan software] was explicitly rejected in the initiative as inconsistent with the legal requirements… regarding network design and not sufficiently transparent to policy makers or stakeholders” [56:4]. Lagabrielle et al. also stated that they had to abandon a planned aspect of stakeholder consultation alongside their development of prioritisations in Marxan because “…this tool embeds strong hypothesis about land-use management and conservation, such as, for instance, the attribution of a value to biodiversity features. The participants globally disagreed with this approach and thus rejected the tool” [45:1424].

Another insight revealed in this study was the high number of plans that focused on reporting the representativeness of protected area networks (or similar) rather than measures of species’ persistence or other measures of impact (and therefore had to be excluded). A protected area network might contain representations of each habitat type, but still fail to ensure species persistence, perhaps because of the inadequate size of, or connectivity between, habitat patches or inadequate representation of the habitats most in need of protection [58]. Too narrow a focus on representativeness in systematic conservation planning risks plans failing to secure healthy populations and species as intended.

Comparison with related reviews of systematic conservation planning

In a recent survey, conservation planning practitioners reported much higher rates of plan implementation and downstream outcomes than is observable in the published literature [59]. This finding is supported by unpublished outcomes provided by the lead author of one of the included studies “Our research worked so well (in terms of impacts) that I was recruited at the university of Réunion Island and the PhD co-student I worked with… is now in charge of developing and monitoring official regional land-use and risk management plans for the Regional Councile of Réunion Island” (Personal communication, Erwann Lagabrielle via email; 4 December 2017). Following up related studies (like Lestrelin et al. [60] in this case) can also indirectly demonstrate how institutional approaches to planning and stakeholder participation evolve following a systematic conservation planning exercise and how new knowledge and data can benefit subsequent decision-making. However, it is often necessary to ask authors which studies are directly linked.

In contrast to a recent review on the same topic [61], albeit more limited in scope, our results suggest there is insufficient evidence to claim whether systematic conservation plans are or are not achieving conservation goals. Through application of rigorous systematic mapping methodology, we identified two relevant terrestrial studies which the authors of that review did not appear to locate. However, we concur that more detailed examinations of how plans are implemented in specific regions would be useful and reaffirm statements made by Bottrill et al. in 2012 [31], “Empirical evidence is not available to support the belief in the benefits of planning”.

Other systematic maps and reviews in the environmental sciences have also reported few or no relevant studies, despite also focusing on widely applied, and well-funded disciplines [6264]. It is important to share such results to illuminate knowledge gaps, limitations with study quality and to avoid duplicating enquiries or assumptions [65].

It is useful for practitioners and decision makers to know when an intervention has not been evaluated to support claims of effectiveness. However, the lack of a consensus on the outcomes of systematic conservation planning does not imply there is no evidence at all. Evidence can take multiple forms, including anecdotal observations from subject experts. For example, the 43 studies which reported the outcomes of systematic conservation planning interventions but which did not meet our rigorous inclusion criteria for study design and/or comparator (Additional file 7) are still valuable to decision makers [66]. Examples of anecdotal outcomes included; “…the initial conservation planning map… has already been used…to block proposed afforestation permits that would have destroyed an area of high conservation value…” [67: 10], and “…pooling of resources, expertise, and capabilities was one of the enabling features in delivery of the new zoning plan…” [7: 1742].

Limitations of the map

Comprehensiveness of search

The three included studies were located from different sources, reinforcing the importance of a multi-pronged strategy. The fact that experts in the field confirmed they were not aware of other published studies provided further support for our results (Additional file 1). Therefore, we doubt the inclusion of additional sources, non-English publications or contact with more subject experts would have made a major difference to our results.

A test library of eight studies we had expected to be included was used in the design and testing of the search string (Additional file 1 in the protocol [15]). This test library was also used to test the comprehensiveness of the search results returned after sources were searched and duplicates removed. Six of the test library articles were returned by our searches, two were not. One was an organisational report [68] so does not appear to have been accessible through the grey literature sources we searched. The grey literature is necessarily harder to search (e.g. some databases we hoped to include did not facilitate the mass exporting of search results, Additional file 1). The other test library article [69] appears to have included subject terms other than those in our search string (this issue is addressed further below).

Future revisions to this search strategy should consider broadening the search string. A keyword we recommend for inclusion is ‘ecoregion’ or ‘ecoregional assessment’, a term used by The Nature Conservancy to encompasses systematic conservation planning. These terms were trialled and not included in the search strings because they did not initially appear to contribute additionally to search results. Many ecoregional assessments were returned through our searches, but others may have been missed.

Limitations with our chosen subject terms apparently arose from the discipline of systematic conservation planning being relevant to a diverse array of subjects. Without significantly expanding the subject search terms to include a potentially limitless spatial terms (e.g. ‘protection’, ‘natural resource’, ‘forest’, ‘pasture’ and so on), relevant studies may be missed. Excluding the qualifier search terms also works to broaden the results, but a careful balance is required to avoid returning too many irrelevant search results.

There is a small risk that literature that does not conform with the academic jargon around systematic conservation planning was overlooked during screening. However, our inclusion and exclusion criteria (Table 5) were specifically designed with this risk in mind, focusing instead on key characteristics of plan design and implementation.

Efficiency in the review process

During the process of conducting this review we experimented with various technological approaches to increase the efficiency of screening thousands of articles. This included trialling specialised software to manage the screening process [38]. Automated duplicate checking, and web scraping also led to significant time savings. Despite progress in the use of machine learning to help automate abstract screening [70, 71], we concluded this technology is not sufficiently developed to have been applicable without further testing. Innovative methods are much needed, particularly as the size and scope of maps and reviews continue to increase [72, 73].

Conclusions

Implications for policy and management

The lack of rigorous evidence for the impacts of systematic conservation planning is of considerable concern. Additional studies are urgently needed to understand the work of governments and environmental NGOs applying these methods around the globe. A strong assumption from theory is that systematic conservation planning is more effective at conserving species and habitats than alternative approaches to allocating resources for conservation (be they ad hoc, driven by extractive use considerations, or based primarily on stakeholder negotiation). Given the focus on evidence, it is consistent with the epistemology of the discipline to ask whether there is evidence for effectiveness, and to suggest ways in which such evidence might be made more easily available. However, there are many theoretical and practical reasons to believe systematic conservation planning is preferable to alternative approaches (or doing nothing) [74]. The race is on to protect sufficient areas of the land and ocean, increasing the importance of core systematic conservation planning principles like representation of species and habitats, clear objective-setting and well-designed stakeholder engagement [11].

Implications for research

In this study, we confirmed a growing evidence-base for the suitability of different methodological approaches for spatial conservation prioritisations [75, 76]. We also found some evidence of how systematic conservation plans are being implemented, including lessons learnt [8, 67, 77, 78]. However, rigorous impact evaluations are lacking. A full set of guidance is likely beyond the scope of this study, but some recommendations are provided below.

To improve the quality of future evaluations of conservation plans, we suggest conservation planning organisations provide incentives and time for staff to write up their findings and make them publicly available (even if not in an academic journal). In addition, conservation planners and academics would benefit from collaborations to leverage the additional resources and skills required to complete evaluation studies. Long-term ecological monitoring and reporting combined with adaptive management [79] and standardised metrics of management effectiveness [80, 81] are extremely valuable. More examples of ways to improve evaluation in systematic conservation planning are offered by McIntosh et al. [16].

To improve project documentation, we suggest including: (a) explicit statements about the objectives of a systematic conservation planning process and quantitative or qualitative evidence for whether those objectives were met; and (b) detail about the intervention i.e., how the planning process was conducted, to enable readers to determine whether the process has the attributes of a systematic conservation planning project. To improve evaluation documentation, we suggest including: (a) theories of change about how specific aspects of the planning process were expected to lead to particular outcomes (and where possible, whether results led to a modified understanding of the theory of change); (b) clear descriptions of the study design (with reference to existing classifications e.g. experimental or qualitative sampling [82]) and discussion about any limitations with the chosen study design; and (c) where possible, presentation of a range of perspectives on the outcomes and potential causal links with the planning process.

When designing an evaluation in these contexts, it is preferable to focus on improving the minimum standards of evaluation, rather than expecting to achieve ‘best practice’ evaluation methodologies [48, 83]. Based on our included studies and related literature on barriers to evaluation [48], the most practical counterfactual study designs for conservation planning are likely to include comparisons before/after the planning process. As a minimum, authors should identify potential alternative explanations (external to the planning process) for observed outcomes and explain their understanding of the relative importance of different potential causal processes. Examples include stakeholder perceptions of the relative importance of a local election in promoting political action, or a natural disaster having reduced interest in planning versus on-ground action. For more see McIntosh et al. [16]. Given that systematic conservation planning constitutes a form of ‘dynamic planning’ [77], dynamic approaches to evaluation may also be necessary. This could include elements of development evaluation, where an evaluator is integrated with the design team [84].

These styles of studies are not impossible to undertake or report on. An increased uptake of before-after and with-without designs in the conservation policy field has been reported by conservation experts [48]. In one of our included studies, a temporal comparator was employed using historical land ownership records. This type of analysis is likely to be feasible in many conservation planning scenarios, particularly where the use of experimental study designs and controls would not be cost effective or ethical.

During full-text screening we encountered several interesting study designs and uses of comparators that have been undertaken in related disciplines. For example, qualitative case study evaluations compared different planning approaches in a single region [45, 85], and another compared two neighbouring regions, one that participated in a community zoning project, and another that did not [86].

In the short term, it is unlikely that enough new studies will be produced to warrant a revision of this systematic map. Future research may consider exploring the evidence for the effectiveness of specific aspects rather than evaluating systematic conservation planning in its entirety (e.g., stakeholder attitudes of consultation processes).

Overall conclusion

Remarkably few rigorous evaluations of systematic conservation plans have been conducted to date, despite many claims about their effectiveness. This does not imply systematic conservation planning is, or is not, effective, but highlights the gap in our understanding of how, when and why it may or may not be effective. It also raises important questions about the challenges of conducting rigorous evaluations in relation to a non-linear and multi-dimensional intervention such as conservation planning. We have provided some suggestions as to how these challenges can be overcome. We recommend more focus is required in this area. We urge academics and practitioners alike to publish the results of systematic conservation planning exercises and to employ robust evaluation methodologies when reporting project outcomes.

Declarations

Authors’ contributions

This study was led by EJM. The research question was independently conceived by MCM and RLP, who advised throughout the project. Title screening was conducted by EJM and SC, abstract screening by EJM, SC, SGK, BW, GA and JPRT, and full-text screening by EJM, SC and SGK. Full-text coding was conducted by EJM and BW. RG advised on the design and execution of the study. All authors read and approved the final manuscript.

Acknowledgements

We thank Christian Kohl and Stefan Unger their assistance with CADIMA and Jeff Brunton and Zak Ghouze for their assistance with EPPI-Reviewer. Discussions with Jorge G. Álvarez-Romero improved the data extraction categories and Neal Haddaway and Gillian Petrokovsky advised on systematic mapping methodologies. This manuscript has also benefited greatly from the comments of two anonymous reviewers.

Competing interests

The authors declare that they have no competing interests (both financial and non-financial).

Availability of data and materials

All data generated or analysed during this study are included in this published article and its supplementary information files, which will be made available for download from the Environmental Evidence website.

Consent for publication

Permission has been granted for all direct quotes and personal communications included.

Ethics approval and consent to participate

Not applicable.

Funding

EJM is supported by the General Sir John Monash Foundation. RG thanks the John Fell Fund of the University of Oxford and Jesus College Oxford for support. RLP acknowledges the support of the Australian Research Council.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
School of Geography and the Environment, University of Oxford, Oxford, UK
(2)
School of Earth and Environmental Sciences, University of Queensland, St Lucia, QLD, Australia
(3)
Department of Ecosystem Science & Sustainability, Colorado State University, Fort Collins, CO, USA
(4)
Australian Research Council Centre of Excellence for Coral Reef Studies, James Cook University, Townsville, QLD, Australia
(5)
Paul G Allen Philanthropies, Seattle, WA, USA

References

  1. Groves CR, Game ET. Conservation planning: informed decisions for a healthier planet. Colorado: Roberts and Company Publishers; 2016.Google Scholar
  2. Kukkala AS, Moilanen A. Core concepts of spatial prioritisation in systematic conservation planning. Biol Rev. 2013;88:443–64.View ArticleGoogle Scholar
  3. Sarkar S, Pressey RL, Faith DP, Margules CR, Fuller T, Stoms DM, et al. Biodiversity conservation planning tools: present status and challenges for the future. Annu Rev Environ Resour. 2006;31:123–59.View ArticleGoogle Scholar
  4. Morrison J, Loucks C, Long B, Wikramanayake E. Landscape-scale spatial planning at WWF: a variety of approaches. Oryx. 2009;43:499–507.View ArticleGoogle Scholar
  5. Groves CR, Jensen DB, Valutis LL, Redford KH, Shaffer ML, Scott JM, et al. Planning for biodiversity conservation: putting conservation science into practice. Bioscience. 2002;52:499.View ArticleGoogle Scholar
  6. Cowling R, Pressey R. Introduction to systematic conservation planning in the Cape Floristic Region. Biol Conserv. 2003;112:1–13.View ArticleGoogle Scholar
  7. Fernandes L, Day J, Lewis A, Slegers S, Kerrigan B, Breen D, et al. Establishing representative no-take areas in the Great Barrier Reef: large-scale implementation of theory on marine protected areas. Conserv Biol. 2005;19:1733–44.View ArticleGoogle Scholar
  8. Brownlie S, De Villiers C, Driver A, Job N, Von Hase A, Maze K. Systematic conservation planning in the Cape Floristic Region and Succulent Karoo, South Africa: enabling sound spatial planning and improved environmental assessment. J Environ Assess Policy Manag. 2005;07:201–28.View ArticleGoogle Scholar
  9. Margules CR, Pressey RL. Systematic conservation planning. Nature. 2000;405:243–53.View ArticleGoogle Scholar
  10. Pressey RL. Ad hoc reservations: forward or backward steps in developing representative reserve systems? Conserv Biol. 1994;8:662–8.View ArticleGoogle Scholar
  11. Venter O, Magrach A, Outram N, Klein CJ, Di Marco M, Watson JEM. Bias in protected-area location and its effects on long-term aspirations of biodiversity conventions. Conserv Biol. 2017;32:127–34.View ArticleGoogle Scholar
  12. Pressey RL, Nicholls AO. Efficiency in conservation evaluation: scoring vs. iterative approaches. Biol Conserv. 1989;50:199–218.View ArticleGoogle Scholar
  13. Watson JEM, Grantham HS, Wilson KA, Possingham HP. Systematic conservation planning: past, present and future. In: Ladle RJ, Whittaker RJ, editors. Conservational biogeography. Chichester: Blackwell Publishing; 2011. p. 136–60.View ArticleGoogle Scholar
  14. Carwardine J, Klein CJ, Wilson KA, Pressey RL, Possingham HP. Hitting the target and missing the point: target-based conservation planning in context. Conserv Lett. 2009;2:4–11.View ArticleGoogle Scholar
  15. McIntosh EJ, McKinnon MC, Pressey RL, Grenyer R. What is the extent and distribution of evidence on conservation outcomes of systematic conservation planning around the globe? A systematic map protocol. Environ Evid. 2016;5:1–13.View ArticleGoogle Scholar
  16. McIntosh EJ, Pressey RL, Lloyd S, Smith RJ, Grenyer R. The impact of systematic conservation planning. Annu Rev Environ Resour. 2017;42:677–97.View ArticleGoogle Scholar
  17. Ball I, Possingham HP. Marxan (v1.8.2): Marine reserve design using spatially explicit annealing, a manual prepared for the Great Barrier Reef Marine Park Authority. Brisbane: University of Queensland; 2000.Google Scholar
  18. Watts ME, Ball IR, Stewart RS, Klein CJ, Wilson K, Steinback C, et al. Marxan with Zones: software for optimal conservation based land- and sea-use zoning. Environ Model Softw. 2009;24:1513–21.View ArticleGoogle Scholar
  19. Lehtomäki J, Moilanen A. Methods and workflow for spatial conservation prioritization using Zonation. Environ Model Softw. 2013;47:128–37.View ArticleGoogle Scholar
  20. Pressey RL, Watts ME, Barrett TW, Ridges MJ. The C-Plan conservation planning system: Origins, applications and possible futures. In: Moilanen A, Wilson KA, Possingham HP, editors. Spatial conservation prioritization: quantitative methods and computational tools. New York: Oxford University Press; 2009. p. 211–34.Google Scholar
  21. Knight AT, Grantham HS, Smith RJ, McGregor GK, Possingham HP, Cowling RM. Land managers’ willingness-to-sell defines conservation opportunity for protected area expansion. Biol Conserv. 2011;144:2623–30.View ArticleGoogle Scholar
  22. Jones KR, Watson JEM, Possingham HP, Klein CJ. Incorporating climate change into spatial conservation prioritisation: a review. Biol Conserv. 2016;194:121–30.View ArticleGoogle Scholar
  23. Visconti P, Joppa L. Building robust conservation plans. Conserv Biol. 2014;29:503–12.View ArticleGoogle Scholar
  24. Kullberg P, Moilanen A. How do recent spatial biodiversity analyses support the convention on biological diversity in the expansion of the global conservation area network? Nat Conserv. 2014;12:3–10.View ArticleGoogle Scholar
  25. Watts MEJ. Global users. Marxan.net. 2018 http://marxan.net/index.php/globe. Accessed 3 Apr 2018.
  26. Álvarez-Romero JG, Mills M, Adams VM, Gurney GG, Pressey RL, Weeks R, et al. Research advances and gaps in marine planning: towards a global database in systematic conservation planning. Biol Conserv. 2018. https://doi.org/10.1016/j.biocon.2018.06.027.View ArticleGoogle Scholar
  27. Pressey RL, Bottrill MC. Approaches to landscape- and seascape-scale conservation planning: convergence, contrasts and challenges. Oryx. 2009;43:464–75.View ArticleGoogle Scholar
  28. Bottrill MC, Pressey RL. The effectiveness and evaluation of conservation planning. Conserv Lett. 2012;5:407–20.View ArticleGoogle Scholar
  29. James KL, Randall NP, Haddaway NR. A methodology for systematic mapping in environmental sciences. Environ Evid. 2016;5:7.View ArticleGoogle Scholar
  30. Bottrill MC, Pressey RL. Systematic Review No. 74. Working title: is systematic conservation planning an effective approach for designing and implementing areas for biodiversity conservation? Review protocol for the Collaboration for Environmental Evidence. Bangor, UK: Collaboration for Environmental Evidence, 2009.Google Scholar
  31. Bottrill MC, Mills M, Pressey RL, Game ET, Groves C. Evaluating perceived benefits of ecoregional assessments. Conserv Biol. 2012;26:851–61.View ArticleGoogle Scholar
  32. Garnett ST, Sayer J, du Toit J. Improving the effectiveness of interventions to balance conservation and development: a conceptual framework. Ecol. Soc. 2007;12:2.View ArticleGoogle Scholar
  33. Haddaway NR, Bayliss HR. Shades of grey: two forms of grey literature important for reviews in conservation. Biol Conserv. 2015;191:8–11.View ArticleGoogle Scholar
  34. EPPI-Centre. Export your search file to RIS format. http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=2934. Accessed 8 May 2018.
  35. Haddaway NR, Collins AM, Coughlin D, Kirk S. A rapid method to increase transparency and efficiency in web-based searches. Environ Evid. 2017;6:1.View ArticleGoogle Scholar
  36. Stansfield C, Dickson K, Bangpan M, Oliver S, Bangpan M, Stansfield C, et al. Exploring issues in the conduct of website searching and other online sources for systematic reviews: how can we be systematic? Syst Rev. 2016;5:191.View ArticleGoogle Scholar
  37. Thomas J, Brunton J, Graziosi S. EPPI-Reviewer 4: software for research synthesis. EPPI-Centre Software. London: Social Science Research Unit, Institute of Education; 2010.Google Scholar
  38. Kohl C, McIntosh EJ, Unger S, Haddaway N, Kecke S, Schiemann J, et al. Online tools supporting the conduct and reporting of systematic reviews and systematic maps: a case study on CADIMA and review of existing tools. Environ Evid. 2018;7:e1–17.View ArticleGoogle Scholar
  39. Weeks R, Aliño PM, Atkinson S, Beldia P, Binson A, Campos WL, et al. Developing marine protected area networks in the coral triangle: good practices for expanding the coral triangle marine protected area system. Coast Manag. 2014;42:183–205.View ArticleGoogle Scholar
  40. Osmond M, Airame S, Caldwell M, Day J. Lessons for marine conservation planning: a comparison of three marine protected area planning processes. Ocean Coast Manag. 2010;53:41–51.View ArticleGoogle Scholar
  41. Carletta J. Assessing agreement on classification tasks: the kappa statistic. Comput Linguist. 1996;22:249–54.Google Scholar
  42. CEE. Guidelines for systematic review and evidence synthesis in environmental management. Version 4.2. Bangor, UK: Collaboration for Environmental Evidence; 2013.Google Scholar
  43. Fisher JRB, Dills B. Do private conservation activities match science-based conservation priorities? PLoS ONE. 2012;7:e46429.View ArticleGoogle Scholar
  44. The Nature Conservancy. World Wildlife Fund. Arlington: Standards for ecoregional assessments and biodiversity visions. World; 2006.Google Scholar
  45. Lagabrielle E, Botta A, Daré W, David D, Aubert S, Fabricius C. Modelling with stakeholders to integrate biodiversity into land-use planning—lessons learned in Réunion Island (Western Indian Ocean). Environ Model Softw UK. 2010;25:1413–27.View ArticleGoogle Scholar
  46. Álvarez-Romero JG, Pressey RL, Ban NC, Torre-Cosio J, Aburto-Oropeza O. Marine conservation planning in practice: lessons learned from the gulf of California. Aquat Conserv. 2013;23:483–505.View ArticleGoogle Scholar
  47. Ulloa RJ, Torre L, Bourillón A, Gondor A, Alcantar N. Planeación ecorregional para la conservación marina: Golfo de California y costa occidental de Baja California Sur. Informe final a The Nature Conservancy. Guaymas (México); 2006.Google Scholar
  48. Curzon HF, Kontoleon A. From ignorance to evidence? The use of programme evaluation in conservation: evidence from a Delphi survey of conservation experts. J Environ Manage. 2016;180:466–75.View ArticleGoogle Scholar
  49. Carter SK, Keuler NS, Pidgeon AM, Radeloff VC. Evaluating the influence of conservation plans on land protection actions in Wisconsin, USA. Biol Conserv. 2014;178:37–49.View ArticleGoogle Scholar
  50. Morandi B, Piégay H, Lamouroux N, Vaudor L. How is success or failure in river restoration projects evaluated? Feedback from French restoration projects. J Environ Manage. 2014;137:178–88.View ArticleGoogle Scholar
  51. Ferraro PJ, Pattanayak SK. Money for nothing? A call for empirical evaluation of biodiversity conservation investments. PLoS Biol. 2006;4:e105.View ArticleGoogle Scholar
  52. Nutters HM, Pinto da Silva P. Fishery stakeholder engagement and marine spatial planning: lessons from the Rhode Island Ocean SAMP and the Massachusetts Ocean Management Plan. Ocean Coast Manag. 2012;67:9–18.View ArticleGoogle Scholar
  53. Pressey R, Whish G, Barrett T, Watts M. Effectiveness of protected areas in north-eastern New South Wales: recent trends in six measures. Biol Conserv. 2002;106:57–69.View ArticleGoogle Scholar
  54. Sola P, Cerutti PO, Zhou W, Gautier D, Iiyama M, Schure J, et al. The environmental, socioeconomic, and health impacts of woodfuel value chains in Sub-Saharan Africa: a systematic map. Environ Evid. 2017;6:4.View ArticleGoogle Scholar
  55. Knight AT, Cowling RM, Rouget M, Balmford A, Lombard AT, Campbell BM. Knowing but not doing: selecting priority conservation areas and the research-implementation gap. Conserv Biol. 2008;22:610–7.View ArticleGoogle Scholar
  56. Kirlin J, Caldwell M, Gleason M, Weber M, Ugoretz J, Fox E, et al. California’s Marine Life Protection Act Initiative: supporting implementation of legislation establishing a statewide network of marine protected areas. Ocean Coast Manag. 2013;74:3–13.View ArticleGoogle Scholar
  57. Baskent EZ, Başkaya S, Terzioǧlu S. Developing and implementing participatory and ecosystem based multiple use forest management planning approach (ETÇAP): Yalnizçam case study. For Ecol Manage. 2008;256:798–807.View ArticleGoogle Scholar
  58. Pressey RL, Weeks R, Gurney GG. From displacement activities to evidence-informed decisions in conservation. Biol Conserv. 2017;212:337–48.View ArticleGoogle Scholar
  59. Sinclair SP, Milner-Gulland EJ, Smith RJ, McIntosh EJ, Possingham H, Vercammen A, et al. The use, and usefulness, of spatial conservation prioritizations. Conserv Lett. 2018;1:e12459.View ArticleGoogle Scholar
  60. Lestrelin G, Augusseau X, David D, Bourgoin J, Lagabrielle E, Lo Seen D, et al. Collaborative landscape research in Reunion Island: using spatial modelling and simulation to support territorial foresight and urban planning. Appl Geogr. 2017;78:66–77.View ArticleGoogle Scholar
  61. Wiersma YF, Sleep DJH. A review of applications of the six-step method of systematic conservation planning. For Chron. 2016;92:322–35.View ArticleGoogle Scholar
  62. Bayliss HR, Schindler S, Essl F, Rabitsch W, Pullin AS. What evidence exists for changes in the occurrence, frequency or severity of human health impacts resulting from exposure to alien invasive species in Europe? Environ Evid. 2017;4:1–6.Google Scholar
  63. Da Silva NR, Stewart R, van Rooyen C. Gaining from the nothing: the value of an empty review. First Int Conf Collab Environ Evid. Stockholm: Collaboration for Environmental Evidence; 2016; p 34.Google Scholar
  64. Bowler DE, Buyung-Ali LM, Healey JR, Jones JPG, Knight TM, Pullin AS. Does community forest Management provide global environmental benefits and improve local welfare? Front Ecol Environ. 2012;10:29–36.View ArticleGoogle Scholar
  65. Alderson P. Should journals publish systematic reviews that find no evidence to guide practice? Examples from injury research. BMJ. 2000;320:376–7.View ArticleGoogle Scholar
  66. Greenhalgh T, Russell J. Reframing evidence synthesis as rhetorical action in the policy making drama. Healthc Policy| Polit Santé. 2006;1:34–42.View ArticleGoogle Scholar
  67. Smith R, Leader-Williams N. Transnational conservation planning in the Maputaland ecoregion of southern Africa. Kent: Darwin Initiative for the Survival of Species Annual Report; 2004.Google Scholar
  68. Raab J. California Marine Life Protection Act: evaluation of the Central Coast regional stakeholder group process. Boston; 2006.Google Scholar
  69. Gelderblom CM, van Wilgen BW, Nel JL, Sandwith T, Botha M, Hauck M. Turning strategy into action: implementing a conservation action plan in the Cape Floristic Region. Biol Conserv. 2003;112:291–7.View ArticleGoogle Scholar
  70. O’Mara-Eves A, Thomas J, McNaught J, Miwa M, Ananiadou S. Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Rev. 2015;4:5.View ArticleGoogle Scholar
  71. Gates A, Johnson C, Hartling L. Technology-assisted title and abstract screening for systematic reviews: a retrospective evaluation of the Abstrackr machine learning tool. Syst Rev. 2018;7:1–9.View ArticleGoogle Scholar
  72. Westgate MJ, Haddaway NR, Cheng SH, McIntosh EJ, Marshall C, Lindenmayer DB. Software support for environmental evidence synthesis. Nat Ecol Evol. 2018;2:588–90.View ArticleGoogle Scholar
  73. Cheng SH, Augustin C, Bethel A, Gill D, Anzaroot S, Brun J, et al. Using machine learning to advance synthesis and use of conservation and environmental evidence. Conserv Biol. 2018;32(4):762–4.View ArticleGoogle Scholar
  74. Barnes MD, Glew L, Wyborn C, Craigie ID. Prevent perverse outcomes from global protected area policy. Nat Ecol Evol. 2018;2018:1.Google Scholar
  75. Klein CK, Tulloch VJ, Halpern BS, Selkoe KA, Watts ME, Steinback C, et al. Tradeoffs in marine reserve design: habitat condition, representation, and socioeconomic costs. Conserv Lett. 2013;6:324–32.Google Scholar
  76. Hermoso V, Kennard MJ, Linke S. Integrating multidirectional connectivity requirements in systematic conservation planning for freshwater systems. Divers Distrib. 2012;8:448–58.View ArticleGoogle Scholar
  77. Pressey RL, Mills M, Weeks R, Day JC. The plan of the day: managing the dynamic transition from regional conservation designs to local conservation actions. Biol Conserv. 2013;166:155–69.View ArticleGoogle Scholar
  78. Reyers B, Rouget M, Jonas Z, Cowling RM, Driver A, Maze K, et al. Developing products for conservation decision-making: lessons from a spatial biodiversity assessment for South Africa. Divers Distrib. 2007;13:608–19.View ArticleGoogle Scholar
  79. Westgate MJ, Likens GE, Lindenmayer DB. Adaptive management of biological systems: a review. Biol Conserv. 2013;158:128–39.View ArticleGoogle Scholar
  80. Geldmann J, Coad L, Barnes MD, Craigie ID, Woodley S, Balmford A, et al. A global analysis ofmanagement capacity and ecological outcomes in terrestrial protected areas. Conserv. Lett. 2018;11:1–10.View ArticleGoogle Scholar
  81. Hockings M. Evaluating management of protected areas: integrating planning and evaluation. Environ Manage. 1998;22:337–45.View ArticleGoogle Scholar
  82. Margoluis R, Stem C, Salafsky N, Brown M. Design alternatives for evaluating the impact of conservationprojects. New Dir. Eval. 2009;2009:85–96.View ArticleGoogle Scholar
  83. Smith GCS, Pell JP. Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. BMJ. 2003;327:1459–61.View ArticleGoogle Scholar
  84. Patton MQ. Developmental evaluation. Am J Eval. 1994;15:311–9.View ArticleGoogle Scholar
  85. Gleason M, Fox E, Ashcraft S, Vasques J, Whiteman E, Serpa P, et al. Designing a network of marine protected areas in California: achievements, costs, lessons learned, and challenges ahead. Ocean Coast Manag. 2013;74:90–101.View ArticleGoogle Scholar
  86. Leisher C, Brouwer R, Boucher TM, Vogelij R, Bainbridge WR. Striking a balance: socioeconomic development and conservation in grassland through community-based zoning. PLoS ONE. 2011;6:e28807.View ArticleGoogle Scholar
  87. Costanza R, D’Agre R, de Groot R, Farber S, Grasso M, Hannon B, et al. The value of the world’s ecosystem services and natural capital. Nature. 1997;387:253–60.View ArticleGoogle Scholar
  88. Pretty J, Ward H. Social capital and the environment. World Dev. 2001;29:209–27.View ArticleGoogle Scholar
  89. DfID. Sustainable livelihoods guidance sheets: framework. London: Context; 1999.Google Scholar
  90. Platje J. An institutional capital approach to sustainable development. Manag Environ Qual. 2008;19:222–33.View ArticleGoogle Scholar
  91. Harzing A-W. Publish or perish. Tarma: Software Research Ltd; 2007.Google Scholar
  92. Ramsay C, Matowe L, Grilli R, Grimshaw J, Thomas R. Interrupted time series designs in health technology assessment: lessons from two systematic reviews of behavior change strategies. Int J Technol Assess Health Care. 2003;19:613–23.View ArticleGoogle Scholar
  93. Moher D, Liberati A, Tetzlaff J, Altman D. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement (reprinted from Annals of Internal Medicine). Phys Ther. 2009;89:873–80.Google Scholar

Copyright

© The Author(s) 2018

Advertisement