- Systematic Review Protocol
- Open Access
Review of the evidence base for ecosystem-based approaches for adaptation to climate change
Environmental Evidence volume 1, Article number: 13 (2012)
Ecosystem-based approaches for adaptation (EbA) integrate the use of biodiversity and ecosystem services into an overall strategy for helping people adapt to climate change. To date, insight into these approaches has often been based on reports from isolated anecdotal case studies. Although these are informative, and provide evidence that people are using ecosystems to adapt, they provide rather limited insight in terms of measuring and evaluating the effectiveness of EbA, especially when compared with technical or structural adaptation interventions. The body of scientific evidence indicating how effective such approaches are is lacking in some aspects. Where evidence does exist it is often dispersed across a range of related fields, such as natural resource management, disaster risk reduction and agroecology. To date, there has been little attempt to systematically assemble and analyse this evidence. Therefore, the current state of evidence regarding the merits or otherwise of EbA is unknown and it has not been possible to identify prevailing knowledge gaps to inform research and analysis, which will enable policymakers to compare EbA with other adaptation options.
This protocol details the methodology to be used to conduct a systematic map of peer-reviewed published journal papers and a limited selection of grey literature, to give a methodical overview of the state of the evidence base for EbA effectiveness, and to identify the current knowledge gaps. It addresses the following question: What is the state of the evidence base regarding the ability of ecosystem-based approaches for adaptation to help people adapt to the impacts of climate change?
Adaptation to climate change may be achieved in many different ways. One way is through ecosystem-based approaches for adaptation (EbA) – defined by the Convention on Biological Diversity (CBD) 2nd Ad Hoc Technical Expert Group on Biodiversity and Climate Change  as ‘the use of biodiversity and ecosystem services to help people adapt to the adverse effects of climate change’. This definition was elaborated by the CBD 10th Conference of the Parties (CoP) in October 2010, as including ‘sustainable management, conservation and restoration of ecosystems, as part of an overall adaptation strategy that takes into account the multiple social, economic and cultural co-benefits for local communities’ .
EbA is being enthusiastically promoted by conservation and development organisations that recognise the integral relationship between ecosystems and livelihoods and the threat that climate change poses to these. As a result there are numerous anecdotal case studies of its apparent success describing how local communities are already practicing EbA [3–6]. However, such case studies largely lack a scientific assessment that provides reliable and robust evidence indicating how effective these approaches are . In particular, there are very few quantitative assessments of EbA effectiveness which use controls to examine the impact of the intervention, and few which provide plausible counterfactual arguments about likely causal mechanisms involved. There are also few case studies on either the cost-effectiveness of EbA or on the assessment of its social value. Importantly, the UNFCCC Secretariat  notes that, ‘Besides financial and economic assessments, social appraisal is crucial in any assessment of costs and benefits of adaptation options, because the impacts of climate change often disproportionately affect the most vulnerable communities and groups.’
In other cases, empirical material that is relevant to EbA is dispersed across a range of related fields such as natural resource management, disaster risk reduction and agroecology. Evidence and lessons learnt from these disciplines have not previously been collated to inform EbA, nor have they been consolidated so that policymakers can compare EbA with other possible adaptation options.
Beyond the gaps in the evidence base, a further difficulty has been the lack of general consensus on what successful or effective adaptation actually means – although assessment frameworks are now emerging [9–12].
This protocol details the methodology to be used to conduct a systematic map of peer-reviewed published journal papers and a limited selection of grey literature (i.e. published and unpublished documents that do not pass through the scientific peer-review system), to give a methodical overview of the state of the evidence base for EbA effectiveness, and to identify the current knowledge gaps.
Objective of the review
This review seeks to address the primary question: What is the state of the evidence base regarding the ability of ecosystem-based approaches for adaptation to help people adapt to the impacts of climate change? (See ‘Table 1 – Elements of the systematic map question’).The secondary question is: What are the critical knowledge gaps in the evidence?
Experts were invited to a workshop in Cambridge from organisations working on EbA or related activities that BirdLife International have had exposure to, or worked with, on ecosystems and adaptation during the UNFCCC negotiations. Focus groups were held during this expert workshop to guide the identification of key terms to be used for literature searches of peer-reviewed and grey literature. As a starting point, search terms used in a recent study by CIFOR (personal communication with Emilia Pramova, see ‘Table 2 – CIFOR study search terms’) were built upon to meet project needs.
Concern about climate change – and how to adapt to it – has only received widespread international political attention in the last two decades. However, people have been adapting to, and managing their environment and natural resource use in response to, climatic variability, climatic shocks and change for centuries. Many of the impacts we can expect to see because of climate change will be the same as those associated with climate variability and shocks , so this wider literature presents a large relevant knowledge base to draw upon. Our selection of search terms to be used also covered those associated with climatic variability, extremes or other natural hazards that can provide analogues to events linked to climate change.
Trial literature searches were conducted through the ISI Web of Knowledge (WoK: http://wok.mimas.ac.uk/) to hone search terms by testing whether a short list of papers known by experts to contain information relevant to this review appeared in the outputs of the search.
The resulting final search terms (see ‘Table 3 – Search term scoping and evolution and search string’) have been selected to provide broad, but manageable coverage related to the review questions (see ‘Objective of the review’ above). Four sets of search terms will be used with individual terms (and Wildcard symbols (*) where appropriate) separated by Boolean “OR” operators and sets combined using “AND”. A fifth set will be used and combined with “AND NOT” (to exclude solely ecosystem-based approaches for climate change mitigation, and other mitigation studies that refer to land use emissions or mitigation approach (e.g. renewable energy) impacts on the environment). Searches will be conducted in English only.
Publication database searches
Two databases will be searched: WoK and Scopus. This follows the recommendation of the Centre for Evidence-based Conservation  who suggest using more than one database of peer-reviewed papers. Both these databases cover science (including life science, chemistry, physics, mathematics, biology, and environmental science) and social sciences (although WoK does so regarding the latter to a greater degree) . In each database the search will be refined by selecting key subject areas. For WoK these will be: “environmental sciences and ecology”, “agriculture”, “water resources”, “forestry”, “engineering”, “biodiversity conservation”, “science technology other topics”, and “social science other topics”; and for Scopus: “environmental science”, “earth and planetary sciences”, “agricultural and biological sciences”, “social sciences”, “engineering”, “energy”, “economics, econometrics and finance”, and “multidisciplinary”. The Field Code a ‘Topic’ does not exist on Scopus, so the nearest equivalent, ‘abstract’ (returns searches where the search terms are found in the abstract of the papers), will be used as a replacement (see ‘Table 3 – Search term scoping and evolution and search string’). The results from the Scopus search will be combined with the WoK search in Reference Manager and overlaps will be excluded.
In addition, to WoK and Scopus, Google Scholar will be searched using a more limited range of terms from the four sets of search terms. The first 50 ‘hits’ will be compared with the WoK and Scopus search returns to test their comprehensiveness. References returned by the Google Scholar search, but not found in the WoK and Scopus searches, will be added to the reference list.
Specialist searches – searches for grey literature
A limited selection of ‘grey’ literature (i.e. published and unpublished documents that do not pass through the scientific peer-review system) will be identified by going directly to relevant organisation websites informed by the list of relevant sources determined by expert input:
Adaptation Learning Mechanism
Ecosystems and Livelihoods Adaptation Network
Global Environment Facility – Project Database
International Union for Conservation of Nature
Overseas Development Institute
The Nature Conservancy
United Nations Convention on Biological Diversity
United Nations Convention to Combat Desertification
United Nations Development Programme
United Nations Environment Programme
United Nations Framework Convention on Climate Change
United Nations Framework Convention on Climate Change – Nairobi Work Programme Database on Ecosystem-based approaches to adaptation
United Nations International Strategy for Disaster Reducation
World Bank – Project Database
World Conservation Society
World Resources Institute
World Resources Institute – World Resources Report
World Wildlife Fund
Finally, participants in the expert workshop will be asked to review the list of references generated by the peer-reviewed and grey literature searches to identify additional key references.
Estimating the comprehensiveness of the search
We consider that the search will be comprehensive as a result of combining objective database searches and subjective expert advice (including trial searches as described in the ‘Search terms’ section above, as well as a test Google Scholar search as described in the ‘Internet searches’ section above), and taking into account both peer-reviewed and grey literature.
Limitations of the review’s comprehensiveness include that literature/topic review papers will be excluded (see ‘Study inclusion/exclusion criteria’ section below for reasoning) despite their possible use as a source of additional references. The comprehensiveness of the search could have been furthered by searching for relevant literature within the bibliographies of the captured literature.
The search will not include a body of relevant biophysical science literature (e.g. on intertidal vegetation controls on wave energy dissipation, erosion and sedimentation) as this does not make explicit links to human adaptation despite being relevant to such adaptation. This literature could provide significant information on biophysical thresholds or ‘tipping points’ that control the degree to which an ecosystem (and the services it provides to the community) can continue to act in that capacity under an increase in physical stress (e.g. climate change), and on boundary conditions (minimum size or the state of ecosystem necessary to provide adaptation benefits).
The comprehensiveness of the grey literature search will be negatively affected by the lack of a single comprehensive database/portal housing relevant grey literature documents. Moreover, our search will not include (because of time constraints) unpublished project documentation that is not available online.
Study inclusion/exclusion criteria
After combing search outputs from WoK and Scopus and removing duplicates, the inclusion/exclusion process will be undertaken through a stepwise process, by applying the primary inclusion and exclusion criteria (see ‘Table 4 – Primary inclusion and exclusion criteria’) to: a) the article title, b) the abstract and c) the full text, of each of the articles (‘articles’ used hereafter to cover peer-reviewed published journal papers, grey literature and any papers/literature which includes multiple studies/pieces of evidence which will be recorded separately – see ‘Data extraction strategy’ below for more information on multiple studies within the same source). Exclusion will be conservative during phase a) and b) if there is any doubt that the criteria for inclusion or exclusion are being met. Step a) will be undertaken by a single reviewer, while step b) and c) will involve more than one reviewer.
Reviewer bias will be tested for at the start of the selection process of step b) with a Kappa analysis . Two reviewers will review a common, random 10 per cent sample of the abstracts. Level of agreement between the number of articles rejected or accepted by the reviewers will be calculated using the Kappa statistic . Values can range from +1 (perfect agreement) to −1 (strong disagreement).
During step c), reasonable effort will be made to secure the relevant articles, contacting authors if necessary. However, it may not be feasible within the time and budget constraints to secure all articles (a list of these will be made available when the results are published).
In order to make the review manageable in the time available, secondary exclusion criteria have been developed. One set of articles excluded will be those focussed on ‘mal-adaptation’. These studies typically give evidence on the negative impact of other types of adaptation (e.g. sea walls) on ecosystems and then suggest that using ecosystems would be a more appropriate form of response, rather than testing the effectiveness of the identified ecosystem-based approaches for adaptation. The other exclusion criteria have been based on the types of intervention. Specifically, articles covering traditional agricultural practices (such as minimal tillage, cropping date adjustment, water management like the creation of artificial ponds for water storage) without consideration of the surrounding ecosystem will be excluded. By contrast, agricultural practices that have the explicit objective of protecting the surrounding ecosystem to provide adaptation benefits will not be excluded (e.g. agroforestry, agrobiodiversity conservation, management of agricultural lands for watershed management/runoff management). Approaches focussed on urban interventions, such as urban green roofs, green infrastructure and green and blue space within a city, will be excluded. By contrast, urban approaches which consider a city’s relationship with its surrounding ecosystem(s)/wider landscape will be included. This includes watershed management working towards providing protection to densely populated areas on floodplains or at the foot of steep hillsides.
These secondary exclusions criteria were based on the following:
– The role that biodiversity and ecosystems play in the adaptation approach are not always made clear in such studies.
Despite these exclusions, basic information such as climatic hazards/impacts addressed, location and ecosystem, will still be extracted from these articles (see questions outlined in the first section of the Assessment Framework  – see ‘Data extraction strategy’ section below) to allow the possibility for future assessments of those studies.
Furthermore, all of the articles identified prior to secondary exclusion will be recorded (and will be included in the published results) so that future reviewers can follow-up on these excluded articles if required.
For the grey literature, the primary and secondary exclusion criteria will be merged (because this will only be a limited selection of the grey literature) to become general exclusion criteria. Like the peer-reviewed literature, a stepwise process to inclusion/exclusion will be adopted: a) title and introduction, b) a scan of the full text and c) a read of the full text.
The number of articles found in the search and included at each inclusion/exclusion level will be reported in the published results.
Study quality assessment
The design of each included study will be recorded based on general categories, see ‘Table 5 – Study quality assessment categories’), and qualitative detail on the measure of intervention success used. This will allow an overview to be given of the quality of included studies.
Data extraction strategy
An Assessment Framework has been developed to extract data from the studies that are reviewed  to enable critical appraisal. This consists of a series of 39 questions separated into 4 parts:
Basic information (including climate hazard/impact addressed, location, ecosystem, and adaptation target);
Description of evidence (including type and quality of evidence, and constituent parts of effectiveness addressed);
Detail of evidence (including measure of success used, result, and costs and benefits); and
Project design section for practical projects (including assessments undertaken, monitoring, and sustainability).
The questions (or variables) included in the framework were identified at the expert workshop through discussions on what information is required by policy makers to make informed decisions about the use of EbA as an adaptation option in any given context. This expert opinion was supplemented through a review of the monitoring and evaluation frameworks of a number of major climate change adaptation funds [12, 18] and other adaptation evaluation literature [9, 19].
Data input will be for the most part categorical (i.e. follow a numerical coding). This will allow for the quantitative description of the results. Some qualitative descriptive elements are also requested by the framework to capture fuller details where required. Attempts to minimise reviewer subjectivity will be made by using limited free text answers to questions, by asking reviewers to select from lists of pre-defined answers, and by holding reviewer meetings on a regular basis to discuss queries on data extraction to ensure consistency of input.
Data will be recorded in a dedicated Excel spreadsheet (‘logbook’, see ‘Additional file 1 – Excel blank logbook’). Each study/source of evidence will be recorded in the logbook and allocated a unique article ID. Articles that include multiple studies/pieces of evidence (e.g. two or more case studies with the evidence presented separately for each) will be recorded by allocating a separate ID to each study within the article. The logbook columns correspond to the framework questions/evidence variables and separate worksheets correspond to the four discrete sections described above. More information on how to use the Assessment Framework and logbook is available in Munroe et al. .
Data synthesis and presentation
Descriptive and basic inferential statistics will be used to summarise quantitative trends in the information provided by the literature, mapping the quantity and quality of articles relevant to the question. An overview of the evidence will be given according to a variety of categories (see ‘Table 6 – Categories of data analysed’), based on the Assessment Framework, to highlight knowledge gaps in the evidence base (the secondary question of the review). Implications of the state of the evidence base for policy, implementation and research related to EbA, as captured by the systematic map, will be given in the published results.
aField Codes or “Tags” restrict a search to a particular part (“field”) of a record, such as title, author or subject; bA form of assessing/further populating the paper list would be to extract the references that have provided the review paper with hard evidence; cThese categories have been developed drawing upon the consensus categories used by the Intergovernmental Panel on Climate Change (IPCC) to determine level of scientific understanding of radiative forcing [http://www.ipcc.ch/publications_and_data/ar4/wg1/en/ch2s2-9-1.html].
RM is Climate Change Officer at BirdLife International.
DR is Senior Researcher (Biodiversity) at the International Institute for Environment and Development.
ND is Programme Officer, Climate Change and Biodiversity at UNEP-WCMC.
TS is Reader in Coastal Ecology and Geomorphology and Director, Cambridge Coastal Research Unit, Department of Geography, University of Cambridge.
IM is College Lecturer in Physical Geography, Fitzwilliam College, Cambridge, and Deputy Director, Cambridge Coastal Research Unit, Department of Geography, University of Cambridge.
BV is University Senior Lecturer, Department of Geography, University of Cambridge.
HR is a consulting researcher with the International Institute for Environment and Development.
AK is University Lecturer, Department of Land Economy, University of Cambridge.
AG is a Researcher (Biodiversity) at the International Institute for Environment and Development.
IC is a consulting researcher working for UNEP-WCMC during this project.
JS is a consulting researcher paid for by the United States Agency for International Development’s (USAID) Sustainable Conservation Approaches in Priority Ecosystems (SCAPES) programme during this project.
Secretariat of the Convention on Biological Diversity: Biodiversity and Climate-Change Adaptation. Technical Series. In Connecting Biodiversity and Climate Change Mitigation and Adaptation: Report of the Second Ad Hoc Technical Expert Group on Biodiversity and Climate Change. Montreal: Secretariat of the Convention on Biological Diversity; 2009:41.
Convention on Biological Diversity Decision Adopted by the Conference of the Parties to the Convention on Biological Diversity at its Tenth Meeting; UNEP/CBD/COP/DEC/x/33; 29 October 2010. In X/33 Biodiversity and climate change. Nagoya, Japan: Secretariat of Convention on Biological Diversity; 2010:2.
Colls A, Ash N, Ikkala N: Ecosystem-based Adaptation: a natural response to climate change. Gland, Switzerland: IUCN; 2009.
BirdLife International: Partners with nature: how healthy ecosystems are helping the world’s most vulnerable adapt to climate change. Cambridge: BirdLife International; 2009.
Reid H, Swiderska K: Biodiversity, climate change and poverty: exploring the links. London: IIED; 2008.
Ecosystems and Livelihoods Adaptation Network (IUCN, WWF, CARE, IIED) case studies: good practices. http://www.elanadapt.net/good-practices
Reid H: Improving the evidence for ecosystem-based adaptation. Sustainable Development Opinion Paper. London: IIED; 2011.
UNFCCC Secretariat: Synthesis report on efforts undertaken to assess the costs and benefits of adaptation options, and views on lessons learned, good practices, gaps and needs, FCCC/SBSTA/2010/3: 22 March 2010. Bonn: UNFCCC Secretariat; 2010:5.
Brooks N, Anderson S, Ayers J, Burton I, Tellam I: Tracking Adaptation and Measuring Development. IIED Climate Change Working Paper No.1. London: IIED; 2011.
Spearman M, McGray H: Making Adaptation Count: Concepts and Options for Monitoring and Evaluation of Climate Change Adaptation. Eschborn: The Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ); 2011.
Lamhauge N, Lanzi E, Agrawala S OECD Environment Working Papers, No.38. OECD Publishing. Monitoring and Evaluation for Adaptation: Lessons from Development Co-operation Agencies 2011. http://dx.doi.org/10.1787/5kg20mj6c2bw-en
Adaptation Fund: Project level results framework and baselines guidance. http://www.adaptation-fund.org/sites/default/files/Results%20Framework%20and%20Baseline%20Guidance%20final.pdf/
Intergovernmental Panel on Climate Change: Climate Change 2007: Synthesis report. Contributions of working groups I, II and III to the fourth assessment report of the IPCC. Geneva, Switzerland: IPCC; 2007.
Center for Evidence-Based Conservation: Guidelines for systematic review in the environmental management, Version 4.0. Environmental Evidence. Bangor: Centre for Evidence-Based Conservation; 2010:37–38.
Jasco P: As we may search – Comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases. Current Contents 2005,89(9):1537–1547.
Kappa test online calculator. http://www.chestx-ray.com/Statistics/kappa.html
Munroe R, Doswald N, Roe D, Reid H, Giuliani A, Castelli I: Framework for Assessing the Evidence for the Effectiveness of Ecosystem-based Approaches to Adaptation. Cambridge: BirdLife International, UNEP-WCMC, IIED; 2011. http://elanadapt.net/sites/default/files/siteimages/cci_and_elan_framework.pdf
Pilot Programme for Climate Resilience (PPCR): Results Framework. PPCR/SC.7/7: 28 October 2010. Washington DC: Meeting of the PPCR Sub-Committee, Climate Investment Funds; 2010.
Africa Climate Change Resilience Alliance: Consultation Document: [Draft] The ACCRA Local Adaptive Capacity Framework (LAC). Kampala: Oxfam GB; 2010. http://community.eldis.org/.59d669a7/LACFconsult.pdf
Munroe R, Doswald N, Roe D, Reid H, Giuliani A, Castelli I: Guidance on Applying the Framework for Assessing the Evidence for the Effectiveness of Ecosystem-based Approaches to Adaptation. Cambridge, UK: BirdLife International, UNEP-WCMC, IIED; 2011.
Our thanks to the expert workshop participants for their input to the development of the methodology and framework, especially Edmund Barrow (IUCN), Dave Hole (Conservation International), and Emilia Pramova (CIFOR).
The project ‘Effectiveness of ecosystem-based approaches for adaptation: Critical review of current evidence’ for which this protocol and systematic map was designed, will be supported by the Collaborative Fund of the Cambridge Conservation Initiative funded by Arcadia, and the Ecosystem, Livelihoods and Adaptation Network (ELAN). The grey literature review will be supported through a consultancy paid for by the United States Agency for International Development’s (USAID) Sustainable Conservation Approaches in Priority Ecosystems (SCAPES) programme.
The authors declare that they have no competing interests.
RM jointly conceived the study and co-drafted the manuscript, and will coordinate and participate in the review. DR jointly conceived the study and co-drafted the manuscript, and will advise on the review. ND jointly conceived the study and provided comments on the manuscript, and will coordinate and participate in the review. TS advised on the study and provided detailed edits on the manuscript, and will advise on the review. IM advised on the study, and provided detailed edits on the manuscript, and will advise on and participate in the review. BV advised on the study and provided detailed edits on the manuscript, and will advise on the review. HR jointly conceived the study and provided edits on the manuscript. AK provided comments and edits on the manuscript. AG provided edits on the manuscript and will participate in the review. IC provided edits on the manuscript and will participate in the review. JS provided comments on the manuscript and will participate in the review. All authors read and approved this final manuscript.
Electronic supplementary material
About this article
- Climate change
- Ecosystem management
- Ecosystem services
- Systematic map
- Ecosystem-based approaches for adaptation
- Ecosystem-based adaptation