- Systematic Map Protocol
- Open Access
Evidence for the impacts of agroforestry on agricultural productivity, ecosystem services, and human well-being in high-income countries: a systematic map protocol
Environmental Evidence volume 7, Article number: 24 (2018)
Agroforestry bridges the gap that often separates agriculture and forestry by building integrated systems that address both environmental and socio-economic objectives. Agroforestry can improve the resiliency of agricultural systems and mitigate the impacts of climate change. Existing research suggests that integrating trees on farms can prevent environmental degradation, improve agricultural productivity, increase carbon sequestration, generate cleaner water, and support healthy soil and healthy ecosystems while providing stable incomes and other benefits to human welfare. Although these claims are becoming more widely accepted as the body of agroforestry research increases, systematic understanding of the evidence supporting them remains lacking for high-income countries. This systematic map will address this research need by providing a tool for identifying and visualizing the existing evidence demonstrating the impacts of agroforestry practices and interventions on agricultural productivity, ecosystem services, and human well-being. The results will be useful for informing policy decisions and future research by making the evidence easily accessible and highlighting the gaps in knowledge as well as areas with enough evidence to conduct systematic reviews.
This systematic map will identify, collect, display, and describe available evidence on the impacts of agroforestry on agricultural productivity, ecosystem services, and human well-being in high-income countries. The search strategy will cover 5 primary databases and 24 organizational websites using a pre-defined search string designed to capture studies relating agroforestry practices and interventions to outcomes in high-income countries. The searches will all be conducted in English. We will screen the identified studies for inclusion or exclusion in stages, first on title and abstract and then on full-text. We will collect data from studies included at the full-text stage to form the map and associated database. For inclusion, the study in question must assess the impacts of the deliberate promotion and/or actual integration of woody perennials (trees, shrubs, palms, bamboos, etc.) on the same land management unit as agricultural crops and/or animals.
Agroforestry has risen to prominence as a land-use strategy to help address global climate change and provide other environmental, economic, and social benefits [1,2,3,4,5,6]. However, systematic knowledge on the human–environment impacts of agroforestry practices and interventions remains lacking. Agroforestry is promoted for its potential for carbon sequestration, soil erosion and runoff control, and improved nutrient and water cycling, as well as for offering socio-economic benefits and greater agricultural productivity [3,4,5,6,7,8,9,10,11]. While researchers and policy makers have long studied and supported agroforestry practices in low- and middle-income countries (L&MICs), particularly in tropical regions, recognition and promotion of agroforestry in the temperate climates typical of developed countries gained steam only more recently [2, 12]. As the conversation discussing the potential and future for agroforestry continues to evolve, we note an increased study of and policy support for agroforestry in high-income countries (HICs) [2, 8, 13, 14]. Evidence of the socio-economic and biophysical impacts of various agroforestry interventions and practices in HICs spans many disciplines and addresses a broad range of outcomes, thus creating an opportunity and need to synthesize the evidence for easier exchange of knowledge and ideas.
This study therefore aims to assemble the research showing the impacts of agroforestry practices and interventions in HICs to provide an evidence map of the literature to aid researchers and policy-makers in developing strategies for future research initiatives and policy formation. This systematic map (SM) directly parallels an evidence gap map (EGM) of the impacts of agroforestry in L&MICs that is currently in progress by members of same research group . This protocol draws heavily from the L&MIC EGM protocol since the methods are intentionally aligned. Although these two maps are intended to directly parallel one another, we acknowledge that there are differences in the types of agroforestry practiced and studied between HICs and L&MICs. These differences may in part be explained by greater wealth and resources associated with the socio-political histories in HICs than in L&MICs as well as different types of climates, with HICs being predominantly in temperate climates and L&MICs being predominantly in tropical climates. Furthermore, we note that the L&MIC EGM has a stronger emphasis on agroforestry interventions since it was conducted through the International Initiative for Impact Evaluation (3ie), which focuses more on synthesizing evidence on the impacts of interventions. This SM intends to capture studies on the impacts of both agroforestry interventions as well as agroforestry practices in general, without placing emphasis on one or the other.
Simply defined, agroforestry is the intentional integration of woody vegetation, such as trees and shrubs, with crops and/or livestock simultaneously or sequentially on a land management unit. This integration is intended to diversify production systems to create environmental, economic, and social benefits through complementary interactions between the system components [16,17,18]. The general types of agroforestry include agrisilviculture (also called silvoarable, defined as trees integrated with cropping systems), silvopasture (trees integrated with livestock systems), agrosilvipasture (trees integrated with both crops and livestock as a system), forest farming (crop or livestock production within a forested area), urban agroforestry (often referred to as homegardens, defined as integrating trees with crops near the homestead), and other types, such as integrating trees in fisheries or beekeeping operations . Common agroforestry practices are identified and presented in Table 1. We note that these practices are meant to be mutually exclusive (i.e., an agrosilvopasture practice would not also be classified as an agrisilviculture practice and a silvopasture practice).
We further define several types of interventions that may be used to promote any one or more of these agroforestry practices. Agroforestry intervention types are described in Table 2, and they represent types of support policy-makers could provide to promote adoption of one or more of the agroforestry practices described in Table 1.
This systematic map will denote whether a study is an impact evaluation of an agroforestry-related intervention or is an evaluation of the impact only of an agroforestry practice.
Agroforestry research began with the study of the existing traditional practices of local populations, which formed the basis for conducting more rigorous experimental research . As agroforestry research developed, researchers found a high potential for agroforestry to address many current environmental and social concerns, such as climate change and food security . From this knowledge base, agroforestry advocates began pushing for the establishment of policies and programs to support the integration of trees on agricultural lands.
Broadly speaking, however, governmental policies for landowners have often lacked incentives to take up agroforestry practices . Historically, there was the assumption that land must remain segregated between agriculture and other uses to optimize planning and productive efficiency (as opposed to establishing integrative land management techniques), which has limited the development of agroforestry . To support the progress of industrial agriculture, governments designed national policies to promote specialization and intensification, which works to enforce this strategy of separation . Industrial agriculture, however, is now associated with many negative social and environmental consequences [25,26,27,28]. Agroforestry has the potential to help address these consequences, and thus individuals familiar with agroforestry have started proposing and implementing a range of education and extension programs, financial incentives and cost-sharing initiatives, and support for the creation of markets for non-timber forest products to facilitate agroforestry adoption [16, 21, 29, 30]. Such interventions have the potential to provide the incentives and support necessary to establish agroforestry as a thriving alternative land use strategy, by way of the following conceptual framework.
Figure 1, developed and presented previously in , illustrates a generic theory of change which may underlie an effective agroforestry intervention. It identifies two initial preconditions: (1) successful mobilization and engagement of farmers; and (2) facilitating farmer capacity development and/or access to appropriate tree germplasm. The first of these and, in many cases, both, are required for significant and effective adoption of promoted agroforestry practices. Following such adoption, several intermediary outcomes are then expected. For example, farmers may see improved soil health and other ecosystem services, such as water infiltration, which then increase crop productivity or reduce production costs and, therefore, increase returns. Some participants may find that increased use and availability of tree/shrub fodder leads to increases in milk production and returns. Selling other agroforestry products such as timber, firewood, and fruit, can increase and diversify income and food sources [3, 31, 32]. These changes may have differential effects depending on gender, socio-economic status, race/ethnicity, or education/literacy level. Together, these intermediate outcomes are expected to bolster resilience to shocks, as well as boost overall household income and food security. These positive benefits along with features of the broader context in which participants operate will shape household investment in agroforestry. Our theory of change diagram presents positive pathways linking agroforestry interventions, adoption, and beneficial impacts; however, we also note that there are potentially negative tradeoffs to agroforestry, such as a reduction in area of crop production and negative tree-crop interactions.
By mapping the existing evidence of agroforestry practices in high-income countries with their impacts on agricultural productivity, ecosystem services, and human well-being, we will create an easily-navigable database of relevant research related to agroforestry impacts as well as form a clearer picture on key areas of interest for further research. The results will encompass research from all high-income countries, which will allow policymakers to utilize knowledge gained from around the globe as well as make the study relevant to all developed nations.
Why this systematic map is important to do for high income countries
A large body of evidence around agroforestry has accumulated over the past three decades through research across the high-income countries (HICs) of the world [5,6,7, 33]. These HICs are listed in Additional file 1, as defined by the World Bank for the 2018 fiscal year . Figure 2 provides a map showing the global HICs.
To date, however, there has not been a comprehensive synthesis of evidence of what agroforestry practices and interventions have been effective, under what circumstances, and by what measures in HIC contexts. Recent literature reviews have given overviews of the evidence for the impacts of agroforestry on ecosystem services and environmental benefits, climate change adaptation and mitigation, carbon sequestration, biomass production, soil health, and food production [5, 9,10,11, 35, 36]; however, they did not follow systematic review protocols. There are several recent efforts to systematically map and review aspects of agroforestry. Notably, one group mapped the evidence on agroforestry impacts on biodiversity and ecosystem services across Europe [7, 37]. Other systematic reviews include aspects of agroforestry, such as a systematic map on the impacts of vegetated strips—including windbreaks, hedgerows, and shelterbelts—on nutrients, pollutants, socioeconomics, biodiversity, and soil retention in boreo-temperate systems . Another study maps the impacts of Ecological Focus Area options (including agroforestry) in European farmed landscapes on climate regulation and pollination services . Finally, we note that a systematic map of the effects of nature conservation on human well-being  and one on forests and poverty globally  include some studies on the impacts of agroforestry. We are not aware, however, of any effort to systematically map evidence on the impacts of agroforestry interventions and practices on the broad range of agricultural productivity, ecosystem services, and human well-being outcomes across HICs. Lack of such evidence synthesis constrains the ability of policymakers, practitioners, and researchers to make effective decisions relating to agroforestry.
Though it is easy to find examples of agroforestry practiced throughout the world, the initiatives to create policies and programs that formalize and promote agroforestry are relatively new in most HICs. International groups have invested in agroforestry projects in low- and middle-income countries (L&MICs) for decades (emerging in the 1960s and 1970s) as a solution to address environmental degradation, boost food security, and contribute to a range of other development policy objectives [3, 42]. By contrast, agroforestry policy in the US, for instance, was first introduced in the mid-1980s (though promotion of windbreaks to reduce soil-erosion during the 1930s Dust Bowl era may be considered a precursor), with more formalized agroforestry policy emerging only in the 1990s with the Forest Stewardship Act of 1990 establishing a Center for Semiarid Agroforestry (renamed the National Agroforestry Center in 1994, broadening its scope to include the entire country). Similarly in the EU, agroforestry promotion began in the early 1990s with the 1992 reform of the EU Common Agricultural Policy (CAP), which formerly encouraged practices that discouraged farmers from integrating trees on farms . Only within the last decade has there been a significant uptake of agroforestry projects in HICs in the context of institutionalized support for agroforestry as an alternative land use approach to address conservation and sustainable agricultural development objectives .
Major agroforestry initiatives in high-income countries include the USDA Agroforestry Strategic Framework Plan (FY 2011–2016) in the United States , the European Commission’s AGFORWARD program in Europe (FY 2014–2017) , Agriculture and Agri-Food Canada’s Agroforestry Development Centre in Canada , and the Farm Forestry National Action Statement and the Master TreeGrower Program (supported by the Australian Agroforestry Foundation) in Australia . In Japan, the Satoyama Initiative includes agroforestry concepts, though it covers a broad range of practices .
One of the initial goals set out by these projects is to map out trees on farms and existing agroforestry practices within their respective countries. Several countries produced formalized documentation of the existing extent of agroforestry in their respective regions, such as the USDA in the United States  and the AGFORWARD project in Europe . The USDA report, however, is limited to practices established with USDA technical and financial aid and a comprehensive mapping is yet to be completed and released. This SM will add to the toolset of resources supporting these initiatives by compiling existing knowledge of agroforestry impacts, identifying research needs, and making evidence accessible and customizable for diverse audiences. Furthermore, the SM will highlight any existing studies evaluating the impacts of these projects or any other agroforestry-related programs and policies in HICs.
There is evidence showing that agroforestry offers many ecological benefits—environmental, economic, and social—which give reason to incentivize and empower landowners to adopt such practices; however, it is also important to consider the evidence of the tradeoffs associated with agroforestry [4,5,6,7,8,9,10,11, 23, 48]. There is a growing interest in the potential of agroforestry and an increasing awareness of the role agroforestry can play in creating a diversified, multi-dimensional farming system [2, 14, 36, 49]. Nevertheless, viewed in broader perspective, the integration of agroforestry into practice is still relatively low. For instance, the USDA estimates that agroforestry is applied on less than 1% of agricultural land with the potential for agroforestry through USDA assisted programs . This SM will therefore provide important evidence synthesis that may support initiatives to disseminate agroforestry knowledge and promote broader adoption of agroforestry as an alternative land use strategy across different HIC contexts. Additionally, it will help to find evidence of potential tradeoffs that come with the establishment of agroforestry practices.
There are two primary audiences for this SM. First, we expect that researchers on agroforestry and broader sustainability issues will use the results to inform further investigations on these topics, including new empirical research, as well as systematic reviews of specific linkages and further evidence synthesis. Results should be of wide interest to researchers in a range of institutions, particularly national programs (USDA, AGFOWARD, etc.), national and regional agroforestry associations and extension programs, and universities. The second main anticipated audience is decision-makers for whom agroforestry is already or potentially of interest. This includes relevant government ministries and agencies, non-governmental organizations (NGOs), and other advocacy and implementing organization staff.
In developing the parallel L&MIC EGM, our team engaged with an advisory group comprised of 3ie members, donor agency staff, International Development Coordinating Group (IDCG) members and other evidence synthesis experts, International Centre for Research in Agroforestry (ICRAF) scientists and other agroforestry subject experts. We published the L&MIC EGM protocol with the Campbell Collaboration . In preparing that protocol, we coordinated with the advisory group as well as colleagues involved in two related evidence maps [40, 50], and we presented the work at several conferences with opportunities for discussion, see . The HIC SM protocol was presented as a poster presentation at the Green Lands Blue Waters conference in Madison, Wisconsin in November 2017 and discussed with interested agroforestry experts. Feedback and suggestions given to the authors were incorporated into the HIC and L&MIC protocols. Finally, we expect to engage with additional reviewers through our efforts to publish this protocol and the resultant SM in peer-reviewed journals.
Objective of the map
The primary aim of this systematic map is to identify, map and describe existing evidence on the effects of agroforestry practices and interventions on agricultural productivity, ecosystem services, and human well-being in HICs.
In doing so, it addresses the following research questions:
What are the extents and characteristics of empirical evidence on the effects of agroforestry practices and interventions on agricultural productivity, ecosystem services, and human well-being in HICs?
What are the major gaps in the primary evidence base?
What are the agroforestry practice/intervention and outcome areas with potential for evidence synthesis?
To address these research questions, the scope is defined by the Population (Subject), Intervention (or Practice), Comparator, and Outcome (PICO) components to be examined, which are presented in Table 3.
The methods for the searches, screening, and eligibility criteria replicates those used for the L&MIC EGM , with modifications to adapt the process to account for differences between HIC and L&MIC concepts of agroforestry.
We will undertake a comprehensive search across 5 bibliographic databases and 24 organizational websites for grey literature to best capture an unbiased representation of existing literature. Studies from January 1, 1990 to the time of the search (mid-2018) will be included in the search. We begin the study period in 1990 as this is roughly the time that HICs saw increased support for agroforestry and other approaches designed to further environmental goals, as discussed earlier. The search will be done through use of search engines, based on key words within the identified databases. When such a strategy is not possible (e.g. for some topical databases and organizational websites), hand searches will be performed to extract all potentially relevant studies. Due to resource constraints, the focus will be on studies published in English, which we acknowledge as a limitation of the comprehensiveness of this study.
The bibliographic databases that will be searched for publications are:
EBSCO: Agricola, Econlit
Web of Science: Core Collection
CAB Abstracts and Global Health
The search terms to be used in full to conduct a topic search in the Web of Science: Core Collection and CAB Abstracts and Global Health databases are presented in Table 4. The search string includes each of the agroforestry practices from Table 1. We note that the intervention types are more generic, including topics well beyond agroforestry. Our search terms will therefore focus on practices, but in doing this, we capture the range of relevant interventions studied as well. We include relevant study area terms (study country terms) to limit the number of search results returned, and this decision did not affect our study retrieve performance, as detailed in the section on Assessing Retrieval Performance below. We make use of the Boolean operators and wildcards where possible, as shown in Table 4 and described in further detail in Additional file 2. Search strings are simplified for databases that have limited characters or lack Boolean functionality. The search strings used for each of the databases are detailed in Additional file 2.
Additionally, to identify the existing grey literature, the websites of various organizations that are likely to produce published and unpublished research will be searched, using the search terms from We include relevant study area terms (study country terms) to limit the number of search results returned, and this decision did not affect our study retrieve performance, as detailed in the section on Assessing Retrieval Performance below. We make use of the Boolean operators and wildcards where possible, as shown in Table 4 and described in further detail in Additional file 2. Search strings are simplified for databases that have limited characters or lack Boolean functionality. The search strings used for each of the databases are detailed in Additional file 2.
The list of relevant research organizations (Table 5) has been constructed from cross-validation of websites listed in the systematic mapping protocols of agroforestry related studies [41, 51]. To optimize the scope of the search while ensuring transparency in our methods, we will follow the approach developed by Haddaway et al. , which will allow us to search multiple websites simultaneously and to extract the relevant information from each website into a single database.
A search of literature through web-based search engines will also be performed. A search in Google Scholar, using the search terms from Table 4 will be performed and the first 300 results sorted by relevance will be reviewed, following the findings from Haddaway et al. . The search string for Google Scholar will be simplified to not include the list of high-income countries (relevant study locations). The online literature review and reference management software, EPPI-Reviewer 4, will be used to upload relevant titles and abstracts for candidate studies identified through the search strategy. We will create a project workspace using Box (https://www.box.com/home; accessed 2 September 2018) to assist in organizing and managing documentation files as well as a project workspace using Slack (https://slack.com/; accessed 2 September 2018; ) to manage communication on decisions, changes, and questions and provide a platform for all team members to have access to all relevant documents.
Assessing retrieval performance
The comprehensiveness of the search string was evaluated based on a test list of studies that meet the eligibility criteria. This test list consists of 44 studies, of which 22 are impact studies (18 primary studies, 4 systematic meta-analyses, and 1 farmer-managed field trial) and 22 researcher-managed field trials (which would not be included in this systematic map, per above, but which we would like to identify in the screening process for future work). The list was created based on personal knowledge and a snowballing method reviewing bibliographies and citations of known agroforestry papers. The test list was formed independently, without using the search string. The search string was tested and modified as needed by running it against this test list. The test list and results of the scoping process are reported in Additional file 2. The search string retrieved 43 of the 44 test studies (97.7% of the studies in the test list), which was deemed acceptable.
We designed our search string to balance between specificity (proportion of relevant information) and sensitivity (finding all relevant information), per . Our decision to include the country search terms is to avoid duplication of efforts from the parallel L&MIC EGM study and reduce the number of studies retrieved by our search to make the study feasible. We reviewed a sample of 500 studies omitted and found that the decision to include country terms omits primarily studies relating to L&MICs along with studies not relevant to our SM. Of the 500 omitted studies reviewed, we only found two relevant for inclusion (0.4%). Limiting the search by country terms reduces returned results from 92,293 to 30,014 (see Additional file 2). Furthermore, our retrieval rate of our test list studies does not change when the countries are removed from the search string (the one study that was missed did not contain any agroforestry-related terms that could be added to the search string). The topic search in Web of Science searches for the search terms in the title, abstract, keywords, and Web of Science KeyWords Plus. We found that with this type of topic search, a country term is almost always picked up, returning a sufficient percentage of the body of HIC agroforestry literature. Our assessment of study retrieval performance gives us confidence that we are capturing the majority of literature while not extending the scope of our study beyond feasibility.
Article screening and study eligibility criteria
We will first review search results at the level of title and abstract to determine inclusion or exclusion. Any study that we are unsure of whether it should be included or not during the title and abstract stage will be included for full-text review. We will keep a full list of excluded studies and record reasons for exclusion for each. Studies that meet the eligibility criteria at both the title and abstract stages will be reviewed at the full text stage. Those excluded at this stage will also be recorded along with reasons for the exclusion. A full list of studies excluded at full-text will be provided with the systematic map, along with the reasons for exclusion.
We will use double screening for a small subset of 100 training studies at the title and abstract stage and then use the approach in Snilstveit et al. for securing agreement among coders . We will use a training set consisting of 100 studies randomly selected from an initial search using our search string in Web of Science to assess agreement among coders. The reviewers will discuss any discrepancies between coding for this subset to reach agreement. Based on a training set of studies screened by all reviewers, inter-rater reliability will be calculated using a Kappa statistic for all studies double screened at title and abstract levels . If the Kappa test agreement falls below 0.6, indicating moderate agreement, an additional reviewer will be consulted and an additional set of 100 test studies will be screened by all reviewers, as in [39, 51].
During the screening process, when a rater is uncertain about study eligibility, the relevant study will be marked for a second opinion and screening by a second rater will be conducted. The lead reviewer will check the consistency of coding periodically throughout the coding process for a subset of studies at the title and abstract screening phase and at the full-text screening stage. At both the title and abstract screening phase and the full-text data extraction stage, a subset of 10% of the studies will be assessed by at least two reviewers. Studies where there is inconsistency or disagreement will be marked as “Re-evaluate” in EPPI-Reviewer 4 and will be discussed by reviewers to reach agreement.
Relevant type of study
Given that we seek to provide a resource for decision-makers, as well as identify gaps and well-researched areas in the current evidence base, we will include both primary studies and systematic reviews. Primary studies that measure the effect of agroforestry practices and interventions on the different outcomes of interest will be included, as will systematic reviews of the literature that synthesize and analyze these same relationships. We will include English-language studies conducted from 1990 onwards, through until the time of search (mid-2018).
Included studies must explicitly examine the outcomes of specific agroforestry practices and interventions on farm settings. Further, they must use a comparator, which may be temporal, spatial, between group, or some combination of these (see below). We will exclude theoretical or modeling studies (unless they include a relevant empirical example with design that meets eligibility criteria), and editorials and commentaries. Experimental trials managed by researchers will not be included due to time and resource constraints and since the population of interest for this systematic map is farmers and farmer’s land. These experimental off-farm trials, however, will be excluded into a separate bin in EPPI-Reviewer 4 and be available as a base for future work and synthesis. On-farm field trials will be included if all other eligibility criteria are met.
Relevant types of study design
We will include four kinds of studies: (1) quantitative impact evaluations, (2) systematic reviews, (3) on-farm field trials that test specific agroforestry techniques and approaches, and (4) observational studies on the effect of agroforestry practices.
Impact evaluations are studies that measure changes that occur due to an intervention. Such studies will use an experimental or quasi-experimental study design to conduct a counterfactual analysis to allow for attribution of changes in an outcome to a specific intervention, or compare the effects of different types of programs . Specifically, we will include the following types of impact evaluation studies:
Studies where participants or sites/plots of land (farmers, or land management areas on a farmer’s land) are randomly assigned to treatment and comparison group (experimental study designs);
Studies where assignment to treatment and comparison groups is based on other known allocation rules, including a threshold on a continuous variable (regression discontinuity designs) or exogenous geographical variation in the treatment allocation (natural experiments);
Studies with non-random assignment to treatment and comparison group that include pre-and post-test measures of the outcome variables of interest to ensure equity between groups on the baseline measure, and that use appropriate methods to control for selection bias and confounding. Such methods include statistical matching (for example, propensity score matching, or covariate matching), regression adjustment (for example, difference-in-differences, fixed effects regression, single difference regression analysis, instrumental variables, and ‘Heckman’ selection models).
Studies with non-random assignment to treatment and comparison group that include post-test measures of the outcome variables of interest only and use appropriate methods to control for selection bias and confounding, as above.
Ideally, studies would include baseline and post-intervention data, but due to our expectation of a small number of studies meeting this criterion, we will include studies with post-intervention outcome data only as long as they use some method to control for selection bias and confounding factors.
Reviews examine the effects of different interventions using transparent and systematic methods to identify, appraise and synthesize findings from studies addressing a specific issue . We will include systematic reviews, meta-analyses, and literature reviews that describe methods used for search, data collection, and synthesis. Literature reviews that do not describe methods used for search, data collection and synthesis will not be included; however, they will be identified and included in a separate bin in EPPI-Reviewer 4 and made available as a list with the report. Systematic maps and evidence gap maps identified where agroforestry is an area of interest will also be noted in the final report.
Field trials in agroforestry are designed to test the effects of experimental treatments or other variables on crop yield or other outcomes of interest in conditions similar to the actual growing conditions experienced by farmers who may adopt the treatment [14, 60]. While impact evaluations measure the changes due to an intervention, field trials measure the changes due to a practice. As for agronomy more generally, field trials can be divided into three types: (1) Researcher managed and researcher implemented; (2) researcher managed and farmer implemented, and (3) farmer managed and farmer implemented. We will include only studies implemented on a farmer’s land, and only if they pertain directly to some aspect of agroforestry, include an experimental research design, and describe the effects of an intervention, technique, or practice on an outcome category relevant to the current study.
Finally, we will include observational studies on the effects of agroforestry practices, provided they are quantitative and include at least one comparison as described below (e.g. before/after; study group/non-study group). We include such studies given that we anticipate a number of potentially interesting studies will not examine the impacts of an agroforestry intervention per se, but a specific practice or set of practices.
The SM will include both completed and ongoing studies, and the ongoing studies will be coded as ongoing and not yet completed.
The subject of interest will be farms and/or the people that live and farm on them that are incorporating any agroforestry practices into their farming system within the high-income countries (Additional file 1).
Relevant intervention or practice
Our study will capture studies evaluating the impacts of interventions to support agroforestry as well as those evaluating agroforestry practices alone. From a policy perspective, it is especially useful to know what kinds of interventions might most effectively promote agroforestry practices to yield desired social-ecological outcomes. Although impact evaluations on agroforestry-related interventions (Table 2) are of particular interest for policy-makers, our study will also include studies on the impacts of specific agroforestry practices (Table 1) without a policy intervention, which will broadly capture the impacts of agroforestry practices. This SM will therefore include any study that meets our criteria that evaluates the impacts of one or more agroforestry practice or intervention. In our map, we will indicate studies that include an evaluation of an agroforestry-related intervention, versus studies that evaluate the impact of only an agroforestry practice without a policy intervention. We will conduct analyses on the body of agroforestry practice impact studies as well as on the body of studies evaluating the impacts of specific agroforestry interventions.
Farm or household that does not adopt a given practice identified in Table 1, or is not exposed to a specific agroforestry intervention,
Farm or household before adopting a given agroforestry practice, or being exposed to a specific agroforestry intervention,
Farm or household that adopts a different agroforestry practice, and/or that is exposed to a different specific agroforestry intervention,
Primary forests, secondary forests, or managed forestry/plantations that not exposed to a specific agroforestry intervention,
A combination of two or more of the above. We will not include studies that only compare agroforestry practices with other agroforestry practices (i.e. studies that only evaluate different implementation of the same agroforestry practice, or studies that only evaluate multiple types of agroforestry practice).
The columns of the SM matrix will be comprised of three broad outcome categories: (1) agricultural productivity, (2) ecosystem services, and (3) human well-being.
Studies that focus exclusively on the adoption of a particular agroforestry technique or species without reference to impact will be excluded. We will, however, note the number of adoption-related studies (and their geographic location) excluded due to lack of evidence on outcomes. The primary outcomes are the three stated above (agricultural productivity, ecosystem services, and human well-being), and secondary outcomes are adoption and behavior change, which will only be reported if the study also reports primary outcomes.
Specific outcome categories under agricultural productivity will comprise farm productivity, including yield, and profitability.
Ecosystem services outcomes will first be classified under three broad categories: (a) provisioning, (b) regulation and maintenance, and (c) cultural services. Outcomes will be further divided into a number of specific categories following the Common International Classification of Ecosystem Services (CICES) developed by the European Environment Agency  and presented in Table 6. CICES builds from the seminal Millennium Ecosystem Assessment , The Economics of Ecosystems and Biodiversity , and other ecosystem services classification schemes.
For human well-being, the final broad outcome we will examine, we adapt the classification published in  to identify a set of key policy-relevant domains of human well-being (Table 7). Based on likely policy interest and goals typically articulated by proponents of agroforestry, we will focus on five dimensions of human well-being: income and household expenditure, housing and material assets, food security and nutrition, health, and cultural and subjective well-being. We will also include the category of “other” which may group some studies focusing on the other dimensions of human well-being identified in McKinnon et al. . In this last category, we will note in particular any mention of adaptive capacity or resilience, especially with reference to the impacts of climate change.
We will present the three outcomes in the SM main matrix in two ways: (1) a simplified typology of broad agroforestry practice/intervention and outcome categories and (2) a more detailed version with the specific agroforestry practice/intervention and outcome categories.
Types of settings
We expect that the agroforestry interventions and outcomes will take place in a range of settings in HICs. These settings will cover a range of ecoregions and are likely to be primarily rural, but potentially also urban areas (e.g. city gardens). We also expect much of the evidence to pertain to smallholders, but some may describe agroforestry practices among larger landholders.
Study quality assessment
Systematic maps do not tend to provide much information on study quality, but rather simply provide the broad overview of knowledge and highlight areas where there is the potential for further review and literature quality assessment . Therefore, we will not conduct study quality assessments on the studies included in this SM.
However, our study will include information about type of study design, referring to the types of study design presented above, including quantitative impact evaluations (experimental or quasi-experimental), systematic reviews, on-farm field trials (farmer-managed or researcher-managed), and observational studies on the effect of agroforestry practices. Furthermore, the type of quasi-experimental methods used, if applicable, will be documented. This data is not intended to offer an assessment of study quality, but rather provide basic information to get a broad perspective of the type of research being conducted in each area of the typology. As in the L&MIC EGM, we will break our results into three sections: a discussion of all included primary studies, a discussion of the subset consisting of only quantitative impact evaluation studies on interventions, and a discussion of included systematic reviews. We will present the distribution of study types for included studies and provide a list of all studies included at full-text with their assigned study type.
We do not expect to find any included studies authored by the coders of this systematic map. However, in the case that a study authored by one of the reviewers is included, those involved with authoring the studies will not be involved in decisions regarding inclusion or critical appraisal of that study.
Data coding strategy
Our research team will be led by the first author of this protocol (SEB), and the point of contact for any disputes on coding strategy will be the second author (DCM). The research team will consist of SEB, DCM, and between two and three hired research assistants, who are students at the University of Illinois at Urbana-Champaign. We will use a standardized data extraction form, attached as Additional file 3, to extract descriptive data from all studies meeting our eligibility criteria. We will create a codebook describing the scope of each question in the data extraction form. We will conduct a pilot with a small subset of studies by everyone in the research team to ensure consistency and to resolve any issues or ambiguities. Given the likely volume of studies (based on previous SR and SM experience such as in [15, 40]), we do not plan to carry out extensive side-by-side double extraction of data at the full text stage. Instead, we will conduct random spot checks of a small percentage of included articles to ensure consistency between raters. We will measure consistency using percent disagreement of spot-checking with the primary rater. We note that in our test studies, we found that some studies only specify a general practice (silvoarable, silvopasture, etc.) without detailing a specific practice (alley-cropping, improved fallow, riparian buffer strips, etc.). We address this in our data extraction spreadsheet by allowing a selection of “not specified” for the practice type.
Study mapping and presentation
We expect to perform several analyses based on the data collected and to summarize results visually and in various written forms to effectively communicate with intended audiences. A final report will present the map as a detailed data set with figures of descriptive statistics derived from the data set, as in . We intend to publish our systematic map in Environmental Evidence and upload the dataset online as an open-access, interactive site, as in . To communicate our results and visualize our data, we intend to create at minimum the following:
We will create a flow diagram of the systematic mapping process, detailing the number of studies returned by our search, included and excluded at each stage, and the number of studies included at the final data extraction stage.
We will create a tabular visualization reflecting agroforestry practices on the intervention axis and outcomes of the interventions on the outcome axis.
We will create a second tabular visualization reflecting agroforestry interventions on the intervention axis and outcomes of the interventions on the outcome axis.
We will provide descriptive statistics on geographical distribution of study location by country and world region, the type of studies, and quality of the SRs.
We will visually present the included studies in a matrix. The matrix will be stylized as a topography that notes whether the study is a review/SR, an impact evaluation of agroforestry interventions, or if the paper studies the impacts of specific agroforestry practices.
Based on these maps, we will perform gap analysis to identify areas for systematic review or primary research.
We will create heatmaps as in  to visualize and identify knowledge gaps and clusters.
We will upload our dataset online on an open-access, interactive map server, as in . Users will be able to visualize our results, filter the dataset per our data coding criteria, and automatically interact with the most updated version.
Furthermore, the reviewers will formally discuss and collectively identify areas of knowledge gaps and clusters of higher-quality literature based on their experience from screening full-texts after coding is completed. We will also comment specifically on the extent to which the literature examines interventions vs. specific practices or both simultaneously.
Smith J, Pearce BD, Wolfe MS. Reconciling productivity with protection of the environment: is temperate agroforestry the answer? Renew Agric Food Syst. 2012;28:80–92.
Jose S, Gold MA, Garrett HE. The future of temperate agroforestry in the United States. In: Nair PKR, Garrity D, editors. Agroforestry—the future of global land use. Dordrecht: Springer Netherlands; 2012. p. 217–45.
Waldron A, Garrity D, Malhi Y, Girardin C, Miller DC, Seddon N. Agroforestry can enhance food security while meeting other sustainable development goals. Trop Conserv Sci. 2017;10:1–6.
Garrity DP. Agroforestry and the achievement of the millennium development goals. Agrofor Syst. 2004;61:5–17.
Jose S. Agroforestry for ecosystem services and environmental benefits: an overview. Agrofor Syst. 2009;76:1–10.
Jose S, Bardhan S. Agroforestry for biomass production and carbon sequestration: an overview. Agrofor Syst. 2012;86:105–11.
Fagerholm N, Torralba M, Burgess PJ, Plieninger T. A systematic map of ecosystem services assessments around European agroforestry. Ecol Ind. 2016;62:47–65.
Wilson M, Lovell S. Agroforestry—the next step in sustainable and resilient agriculture. Sustainability. 2016;8:574.
Holzmueller EJ, Jose S. Biomass production for biofuels using agroforestry: potential for the North Central Region of the United States. Agrofor Syst. 2012;85:305–14.
Tsonkova P, Böhm C, Quinkenstein A, Freese D. Ecological benefits provided by alley cropping systems for production of woody biomass in the temperate region: a review. Agrofor Syst. 2012;85:133–52.
Udawatta RP, Jose S. Agroforestry strategies to sequester carbon in temperate North America. Agrofor Syst. 2012;86:225–42.
Buck LE. Agroforestry policy issues and research directions in the US and less developed countries: insights and challenges from recent experience. Agrofor Syst. 1995;30:57–73.
Lassoie JP, Buck LE, Current D. The development of agroforestry as an integrated land use management strategy. In: Gene Garrett HE, editor. North American agroforestry: an integrated science and practice. 2nd ed. Washington: American Society of Agronomy; 2009. p. 1–24.
Lovell ST, Dupraz C, Gold M, Jose S, Revord R, Stanek E, Wolz KJ. Temperate agroforestry research: considering multifunctional woody polycultures and the design of long-term field trials. Agrofor Syst. 2018;92:1397–415.
Miller DC, Ordonez PJ, Baylis K, Hughes K, Rana P. Protocol for an evidence and gap map. The impacts of agroforestry on agricultural productivity, ecosystem services, and human well-being in low- and middle-income countries: an evidence and gap map. In: Campbell collaboration; 2017.
USDA. USDA agroforestry strategic framework, fiscal year 2011–2016. Washington, D.C.: U. S. Department of Agriculture; 2011.
Leakey RRB. Definition of agroforestry revisited. Agrofor Today. 1996;8:5–7.
Atangana A, Khasa D, Chang S, Degrande A. Definitions and classification of agroforestry systems. In Tropical agroforestry. Dordrecht: Springer; 2014. p. 35–47.
Huxley P, van Houten H. Glossary for agroforestry. Nairobi: World Agroforestry Center (ICRAF); 1997.
USDA. Agroforestry USDA reports to America, fiscal years 2011–2012—comprehensive version. Washington, D.C.: Department of Agriculture; 2013.
Mosquera-Losada MR, Santiago-Freijanes J, Pisanelli A, Rois M, Smith J, Herder M, Moreno G, Malignier N, Mirazo JR, Lamersdorf N, et al. Extent and success of current policy measures to promote agroforestry across Europe. 2016.
Nair PKR, Garrity D. Agroforestry—the future of global land use. 2012.
Garrett HE, Buck L. Agroforestry practice and policy in the United States of America. For Ecol Manag. 1997;91:5–15.
IPES-Food. From uniformity to diversity: a paradigm shift from industrial agriculture to diversified agroecological systems. International Panel of Experts on Sustainable Food Systems; 2016.
Matson PA, Parton WJ, Power AG, Swift MJ. Agricultural intensification and ecosystem properties. Science. 1997;277:504–9.
Tilman D, Cassman KG, Matson PA, Naylor R, Polasky S. Agricultural sustainability and intensive production practices. Nature. 2002;418:671.
Lobao L, Meyer K. The great agricultural transition: crisis, change, and social consequences of twentieth century US farming. Ann Rev Sociol. 2001;27:103–24.
Foley JA, DeFries R, Asner GP, Barford C, Bonan G, Carpenter SR, Chapin FS, Coe MT, Daily GC, Gibbs HK, et al. Global consequences of land use. Science. 2005;309:570–4.
Valdivia C, Barbieri C, Gold MA. Between forestry and farming: policy and environmental implications of the barriers to agroforestry adoption. Can J Agric Econ. 2012;60:155–75.
Jacobson M, Kar S. Extent of agroforestry extension programs in the United States. J Ext. 2013;51:4RIB4.
Mbow C, Van Noordwijk M, Luedeling E, Neufeldt H, Minang PA, Kowero G. Agroforestry solutions to address food security and climate change challenges in Africa. Curr Opin Environ Sustain. 2014;6:61–7.
Sharma N, Bohra B, Pragya N, Ciannella R, Dobie P, Lehmann S. Bioenergy from agroforestry can lead to improved food security, climate change, soil quality, and rural development. Food Energy Secur. 2016;5:165–83.
Garrett HE. North American agroforestry: an integrated science and practice. Washington: American Society of Agronomy; 2009.
World Bank Country and Lending Groups. https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups. Accessed 28 June 2018.
Wolz KJ, Lovell ST, Branham BE, Eddy WC, Keeley K, Revord RS, Wander MM, Yang WH, DeLucia EH. Frontiers in alley cropping: transformative solutions for temperate agriculture. Glob Chang Biol. 2018;24:883–94.
Dollinger J, Jose S. Agroforestry for soil health. Agrofor Syst. 2018;92:213–9.
Torralba M, Fagerholm N, Burgess PJ, Moreno G, Plieninger T. Do European agroforestry systems enhance biodiversity and ecosystem services? A meta-analysis. Agric Ecosyst Environ. 2016;230:150–61.
Haddaway NR, Brown C, Eales J, Eggers S, Josefsson J, Kronvang B, Randall NP, Uusi-Kämppä J. The multifunctional roles of vegetated strips around and within agricultural fields. Environ Evid. 2018;7:14.
Ottoy S, Angileri V, Gibert C, Paracchini ML, Pointereau P, Terres J-M, Van Orshoven J, Vranken L, Dicks LV. Impacts of selected ecological focus area options in European farmed landscapes on climate regulation and pollination services: a systematic map protocol. Environ Evid. 2018;7:10.
McKinnon MC, Cheng SH, Dupre S, Edmond J, Garside R, Glew L, Holland MB, Levine E, Masuda YJ, Miller DC, et al. What are the effects of nature conservation on human well-being? A systematic map of empirical evidence from developing countries. Environ Evid. 2016;5:8.
Cheng SH, Ahlroth S, Onder S, Shyamsundar P, Garside R, Kristjanson P, McKinnon MC, Miller DC. What is the evidence for the contribution of forests to poverty alleviation? A systematic map protocol. Environ Evid. 2017;6:10.
Garrity DP, Akinnifesi FK, Ajayi OC, Weldesemayat SG, Mowo JG, Kalinganire A, Larwanou M, Bayala J. Evergreen agriculture: a robust approach to sustainable food security in Africa. Food Secur. 2010;2:197–214.
Mosquera-Losada MR, Moreno G, Pardini A, McAdam JH, Papanastasis V, Burgess PJ, Lamersdorf N, Castro M, Liagre F, Rigueiro-Rodríguez A. Past, Present and Future of Agroforestry Systems in Europe. In: Nair PKR, Garrity D, editors. Agroforestry—the future of global land use. Dordrecht: Springer Netherlands; 2012. p. 285–312.
Thevathasan NV, Gordon AM, Bradley R, Cogliastro A, Folkard P, Grant R, Kort J, Liggins L, Njenga F, Olivier A, et al. Agroforestry research and development in Canada: the way forward. In: Nair PKR, Garrity D, editors. Agroforestry—the future of global land use. Dordrecht: Springer Netherlands; 2012. p. 247–83.
Reid R. Developing farmer and community capacity in agroforestry: is the Australian Master TreeGrower program transferable to other countries? Agrofor Syst. 2016;91:847–65.
Takeuchi K. Rebuilding the relationship between people and nature: the Satoyama initiative. Ecol Res. 2010;25:891–7.
den Herder M, Moreno G, Mosquera-Losada RM, Palma JHN, Sidiropoulou A, Santiago Freijanes JJ, Crous-Duran J, Paulo JA, Tomé M, Pantera A, et al. Current extent and stratification of agroforestry in the European Union. Agric Ecosyst Environ. 2017;241:121–32.
Montambault JR, Alavalapati JRR. Socioeconomic research in agroforestry: a decade in review. Agrofor Syst. 2005;65:151–61.
Jordan N, Warner KD. Towards multifunctional agricultural landscapes for the Upper Midwest Region of the USA. In: Steve Wratten HS, Ross C, Robert C, editors. Ecosystem services in agricultural and urban landscapes. New York: Wiley; 2013. p. 139–56.
Cheng SH, MacLeod K, Ahlroth S, Onder S, Perge E, Shyamsundar P, Rana P, Garside R, Kristjanson P, McKinnon MC, Miller DC. What is the contribution of forests to poverty alleviation? A systematic map of global evidence. Environmental Evidence in review.
Haddaway NR, Brown C, Eggers S, Josefsson J, Kronvang B, Randall N, Uusi-Kämppä J. The multifunctional roles of vegetated strips around and within agricultural fields. A systematic map protocol. Environ Evid. 2016;5:18.
Haddaway NR, Collins AM, Coughlin D, Kirk S. A rapid method to increase transparency and efficiency in web-based searches. Environ Evid. 2017;6:1.
Haddaway NR, Collins AM, Coughlin D, Kirk S, Larsen P, Ins MV, Pautasso M, Noorden R, Khabsa M, Giles C, et al. The role of google scholar in evidence reviews and its applicability to grey literature searching. PLoS ONE. 2015;10:e0138237.
Perkel JM. How scientists use Slack. Nat News. 2017;541:123.
James KL, Randall NP, Haddaway NR. A methodology for systematic mapping in environmental sciences. Environ Evid. 2016;5:7.
Snilstveit B, Stevenson J, Villar PF, Eyers J, Harvey C, Panfil S, Puri J, McKinnon MC. Land-use change and forestry programmes: evidence on the effects on greenhouse gas emissions and food security. In: Evidence gap map report 3. London; 2016.
Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20:37–46.
Ferraro PJ. Counterfactual thinking and impact evaluation in environmental policy. New Dir Eval. 2009;2009:75–84.
Waddington H, White H, Snilstveit B, Hombrados JG, Vojtkova M, Davies P, Bhavsar A, Eyers J, Koehlmoos TP, Petticrew M, et al. How to do a good systematic review of effects in international development: a tool kit. J Dev Eff. 2012;4:359–87.
Nair PKR. State-of-the-art of agroforestry research and education. Agrofor Syst. 1993;23:95–119.
The Common International Classification of Ecosystem Services (CICES). http://cices.eu. Accessed 28 June 2018.
Millennium Ecosystem Assessment. Ecosystems and human well-being: synthesis. Washington, DC: Island Press; 2005.
Kumar P. The economics of ecosystems and biodiversity (TEEB) ecological and economic foundations. London, Washington: Earthscan; 2010.
Miller D, Ordonez P, Brown S, Forrest S, Nava N, Hughes K, Baylis K. The impacts of agroforestry on agricultural productivity, ecosystem services, and human well-being in low- and middle-income countries: an evidence gap map. In: 3ie evidence gap map report. London: International Initiative for Impact Evaluation (3ie); (in review).
All authors developed the original idea, participated in discussions defining the search and eligibility criteria and data coding strategy, and decided the study mapping and presentation design. SEB contributed to developing the background section and testing the search criteria. DCM, PJO, and KB developed the theory of change. PJO and SEB developed the data coding table. SEB had the lead of writing the manuscript, with contributions from all authors. All authors read and approved the final manuscript.
We thank Karl Hughes and Pushpendra Rana for helping shape this protocol through their development of a parallel project creating an Evidence Gap Map (EGM) for agroforestry impacts in low- and middle- income countries (L&MICs), supported by 3ie . We also thank Festus Amadu and Katia Nakamura for comments on an earlier version of this protocol. Finally, we thank the editor of Environmental Evidence, Dr. Andrew Pullin, and three anonymous reviewers for comments on earlier drafts of this manuscript that have greatly improved this protocol.
The authors declare that they have no competing interests.
Availability of data and materials
Consent for publication
Ethics approval and consent to participate
This project is funded by the USDA National Institute of Food and Agriculture, Hatch Project #1009327.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Additional file 1. List of high-income countries (according to the World Bank 2018 fiscal year classification ). Tabular list of high-income countries according to the World Bank 2018 fiscal year classification.
About this article
Cite this article
Brown, S.E., Miller, D.C., Ordonez, P.J. et al. Evidence for the impacts of agroforestry on agricultural productivity, ecosystem services, and human well-being in high-income countries: a systematic map protocol. Environ Evid 7, 24 (2018). https://doi.org/10.1186/s13750-018-0136-0
- Alley cropping
- Riparian buffer
- Forest farming
- Socio-economic impact
- Impact evaluation