What are the socio-economic impacts of genetically modified crops worldwide? A systematic map protocol

Abstract

Background

Genetically modified (GM) crops have generated a great deal of controversy. Since commercially introduced to farmers in 1996, the global area cultivated with GM crops has increased 94-fold. The rapid adoption of GM technology has had substantial socio-economic impacts which a vast amount of technical and non-technical literature has addressed in the last two decades. However, contradictory results between individual studies abound. Extensive and transparent reviews concerning this contentious and complex issue could help promote evidence-based dialogue among the diverse parties involved.

Methods

This protocol specifies the methodology for identifying, evaluating, and mapping evidence related to the main review question: what are the socio-economic impacts of genetically modified crops worldwide? This question has been subdivided into the following topics: (a) farm-level impacts; (b) impacts of coexistence regulations; (c) impacts along the supply chain; (d) consumer-level impacts; (e) impacts on food security; and (f) environmental economic impacts. The search strategy includes the identification of primary studies from general scientific databases; global, regional, and national specialist databases; an on-line search engine; institutional websites; journal websites; subject experts/researchers; and serendipity. Searches will be conducted in six languages (Chinese, English, French, German, Portuguese, and Spanish). Identified studies will be screened for inclusion/exclusion criteria by a group of multi-language reviewers. Finally, pre-defined data from the studies will be extracted, mapped, and presented in a report. Potential research gaps will be identified and discussed, and the review process will be documented in an open-access database (i.e. CADIMA, http://www.cadima.info/).

Background

Genetically modified (GM) crops have generated a great deal of controversy. The use of biotechnology in agriculture has caused major ideological and scientific concerns that continue to be echoed in the media and academic press [1]. Since commercially introduced to farmers in 1996, the global area cultivated with GM crops has increased 94-fold, from 1.7 million hectares to 160 million hectares in 2011 [2]. The rapid adoption of this technology has had substantial socio-economic impacts [3]. Consequently, a vast amount of technical and non-technical literature addressing this topic has accumulated over the last two decades [4]. Moreover, groups of stakeholders characteristically advocate opposing opinions, which may not be based on best available evidence. Therefore, the availability of transparent and reliable reviews of studies on the socio-economic impacts of GM crops could help promote evidence-based dialogue among the diverse parties involved. Systematic maps employ structured procedures that can be particularly useful for minimizing potential biases that may arise during the process of identification, selection, and analysis of evidence involved in controversial topics. Systematic maps provide an opportunity to gather and describe evidence relevant to a broad field of policy and management relevance^a. The breadth of the evidence captured in a systematic map helps to clearly identify potential research gaps and guide future research efforts [5]. In addition, systematic maps make relevant evidence readily accessible to researchers and stakeholders through the development of extensive databases, the content of which can be relatively easily updated as needed.

Currently, numerous literature reviews and meta-analysis studies have assessed the socio-economic impacts of GM crops (a non-comprehensive list of 20 studies is included in the Additional file 1). Nevertheless, none is a systematic map, and only one is a systematic review (see Hall et al. [6])^b. That systematic review focused on the costs and profits of GM agriculture in comparison with conventional agriculture. One shortcoming of the document, as stated by the authors, was the exclusion of studies conducted before 2006, which disregards valuable earlier literature. The authors also clarified that [6]: “Additional time for conducting a systematic review such as this one would allow the inclusion in the search process of additional databases that were excluded because it was not possible to directly export results to Reference Manager Database. An extended review on this topic would be a potentially valuable contribution to the ‘GM debate’”.

Through the EU project “GMO Risk Assessment and Communication of Evidence” (GRACE, 2012–2015), comprehensive reviews of existing evidence of potential health, environmental, and socio-economic impacts of GM crops worldwide will be conducted [7]. As members of GRACE, the authors of this protocol (Technische Universitaet Muenchen, TUM) will undertake a systematic map on the socio-economic impacts of genetically modified (GM) crops. In particular, the Description of Work (DoW) for GRACE states that TUM is responsible for carrying out reviews on the following key topics: (1) farm-level economic impacts of GM crops; (2) economics of coexistence; (3) economics of segregation at the level of supply-chains; and (4) consumer acceptance of GM crops.^c GRACE is following a participatory approach, and stakeholders are being consulted during each of the project’s steps. The stakeholders include members of industry and civil society organizations, as well as competent authorities on GM crops in the EU Member States and scientific experts from academia^d. Two new topics were added based on stakeholder requests: environmental economic impacts^e of GM crops and the impacts of GM crops on food security (for more information about the participatory process, see GRACE [8]). Therefore, TUM will produce a systematic map covering the six topics stated above, the overall conceptual model of which is outlined in Figure  1. The extensive systematic map will address the broad review question: what are the socio-economic impacts of genetically modified crops worldwide?

Figure 1

Overall conceptual model of the topics to be covered in the systematic map.

The systematic map undertaken will provide an important overview of the existing literature related to the socio-economic impacts of GM crops available in six languages (Chinese, English, French, German, Spanish, and Portuguese). These languages are among the top nine used for publication of research^f [9] and also the primary languages spoken in 23 of the 28 countries currently cultivating GM crops [10].

The description of the topics to be covered in the systematic map is provided below.

Farm-level impacts

Farmers have different socio-economic motivations for adopting GM crops. Significant socio-economic determinants include: gender associated aspects (e.g., [11]); individual and social learning (e.g., [12]); educational level (e.g., [13]); and expected benefits and uncertainty (e.g., [14–16]). For GM adopters, potential changes in yield and economic returns depend on current and previous crops and specific trait characteristics; agricultural practices; incidence of pest infestation; seed costs; and market characteristics (e.g., [17, 18]). Farmers’ production efficiency (farmers’ ability to produce more with less than or equal inputs/resources) would also be affected (e.g., [19]), as well as the frequency of pesticide poisoning incidents and health impacts (e.g., [20]). Consumption of new bio-fortified GM crops are expected to increase farmers’ nutrition status and as such, they could significantly contribute to farmers’ well-being (e.g., [21]). Most of the world’s poor depend mainly on farming for their subsistence. The adoption of GM crops could have different impacts on wealthier and poorer farmers (e.g., [22]), which could exacerbate/mitigate social problems. Ethical aspects may also be affected, as it has been demonstrated that ethical values can change over time (e.g., changing views on euthanasia in the U.S. and Japan [23]). A change in acceptability of GM crops may imply a change in adopters’ values. Finally, cultural aspects may be impacted as well; for example, GM seeds need to be purchased, causing a disturbance in the traditional exchange of seeds among indigenous farmers (along with potential changes in identity and trust among involved farmers).The main aspects considered within this topic are presented graphically in a conceptual model (Figure  2). This conceptual model shows that socio-economic factors influence farmer decisions regarding the adoption of GM crops. GM adoption is expected to impact aspects related to farmers’ income and also intangible aspects. The potential income-related impacts include changes in the use of inputs; associated costs; output (quantity and quality); and gross income. Some farmers could experience changes in time available for conducting off-farm income-generating activities. A farm’s efficiency could deteriorate or improve with use of new technologies impacting the farmer’s income. Intangible aspects that may be affected after GM adoption relate to health safety issues associated with changes in pesticide use and farmers’ nutritional status if they cultivate and consume bio-fortified crops. Primary social, ethical, and cultural aspects are also depicted in the conceptual model.

Figure 2

Conceptual model of socio-economic impacts on farmers.

Coexistence related impacts

The possibility that GM farms contaminate non-GM farms via unintentional or inadvertent gene flow constitutes a challenge for the coexistence of GM farming and conventional agriculture, including organic certified agricultural systems. Several studies have analysed the effects that the introduction of ex-ante regulatory and ex-post liability aspects would have on farm-level costs and GM spatial configuration and adoption dynamics (e.g., [24–26]). In addition, potential benefits due to higher price premiums for non-GM products have also been evaluated (e.g., [27]).The main aspects considered within this topic are presented graphically in a conceptual model (Figure  3). This conceptual model shows that GM plants and crops can be introduced under alternative coexistence systems (separation between GM and non-GM farms and dual GM/non-GM farms) and regulatory frameworks, including ex-ante (e.g., mandatory segregation, traceability, minimum GM tolerance levels, rigid and flexible refuge areas, and voluntary GM-free zones) and ex-post liability aspects (e.g., compensation funds, insurance schemes, and marketplace liability). The different coexistence options are expected to influence in different manners GM and non-GM farm-level costs, particularly operational; transaction; opportunity; and testing and remediation costs. GM adoption dynamics could change as well, such as the rate of adoption, spatial configuration, and speed and stability of GM expansion. GM-farmers would also generate externalities and directly influence the economic benefits of non-GM farmers due to inadvertent gene flow from GM to non-GM fields which may create problems for non-GM farmers willing to sell their products in specific markets (e.g., organic certified markets). Finally, social factors, such as the level of trust between neighbors, would influence farm-level costs (e.g., lower/higher negotiation costs) and adoption dynamics of GM crops (e.g., stronger/lower imitation or neighboring effects) in each of the ex-ante and ex-post regulatory regimes under evaluation (social aspects not pictured in the figure).

Figure 3

Conceptual model of socio-economic impacts of coexistence.

Supply chain impacts

The focus of this section is on the supply chain or organization network as unit of analysis. It aims to analyse the socio-economic impacts of the commercialization of GM crops on supply chain structure and performance dynamics, as well as cost and benefit distribution along different actors in the supply chain.

In general, the basic elements of the structure of the supply chain include:

• (a) Vertical relations. These refer to the sequence of value adding activities. Actors performing different functions within the supply chain are vertically linked through buying and selling relationships. Vertical relations highlight the level of cooperation, coordination, trust, and governance (or power) along the chain.

• (b) Horizontal relations. These reflect the relationships among actors performing the same function within the chain. Horizontal relations can be formal (e.g., cooperatives and associations) or informal.

The main factors related to supply chain performance are:

• (a) Efficiency or the ability to deliver value at a minimum of total costs.

• (b) Effectiveness or the ability of the chain to provide superior value.

• (c) Innovation or the ability to respond to changes in consumer demand or the external environment.

Several studies have analysed the effect that the commercialization of GM crops would have on the supply chain structure, as well as the distribution of costs and benefits of different actors along the supply chain (e.g., [28–32]). Moreover, governance mechanisms and market power of different actors would also be affected (e.g., [33, 34]). The main aspects considered under this topic are presented graphically in a conceptual model (Figure  4). This conceptual model shows that the commercialization of GM products under different enforced coexistence rules, labeling schemes, and protection of intellectual property rights would have impacts on the supply chain structure (e.g., vertical and horizontal relations) and performance (e.g., efficiency, effectiveness, and innovation ability). This in turn would affect the distribution of costs and benefits for the different actors along the supply chain, as well as their market power (ability to influence the price of a commercialized item).

Figure 4

Conceptual model of socio-economic impacts along the supply chain.

Consumer-level impacts

The socio-economic determinants for consumers’ acceptance of GM food and the associated price premiums for non-GM products have been evaluated under different mandatory and voluntary GM-related label schemes (e.g., [35–37]). Other studies have evaluated the option values of a moratorium or ban on GM products (e.g., [38]). Those price premiums and option values have been used to calculate economic welfare effects (e.g., [39]). These and other main aspects related to the impacts of GM products on consumers are presented graphically in a conceptual model (Figure  5). The conceptual model shows that GM products can be introduced into the market under mandatory and voluntary GM-related labels, including different tolerance levels (or percentage of GM ingredients in the final products) or can be subject to moratorium or ban. The decision or intention to buy those products is based on consumers’ socio-economic characteristics (e.g., age, gender, and educational level). Potential buyers can indicate their willingness to pay (WTP) for these products, and changes in social welfare can be calculated based on the differences between the WTP and actual or expected prices (price premiums). If there is a moratorium or ban on GM products, option values can be calculated based on a (hypothetical) WTP to preserve or maintain this situation. Social welfare can be estimated by the difference between the WTP and the opportunity costs of forgoing economic growth associated with the commercialization of GM products. GM products can have an impact on consumers’ health, for example in the case of bio-fortified food. Social, ethical, and cultural aspects were added as requested by stakeholders.

Figure 5

Conceptual model of socio-economic impacts at consumer level.

Environmental economic impacts of GM crops

GM crops may substitute for agricultural inputs and practices that are environmentally harmful. The study by Brookes & Barfoot [40] suggest that “since 1996 the use of pesticides (counted as active ingredients) on the GM crop area was reduced by 448 million kg (9% reduction), and the environmental impact quotient — an indicator measuring the environmental impact associated with herbicide and insecticide use on these crops — fell by 17.9%. In 2010, the total carbon dioxide emission savings associated with GM crop adoption were equal to the removal from the roads of 8.6 million cars due to reduced fuel use and additional soil carbon sequestration”.

GM crops can cause environmental harm as well (although there is considerable uncertainty and no consensus among scientists) [41]. In particular, the protection of biodiversity and ecosystem services ought to be a top priority when taking into consideration the dependency on a healthy environment of all human activity, now and in the future [42]. For those opposed to GM technology, GM crops are exotic species being introduced into open complex ecosystems of which we have limited understanding [43], and as such it is impossible to anticipate all impacts of GM technology on the environment.

The effects of GM crop adoption on the environment will depend not only on human behavior but on biological, ecological, and chemical interactions as well. Many disciplines are needed to evaluate these kinds of impacts [41]. In addition, there is the possibility of irreversible ecosystem disruptions due in part to the unpredictable and novel effects of gene mixing [43].

Figure  6 shows a basic conceptual model of the potential environmental economic impacts of GM crops (based on information obtained from [40–46]). Depending on the type of genetic modification, the cultivation of GM crops can change the type or quantity of herbicide/insecticide used, improve the crops’ resistance to external climate stress (e.g., drought and salinization), or cause an undesired gene flow (e.g., from GM crops to wild relatives).

Figure 6

Conceptual model of environmental economic impacts of GM crops.

Changes in the type or quantity of herbicide/insecticide could create or alter herbicide resistance in weeds or pesticide resistance in pests. Soil, water, and air contamination is reduced if the substituted herbicide/pesticide was more toxic than the new herbicide/pesticide. Further, if less herbicide/pesticide is required, resources like fuel could be saved. Changes in herbicide/insecticide use could also modify agricultural practices, such as encourage tillage, weed management, or monoculture. New alternative agricultural practices could change the use of resources and fuel consumption, which in turn would have impacts on soil, water, and air contamination and soil organisms and biodiversity. In addition, there could be improvements in crop yields using existing land and water resources, which in turn could reduce land use; water and air contamination; minimize the impacts on biodiversity; and save resources and fuel consumption. In a similar manner, the cultivation of drought- and salinity-tolerant GM crops would also impact soil, water, air, biodiversity, and modify the use of resources and fuel consumption. Finally, there could be a gene flow from GM crops to wild relatives with unknown consequences to the environment.

It is worth mentioning that this protocol contemplates the environmental economic impacts of GM crops. Therefore, only primary studies incorporating an economic assessment of these and similar environmental impacts will be considered. The environmental impact assessment component of the included primary studies will be taken as given.

Food security at household level

The estimated number of undernourished people has continued to decrease, but the rate of progress still appears insufficient to reach international goals for hunger reduction [47]. Currently, about 842 million people (one in eight people in the world) suffer chronic hunger, unable to obtain the amount of food necessary to conduct an active life [47]. The vast majority of hungry people live in developing countries, where the prevalence of undernourishment is estimated at 14 percent [47].

Food security exists when all people, at all times, have physical and economic access to sufficient, safe, and nutritious food that meets dietary needs and food preferences for an active and healthy life [47]. There are four dimensions of food security: food availability (e.g., food production and processing); food access (e.g., having the economic resources to buy the right food); food utilization (e.g., education to individuals to make proper use of healthy food); and food system stability (e.g., adequate access to food at all times). For food security objectives to be realised, all four dimensions must be fulfilled simultaneously [47, 48].

Therefore, food security is a multidimensional concept, and data on all dimensions are rarely available and frequently unreliable [49]. Moreover, the international community lacks a consensus on core household food security indicators needed in order to properly measure and monitor food security worldwide. The indicators also vary on level of analysis, ranging from the regional or national level to the household or individual level, depending on data availability and the design of the instruments used to collect the data (e.g., surveys) [49].

In relation to GM crops, reports from expert governmental and nongovernmental bodies increasingly include GM crops as part of a wider approach to food security [50]. GM crops could help to mitigate expected food shortages related to population growth and the effects of climate change in specific regions worldwide. For example, GM crops could impact food availability by providing seeds which are resistant to adverse climate conditions; have an effect on food access by increasing farmers’ incomes; and, under the same food utilization conditions, bio-fortified crops could increase the nutritional status of households worldwide. (Figure  7 illustrates this example).

Figure 7

Conceptual model of food security at household level.

In the approach followed in this protocol, the ultimate goal of food security is to improve the nutritional status of households. It is worth mentioning that several of the multidimensional aspects of food security have been already covered by other topics in this protocol (e.g., impacts of GM crops on farm-level income). Nevertheless, there are a growing number of socio-economic studies which specifically evaluate the impacts of GM crops on (at least one component of) food security and explicitly indicate that as so.

Objective of the systematic map

The main objective of the systematic map is to identify the breadth of knowledge related to the socio-economic impacts of GM crops worldwide. Our question related to the overall objective of the systematic map is:

What research evidence exists (number of studies and the current state of research studies) on the socio-economic impacts of GM crops worldwide (in Chinese, English, French, German, Spanish, and Portuguese languages)?

Relating to the secondary objectives, the systematic map will identify the types of socio-economic impacts; populations; crops and GM traits; geographical focus; research methodologies; evidence gaps; and the particular topics that could be subjects of further analyses or subsequent systematic reviews. Therefore, the questions related to the secondary objectives of the systematic map are:

• a) What types of socio-economic impacts have been addressed?

• b) What types of populations have been addressed?

• c) What types of crops and GM traits have been addressed?

• d) What is the geographical focus of the evidence?

• e) What research methods have been used to collect and analyse the evidence?

• f) What evidence gaps exist that could/ should be addressed in future primary research?

• g) Which particular topics could be the subject of further analysis (e.g., meta-analyses and meta-regressions) or subsequent systematic reviews?

Methods

Search strategy

Systematic maps require an objective and reproducible search of a range of sources to identify as many relevant studies as possible (within resource and time limits). A search strategy that includes extensive search terms and a combination of multiple data sources can increase the likelihood of capturing most of the relevant references. Our search terms consider a list of intervention-, outcome- and population-related keywords in six languages. Our data sources include: general scientific databases; an on-line search engine (Google Scholar); global, regional, and national specialist databases; institutional websites (to be accessed through one of the largest institutional repository search engines: Bielefeld Academic Search Engine, BASE); journal websites; subject experts/researchers; and serendipity (e.g., finding relevant documents by accidental discovery or by chance). The reference management software to be used for exporting/importing the references is Citavi, which is freely available at TUM.

We aim to identify as many of the available relevant studies as possible (based on time and budget constraints). Sensitivity will be favored over specificity. Sensitivity implies that the emphasis of the search procedure will be in obtaining most of the relevant articles at the risk of obtaining a high number of non-relevant ones (which would need to be depurated later during the screening stage). On the other hand, specificity emphasizes the retrieval of relevant articles with the lowest number of non-relevant ones as possible (at the risk of omitting some/ many of the relevant articles).

Search terms

Search terms related to the intervention

The selected search terms related to the intervention (GM crops) in the English language are presented in Table  1. These search terms were derived from a preliminary list of 29 GM crop related terms compiled by experts from the GRACE project (see Additional file 2). IDEAS/REPEC, the largest freely-available bibliographic database dedicated to economics, was used to test this preliminary list. Search terms which did not retrieve relevant references (e.g., cisgenesis) or which retrieved similar references as other search terms (e.g., glufosinate tolerant did not retrieve additional relevant references in comparison to herbicide tolerant) were dropped from the list. The searches were conducted on title, abstract, and keywords. Then AGROVOC, the corporate thesaurus of the Food and Agriculture Organization (FAO), was searched for controlled terms^g which were also included as search terms (e.g., biosafety, biosecurity). Finally the titles from the reference lists of the reviews and meta-analysis included in the Additional file 1 were visually examined to evaluate the completeness of our selected search terms, and new search terms were added when needed (e.g., Bollgard, drought resistant).

Table 1 List of search terms in English language

Additional searches using the selected terms were conducted in Web of Science (All databases) in Topic (Title, abstract, keywords). Some of the terms retrieved high number of records (e.g., bt retrieves 78,027 references; and 3,310 when restricted by research domains: social sciences and arts humanities), most of them were not related to GM crops. Therefore the final intervention terms include a descriptor (e.g., bt crop*) or the type of crop (e.g., bt cotton). The types of crops were compiled from the list of approved crops by GM trait reported in the “GM approval database” by the International Service for the Acquisition of Agri-biotech Applications (ISAAA, http://www.isaaa.org/gmapprovaldatabase/default.asp). In this way, we obtained a reasonable number of references. For example, for bt as a group (“bt crop*” or “bt seed*” or “bt cotton” or “bt maize” or “bt corn” or “bt soybean” or “bt tomato” or “bt eggplant” or “bt rice” or “bt potato”), we obtained 300 references (restricted by research domains: social sciences and art humanities). Importantly, most of those references were relevant.

In the case of non-English languages, the preliminary list of GM related terms was translated to Chinese, French, German, Spanish, and Portuguese, and each term was then tested in Google Scholar, which allows for the retrieval of a comprehensive number of references in each of the non-English languages considered. Before conducting the independent test runs per keyword and language, Google Scholar was set up for retrieving results in the corresponding language, and the search history personalization (customization of results based on previous search activity) was deactivated Searches were conducted in full-text. Terms which retrieved relevant studies among the top 10 percent of the records obtained per search term (ordered by relevance) were selected as search terms, and the remaining were dropped. AGROVOC was also searched for controlled terms in those languages, but additional relevant terms were not found. In addition, reviews of the socio-economic impacts of GM crops using literature in the languages selected in this protocol were not found; however, a visual evaluation of the reference list of some relevant articles in those languages suggested that the selected terms are adequate. The list of selected search terms in non-English languages is included in the Additional file 2.

Search terms related to the outcome and population

A comprehensive list of 380 terms related to the outcome and population was compiled in the English language and translated into the non-English languages considered in this protocol. Then, we selected the outcome and population terms that retrieved the largest number of relevant references, based on visual inspections from the results of IDEAS/REPEC for English searches. The descriptors per topic considered in this protocol (farmers, consumers, supply chain, coexistence, environmental economics, and food security) were also included as additional search terms. The selected outcome and population terms for searches in the English language are included in Table  1. Note that outcome and population terms will be joined by OR while conducting the actual searches. Thus, “environmental economics” is not directly included in the list as a population term since econ*, a more general term, is already in the list of search terms. Any potential results of searches for “environmental economics” will be already captured in the results of the searches for econ*.

The original list of the 380 terms and the corresponding translations in non-English languages are included in the Additional file 3.

Database searches

Database selection

Little evidence exists to guide prioritization of databases for reviewers [51]. Our criteria for database selection considered the following aspects:

• a) subject area (socio-economics);

• b) geographic coverage of the studies (e.g., databases covering developing countries); and

• c) inclusion of primary studies.

We gave preference to databases that allow for directly exporting batch results to the reference software, can retrieve full text documents, or provide links to access those studies. However, relevant databases in non-English languages do not provide those facilities (e.g., a number of the Spanish-language databases), and therefore we will not strictly restrict our searches to “user-friendly” databases.

We excluded databases that do not correspond to our subject areas (e.g., biochemistry); do not focus on primary studies (e.g., newspapers); are redundant (included in other databases or platforms; e.g., BIOSIS Previews and Current Contents Connect which are included under TUM’s subscription to Web of Science); require additional payment; or databases that are unavailable or inaccessible at the present time (e.g., databases undergoing major restructuring).

The list and a general description of the selected databases, platforms, and search engines for searches in the English language are provided in Table  2, while the selected databases for searches in non-English languages are included in the Additional file 4.

Table 2 Selected databases, platforms, and search engines for searches in English language

It is expected that the selected databases will offer good coverage of the literature available in each of the six languages included in this protocol. An overview of the content of all the selected databases, platforms, and search engines (in English and non-English) is provided in the Additional file 5.

Overall search procedure for databases

Our overall search strategy for searches in the English language considers the following:

• a) searches will be conducted on “title, abstract or keyword”, when this option is available. If the database does not offer this facility, searches will be conducted in the common default option “all fields”. Nevertheless, searches in Google Scholar, BASE, and Agricola will be conducted in “title” due to the large number of references obtainable;

• b) searches will be limited to the time period from 1996 (the year GM crops were commercially introduced to farmers) to present;

• c) searches will be filtered by type of document (article, chapter, book, thesis, manuscript, and conference paper) and socio-economic subjects or disciplines (if the database provides these facilities);

• d) searches will be conducted using only the intervention terms in socio-economic related databases (IDEAS/REPEC, British Library for Development, IFPRI, JOLIS, and OECD iLibrary) and in databases providing socio-economic filters (Web of Science, Scopus, ELDIS, Agricola, BASE, and Open Grey). There is no need to combine the intervention terms with the outcome and population terms in these databases. Without doing so, they provide a manageable number of results primarily related to the socio-economic issues of GM crops;

• e) intervention terms will be combined with the outcome and population terms in searches conducted in databases not related to socio-economic issues or without socio-economic filters (Google Scholar and AGRIS) in order to avoid obtaining a large number of irrelevant results.

The procedure for searches in non-English languages is described in the Additional file 6.

Estimating the comprehensiveness of the search

Specific search strategies must be constructed for each database indicated above. Some databases allow truncation, stemming, and searches with strings, while others only partially or do not allow doing so. When a database does not allow truncation or stemming, different words endings (or suffixes) need to be used for conducting the searches (for example, with Google Scholar). During our scoping exercise, we identified the total number of records obtained from searches in each of the databases included in our protocol. The results for the searches in English are reported in Table  3. The details of the searches per database in English, and the results of the searches for non-English languages are included in the Additional file 7.

Table 3 Number of references identified per database in English language

The results suggest that it is possible to follow the proposed search strategy. The total number of references (without eliminating duplicates) from all the databases in the English language is 38,781. Based on visual inspections, we roughly estimate that about one third will be duplicates. (Many searches will be conducted per keyword and then joined, which will generate many duplicates in the final list of references). In the end, we expect to manually screen about 26,000 references from searches in databases in the English language.

Validation of search strategies

Following Hausner et al., we will validate our search strategy by checking if the relevant references from the reviews and meta-analyses included in the Additional file 1 are among our included studies. In the event that the references were not retrieved, we will refine the search strategies until we are able to retrieve those references.

Searches in journals

No database is capable of exhaustively monitoring all existing journals. We retrieved a list of journals, which contained articles related to the socio-economic impacts of GM crops based on the results of a search conducted in IDEAS/REPEC using the search terms “genetically modified” and “transgenic” in 2012. It is expected that some of these journals are already indexed in one or more of the databases considered in this protocol, especially the high ranked journals. (For example, Scopus indexes the journals published by Elsevier and other selected journals based on their quality. See the full list of 33,635 journals and conferences at http://www.elsevier.com/online-tools/scopus/content-overview). We will check if the journals in our list mentioned above are already fully indexed in our selected databases, and those which are not indexed or partially indexed (e.g., no ‘cover to cover’ or some years missing) will be searched manually. The complete list of (93) journals (in English) is included in the Additional file 8.

Subject expert consultation

When eligible and appropriate, including completed yet unpublished studies in a systematic map helps to minimize bias [56]. Also requesting and obtaining lists of publications from experts could allow us to verify if we have retrieved all the relevant information and could help us to fill potential information gaps in our data collection. Finally, it is also important to identify ongoing studies/research related to the socio-economic impacts of GM crops worldwide. Therefore, we expect to conduct an online survey requesting experts for their published and unpublished studies; and for information about previous and current projects, which are related to socio-economic impacts of GM crops. The draft version of the questionnaire is included in the Additional file 9. The list of experts will be primarily compiled using lists of authors given in the included studies and information gained from worldwide economic organizations about authors in this field (see http://edirc.repec.org/alphabet.html).

Study inclusion criteria

Our inclusion criteria specify the types of populations, interventions, comparators, outcomes, and study designs, to be addressed in the systematic map. The identified studies will be screened against these criteria in order to be included in the systematic map. Our screening process will be conducted stepwise (see Figure  8). First, the studies will be screened against the inclusion criteria by title (and abstract when available). The studies which do not fulfill the inclusion criteria will be excluded. In case of doubt, the study will be retained for further evaluation. Second, we will review the abstract and full text of the articles, and in a similar way, the studies that do not fulfill the general inclusion criteria will be excluded. In case of doubt, the study will be retained for further analysis.

Figure 8

Procedure for screening of studies.

The general inclusion criteria are the following:

• Relevant Population: Global human civilizations and their economies

• Relevant Intervention: availability/adoption/commercialization of any type of crop (e.g., maize, soybean, cotton, canola) with any type of genetic modification (e.g., herbicide resistance, insect resistance)

• Relevant Comparator: situation before the availability/adoption/commercialization or without the intervention for a comparable group of population^h

• Relevant Outcome: economic quantification or social analysis of the effects of the intervention

• Relevant Study design: primary study (survey/interview, observational/ethnographic, model or experiment)

The exclusion criteria consider studies which are not related to GM crops (e.g., animals and microorganisms); do not include a comparator (which allow for an impact assessment); are in a language not considered in this protocol; are published before 1996; are not primary studies (e.g., newspapers, editorials, opinions, literature reviews); or are not accessible.

The included evidence will be coded using criteria for classifying the included studies. The elements of these criteria are related to the topics considered in this protocol and are summarized in Table  4 below.

Table 4 Criteria for classifying included studies

It is important to mention that we will make our best effort to retrieve the full text of all potential relevant studies after the first screening, given time and budget constraints. The maximum amount of time we are considering for finishing collecting all the data for the systematic map is six months, which includes the time we will dedicate to contacting the authors of missing references.ⁱ In case we are not able find the full texts of potential relevant references, these references will be excluded from the analysis but included in a list of potential relevant studies (with full text not available) in the final systematic map.

Our team currently composed of 10 reviewers will screen the studies identified during the systematic searches in different languages to exclude irrelevant titles. The reviewers will also perform a random screening of 10 percent of the studies from one of the other reviewers, and a Cohen’s Kappa coefficient will be calculated to measure the degree of inter-reviewer agreement. If the Kappa value is less than 0.5, the reviewers will examine their differences, and possible errors will be corrected to ensure a reliable screening procedure.

Data extraction strategy

Studies that pass the inclusion criteria will be imported into a database. Each study will be coded based on the following information:

• General information about the study (authors, year of publication, affiliation, donor)

• Type of publication (e.g., peer-review article, non peer-review manuscript/article, book, book chapters)

• Location of the study (e.g., region, state, country, locality)

• Description of the population (e.g., average age, gender, education)

• Type of crop and GM traits (e.g., Bt cotton)

• Type of evaluation method (e.g., propensity score matching, differences in differences)

• Other relevant qualitative information, especially when the study design is only qualitative (e.g., descriptions from ethnographic studies).

Data presentation

The data presentation will include descriptive statistics by type of socio-economic impact; population; geographical focus of the evidence (e.g., developed and developing countries); research methods; and changes on time (or time trends) when possible.

The final outcomes of this protocol will be a systematic map report on the socio-economic impacts of GM crops worldwide based on the evidence available in six languages, and a searchable database (including the list of references of the included studies, along with the information extracted from those studies; the list of excluded studies and reasons for exclusion; and the list of potentially relevant studies with full text not available). Also the review process will be documented and included in an open-acess database named “Central Access Database for Impact Assessment of Crop Genetic Improvement Technologies” (CADIMA, http://www.cadima.info/) which is currently under development by members of the GRACE project.

Endnotes

^aNevertheless, as indicated by a reviewer, systematic maps can also be used to map narrow questions, especially where the studies pertaining to a particular topic are unlikely to meet the criteria for quantitative synthesis (or meta-analysis).

^bSystematic maps and systematic reviews follow the same structured methodologies. However, systematic reviews, in contrast to systematic maps, include an evidence synthesis and are set out to critically appraise the evidence. Both systematic maps and systematic reviews are considered stand-alone pieces of review of the evidence. Nevertheless, systematic maps can also be undertaken as the first step before conducting systematic reviews, which would be only undertaken if there is sufficient quantity and quality of evidence on specific sub-topics (for example, the CASE project [57] conducted first a broad systematic map and then subsequent systematic reviews on particular sub-topics identified from the systematic map). TUM may conduct subsequent systematic reviews, which will depend on the amount of evidence found in particular sub-topics and time availability after finishing the systematic map. Given the case, TUM will elaborate additional independent protocols before conducting the systematic reviews. The total duration of the GRACE project is from 2012 to 2015.

^cAnother institution, the Center for European Policy Studies (CEPS), is in charge of reviewing the evidence at the “macro-level” (socio-economic impacts at the sectoral and macro level, trade impacts of GM crops, and politics of GM crops).

^dThe stakeholder consultation process is being facilitated by the working package/group “Stakeholder and user involvement” of the GRACE project (see GRACE [7]).

^eIn the context of the GRACE project, environmental economics is defined as the economic effects or consequences of current or potential environmental impacts.

^fThe other three top languages used for publication of research are Dutch, Italian, and Russian [9].

^gControlled terms are standardized subject terms used by a database to categorize articles based on the content. In contrast, free terms are natural language terms (i.e., terms included in the title of a document).

^hThe comparators are associated with “before-after” and “with-without” evaluations, which allows controlling for selectivity bias.

ⁱWe expect to finish the whole systematic map in a maximum of fifteen months.