Genetically modified (GM) crops have generated a great deal of controversy. Since commercially introduced to farmers in 1996, the global area cultivated with GM crops has increased 94-fold. The rapid adoption of GM technology has had substantial socio-economic impacts which a vast amount of technical and non-technical literature has addressed in the last two decades. However, contradictory results between individual studies abound. Extensive and transparent reviews concerning this contentious and complex issue could help promote evidence-based dialogue among the diverse parties involved.
Methods
This protocol specifies the methodology for identifying, evaluating, and mapping evidence related to the main review question: what are the socio-economic impacts of genetically modified crops worldwide? This question has been subdivided into the following topics: (a) farm-level impacts; (b) impacts of coexistence regulations; (c) impacts along the supply chain; (d) consumer-level impacts; (e) impacts on food security; and (f) environmental economic impacts. The search strategy includes the identification of primary studies from general scientific databases; global, regional, and national specialist databases; an on-line search engine; institutional websites; journal websites; subject experts/researchers; and serendipity. Searches will be conducted in six languages (Chinese, English, French, German, Portuguese, and Spanish). Identified studies will be screened for inclusion/exclusion criteria by a group of multi-language reviewers. Finally, pre-defined data from the studies will be extracted, mapped, and presented in a report. Potential research gaps will be identified and discussed, and the review process will be documented in an open-access database (i.e. CADIMA, http://www.cadima.info/).
Genetically modified (GM) crops have generated a great deal of controversy. The use of biotechnology in agriculture has caused major ideological and scientific concerns that continue to be echoed in the media and academic press
[1]. Since commercially introduced to farmers in 1996, the global area cultivated with GM crops has increased 94-fold, from 1.7 million hectares to 160 million hectares in 2011
[2]. The rapid adoption of this technology has had substantial socio-economic impacts
[3]. Consequently, a vast amount of technical and non-technical literature addressing this topic has accumulated over the last two decades
[4]. Moreover, groups of stakeholders characteristically advocate opposing opinions, which may not be based on best available evidence. Therefore, the availability of transparent and reliable reviews of studies on the socio-economic impacts of GM crops could help promote evidence-based dialogue among the diverse parties involved. Systematic maps employ structured procedures that can be particularly useful for minimizing potential biases that may arise during the process of identification, selection, and analysis of evidence involved in controversial topics. Systematic maps provide an opportunity to gather and describe evidence relevant to a broad field of policy and management relevancea. The breadth of the evidence captured in a systematic map helps to clearly identify potential research gaps and guide future research efforts
[5]. In addition, systematic maps make relevant evidence readily accessible to researchers and stakeholders through the development of extensive databases, the content of which can be relatively easily updated as needed.
Currently, numerous literature reviews and meta-analysis studies have assessed the socio-economic impacts of GM crops (a non-comprehensive list of 20 studies is included in the Additional file
1). Nevertheless, none is a systematic map, and only one is a systematic review (see Hall et al.
[6])b. That systematic review focused on the costs and profits of GM agriculture in comparison with conventional agriculture. One shortcoming of the document, as stated by the authors, was the exclusion of studies conducted before 2006, which disregards valuable earlier literature. The authors also clarified that
[6]: “Additional time for conducting a systematic review such as this one would allow the inclusion in the search process of additional databases that were excluded because it was not possible to directly export results to Reference Manager Database. An extended review on this topic would be a potentially valuable contribution to the ‘GM debate’”.
Through the EU project “GMO Risk Assessment and Communication of Evidence” (GRACE, 2012–2015), comprehensive reviews of existing evidence of potential health, environmental, and socio-economic impacts of GM crops worldwide will be conducted
[7]. As members of GRACE, the authors of this protocol (Technische Universitaet Muenchen, TUM) will undertake a systematic map on the socio-economic impacts of genetically modified (GM) crops. In particular, the Description of Work (DoW) for GRACE states that TUM is responsible for carrying out reviews on the following key topics: (1) farm-level economic impacts of GM crops; (2) economics of coexistence; (3) economics of segregation at the level of supply-chains; and (4) consumer acceptance of GM crops.c GRACE is following a participatory approach, and stakeholders are being consulted during each of the project’s steps. The stakeholders include members of industry and civil society organizations, as well as competent authorities on GM crops in the EU Member States and scientific experts from academiad. Two new topics were added based on stakeholder requests: environmental economic impactse of GM crops and the impacts of GM crops on food security (for more information about the participatory process, see GRACE
[8]). Therefore, TUM will produce a systematic map covering the six topics stated above, the overall conceptual model of which is outlined in Figure
1. The extensive systematic map will address the broad review question: what are the socio-economic impacts of genetically modified crops worldwide?
Figure 1
Overall conceptual model of the topics to be covered in the systematic map.
The systematic map undertaken will provide an important overview of the existing literature related to the socio-economic impacts of GM crops available in six languages (Chinese, English, French, German, Spanish, and Portuguese). These languages are among the top nine used for publication of researchf
[9] and also the primary languages spoken in 23 of the 28 countries currently cultivating GM crops
[10].
The description of the topics to be covered in the systematic map is provided below.
Farm-level impacts
Farmers have different socio-economic motivations for adopting GM crops. Significant socio-economic determinants include: gender associated aspects (e.g.,
[11]); individual and social learning (e.g.,
[12]); educational level (e.g.,
[13]); and expected benefits and uncertainty (e.g.,
[14–16]). For GM adopters, potential changes in yield and economic returns depend on current and previous crops and specific trait characteristics; agricultural practices; incidence of pest infestation; seed costs; and market characteristics (e.g.,
[17, 18]). Farmers’ production efficiency (farmers’ ability to produce more with less than or equal inputs/resources) would also be affected (e.g.,
[19]), as well as the frequency of pesticide poisoning incidents and health impacts (e.g.,
[20]). Consumption of new bio-fortified GM crops are expected to increase farmers’ nutrition status and as such, they could significantly contribute to farmers’ well-being (e.g.,
[21]). Most of the world’s poor depend mainly on farming for their subsistence. The adoption of GM crops could have different impacts on wealthier and poorer farmers (e.g.,
[22]), which could exacerbate/mitigate social problems. Ethical aspects may also be affected, as it has been demonstrated that ethical values can change over time (e.g., changing views on euthanasia in the U.S. and Japan
[23]). A change in acceptability of GM crops may imply a change in adopters’ values. Finally, cultural aspects may be impacted as well; for example, GM seeds need to be purchased, causing a disturbance in the traditional exchange of seeds among indigenous farmers (along with potential changes in identity and trust among involved farmers).The main aspects considered within this topic are presented graphically in a conceptual model (Figure
2). This conceptual model shows that socio-economic factors influence farmer decisions regarding the adoption of GM crops. GM adoption is expected to impact aspects related to farmers’ income and also intangible aspects. The potential income-related impacts include changes in the use of inputs; associated costs; output (quantity and quality); and gross income. Some farmers could experience changes in time available for conducting off-farm income-generating activities. A farm’s efficiency could deteriorate or improve with use of new technologies impacting the farmer’s income. Intangible aspects that may be affected after GM adoption relate to health safety issues associated with changes in pesticide use and farmers’ nutritional status if they cultivate and consume bio-fortified crops. Primary social, ethical, and cultural aspects are also depicted in the conceptual model.
Figure 2
Conceptual model of socio-economic impacts on farmers.
Coexistence related impacts
The possibility that GM farms contaminate non-GM farms via unintentional or inadvertent gene flow constitutes a challenge for the coexistence of GM farming and conventional agriculture, including organic certified agricultural systems. Several studies have analysed the effects that the introduction of ex-ante regulatory and ex-post liability aspects would have on farm-level costs and GM spatial configuration and adoption dynamics (e.g.,
[24–26]). In addition, potential benefits due to higher price premiums for non-GM products have also been evaluated (e.g.,
[27]).The main aspects considered within this topic are presented graphically in a conceptual model (Figure
3). This conceptual model shows that GM plants and crops can be introduced under alternative coexistence systems (separation between GM and non-GM farms and dual GM/non-GM farms) and regulatory frameworks, including ex-ante (e.g., mandatory segregation, traceability, minimum GM tolerance levels, rigid and flexible refuge areas, and voluntary GM-free zones) and ex-post liability aspects (e.g., compensation funds, insurance schemes, and marketplace liability). The different coexistence options are expected to influence in different manners GM and non-GM farm-level costs, particularly operational; transaction; opportunity; and testing and remediation costs. GM adoption dynamics could change as well, such as the rate of adoption, spatial configuration, and speed and stability of GM expansion. GM-farmers would also generate externalities and directly influence the economic benefits of non-GM farmers due to inadvertent gene flow from GM to non-GM fields which may create problems for non-GM farmers willing to sell their products in specific markets (e.g., organic certified markets). Finally, social factors, such as the level of trust between neighbors, would influence farm-level costs (e.g., lower/higher negotiation costs) and adoption dynamics of GM crops (e.g., stronger/lower imitation or neighboring effects) in each of the ex-ante and ex-post regulatory regimes under evaluation (social aspects not pictured in the figure).
Figure 3
Conceptual model of socio-economic impacts of coexistence.
Supply chain impacts
The focus of this section is on the supply chain or organization network as unit of analysis. It aims to analyse the socio-economic impacts of the commercialization of GM crops on supply chain structure and performance dynamics, as well as cost and benefit distribution along different actors in the supply chain.
In general, the basic elements of the structure of the supply chain include:
(a)
Vertical relations. These refer to the sequence of value adding activities. Actors performing different functions within the supply chain are vertically linked through buying and selling relationships. Vertical relations highlight the level of cooperation, coordination, trust, and governance (or power) along the chain.
(b)
Horizontal relations. These reflect the relationships among actors performing the same function within the chain. Horizontal relations can be formal (e.g., cooperatives and associations) or informal.
The main factors related to supply chain performance are:
(a)
Efficiency or the ability to deliver value at a minimum of total costs.
(b)
Effectiveness or the ability of the chain to provide superior value.
(c)
Innovation or the ability to respond to changes in consumer demand or the external environment.
Several studies have analysed the effect that the commercialization of GM crops would have on the supply chain structure, as well as the distribution of costs and benefits of different actors along the supply chain (e.g.,
[28–32]). Moreover, governance mechanisms and market power of different actors would also be affected (e.g.,
[33, 34]). The main aspects considered under this topic are presented graphically in a conceptual model (Figure
4). This conceptual model shows that the commercialization of GM products under different enforced coexistence rules, labeling schemes, and protection of intellectual property rights would have impacts on the supply chain structure (e.g., vertical and horizontal relations) and performance (e.g., efficiency, effectiveness, and innovation ability). This in turn would affect the distribution of costs and benefits for the different actors along the supply chain, as well as their market power (ability to influence the price of a commercialized item).
Figure 4
Conceptual model of socio-economic impacts along the supply chain.
Consumer-level impacts
The socio-economic determinants for consumers’ acceptance of GM food and the associated price premiums for non-GM products have been evaluated under different mandatory and voluntary GM-related label schemes (e.g.,
[35–37]). Other studies have evaluated the option values of a moratorium or ban on GM products (e.g.,
[38]). Those price premiums and option values have been used to calculate economic welfare effects (e.g.,
[39]). These and other main aspects related to the impacts of GM products on consumers are presented graphically in a conceptual model (Figure
5). The conceptual model shows that GM products can be introduced into the market under mandatory and voluntary GM-related labels, including different tolerance levels (or percentage of GM ingredients in the final products) or can be subject to moratorium or ban. The decision or intention to buy those products is based on consumers’ socio-economic characteristics (e.g., age, gender, and educational level). Potential buyers can indicate their willingness to pay (WTP) for these products, and changes in social welfare can be calculated based on the differences between the WTP and actual or expected prices (price premiums). If there is a moratorium or ban on GM products, option values can be calculated based on a (hypothetical) WTP to preserve or maintain this situation. Social welfare can be estimated by the difference between the WTP and the opportunity costs of forgoing economic growth associated with the commercialization of GM products. GM products can have an impact on consumers’ health, for example in the case of bio-fortified food. Social, ethical, and cultural aspects were added as requested by stakeholders.
Figure 5
Conceptual model of socio-economic impacts at consumer level.
Environmental economic impacts of GM crops
GM crops may substitute for agricultural inputs and practices that are environmentally harmful. The study by Brookes & Barfoot
[40] suggest that “since 1996 the use of pesticides (counted as active ingredients) on the GM crop area was reduced by 448 million kg (9% reduction), and the environmental impact quotient — an indicator measuring the environmental impact associated with herbicide and insecticide use on these crops — fell by 17.9%. In 2010, the total carbon dioxide emission savings associated with GM crop adoption were equal to the removal from the roads of 8.6 million cars due to reduced fuel use and additional soil carbon sequestration”.
GM crops can cause environmental harm as well (although there is considerable uncertainty and no consensus among scientists)
[41]. In particular, the protection of biodiversity and ecosystem services ought to be a top priority when taking into consideration the dependency on a healthy environment of all human activity, now and in the future
[42]. For those opposed to GM technology, GM crops are exotic species being introduced into open complex ecosystems of which we have limited understanding
[43], and as such it is impossible to anticipate all impacts of GM technology on the environment.
The effects of GM crop adoption on the environment will depend not only on human behavior but on biological, ecological, and chemical interactions as well. Many disciplines are needed to evaluate these kinds of impacts
[41]. In addition, there is the possibility of irreversible ecosystem disruptions due in part to the unpredictable and novel effects of gene mixing
[43].
Figure
6 shows a basic conceptual model of the potential environmental economic impacts of GM crops (based on information obtained from
[40–46]). Depending on the type of genetic modification, the cultivation of GM crops can change the type or quantity of herbicide/insecticide used, improve the crops’ resistance to external climate stress (e.g., drought and salinization), or cause an undesired gene flow (e.g., from GM crops to wild relatives).
Figure 6
Conceptual model of environmental economic impacts of GM crops.
Changes in the type or quantity of herbicide/insecticide could create or alter herbicide resistance in weeds or pesticide resistance in pests. Soil, water, and air contamination is reduced if the substituted herbicide/pesticide was more toxic than the new herbicide/pesticide. Further, if less herbicide/pesticide is required, resources like fuel could be saved. Changes in herbicide/insecticide use could also modify agricultural practices, such as encourage tillage, weed management, or monoculture. New alternative agricultural practices could change the use of resources and fuel consumption, which in turn would have impacts on soil, water, and air contamination and soil organisms and biodiversity. In addition, there could be improvements in crop yields using existing land and water resources, which in turn could reduce land use; water and air contamination; minimize the impacts on biodiversity; and save resources and fuel consumption. In a similar manner, the cultivation of drought- and salinity-tolerant GM crops would also impact soil, water, air, biodiversity, and modify the use of resources and fuel consumption. Finally, there could be a gene flow from GM crops to wild relatives with unknown consequences to the environment.
It is worth mentioning that this protocol contemplates the environmental economic impacts of GM crops. Therefore, only primary studies incorporating an economic assessment of these and similar environmental impacts will be considered. The environmental impact assessment component of the included primary studies will be taken as given.
Food security at household level
The estimated number of undernourished people has continued to decrease, but the rate of progress still appears insufficient to reach international goals for hunger reduction
[47]. Currently, about 842 million people (one in eight people in the world) suffer chronic hunger, unable to obtain the amount of food necessary to conduct an active life
[47]. The vast majority of hungry people live in developing countries, where the prevalence of undernourishment is estimated at 14 percent
[47].
Food security exists when all people, at all times, have physical and economic access to sufficient, safe, and nutritious food that meets dietary needs and food preferences for an active and healthy life
[47]. There are four dimensions of food security: food availability (e.g., food production and processing); food access (e.g., having the economic resources to buy the right food); food utilization (e.g., education to individuals to make proper use of healthy food); and food system stability (e.g., adequate access to food at all times). For food security objectives to be realised, all four dimensions must be fulfilled simultaneously
[47, 48].
Therefore, food security is a multidimensional concept, and data on all dimensions are rarely available and frequently unreliable
[49]. Moreover, the international community lacks a consensus on core household food security indicators needed in order to properly measure and monitor food security worldwide. The indicators also vary on level of analysis, ranging from the regional or national level to the household or individual level, depending on data availability and the design of the instruments used to collect the data (e.g., surveys)
[49].
In relation to GM crops, reports from expert governmental and nongovernmental bodies increasingly include GM crops as part of a wider approach to food security
[50]. GM crops could help to mitigate expected food shortages related to population growth and the effects of climate change in specific regions worldwide. For example, GM crops could impact food availability by providing seeds which are resistant to adverse climate conditions; have an effect on food access by increasing farmers’ incomes; and, under the same food utilization conditions, bio-fortified crops could increase the nutritional status of households worldwide. (Figure
7 illustrates this example).
Figure 7
Conceptual model of food security at household level.
In the approach followed in this protocol, the ultimate goal of food security is to improve the nutritional status of households. It is worth mentioning that several of the multidimensional aspects of food security have been already covered by other topics in this protocol (e.g., impacts of GM crops on farm-level income). Nevertheless, there are a growing number of socio-economic studies which specifically evaluate the impacts of GM crops on (at least one component of) food security and explicitly indicate that as so.
Objective of the systematic map
The main objective of the systematic map is to identify the breadth of knowledge related to the socio-economic impacts of GM crops worldwide. Our question related to the overall objective of the systematic map is:
What research evidence exists (number of studies and the current state of research studies) on the socio-economic impacts of GM crops worldwide (in Chinese, English, French, German, Spanish, and Portuguese languages)?
Relating to the secondary objectives, the systematic map will identify the types of socio-economic impacts; populations; crops and GM traits; geographical focus; research methodologies; evidence gaps; and the particular topics that could be subjects of further analyses or subsequent systematic reviews. Therefore, the questions related to the secondary objectives of the systematic map are:
a)
What types of socio-economic impacts have been addressed?
b)
What types of populations have been addressed?
c)
What types of crops and GM traits have been addressed?
d)
What is the geographical focus of the evidence?
e)
What research methods have been used to collect and analyse the evidence?
f)
What evidence gaps exist that could/ should be addressed in future primary research?
g)
Which particular topics could be the subject of further analysis (e.g., meta-analyses and meta-regressions) or subsequent systematic reviews?
Methods
Search strategy
Systematic maps require an objective and reproducible search of a range of sources to identify as many relevant studies as possible (within resource and time limits). A search strategy that includes extensive search terms and a combination of multiple data sources can increase the likelihood of capturing most of the relevant references. Our search terms consider a list of intervention-, outcome- and population-related keywords in six languages. Our data sources include: general scientific databases; an on-line search engine (Google Scholar); global, regional, and national specialist databases; institutional websites (to be accessed through one of the largest institutional repository search engines: Bielefeld Academic Search Engine, BASE); journal websites; subject experts/researchers; and serendipity (e.g., finding relevant documents by accidental discovery or by chance). The reference management software to be used for exporting/importing the references is Citavi, which is freely available at TUM.
We aim to identify as many of the available relevant studies as possible (based on time and budget constraints). Sensitivity will be favored over specificity. Sensitivity implies that the emphasis of the search procedure will be in obtaining most of the relevant articles at the risk of obtaining a high number of non-relevant ones (which would need to be depurated later during the screening stage). On the other hand, specificity emphasizes the retrieval of relevant articles with the lowest number of non-relevant ones as possible (at the risk of omitting some/ many of the relevant articles).
Search terms
Search terms related to the intervention
The selected search terms related to the intervention (GM crops) in the English language are presented in Table
1. These search terms were derived from a preliminary list of 29 GM crop related terms compiled by experts from the GRACE project (see Additional file
2). IDEAS/REPEC, the largest freely-available bibliographic database dedicated to economics, was used to test this preliminary list. Search terms which did not retrieve relevant references (e.g., cisgenesis) or which retrieved similar references as other search terms (e.g., glufosinate tolerant did not retrieve additional relevant references in comparison to herbicide tolerant) were dropped from the list. The searches were conducted on title, abstract, and keywords. Then AGROVOC, the corporate thesaurus of the Food and Agriculture Organization (FAO), was searched for controlled termsg which were also included as search terms (e.g., biosafety, biosecurity). Finally the titles from the reference lists of the reviews and meta-analysis included in the Additional file
1 were visually examined to evaluate the completeness of our selected search terms, and new search terms were added when needed (e.g., Bollgard, drought resistant).
Table 1
List of search terms in English language
Intervention (GM crops)
Outcome and population
1) BT
1) Econ*
bt crop*
2) Socio*
bt seed*
3) Social*
bt cotton
4) Cost*
bt maize
5) Benefit*
bt corn
6) Regulat*
bt soybean
7) Farmer*
bt tomato
8) Consumer*
bt eggplant
9) Supply chain
bt rice
10) Coexist*
bt potato
11) Food security
2) Insect* resistan*
insect* resistan* crop*
insect* resistan* seed*
insect* resistan* cotton
insect* resistan* maize
insect* resistan* corn
insect* resistan* soybean
insect* resistan* tomato
insect* resistan* eggplant
insect* resistan* rice
insect* resistan* potato
3) Biotech*
Agricult* biotech*
Biotech* crop*
Biotech* food*
Biotech* seed*
4) Bioeng*
Bioeng* crop*
Bioeng* food*
Bioeng* seed*
5) GM
GM* crop*
GM* food*
GM* seed*
GM* free
GM* label*
GM* product*
6) GMO*
7) Transgenic*
Transgenic* crop*
Transgenic* food*
Transgenic* seed*
8) Genetic* engineer*
Genetic* engineer* agricult*
Genetic* engineer* crop*
Genetic* engineer* food*
Genetic* engineer* seed*
9) Genetic* modif*
Genetic* modif* agricult*
Genetic* modif* crop*
Genetic* modif* food*
Genetic* modif* seed*
10) HT
HT crop*
HT seed*
HT alfalfa
HT canola
HT rapeseed
HT chicory
HT cotton
HT flax*
HT maize
HT corn
HT soybean
HT sugar beet
HT rice
HT potato
HT wheat
11) Herbicide* resistan*
herbicide* resistan* crop*
herbicide* resistan* seed*
herbicide* resistan* alfalfa
herbicide* resistan* canola
herbicide* resistan* rapeseed
herbicide* resistan* chicory
herbicide* resistan* cotton
herbicide* resistan* flax*
herbicide* resistan* maize
herbicide* resistan* corn
herbicide* resistan* soybean
herbicide* resistan* sugar beet
herbicide* resistan* rice
herbicide* resistan* potato
herbicide* resistan* wheat
12) Virus resistan*
virus resistan* crop*
virus resistan* seed*
virus resistan* bean
virus resistan* papaya
virus resistan* squash
virus resistan* sweet pepper
virus resistan* plum
virus resistan* potato
13) Drought resistan*
drought resistan* crop*
drought resistan* seed*
drought resistan* corn
drought resistan* maize
drought resistan* sugarcane
14) Biofortif*
Biofortif* crop*
Biofortif* food*
Biofortif* seed*
Biofortif* cassava
Biofortif* corn
Biofortif* maize
Biofortif* rice
Biofortif* sorghum
15) Biosafe*
Biosafe* agricult*
Biosafe* crop*
Biosafe* food*
Biosafe* seed*
16) Biosecur*
Biosecur* agricult*
Biosecur* crop*
Biosecur* food*
Biosecur* seed*
17) Roundup ready*
18) Liberty link*
19) Starlink*
20) Bollgard*
21) Golden rice
*Indicates a truncation/wild card symbol (i.e. any character(s) permitted).
Additional searches using the selected terms were conducted in Web of Science (All databases) in Topic (Title, abstract, keywords). Some of the terms retrieved high number of records (e.g., bt retrieves 78,027 references; and 3,310 when restricted by research domains: social sciences and arts humanities), most of them were not related to GM crops. Therefore the final intervention terms include a descriptor (e.g., bt crop*) or the type of crop (e.g., bt cotton). The types of crops were compiled from the list of approved crops by GM trait reported in the “GM approval database” by the International Service for the Acquisition of Agri-biotech Applications (ISAAA, http://www.isaaa.org/gmapprovaldatabase/default.asp). In this way, we obtained a reasonable number of references. For example, for bt as a group (“bt crop*” or “bt seed*” or “bt cotton” or “bt maize” or “bt corn” or “bt soybean” or “bt tomato” or “bt eggplant” or “bt rice” or “bt potato”), we obtained 300 references (restricted by research domains: social sciences and art humanities). Importantly, most of those references were relevant.
In the case of non-English languages, the preliminary list of GM related terms was translated to Chinese, French, German, Spanish, and Portuguese, and each term was then tested in Google Scholar, which allows for the retrieval of a comprehensive number of references in each of the non-English languages considered. Before conducting the independent test runs per keyword and language, Google Scholar was set up for retrieving results in the corresponding language, and the search history personalization (customization of results based on previous search activity) was deactivated Searches were conducted in full-text. Terms which retrieved relevant studies among the top 10 percent of the records obtained per search term (ordered by relevance) were selected as search terms, and the remaining were dropped. AGROVOC was also searched for controlled terms in those languages, but additional relevant terms were not found. In addition, reviews of the socio-economic impacts of GM crops using literature in the languages selected in this protocol were not found; however, a visual evaluation of the reference list of some relevant articles in those languages suggested that the selected terms are adequate. The list of selected search terms in non-English languages is included in the Additional file
2.
Search terms related to the outcome and population
A comprehensive list of 380 terms related to the outcome and population was compiled in the English language and translated into the non-English languages considered in this protocol. Then, we selected the outcome and population terms that retrieved the largest number of relevant references, based on visual inspections from the results of IDEAS/REPEC for English searches. The descriptors per topic considered in this protocol (farmers, consumers, supply chain, coexistence, environmental economics, and food security) were also included as additional search terms. The selected outcome and population terms for searches in the English language are included in Table
1. Note that outcome and population terms will be joined by OR while conducting the actual searches. Thus, “environmental economics” is not directly included in the list as a population term since econ*, a more general term, is already in the list of search terms. Any potential results of searches for “environmental economics” will be already captured in the results of the searches for econ*.
The original list of the 380 terms and the corresponding translations in non-English languages are included in the Additional file
3.
Database searches
Database selection
Little evidence exists to guide prioritization of databases for reviewers
[51]. Our criteria for database selection considered the following aspects:
a)
subject area (socio-economics);
b)
geographic coverage of the studies (e.g., databases covering developing countries); and
c)
inclusion of primary studies.
We gave preference to databases that allow for directly exporting batch results to the reference software, can retrieve full text documents, or provide links to access those studies. However, relevant databases in non-English languages do not provide those facilities (e.g., a number of the Spanish-language databases), and therefore we will not strictly restrict our searches to “user-friendly” databases.
We excluded databases that do not correspond to our subject areas (e.g., biochemistry); do not focus on primary studies (e.g., newspapers); are redundant (included in other databases or platforms; e.g., BIOSIS Previews and Current Contents Connect which are included under TUM’s subscription to Web of Science); require additional payment; or databases that are unavailable or inaccessible at the present time (e.g., databases undergoing major restructuring).
The list and a general description of the selected databases, platforms, and search engines for searches in the English language are provided in Table
2, while the selected databases for searches in non-English languages are included in the Additional file
4.
Table 2
Selected databases, platforms, and search engines for searches in English language
General scientific databases and platforms
Web of Science (WoS) (Thomson Reuters) (includes Web of Science Core Collection, BIOSIS Citation Index, BIOSIS Previews, Current Contents Connect, Derwent Innovations Index, Inspec, MEDLINE, and SciELO Citation Index)
Scopus (Elsevier)(a)
On-line search engine
Google Scholar (GS)(b)
Organizations with focus on developing countries
British Library for Development Studies (includes African and Indian journals)
ELDIS (information service related with international development issues)
International organizations
AGRIS (maintained by the Food and Agriculture Organization, FAO)
IFPRI (International Food Policy Research Institute)
JOLIS (World Bank, International Monetary Fund, IMF & International Finance Corporation, IFC)
OECD iLibrary (Organization for Economic Co-operation)
Other organizations/institutional repositories
AGRICOLA (US National Agricultural Library)
IDEAS/ REPEC (Largest bibliographic database dedicated to economics freely available. It contains bibliographic information from other open source databases such as AgEcon)
Bielefeld Academic Search Engine (BASE)(c)
Grey literature
Open Grey (system for information on grey literature in Europe)
(a)Scopus is a multidisciplinary database, which along with WoS, is considered the most complete and widely used for scientific information identification and retrieval
[52].
(b)Google scholar retrieves peer-review and non peer-review publications (grey literature). According to Gehanno et al.
[53], one of the advantages of using Google scholar is that it identifies more types of literature compared to a general scientific database. The results of a study conducted by the same authors suggest that the current coverage of Google scholar allows retrieving all the high quality studies identified by other general scientific databases such as WoS, and “could be the first choice for systematic reviews or meta-analysis”
[53]. On the negative side, Google scholar is “constantly-changing content, algorithms and database structure” and Google does not provide details about Google scholar’s database coverage
[54]. The results of the searches will be ordered by relevance and the first 1000 documents will be imported to Citavi. The reason for this is that Google scholar limits the retrieval of search results to 1000 documents for any particular search query.
(c)BASE is one of the largest institutional repository search engines
[55], which allows access to 2762 content sources, such as the National Library of Australia, Institutional Repository of PhD theses from Katholieke Univ. Leuven (Belgium), EMBRAPA (Brazil), University of Saskatchewan (Canada), Peking University Institutional Repository (China), among others. The full list of sources is available at:
It is expected that the selected databases will offer good coverage of the literature available in each of the six languages included in this protocol. An overview of the content of all the selected databases, platforms, and search engines (in English and non-English) is provided in the Additional file
5.
Overall search procedure for databases
Our overall search strategy for searches in the English language considers the following:
a)
searches will be conducted on “title, abstract or keyword”, when this option is available. If the database does not offer this facility, searches will be conducted in the common default option “all fields”. Nevertheless, searches in Google Scholar, BASE, and Agricola will be conducted in “title” due to the large number of references obtainable;
b)
searches will be limited to the time period from 1996 (the year GM crops were commercially introduced to farmers) to present;
c)
searches will be filtered by type of document (article, chapter, book, thesis, manuscript, and conference paper) and socio-economic subjects or disciplines (if the database provides these facilities);
d)
searches will be conducted using only the intervention terms in socio-economic related databases (IDEAS/REPEC, British Library for Development, IFPRI, JOLIS, and OECD iLibrary) and in databases providing socio-economic filters (Web of Science, Scopus, ELDIS, Agricola, BASE, and Open Grey). There is no need to combine the intervention terms with the outcome and population terms in these databases. Without doing so, they provide a manageable number of results primarily related to the socio-economic issues of GM crops;
e)
intervention terms will be combined with the outcome and population terms in searches conducted in databases not related to socio-economic issues or without socio-economic filters (Google Scholar and AGRIS) in order to avoid obtaining a large number of irrelevant results.
The procedure for searches in non-English languages is described in the Additional file
6.
Estimating the comprehensiveness of the search
Specific search strategies must be constructed for each database indicated above. Some databases allow truncation, stemming, and searches with strings, while others only partially or do not allow doing so. When a database does not allow truncation or stemming, different words endings (or suffixes) need to be used for conducting the searches (for example, with Google Scholar). During our scoping exercise, we identified the total number of records obtained from searches in each of the databases included in our protocol. The results for the searches in English are reported in Table
3. The details of the searches per database in English, and the results of the searches for non-English languages are included in the Additional file
7.
Table 3
Number of references identified per database in English language
Database
Number of references (with duplicates)
Web of Science
4,110
Scopus
3,434
Google Scholar
8,763
British Library for Development Studies
850
ELDIS
1,551
AGRIS
3,720
IFPRI
2,125
JOLIS
440
OECD iLibrary
339
AGRICOLA
261
IDEAS/REPEC
4,932
BASE
8,050
Open Grey
206
TOTAL
38,781
Details of the searches can be found in the Additional file
7.
The results suggest that it is possible to follow the proposed search strategy. The total number of references (without eliminating duplicates) from all the databases in the English language is 38,781. Based on visual inspections, we roughly estimate that about one third will be duplicates. (Many searches will be conducted per keyword and then joined, which will generate many duplicates in the final list of references). In the end, we expect to manually screen about 26,000 references from searches in databases in the English language.
Validation of search strategies
Following Hausner et al., we will validate our search strategy by checking if the relevant references from the reviews and meta-analyses included in the Additional file
1 are among our included studies. In the event that the references were not retrieved, we will refine the search strategies until we are able to retrieve those references.
Searches in journals
No database is capable of exhaustively monitoring all existing journals. We retrieved a list of journals, which contained articles related to the socio-economic impacts of GM crops based on the results of a search conducted in IDEAS/REPEC using the search terms “genetically modified” and “transgenic” in 2012. It is expected that some of these journals are already indexed in one or more of the databases considered in this protocol, especially the high ranked journals. (For example, Scopus indexes the journals published by Elsevier and other selected journals based on their quality. See the full list of 33,635 journals and conferences at http://www.elsevier.com/online-tools/scopus/content-overview). We will check if the journals in our list mentioned above are already fully indexed in our selected databases, and those which are not indexed or partially indexed (e.g., no ‘cover to cover’ or some years missing) will be searched manually. The complete list of (93) journals (in English) is included in the Additional file
8.
Subject expert consultation
When eligible and appropriate, including completed yet unpublished studies in a systematic map helps to minimize bias
[56]. Also requesting and obtaining lists of publications from experts could allow us to verify if we have retrieved all the relevant information and could help us to fill potential information gaps in our data collection. Finally, it is also important to identify ongoing studies/research related to the socio-economic impacts of GM crops worldwide. Therefore, we expect to conduct an online survey requesting experts for their published and unpublished studies; and for information about previous and current projects, which are related to socio-economic impacts of GM crops. The draft version of the questionnaire is included in the Additional file
9. The list of experts will be primarily compiled using lists of authors given in the included studies and information gained from worldwide economic organizations about authors in this field (see http://edirc.repec.org/alphabet.html).
Study inclusion criteria
Our inclusion criteria specify the types of populations, interventions, comparators, outcomes, and study designs, to be addressed in the systematic map. The identified studies will be screened against these criteria in order to be included in the systematic map. Our screening process will be conducted stepwise (see Figure
8). First, the studies will be screened against the inclusion criteria by title (and abstract when available). The studies which do not fulfill the inclusion criteria will be excluded. In case of doubt, the study will be retained for further evaluation. Second, we will review the abstract and full text of the articles, and in a similar way, the studies that do not fulfill the general inclusion criteria will be excluded. In case of doubt, the study will be retained for further analysis.
Figure 8
Procedure for screening of studies.
The general inclusion criteria are the following:
Relevant Population: Global human civilizations and their economies
Relevant Intervention: availability/adoption/commercialization of any type of crop (e.g., maize, soybean, cotton, canola) with any type of genetic modification (e.g., herbicide resistance, insect resistance)
Relevant Comparator: situation before the availability/adoption/commercialization or without the intervention for a comparable group of populationh
Relevant Outcome: economic quantification or social analysis of the effects of the intervention
Relevant Study design: primary study (survey/interview, observational/ethnographic, model or experiment)
The exclusion criteria consider studies which are not related to GM crops (e.g., animals and microorganisms); do not include a comparator (which allow for an impact assessment); are in a language not considered in this protocol; are published before 1996; are not primary studies (e.g., newspapers, editorials, opinions, literature reviews); or are not accessible.
The included evidence will be coded using criteria for classifying the included studies. The elements of these criteria are related to the topics considered in this protocol and are summarized in Table
4 below.
Table 4
Criteria for classifying included studies
Group
Characteristics
Farm-level
The study mainly focuses on the impacts of GM crops at farm-level without considering co-existence issues (see below).
Co-existence
The study mainly focuses on the impacts of co-existence regulations
Supply chain
The study mainly focuses on the impacts of GM crops on/ along the supply chain
Consumer level
The study mainly focuses on the impacts of GM crops on consumers.
Environmental economics
The study mainly focuses on the impacts on environmental economics aspects, including economic quantification of:
a) agro-biodiversity conservation
b) land use changes
c) climate change mitigation and production of renewable energy
d) others
Food security
The study mainly focuses on the impacts on food security, including:
a) physical availability of food
b) economic and physical access to food
c) food utilization
d) stability of the other three dimensions over time
Mixed topics
The study indistinctly focuses on two or more topics indicated above
It is important to mention that we will make our best effort to retrieve the full text of all potential relevant studies after the first screening, given time and budget constraints. The maximum amount of time we are considering for finishing collecting all the data for the systematic map is six months, which includes the time we will dedicate to contacting the authors of missing references.i In case we are not able find the full texts of potential relevant references, these references will be excluded from the analysis but included in a list of potential relevant studies (with full text not available) in the final systematic map.
Our team currently composed of 10 reviewers will screen the studies identified during the systematic searches in different languages to exclude irrelevant titles. The reviewers will also perform a random screening of 10 percent of the studies from one of the other reviewers, and a Cohen’s Kappa coefficient will be calculated to measure the degree of inter-reviewer agreement. If the Kappa value is less than 0.5, the reviewers will examine their differences, and possible errors will be corrected to ensure a reliable screening procedure.
Data extraction strategy
Studies that pass the inclusion criteria will be imported into a database. Each study will be coded based on the following information:
General information about the study (authors, year of publication, affiliation, donor)
Type of publication (e.g., peer-review article, non peer-review manuscript/article, book, book chapters)
Location of the study (e.g., region, state, country, locality)
Description of the population (e.g., average age, gender, education)
Type of crop and GM traits (e.g., Bt cotton)
Type of evaluation method (e.g., propensity score matching, differences in differences)
Other relevant qualitative information, especially when the study design is only qualitative (e.g., descriptions from ethnographic studies).
Data presentation
The data presentation will include descriptive statistics by type of socio-economic impact; population; geographical focus of the evidence (e.g., developed and developing countries); research methods; and changes on time (or time trends) when possible.
The final outcomes of this protocol will be a systematic map report on the socio-economic impacts of GM crops worldwide based on the evidence available in six languages, and a searchable database (including the list of references of the included studies, along with the information extracted from those studies; the list of excluded studies and reasons for exclusion; and the list of potentially relevant studies with full text not available). Also the review process will be documented and included in an open-acess database named “Central Access Database for Impact Assessment of Crop Genetic Improvement Technologies” (CADIMA, http://www.cadima.info/) which is currently under development by members of the GRACE project.
Endnotes
aNevertheless, as indicated by a reviewer, systematic maps can also be used to map narrow questions, especially where the studies pertaining to a particular topic are unlikely to meet the criteria for quantitative synthesis (or meta-analysis).
bSystematic maps and systematic reviews follow the same structured methodologies. However, systematic reviews, in contrast to systematic maps, include an evidence synthesis and are set out to critically appraise the evidence. Both systematic maps and systematic reviews are considered stand-alone pieces of review of the evidence. Nevertheless, systematic maps can also be undertaken as the first step before conducting systematic reviews, which would be only undertaken if there is sufficient quantity and quality of evidence on specific sub-topics (for example, the CASE project
[57] conducted first a broad systematic map and then subsequent systematic reviews on particular sub-topics identified from the systematic map). TUM may conduct subsequent systematic reviews, which will depend on the amount of evidence found in particular sub-topics and time availability after finishing the systematic map. Given the case, TUM will elaborate additional independent protocols before conducting the systematic reviews. The total duration of the GRACE project is from 2012 to 2015.
cAnother institution, the Center for European Policy Studies (CEPS), is in charge of reviewing the evidence at the “macro-level” (socio-economic impacts at the sectoral and macro level, trade impacts of GM crops, and politics of GM crops).
dThe stakeholder consultation process is being facilitated by the working package/group “Stakeholder and user involvement” of the GRACE project (see GRACE
[7]).
eIn the context of the GRACE project, environmental economics is defined as the economic effects or consequences of current or potential environmental impacts.
fThe other three top languages used for publication of research are Dutch, Italian, and Russian
[9].
gControlled terms are standardized subject terms used by a database to categorize articles based on the content. In contrast, free terms are natural language terms (i.e., terms included in the title of a document).
hThe comparators are associated with “before-after” and “with-without” evaluations, which allows controlling for selectivity bias.
iWe expect to finish the whole systematic map in a maximum of fifteen months.
Declarations
Acknowledgements
Jiao Xu, Camilo Lopez, and Oliver Etzel provided support with the searches in different databases and languages. Roi Duran provided inputs related to the translation of the search terms in Portuguese. H.Al-Asadi translated the search terms to French, selected the language-specific databases and conducted searches in this language (no longer member of the GRACE team). The authors are grateful to the six reviewers from CEE who provided useful comments and suggestions to this protocol.
Sources of support
The research is funded by the FP7 project GRACE.
Authors’ Affiliations
(1)
Technische Universitaet Muenchen, Chair of Agricultural and Food Economics
(2)
Program on Sustainable Resource Management
(3)
Library of the Technical University of Munich, Branch Weihenstephan
(4)
Chair of Agricultural Economics and Rural Policy, Wageningen University
References
Smale M, Zambrano P, Gruere G, Falck-Zepeda J, Matuschke I, Horna D, Nagarajan L, Yerramareddy I, Jones H: Measuring the Economic Impacts of Transgenic Crops in Developing Agriculture during the First Decade. Approaches, Findings and Future Directions. Washington: USA: Food Policy Review 10. IFPRI; 2009.Google Scholar
Chen H, Lin Y: Promise and issues of genetically modified crops.Curr Opin Plant Biol 2013, 16: 1–6. 10.1016/j.pbi.2013.01.002View ArticleGoogle Scholar
National Research Council Committee on the Impact of Biotechnology on Farm-Level Economics and Sustainability. Board on Agriculture and Natural Resources. In The Impact of Genetically Engineered Crops on farm Sustainability in the United States. USA: The National Academic Press; 2010.Google Scholar
Smale M, Zambrano P, Falck-Zepeda J, Gruere G: Parables: Applied Economic Literature about the Impact of Genetically Engineered Crop Varieties in Developing Countries. Washington: USA: EPT Discussion Paper 158. IFPRI; 2006.Google Scholar
Bragge P, Clavisi O, Turner T, Tavender E, Collie A, Gruen R: The global evidence mapping initiative: scoping research in broad topic areas.Med Res Methodol 2011, 11: 1–12. 10.1186/1471-2288-11-1View ArticleGoogle Scholar
GRACE: GMO risk assessment and communication of evidence - GRACE. (Description of Work - Annex I, Part B). FP7 Collaborative Project. 2012.Google Scholar
GRACE: GRACE Stakeholder Consultation on Good Review Practice in GMO Impact Assessment. Part 2: Stakeholder priorities for review questions- Review questions on socioeconomic impacts. 2013.Google Scholar
Zambrano P, Maldonado J, Mendoza S, Ruiz L, Fonseca L, Cardona I: Women Cotton Farmers. Their Perceptions and Experience with Transgenic Varieties. A Case Study for Colombia. Washington, USA: International Food Policy Research Institute. Discussion Paper 01118; 2011.Google Scholar
Yoo D: Individual and Social Learning in Bio-Technology Adoption: The Case of GM Corn in the U.S. Seattle, USA: Paper presented at Agricultural and Applied Economics Association Annual Meeting; 2012.Google Scholar
Uematsu H, Mishra A: Net Effect of Education on Technology Adoption by U.S. Farmers. Orlando, USA: Paper presented at the Southern Agricultural Economics Association Annual Meeting; 2010.Google Scholar
Jaramillo P, Useche P, Barhan B, Foltz J: The State Contingent Approach to Farmers’ Valuation and Adoption of New Biotech Crops: Nitrogen-Fertilizer Saving and Drought Tolerance Traits. Denver, Colorado: Paper presented at the Agricultural and Applied Economics Association Annual Meeting; 2010.Google Scholar
Gaurav S, Mishra S: To Bt or not to Bt? Risk and Uncertainty Considerations in Technology Assessment. Mumbai, India: Working paper from Indira Gandhi Institute of Development; 2012.Google Scholar
Birol E, Villalba E, Smale M: Farmer preferences for millpa diversity and genetically modified maize in Mexico: a latent class approach.Environ Dev 2009, 14: 521–540. 10.1017/S1355770X08004944View ArticleGoogle Scholar
Krishna V, Qaim V: Bt cotton and sustainability of pesticide reductions in India.Agric Syst 2012, 107: 47–55.View ArticleGoogle Scholar
Mutuc M, Rejesus R, Pan S, Yorobe J: Impact assessment of Bt corn adoption in the Philippines.J Agric Appl Econ 2012, 44: 117–135.Google Scholar
Huang J, Hu R, Rozelle S, Qiao F, Pray C: Transgenic varieties and productivity of smallholder cotton farmers in China.Aust J Agric Resour Econ 2002, 46: 367–387. 10.1111/1467-8489.00184View ArticleGoogle Scholar
Bennett R, Morse S, Ismael Y: The economic impact of genetically modified cotton on South African smallholders: yield, profit and health effects.J Dev Stud 2006, 42: 662–677. 10.1080/00220380600682215View ArticleGoogle Scholar
Anderson K, Jackson L, Nielsen C: Genetically Modified Rice Adoption: Implications for Welfare and Poverty Alleviation. World Bank Policy Research. Working Paper 3380; 2004.View ArticleGoogle Scholar
Lipton M: Plant breeding and poverty: can transgenic seeds replicate the ‘Green Revolution’ as a source of gains for the poor?J Dev Stud 2007, 43: 31–62. 10.1080/00220380601055510View ArticleGoogle Scholar
Davis A, Mitoh T: Dying in the USA and Japan: selected legal and ethical issues.Int Nurs Rev 1999,46(5):135–139. 10.1046/j.1466-7657.46.no.5issue347.10.xView ArticleGoogle Scholar
Groeneveld R, Wesseler J, Berentsen P: Dominos in the dairy: an analysis of transgenic maize in Dutch dairy farming.Ecol Econ 2013, 86: 107–116.View ArticleGoogle Scholar
Consmüller N, Beckmann V, Petrick M: Identifying driving factors for the establishment of cooperative GMO-free zones in Germany. Foz do Iguacu, Brazil: Paper presented at Conference of the International Association of Agricultural Economists; 2012.Google Scholar
Gray E, Ancev T, Drynan R: Coexistence of GM and non-GM crops with endogenously determined separation.Ecol Econ 2011, 70: 2486. 10.1016/j.ecolecon.2011.08.005View ArticleGoogle Scholar
Wilson W, Dahl B: Costs and risks of testing and segregating genetically modified wheat.Rev Agric Econ 2005, 27: 212–228. 10.1111/j.1467-9353.2005.00222.xView ArticleGoogle Scholar
Coleno F, Angevin F, Lecroart B: A model to evaluate the consequences of GM and non-GM segregation scenarios on GM crop placement in the landscape and crosspollination risk management.Agric Syst 2009, 101: 49–56. 10.1016/j.agsy.2009.03.002View ArticleGoogle Scholar
Moss C, Schmitz T, Schmitz A: The brave new world: imperfect information, segregation costs, and genetically modified organisms.Agrarwirtschaft 2004, 53: 303–308.Google Scholar
Gryson N, Eeckhout T, Neijens T: Cost and Benefits for the Segregation of GM and Non-GM Compound Feed. Gent, Belgium: Paper presented at XII EAAE Congress; 2008.Google Scholar
Kalaitzandonakes N, Matsbarger R, Barnes J: Global identity preservation costs in agricultural supply chains.Can J Agric Econ 2001, 49: 605–615. 10.1111/j.1744-7976.2001.tb00330.xView ArticleGoogle Scholar
Murphy J, Yanacopulos H: Understanding governance and networks: EU-US interactions and the regulation of genetically modified organisms.Geoforum 2005, 36: 593–606. 10.1016/j.geoforum.2004.10.001View ArticleGoogle Scholar
Smyth S, Phillips P: Competitors co-operating; establishing a supply chain to manage genetically modified canola.Int Food Agribus Man 2001, 4: 51–66. 10.1016/S1096-7508(01)00070-2View ArticleGoogle Scholar
Aerni P, Scholderer J, Ermen D: How Would Swiss Consumers Decide if they had Freedom of Choice? Evidence from a Field Study with Organic, Conventional and GM Corn Bread, 36. Washington: USA: Food Policy Review 10. IFPRI; 2011:830–838.Google Scholar
Costa-Font M, Tranter R, Gil J, Jones P, Gylling M: Do Defaults Matter? Willingness to Pay to Avoid GM Food vis-à-vis Organic and Conventional Food in Denmark, Great Britain and Spain. Edinburgh, Scotland: Paper presented at the 84th Conference of Agricultural Economics; 2010.Google Scholar
Kikulke E, Birol E, Wesseler J, Falck-Zepeda J: A latent class approach to investigating demand for genetically modified banana in Uganda.Agric Econ 2011, 42: 547–560. 10.1111/j.1574-0862.2010.00529.xView ArticleGoogle Scholar
Donaghy P, Rolfe J, Bennett J: Quasi-Option Values for Enhanced Information Regarding Genetically Modified Foods. Melbourne, Australia: Paper presented at the 48th Australian Agricultural and Resource Economics Society; 2004.Google Scholar
Lusk J, House L, Valli C, Jaeger S, Moore M, Morrow B, Traill B: Consumer welfare effects of introducing and labeling genetically modified foods.Econ Lett 2005, 88: 382–388. 10.1016/j.econlet.2005.03.009View ArticleGoogle Scholar
Brookes G, Barfoot P: GM Crops: Global Socio-Economic and Environmental Impacts 1996–2011. UK: PG Economics Ltd; 2013.Google Scholar
Ando A, Khanna M: Environmental costs and benefits of genetically modified crops. Implications for regulatory strategies.Am Behav Sci 2000, 44: 435–463. 10.1177/00027640021956279View ArticleGoogle Scholar
Raven P: Does the use of transgenic plants diminish or promote biodiversity?New Biotechnol 2010,27(5):528–533. 10.1016/j.nbt.2010.07.018View ArticleGoogle Scholar
Batie S: The environmental impacts of genetically modified plants: challenges to decision making.Am J Agric Econ 2003, 85: 1107–1111. 10.1111/j.0092-5853.2003.00516.xView ArticleGoogle Scholar
Knox O, Hall C, McVittie A, Walker R, Knight B: A systematic review of the environmental impacts of GM crop cultivation as reported from 2006 to 2011.Food Nutr Sci 2013, 4: 28–44.View ArticleGoogle Scholar
Mannion A, Morse S: Biotechnology in agriculture: agronomic and environmental considerations and reflections based on 15 years of GM crops.Prog Phys Geogr 2012, 36: 747–763. 10.1177/0309133312457109View ArticleGoogle Scholar
Lemaux P: Genetically engineered plants and foods: a scientist’s analysis of the issues (Part II).Annu Rev Plant Biol 2009, 60: 511–559. 10.1146/annurev.arplant.043008.092013View ArticleGoogle Scholar
FAO: The State of Food Insecurity in the World. The Multiple Dimensions of Food Security. Rome: FAO; 2013.Google Scholar
Ruane J, Sonnino A: Agricultural biotechnologies in developing countries and their possible contribution to food security.J Biotechnol 2011, 156: 356–363. 10.1016/j.jbiotec.2011.06.013View ArticleGoogle Scholar
Carletto C, Zezza A, Banerjee R: Towards better measurement of household food security: harmonizing indicators and the role of household surveys.Global Food Security 2013, 2: 30–40. 10.1016/j.gfs.2012.11.006View ArticleGoogle Scholar
Beyer F, Wright K: Can we prioritise which databases to search? A case study using a systematic review of frozen shoulder management.Health Inf Libr J 2012, 30: 49–58.View ArticleGoogle Scholar
Delgado E, Repiso R: The impact of scientific journals of communication: comparing Google Scholar metrics, Web of Science and Scopus.Comunicar 2013, 41: 45–52.Google Scholar
Gehanno J, Rollin L, Darmoni S: Is the coverage of google scholar enough to be used alone for systematic reviews.BMC Med Informat Decis Making 2013, 13: 1–5. 10.1186/1472-6947-13-1View ArticleGoogle Scholar
Giustini D, Boulus M: Google Scholar is not enough to be used alone for systematic reviews.Online J Public Health Inform 2013, 5: 1–9.View ArticleGoogle Scholar
Hammerstrom K, Wade A, Jorgensen A: Searching for Studies: A Guide to Information Retrieval for Campbell Systematic Reviews. UK: The Campbell Collaboration; 2010.Google Scholar
Tripney J, Newman M, Bird K, Thomas J, Kalra N, Bangpan M, Vigurs C: Understanding the drivers, impact and value of engagement in culture and sport: Technical report for the systematic review and database. UK: The Case Programme; 2010.Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
