Bt crops have raised environmental concerns over consequences for sustainability of soil biodiversity and ecosystems services in agricultural land. As Bt crops contain insecticidal proteins potential interactions with non-target organisms are of major concern for the risk assessment. In recent years, numerous field and laboratory studies have been conducted to assess the potential adverse effects of the Bt crops on different non- target organisms including microorganisms living in the soil. Soil microorganisms include primarily bacteria and fungi. They interact with plants and animals and play significant roles in nutrient cycling and energy flow in soils and are actively involved in chemical and biological processes. It is likely that any changes accruing in soil will affect soil microorganisms, and so their abundance/diversity and activities can be used as indicators for many aspects of soil quality, and thus for the assessment of the potential effects Bt crops on the environment.
The key review question is: Are abundances/diversity/activities of soil microorganisms changed by Bt crops compared with conventional crops? For this purpose relevant literature will be collected systematically through a comprehensive search strategy in a number of general, specialized and personal databases. The search terms will define the locality or type of laboratory/glasshouse study – the field, soil, rhizosphere, the populations - types of soil microorganisms, the exposure - types of Bt proteins, the assessed outcome -abundances/diversity/activity of microorganisms and the crop, will be used for retrieving of relevant studies. We present the criteria against which studies will be included in the review and how they will be assessed. These criteria include appropriate study designs and comparators. This protocol outlines the type of analyses that will be performed to assess bias of the selected studies and if co-variables describing the heterogeneity of the studies introduces biases. The synthesis will be performed as a quantitative synthesis combining the magnitude of potential effects from the individual studies. The synthesis will include assessments of heterogeneity related to the studied abundances/ diversities/activities, types of exposure, methodology, comparators and publication bias.
The debate over genetically modified (GM) plants is continuous in the European Union as a consequence of high public sensitivity and complexity of safety issues which they raise. Among the primary concerns are the effects on the environment associated with potential gene flow, weediness or invasiveness of GM plants and possible interactions with non-target organisms. Government regulators in many parts of the world require an assessment of environmental impacts prior to commercialization of GM crops. For that reason many studies analysing the interactions between GM plants and the environment have been carried out, which are of particular interest to professionals - risk assessors or regulators, as well as for many public and non-governmental organizations .
The following protocol describes the methodology for conducting a systematic review and evidence synthesis, which will be performed within the EU funded project GRACE (GMO Risk Assessment and Communication of Evidence). The project aims to explore the existing data about potential health, environmental, and socio-economic impacts of GM plants in a comprehensive, systematic and transparent manner. The primary goal of the project is to address issues of high public interest, therefore different stockholder groups were invited to take part in the formulation and prioritization of the questions to be reviewed and the review methodology as well. Through face-to-face consultation and online questionnaires several high priority questions about the environmental impact of GM plants were selected, among which is the one related to the interaction of GM crops producing toxins from the bacterium Bacillus thuringiensis (Bt) with non-target soil microorganisms.
The technology for genetic modification of plants provides a tool for crop breeding and has been applied for the development of varieties with novel or improved traits. Insect resistance has been achieved via introduction of Bt genes and is the second most distributed GM trait worldwide after the herbicide tolerance. The only GM plant which is cultivated commercially in the EU is maize designed to produce Bt toxin - Cry1Ab, which provides protection against corn borers (Lepidoptera). GM crops producing different types of Bt toxins against other pest insects or in combination with other GM traits are also in the industry pipeline [2, 3]. Because Bt crops contain insecticidal proteins potential interactions with non-target organisms are of major concern for the risk assessment . In recent years, numerous field and laboratory studies have been conducted to assess the potential adverse effects of the Bt crops on different non-target organisms, including microorganisms living in the soil. Soil microorganisms include primarily bacteria and fungi. They interact with plants and animals, play significant roles in nutrient cycling and energy flow in soils and are actively involved in chemical and biological processes. It is likely that any changes accruing in soil will affect soil microorganisms, and so their abundance/diversity and activities can be used as indicators for many aspects of soil quality .
There are two possible ways that the Bt crop can influence soil living organisms: firstly, directly through root depositions, as exudates, cells and mucilage and through litter that contains Cry toxin and/or through unintended changes in the plant due to the genetic modification; and secondly, indirectly through the changes in agricultural management practices related to the genetic modification, e.g. changed insecticide regimes. The major concern is related to soil microbial ecology and that any effects of Bt crop cultivation on soil microorganisms may affect soil ecosystem functioning.
Systematic review and meta-analysis have been applied routinely for synthesis of data from medical studies assessing the risk or benefits of treatments or drugs. However the potential to use this methodology for quantitative synthesis of data from impact assessment of GM plants has also been recognized . Meta-analysis of field and laboratory studies assessing the effects of Bt crops to non-target organisms have been performed already [7–11]. Most of them, however scrutinize exclusively non-target arthropods and their communities. Although comprehensive literature reviews exists [12–16] a quantitative synthesis of evidence, with a quality control as required for an SR, of the effects of Bt crops on soil microorganisms is still missing. It will produce firm evidence-based conclusions about impacts on biodiversity of soil living microorganisms and to impacts on the soil microbial ecology in general. The approach to the systematic review as detailed here is performed in compliance with guidelines of CEE .
Objective of the review
This systematic review aims to synthesise the empirical evidence on the potential effects of Bt crops to soil microorganisms and their activities.
The review question (RQ) asked by the present systematic review protocol is:
Are soil microbial endpoints changed by Bt crops compared with conventional crops?
The question has the following components;
Soil microorganisms, including bacteria and fungi, and their associated activities.
Genetically modified Bt crops and their concomitant farming practice through the soil environment and in the rhizosphere.
The near-isogenic line and their concomitant farming practice through the soil environment and in the rhizosphere.
Changes in abundances/diversity and activities of the subject populations contrasted with the comparator.
The systematic review methodology describes the approach which will be used to find and analyse original articles containing data from field experiments or glasshouse assessing the effects of the Bt crop cultivation to soil microorganisms.
The aim of the search is to find studies containing data primarily from field experiments assessing the effect of Bt crop cultivation to soil microorganisms. However glasshouse studies and studies performed in micro- or mesocosms with field collected soil will also be included. The main approach will be to conduct comprehensive electronic searches in web databases, search engines for scholarly literature and specialized databases. Additional source will be the personal data collected by experts and stakeholders from the project GRACE network. In addition, the reference lists of related review papers and datasets from previously conducted reviews will be checked manually for the presence of relevant studies.
Search terms defining the locality or type of laboratory/glasshouse study – the field, soil, rhizosphere , the populations - types of soil microorganisms, the exposure - types of Bt proteins, the assessed outcome -abundances/diversity/activity of microorganisms and the crop, will be used for retrieving of relevant studies. The search terms will be organised in strings which will be modified according to the requirements of each bibliographic database. The used search strings and the results will be documented and presented as an Additional file 1 of the systematic review.
The following search terms will be used:
Locality term - soil* OR field* OR rhizo OR meso* OR micro* or glass* or lab*
Population terms - microorg* OR microb* OR bacteri* OR fung* OR minera*
Exposure terms - Cry1* OR Cry2* OR Cry3* OR thuringiensis OR Bt OR endotoxin
Outcome terms - population* OR biomass OR communit* OR diversity OR respirat* OR enzym* OR decomposition OR biogeochem* OR nitrogen OR immobili* OR coloni* OR abundance OR richness
Plant terms - transgen* OR maize Or Corn OR Zea OR cotton OR rice OR soybean OR eggplant OR potato OR poplar
A search exercise to determine the scope and relevance of the search strings to be used in the systematic review was performed. Pilot searches were conducted using three platforms: Web of knowledge, Scopus and CAB abstracts. Prior to the pilot search, 19 studies which contained the necessary elements for inclusion in the systematic review were identified by simplified and focused search strings in the Web of knowledge platform. They were used as references to evaluate the relevance of the results from the pilot search. The content of the pilot search strings was modified by including or excluding terms until all 19 articles (if present in the database) were found among the hits and in the same time have not resulted in excessive amount of irrelevant studies. The search strings which produced comprehensive and the most relevant results in the three platforms are shown in Table 1.
Web databases containing scientific literature including theses, books, abstracts and articles will be searched using the defined strings. The following search service providers and bibliographic databases will be used:
Web of knowledge - search service including the following citation databases and platforms:
Web of Science® - platform which consist of nine databases containing scholarly journals, books, book series, reports, conferences, and more.
BIOSIS Citation IndexSM - comprehensive reference database for life science research which includes cited references to primary journal literature.
MEDLINE® - database of the U.S. National Library of Medicine (NLM) contains over 12 million records of journal articles in all areas of the life sciences.
CAB Direct - platform for access to all CABI database subscriptions.
Scopus - large abstract and citation database of peer-reviewed literature.
AGRIS - Information system for the agricultural sciences and technology
All searches will be performed in English; however no restriction for language or year of publication will be made. In that way all studies which have published an abstract in English will be within the scope of our search. Original full text in either English or German will be included in the analysis. The search results from each of the used sources will be saved and citations will be imported in EndNote citation manager software. All duplicates will be removed and a list containing the accumulated results will be created and uploaded to the open-access database CADIMA (Central Access Database for Impact Assessment of Crop Genetic Improvement Technologies).
Experts and stakeholders from the consultation network created within the GRACE project will be asked to provide data relevant to the topic. The complete list of articles received by personal communication and the source will be recorded and will be included in the appendix of the systematic review.
Literature datasets or databases from other reviews related to effects of Bt crops on soil microorganisms, as well as the reference lists of the articles found within the electronic search will be searched manually.
Study inclusion criteria
In order to be included the article needs to fulfil each of the following criteria:
Relevant organisms or activities: Soil microorganisms, including bacteria, fungi and microorganisms at other taxonomic levels and their activities, including respiration, decomposition, biogeochemical cycling, enzymatic activities, mineralisation, immobilisation, nitrogen turnover, root colonization etc.
Relevant exposure(s): Exposure to genetically modified Bt crops and their associated farming practice through the soil environment and the rhizosphere.
Relevant comparator(s) (if appropriate): Ideally the near-isogenic line, however less related varieties will be accepted. In field experimental design allowing for any of the following comparisons will be accepted:
Bt with non-Bt plots, neither of which received any additional insecticide treatments.
Unsprayed Bt plots with non-Bt plots that received insecticides.
Bt with non-Bt crops when both are subject to insecticide treatments.
Relevant outcomes: Changes in organism abundance, diversity or activity.
Relevant study design: The studies must ideally be conducted under field conditions designed according to any block or plot randomization method, however other designs might be accepted if the design is explained in details and makes statistical analysis valid. However, glasshouse studies and studies performed in micro- or mesocosms with field sampled soils will also be included, if they apply to the inclusion criteria.
Data: Quantitative data should be available. They include binary and categorical data.
Applying study inclusion criteria
The selection of studies which fulfil the inclusion criteria will be performed at three stages by two reviewers independently. At first, articles will be selected by their title to remove highly irrelevant studies from the overall search results, followed by the second stage in which the inclusion criteria will be applied against the abstracts of the articles. If there is doubt or lack of enough information for judgement of whether the article will pass to the next stage the full-text will be analysed to ensure it meets the inclusion criteria.
At the beginning of the abstract phase, after the first 100 publications are processed, tests for consistency between two reviewers will be made using Kappa statistics. If the Kappa value is less than 0.6 the inconsistencies will be discussed and the criteria for inclusion will be adjusted taking into account the main reasons for disagreement.
In the third stage, the articles will be reviewed in full text for the presence of each of the elements needed for the inclusion. The reasons for including or excluding of each study at that stage will be recorded.
Study quality assessment
As specified above a study must have
An appropriate comparator (e.g. isogenic or near isogenic line)
experimental procedures which address the microbial abundance/diversity/activity without bias
preferentially an experimental design involving replication and randomisation
The lack of any significant outcome between the Bt-crop and the comparator does not exclude any study, and is not a selection criterion.
Studies which fulfil the inclusion criteria will be assessed for possible bias of one reviewer and a second reviewer and studies that have been excluded will be reassessed by a second reviewer and a conclusion will be reached. The aim of this phase is to ensure that the studies are providing evidence which represent true statistical similarity and level of difference. The internal (design, conduct, and analyses) and external (abundance/diversity/activity, interventions, and outcomes) validity will be assessed by applying the five quality domains mentioned below and defined as low, high or uncertain and the results will be reported separately for each domain.
The assessment will be performed using check lists by one reviewer as a random (20%) subsample of the studies will be performed by second reviewer and the results will be compared with Kappa statistics. The inconsistencies (Kappa value less than 0.6) will be discussed and a third reviewer will be asked to judge in case no agreement could be reached. Records with the evaluation results and the reasons for judgment will be made for each article included at this stage.
Pre-treatment differences between the studied groups and in the baseline characteristics of the study will be assessed in the selection bias domain. The following elements will be assessed:
plot location - low risk if the experimental plots of both the intervention and the control treatment are located in one field or other experimental unit with known history. High risk if the experimental units are located at different fields and there is no nformation about the history of the field.
comparator - low risk if the comparison is Bt crop vs corresponding isogenic line. High risk if for the control treatment another variety is used.
randomisation - the randomisation is the best way to avoid selection bias thus studies which are designed by any block or plot randomization method will be considered as low risk. High risk studies will occur if there is no or poor randomization.
replications - low risk if there is at least three replicates per treatment. High risk if there is no replication.
Performance bias arises if the studied groups are influenced by factors different from the intervention, which may have an effect on the measured outcomes. In the field studies, such influences might come from:
plot size - low risk if the plot size is properly defined in relation to the actual measurements; high risk if plot size is very small.
field management - low risk if both control and intervention received the same agro-technical management including tillage, fertilizers, pesticides, irrigation, cultivation etc. High risk if there are differences in the management between the treatments.
The way the measurement of the outcomes is done can influence the true effect estimation if the selected method is not accurate or can be influenced by the human subjectivity.
– sampling of soil microorganisms - the experimental procedures are crucial when assessing the abundance/diversity and/or activity of microorganisms. When the experiments are performed using standardized techniques or other recognized methods the risk of bias will be considered low. If the technique is not suitable for the examined questions, or is prone to human influence the risk of bias, will be considered high.
Imbalance in the final set of selected studies will be assessed. In theory, an imbalance may occur if studies with the following properties are excluded:
sampling size - low risk if the sample size is equal between the treatments. High risk if it differs due to loss of samples.
missing data - low risk if the amount of data for the measured outcomes is equal for all the treatment. High risk if there is imbalance in the presented outcome data between the treatments.
Data extraction strategy
The aim of the data extraction stage is to retrieve information relevant to the design, performance and measured outcomes, which are intended to be used for the quantitative/qualitative synthesis and the analysis of the variability between studies. Details about the experimental sites and design will be extracted from the text in the sections describing the materials and methods of the study, as well as the description of other experimental procedures and statistical analysis. Numeric/binary/qualitative data for the measured outcomes and will be extracted from tables and figures in the result section of the study. The most direct original data are always selected; derived data are only selected if the datasets are restricted to these.
All data will be imported in standardised Excel tables. Each measurement of abundance/diversity/activity of the microorganisms as affected by the different treatment (e.g. Bt crop and comparator) will be included in the table as separate records containing all the defining variables. Data extraction will be performed by one review team member and checks will be performed by other review team members of a random 10% subset of the data. If necessary, discrepancies will be resolved by the involvement of a third reviewer.
Data to be extracted from the papers is presented in Table 2.
The extracted data will be used to synthesize the evidence provided by the individual studies about the effects of Bt crop cultivation to soil microorganisms, as well as to investigate the heterogeneity among the studies.
The data will include abundances as measured by different techniques and at different taxonomic levels, different measures for diversity and microbial activity as measured by different techniques. The included data will be quantitative, but will include categorical as well as binary data. Initially data of the same kind will analysed together, later data synthesis via meta-analysis will be performed. For this purpose data are likely to be transformed.
Assessment of statistical power of included studies
All data that meet the above requirements will be used in the analysis, irrespective of the statistical power of the individual study.
Measures of treatment effect
Depending on the studied measurements the abundance/diversity, activity can be measured per volume of soil (measured as volume or gram dry matter) or per plot. The reported values are usually the mean for the treatment and the calculation of the associated variance.
Dealing with missing data
If the measurements of either the treated plots or the control plots are missing then the study is not included in the analysis. In cases when for reporting the variance other values than standard deviation or standard error is used, e.g. t, F, p or z-values, an appropriate mathematical method will be used to calculate the pooled standard deviation, if appropriate. If this is not feasible, authors will be contacted to provide the missing data.
Quantitative synthesis will be performed to combine the magnitude of the effects from the individual studies. The analysis will include calculation of the pooled effect size for each study accompanied with the corresponding confidence intervals. The results from the analysis will be presented graphically in ‘forest plots’, where the estimated effect size with the confidence interval of each individual study will be plotted horizontally as the combined effect size and confidence interval will be plotted below them.
The complete dataset will be stored in the GRACE CADIMA database for open access after the finalisation of the review. For mixed effects modelling SAS PROC MIXED, PROC GLIMMIX or R ver. 3  will be used.
A range of effect modifiers will be extracted from the selected papers and stored in the database (Table 2). These effect modifiers include comparator properties, pesticide treatment, insect pressure, experimental design, cropping system, crop rotation, tillage date, date of GM experimental cropping system establishment and soil type.
Assessment of heterogeneity
The heterogeneity across the studies that may influence the outcome will be assessed. In field studies, estimating the effects of Bt crop cultivation to soil microorganisms several sources of contextual heterogeneity may be expected:
Heterogeneity in studied abundances/diversities/activities
The examined measurements will vary among the individual studies. They will even differ in whether abundances, diversity or activity are measured and all kinds of combinations will exist. The techniques used will also vary among the studies.
Heterogeneity in type of exposure
A source of heterogeneity related to the type of exposure will be the variability of the GM plant species and the type of Bt toxin, i.e. the CRY event, which they produce. Among the most widely studied Bt crops are maize producing Cry1Ab (against lepidopteran pests) and Cry3Bb1 (against coleopteran pests), followed by Cry1Ac producing cotton, also protecting against lepidopteran pests.
Heterogeneity in methodology
The main source of variability between the studies related with the methodology could be expected from differences in the experiment duration and time of sampling. Some authors report on short-term studies, others on one or two years study with one sampling at the end of the season, while others report multi-year studies with several sampling occasions in the beginning, in the middle and in the end of the season. Another source of variability might be that the effect of different Bt-cultivars differs. Considering the possible cumulative effect of the Bt-toxin in the soils, the measurements carried out in several subsequent years will be compared separately if corresponding data is found. In addition we will examine cumulative data sets in order to distinguish between cumulative effects of Bt-toxins and the cumulative effectseffect of the cultivation of Bt crops.
Heterogeneity in comparisons
The experimental design of the studies can include one or combination of the following comparisons: Bt with non-Bt plots, neither of which received any insecticide treatments; unsprayed Bt-plots with non-Bt plots that received insecticides and Bt with non-Bt fields when both are subject to insecticide treatments.
To deal with the above mentioned contextual heterogeneity sub-group analysis will be performed if data suitable to calculate the effect size is found in minimum three distinct publications, which contain the same Bt crop producing and type of Cry toxin and the same kind of measurements. Analysis on the finest possible level will be made.
Other sources of heterogeneity which may have effect on the outcomes of the studies could come as a result of differences in the methodology such as plot size, sample size, experimental procedure and field management, or to be caused by the different factors of the environment or the experimental setup.
Statistical analysis of heterogeneity
Heterogeneity will be addressed in conventional analyses of correlation, regression and mixed modelling ANOVA to reveal if the heterogeneities have any significant impact on the Bt crop effect estimates. E.g. if the studies can be grouped into soil type categories holding sufficient number of replicates, hypotheses about the effect of soil type on the outcome of effects can be elucidated. This will be one type of sensitivity analysis. In principle all the effects modifiers can be included in sensitivity analyses, i.e. do their inclusion or exclusion affect the assessment.
The degree to which the different sources of heterogeneity influence the results will be estimated.
The following factors will be considered:
weight of the individual study;
Assessment of publication bias
In the medical research it is known that “positive” results are more likely to be published . Publication bias will be assessed using “funnel plot” analysis, where the effect size of an individual study will be plotted on the horizontal axis and the standard error or sample size on the vertical axis. The asymmetry in the funnel plot may indicate publication bias .
The literature searches and data extraction will be conducted by teams at ABI and Aarhus University jointly, while the team at Aarhus University will be the main responsible for data analysis and synthesis. Furthermore, “Sweet Environmental Consultants” will provide internal quality assurance of the review process.
List of studies used for the relevance check in the search exercise.
Title: Field Evaluation of Arbuscular Mycorrhizal Fungal Colonization in Bacillus thuringiensis Toxin-Expressing (Bt) and Non-Bt Maize
Author(s): Cheeke, Tanya E.; Cruzan, Mitchell B.; Rosenstiel, Todd N.
Title: Effect of Cry1Ab Protein on Rhizobacterial Communities of Bt-Maize over a Four-Year Cultivation Period
Author(s): Barriuso, Jorge; Valverde, Jose R.; Mellado, Rafael P.Source: PLOS ONE Volume: 7 Issue: 4 Article Number: e35481 DOI: 10.1371/journal.pone.0035481 Published: APR 30 2012
Title: Effect of vegetation of transgenic Bt rice lines and their straw amendment on soil enzymes, respiration, functional diversity and community structure of soil microorganisms under field conditions
Lavelle P, Decäens T, Aubert M, Barot S, Blouin M, Bureau F, Margerie P, Mora P, Rossi JP: Soil invertebrates and ecosystem services.ICSZ Soil Animals and Ecosystems Services Proceedings of the XIVth International Colloquium on Soil Biology 2006,42(1):S3-S15.
CEE: Collaboration for Environmental Evidence: Guidelines for Systematic Review and Evidence Synthesis in Environmental Management. Version 4.2. Environmental Evidence: .Book Collaboration for Environmental Evidence. Guidelines for Systematic Review and Evidence Synthesis in Environmental Management. Version 4.2. Environmental Evidence 2013. (Editor ed.^eds.). City http://www.environmentalevidence.org/Documents/Guidelines/Guidelines4.2.pdf
This work is funded within EU-FP7 project GRACE, Grant Agreement KBBE-2011-6-311957. We are grateful for the support and methodological guidance provided by the management and members of GRACE consortium. Michael Urbanczyk’s help with the Word-processor was very useful.
None of the members of the review team has any financial interest in the outcome of the systematic review nor are they affiliated with any religion or organisation that have expressed a certain position to GMO.
KK, PHK, JBS, CFD, NBH prepared this review protocol. NBH made the data extraction of the pilot literature search. PHK and JBS provided study quality criteria and GRACE project management. All authors read and approved the final manuscript.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Kostov, K., Krogh, P.H., Damgaard, C.F. et al. Are soil microbial endpoints changed by Bt crops compared with conventional crops? A systematic review protocol.
Environ Evid3, 11 (2014). https://doi.org/10.1186/2047-2382-3-11