- Systematic Review Protocol
- Open Access
What are the effects of the cultivation of GM herbicide tolerant crops on botanical diversity? A systematic review protocol
© Sweet and Kostov; licensee BioMed Central Ltd. 2014
Received: 3 February 2014
Accepted: 7 April 2014
Published: 24 April 2014
There are concerns that the cultivation of genetically modified herbicide tolerant (GMHT) crops treated with broad spectrum herbicides will cause declines in botanical diversity and hence loss of biodiversity. Cultivation systems of these have different levels of inputs and management interventions and some incorporate the use of minimal/no tillage. Research results show a range of effects and the priority is to determine whether research studies show shifts in botanical diversity and/or declines in plant populations in GMHT compared with conventionally managed crops.
We will perform a rigorous review of studies of plant populations in fields and field margins of GMHT crops by complying with CEE requirements for Systematic Reviews (SR) and the EFSA Guidance on Systematic Reviewing. A Review Protocol (RP) is presented for the SR of data from field studies of GMHT crops, comparing the effects of GM crop, herbicide regimes and associated management applied to HT crops with conventional crops and their weed management for impacts on plant populations in fields and field margins as assessment end points or indicators of impacts on botanical diversity and associated food chain and ecosystem services effects. The literature search will include all the main GMHT crops including maize, soya, oilseed rape, sugar beet, cotton and rice. The keywords will be broad and the search strategy is developed to capture all literature relevant to the primary objective of the review. The range of data bases for the searches is described and all articles discovered in the searches will be collated by Endnote. The criteria against which studies will be included in the review and how they will be assessed are described. They include appropriate study designs, statistical power and comparators. The RP outlines the type of analyses that will be performed to assess bias of the selected studies and if covariables describing the heterogeneity of the studies introduce biases. Publications meeting the selection criteria will be filed separately and subjected to more detailed analysis and data on different plant species and types will be analysed separately, to determine outcomes.
Impact of GM Herbicide Tolerant (GMHT) crops
HT crops are widely grown in N and S America and are being developed for many other regions, including Europe. Since use of herbicides has been associated with declines in farmland biodiversity in some regions, there are concerns that GMHT crops treated with broad spectrum herbicides will also cause declines in biodiversity [1–3]. However HT systems have different levels of inputs and management interventions and some incorporate the use of minimal or no tillage. Research results show a range of effects depending on crop type, the specific herbicide tolerance, ai applied, dose, timing and number of herbicide treatments, tillage system and cropping system [4–12]. Consequently it is not clear how GMHT crop management is affecting botanical diversity or affecting diversity of other biota, either through food chain effects or directly, in agricultural land. In order to determine GMHT cropping effects on farmland biodiversity a starting point is to determine effects on botanical diversity since this will indicate likely food chain effects . There are also concerns that repeated use of the herbicides used on HT crops will promote herbicide resistance development  and further inappropriate use of herbicides leading to reductions in biodiversity .
In agroecosystems, sustainable agricultural production, integrated crop production and integrated pest management are broad protection goals associated with sustainable food production, which embrace the exploitation of a range of ecosystem services which are considered desirable to protect. These include pollination, predation, nutrient cycling, etc.… These protection goals are relevant in relation to herbicide usage in conventional and GMHT crops.
The main concern is that the cultivation of GMHT crops will reduce biodiversity and adversely affect ecosystem services in farmland regions . Conceptual models and logic maps for the most significant environmental issues associated with GMHT crops were distributed to stakeholders. At the stakeholder workshop on 23–24.4.2013 presentations were made explaining the conceptual models and the rationale for the systematic review of GMHT crops. Stakeholders (Appendix listed in Table 1) were invited to comment on our proposed approach and the review topics and questions. In addition comments were received by email. The questions were modified in response to these comments and clarifications provided for some aspects of our approaches to the topics. The updated review questions were returned to stakeholders for prioritisation by them.
At the Stakeholder meeting of 23 & 24 April 2013 and subsequent consultation with stakeholders it was agreed that this topic was a high priority, raised public concern and was scientifically controversial because of the mixed results from field studies and cropping experience in different crops in different regions. The EC is currently considering applications to commercialise GMHT crops in Europe and this study will inform them of the current state of knowledge on this topic. The SR questions on GMHT crop effects were scored above 3.5 (out of 5) on average with the highest scores for scientific disagreement. The SR will initially focus on the question receiving the higher score for scientific importance, with the question on weed resistance to herbicides used in GMHT systems considered according to time availability.
Objective of the review
Several narrative reviews of GMHT crops have been conducted and it is considered that a systematic review (SR) would bring some added-value in comparison with a narrative review. The SR would determine whether research studies and field data from commercialised GMHT crops show shifts in botanical diversity or declines in plant populations in GMHT compared with conventionally managed crops.
Primary review question
Are populations of plant species changed by management regimes applied to GM HT crops compared with conventional crop management?
The SR will review data from field studies of GMHT crops and compare effects of GM crop, herbicide regimes and associated management applied to HT crops and conventional crops for impacts on plant species populations in fields, as assessment end points or indicators of impacts on botanical diversity and associated food chain and ecosystem services effects.
Arising from this primary question is the secondary review question of whether there is evidence that weeds are developing resistance to broad spectrum herbicides as a consequence GMHT crop management.
We will perform a rigorous review of studies of plant populations in fields and field margins of GMHT crops by complying with CEE requirements for Systematic Reviews (SR) and the EFSA Guidance on Systematic Reviewing .
The aim of the literature search is to be as comprehensive as possible in revealing all available scientific and technical publications and reports which compare the environmental effects of GMHT and conventionally managed crops, with a specific focus on the effects on the botanical diversity and weed populations in farmland receiving the two comparable managements. The search will be indiscriminate and non-selective and include all the main GMHT crops being grown and studied including maize, soya, oilseed rape, sugar beet, cotton and rice. Commercialised GMHT crops such as maize, soya and rapeseed are being extensively cultivated in N and S America  with more recent introductions of cotton, sugar beet and alfalfa into USA and rapeseed into Australia. Field studies have been conducted extensively in these countries and experimental studies have been conducted in many European and other countries. From existing literature reviews [1–3, 12, 18–23] it can be anticipated that there is considerable literature on this topic.
The literature search strategy will capture the relevant literature for the systematic review by identifying as many relevant datasets as possible. The keywords will be broad and have a high-sensitivity and low-specificity in order to capture a high proportion of existing datasets. The search will include abstracting databases and reviews, as well as full text databases.
Terminology for the effects of the management of genetically modified herbicide tolerant crops will consider the main herbicides used in GMHT systems (glyphosate and glufosinate). Other herbicides such as dicamba 2,4-D, and other sulphonyl ureas will also be captured in the search but not specifically searched for as these GMHT systems have only been recently been developed and it is not likely that there will be sufficient data to review separately. The search terms will be selected from the following relevant terms.
The search strings for the literature databases consist of 3 parts (linked with AND). Terms in each part are linked with “OR”. Asterisks are used to include terms which are plural or have additional letters. Quotation marks denote multi-word terms (e.g. “herbicide toleran*”).
Part 1: (“herbicide toleran*” OR “herbicide resistan*” OR “glyphosate toleran*” OR “glufosinate toleran*” OR “glyphosate resistan*” OR “glufosinate resistan*” OR HT).
Limits the query to the HT crops for which this review is conducted.
Part 2: (field* OR plot* OR trial* OR farm* OR sampl* OR quadrat*).
This provides data from field studies as the terms refer to methods used in field studies.
Part 3: (weed* OR “weed species” OR “weed population” OR “weed abundance” OR “weed biomass” OR “seed bank” OR “seed return”).
This provides the entry for the following full search string:
(“herbicide toleran*” OR “herbicide resistan*” OR “glyphosate toleran*” OR “glufosinate toleran*” OR “glyphosate resistan*” OR “glufosinate resistan*” OR HT) (field* OR plot* OR location* OR trial* OR farm* OR sampl* OR quadrat*) (weed* OR “weed species” OR “weed population” OR “weed biomass” OR “weed cover” OR “seed bank” OR “seed return”).
Search terms for full text literature databases (e.g. Google Scholar, Scirus), are more restricted in order to avoid large amounts of non-relevant data.
Part 1: (herbicide* OR tolerant OR resistan* OR GM OR GE).
Part 2: (field* OR plot* OR location* OR trial* OR farm*).
Part 3: (weed* OR seed OR “seed bank”).
Part 4: (sample OR “weed abundance”).
The primary searches will be in English. Searches in other languages (e.g. Spanish and Portugese as these are relevant for S America where there is extensive cultivation of GMHT crops) will be conducted using search engines that contain non-English information (e.g. Google Scholar). One example might be to use international terms for GMOs and products, such as.
OGM OR glyphosate OR glufosinate
The precise format of the search strings will be adapted to the requirements of each database and other search words listed above will be used to broaden the search or in case some relevant papers contain none of the words in the main search strings. The search strings used will be recorded and descriptions of how each database was searched will be provided.
Data bases for the searches
Google scholar (search engine) and ISI Web of Knowledge were used for the original scoping exercise and were cross referenced against some recent literature reviews on HT crops. For the results of the scoping see below.
The systematic review will search the following abstracting literature databases:
Web of Science (ISI Web of Knowledge 1900 – 2013) (Thomson Reuters, New York, USA), contains peer reviewed scientific publications in English.
CAB Abstracts (1984–2013) (CABI, Wallingford, UK), database that includes local and non-English publications.
AGRICOLA (1970–2013) (National Agricultural Library, U.S. Department of Agriculture, Beltsville, USA): contains records of materials in agricultural and related sciences in the National Agricultural Library and cooperating institutes.
AGRIS (1975–2013) (Food and Agriculture Organization of the United Nations FAO, Rome, Italy): International System for Agricultural Sciences and Technology provides an international bibliographic database of national, and international centers.
Full text search engines:
GOOGLE SCHOLAR: online search engine of scientific and other scholarly works.
JSTOR: Digital library of academic journals, reports, books, and other primary sources.
NTIS (ProQuest), Cochrane Library, SCOPUS (for scientific reports), Medline & EMBASE may also be considered.
The following webpages containing information on effects of GMOs will also be searched:
Gmo-safety.eu: webpage established by the German Federal Ministry of Education and Research describing research projects and with links to reports etc.
EFSA.europa.eu: webpage of the European Food Safety Authority containing information related to GMO applications in EU.
EU GMO Register (http://gmoinfo.jrc.ec.europa.eu/). Lists all deliberate releases of GM plants for any experimental purposes.
Previously published review papers on GMHT crops will be listed and screened for any references that were not picked up in literature searches.
Scientists or organisations that have conducted field studies of GMHT crops will be contacted directly to obtain any relevant data. All contacts and responses will be recorded.
Full-text databases, such as Google Scholar and Scientific Electronic Library Online (http://www.scielo.org) are also suitable for non-English searches (e.g. Spanish, Portugese).
Field studies of GMHT crops in South America and Spain published in English often contain references to Spanish and Portuguese language reports of field studies. These will also be examined for relevance to the systematic review. Any papers that appear to be relevant will be sent to: Professor Ramon Albajes, Universitat de Lleida, Department of Plant Production and Forest Sciences
Ramon has conducted studies of GMHT crops and is in a good position to assess the quality of studies in Spanish and Portuguese against the inclusion criteria.
Field studies of GMHT crops have mostly occurred in N and S America, Australasia and Europe and so very few reports in Oriental languages are anticipated. However if they are revealed by the search and appear relevant steps will be taken to obtain translations.
Data base of publications
All publications discovered in the searches will be collated by Endnote. Subsets will be listed according main topic. Publications meeting the selection criteria (see below) will be filed separately and subjected to more detailed analysis.
Scoping the literature
Preliminary searches made between February and October 2012 using the search terms listed above in Google scholar have revealed 318 articles on GMHT crops of which 51 scientific papers and reports contain comparative data, including 21 papers studying effects on biodiversity, 15 papers on ecological impacts, and 23 papers on weed resistance development. The papers and reports identified were compared with those listed in recent review papers of GMHT crops e.g. [2, 15, 18]. The published reviews did not contain papers on GMHT crops that were not also identified in the scoping exercise. It is anticipated that a more comprehensive and systematic search using the full range of search terms and the data bases listed above will reveal more publications particularly as there are several recent comparative studies of GMHT crops commercialised in N and S America.
The quality of the studies has not been assessed but the volume of existing studies indicates that there is sufficient published information to justify a systematic review of the literature. The review will also reveal where there are gaps in the type of data collected, which in turn will identify where conclusions cannot be reached or conclusions are based on data subsets which might relate more to particular forms of bias.
Study inclusion criteria
The inclusion criteria are as follows:
Population: non-crop plant populations within fields/plots including field margins and cropped areas.
Intervention: the cultivation management of a GMHT crop which includes the use of the herbicide to which it is tolerant and tillage.
Comparator: a conventional crop (i.e. not GM HT) of a similar variety grown using conventional cultivation and weed management techniques in the same or immediately adjacent plots/fields.
Outcome: changes/differences in the non-crop plant and/or seed bank populations occurring in the GMHT and conventionally managed crops.
The studies must be comparisons between conventional weed management and GMHT management systems preferably in studies with split field or paired field systems or plots in fully randomised and replicated systems with sufficient replication or statistical power to show differences at <50% level. The compared fields should be receiving other management inputs which are similar and in similar rotational cropping systems.
The reviewed articles contain original data that has not been published elsewhere. Reviews, summaries, abstracts, comments papers, etc. will be listed separately and examined for their sources of information, data and bibliography (see Search strategy).
Data includes sample sizes, plot/field means and variation (SE, SD), for each treatment.
the study provides original data in the form of tables, figures or directly from the authors if requested.
Data are presented for each sampling time and not pooled through the season or year.
Weed data from studies of non-GM HT systems (e.g. Clearfield) will also be compared with weed data from GMHT crops and conventional crops.
The Comparisons should be of measurement end points (outcomes) such as the following:
Broad leaved weeds species and numbers
Graminaceous weeds species and numbers
Identified plant species totals
Weed seed bank, species populations and total numbers
feed Seed rain, species numbers and total numbers.
Plants assessed as plant numbers/m2 counted after herbicide applications and prior to harvest (earlier season data could also be looked at but the later timing is correlated with weed seed production and return).
Seed assessed as seed rain data collected prior to or at crop harvest, and/or seed numbers/m3 soil or germinated seedlings/m3 soil collected from soil auger sampling, or germinated seedlings/m2 soil in field after harvest of HT crop but before any weed management of the next crop.
If continuous HT cropping systems are studied then annual and accumulated data of these parameters will also be assessed.
The GMHT systems may consist of:
Glufosinate (glu) or glyphosate (gly) only used at Manufacturers Recommended rate
glu or gly only used at other rates and application timings,
glu or gly at various rates and timings used in programmes with other herbicides or weed management practices (eg hoeing)
glu or gly used in conjunction with minimum or no tillage systems.
Each treatment difference will be taken into account when analysing comparative effects.
The conventional (comparator) systems should be any conventional management that reflects routine normal or local farm practise. Conventional weed management will also include non-spraying methods such as mechanical hoeing, tillage etc., and the management practices used in each study will be documented.
Where the effects of the comparable weed management systems could be influenced by changes in other management factors such as crop rotation, irrigation, fertilizer treatments etc., data will be analyse separately.
Screening references: applying inclusion criteria
The references revealed by the searches described above will be entered into to Endnote X5 (Thomson Reuters). Separate Endnote files will be established for each search engine for a record of each search. The files will then be combined into one database which will eliminate duplicates automatically, though this will be checked manually. Some search engines do not allow transfer of articles to Endnote so that the search results will be copied or transfered to another file and checked manually. The inclusion criteria will be applied to titles and abstracts to remove unwanted references. The result will be an Endnote X5 database containing the potentially useful articles. These data will also be transferred to CADIMA (http://www.cadima.info/index.php?r=area/centralAccessPoint) for storage, evaluation and future availability.
The selected articles will have full texts extracted and examined further according to the inclusion criteria. Any articles excluded at this stage of the study for the systematic review will be documented in an Excel table and reasons for exclusion provided.
Screening references: quality assurance process
In this screening process, a random subset of 10% of the articles identified by the search procedure will be independently assessed by a second team member, applying the same inclusion criteria. This will be done at the abstract phase and full text phase as the abstracts often do not describe the type of comparative study conducted and the nature of the data collected. The results of the independent assessments will be analyzed and documented using Kappa statistics (http://www.vassarstats.net/kappa.html). Articles excluded in one assessment, but included by the other reviewer will be recorded and the reasons for the differences will be discussed by the review team. If the kappa-value falls below 0.6 indicating significant differences, a review will be undertaken so that inclusion criteria can be revised and tested for improved reviewer agreement and to optimise the screening process.
Study quality assessment
In order to assess the reliability of the presented results, each study will be assessed for the following possible sources of bias:
Selection bias. This can occur in field study design whereby the design of the study eg the plot size, herbicide treatment timings, sampling timing and method may favour one treatment.
Performance bias. Systematic differences between groups e.g. exposure to factors other than the intended intervention, such as the change in other management inputs (e.g. fertilizer, pest control, irrigation) associated with one treatment but not the other.
Detection bias (the way outcomes are measured differs between treatment groups, application of statistical analysis).
Attrition bias may occur in differences in sampling times following herbicide treatment, frequency, sizes between study groups, selective choice of parameters, and in missing data.
Reporting bias (e.g. preferential reporting of positive or desired outcomes).
Studies will be examined for a clear hypothesis and designs which are appropriate to achieve outcomes which test the hypothesis by testing both for similarities (equivalences) and differences.
Research experimental studies conducted on farm will tend to have farmers using normal managed weed control on conventional crops which give acceptable levels of weed control (to the farmer) while optimising inputs and costs. By contrast the management of the GMHT crops will tend to follow a rather prescriptive or formulaic protocol for the herbicide applications as requested by the researcher and/or biotech company. Thus there will be performance bias in these studies and they will contrast with studies conducted on farms where GMHT systems are well established. In this latter case farmers will be familiar with the requirements of GMHT crops and will thus optimise their inputs. Thus the effects on weed populations may be different between experimental studies and field studies of commercial GMHT crops.
In areas where there are concerns about weed resistance development (eg where there is intensive and repeated use of GMHT crops), farmers will tend to go for maximum weed control to prevent survival of resistant weeds and so this performance bias may result in different GMHT treatments from those where resistance is not perceived as a problem.
Sequential cultivation of GMHT crops is likely to have different consequences than rotational cultivation and so this should be evaluated separately in the analysis.
Experimental and Reporting bias is likely as some studies will be sponsored or conducted by interested organisations (eg: agrochem Cos, biotech Cos) or by organisations antagonistic to GMOs or modern agricultural practices, and therefore there may be some selection of recorded parameters, data, results or reports to show a particular effect or show a “desired” effect or result.
For example v.low or zero weeds would be considered a desirable result by an Agchem company but an adverse effect (reduction in botanical diversity) by an environmental organisation.
The study quality assessments will be independently assessed, as in the references screening process, by a second team member, applying the same quality criteria. The results of the independent assessments will be analyzed and documented using Kappa statistics (http://www.vassarstats.net/kappa.html). Studies assessed differently by reviewers will be recorded and the reasons for the differences will be discussed by the review team. If the kappa-value falls below 0.6 indicating significant differences, a review will be undertaken so that quality criteria can be revised and tested for improved reviewer agreement and to optimise the quality assessment process.
The levels of bias in studies will be characterized and studies which contain bias which has clearly influenced results will be assessed separately and recorded in the systematic review. This will be documented to ensure transparency and reconstructability.
Data extraction strategy
Original data will be described in each article, the method of extraction (including contacts with authors) will also be provided. Any estimated or calculated values will be described or explained. Where presented data are insufficient or inadequate, authors will be directly contacted and requested to provide the appropriate data.
Data will be stored in CADIMA that will be built for this purpose. Tables in Excel format can be easily extracted from the database.
Random samples of 20% of the extracted data will be checked by the second member of the review team.
The variables extracted and recorded are shown in Table 3 in Appendix and will be similar to those proposed by Meissle M, Naranjo SE, Riedel J, Romeis J: Does the growing of Bt maize change populations or ecological functions of non-target animals compared to the growing of conventional non-GM maize? 2014: Submitted to Environmental Evidence. This is in order to have conformity and similarity of data bases for later extraction and interrogation purposes.
The extracted datasets will be collated and analyzed and a narrative summary of the extracted outcomes will be produced. Tables and figures illustrating and summarizing the evidence will be produced to show the products and outcomes of the literature analysis. The data will be grouped and presented in different ways in order to allow a range of analyses and demonstrate a range of outcomes. For example:
Number of years of study.
Location of study.
Spatial scale of experiment.
Commercial field studies.
In crop or field margin.
Taxonomic groups of weeds (e.g. graminae, broadleaved).
Types of weeds (e.g. annual, perennial).
HT system studied (glufosinate or glyphosate).
Studies comparing GM HT crops with minimal tillage vs conventionally cultivated and treated crops.
Weed herbicide resistance.
Suitable quantitative datasets will be analyzed statistically as described below.
For each measurement endpoint the effect size and direction in relation to comparator (ie + or – ve) will be assessed.
Statistical meta-analyses will be conducted for data sets of populations of weed taxonomic groups occurring frequently in studies with similar designs and measured parameters. The scoping exercise indicates that there may be sufficient similar data sets for conducting meta-analyses, but this will not be confirmed until a qualitative evaluation of these data sets has been conducted. The range of weed types and parameters analysed will depend on the quality and amount of appropriate data to allow quantitative assessments.
For meta-analyses, the following inclusion criteria will be applied:
Clearly defined and similar parameters (e.g. abundance of weed taxa) were measured in the field under similar circumstance of time in the season
The taxonomic group of the weed measured was clearly described.
Sampling strategy (e.g. size, frequency, timing) is given for each treatment.
Raw data as well as plot, treatment and site means are presented together with a measure of variation (SE, SD).
Convention crops and their normal routine herbicide treatments are compared with GMHT crops and their specific herbicide treatments (e.g. gly and glu).
Mean measures of effect and variance are reported in the article or can be acquired from the author.
Data are recorded on an annual/seasonal basis (pooled and cumulative data sets from more than one year may be separately analysed).
Measures of treatment effect
The weed response variable to herbicide treatments is abundance. If enough data are available, response variables of individual weed species and weeds of certain taxonomic groups will be analysed. The mean effect size is calculated as the difference between the effects on weed abundance of herbicide treatments applied to GMHT crops and the herbicide treatments applied to conventional crops. This is reported is Hedges’ d weighted mean response, and is calculated as the mean response divided by a pooled standard deviation and multiplied by a correction term to allow for small sample size bias.
GMHT and conventional crop fields will be compared for
Different crop species
different weed species and groups
herbicide resistance development
different herbicide treatments
differences in commercial crop studies and experimental studies
Where there are comparisons of the effects of GMHT systems with those of HT systems for crops developed through non-GM techniques (e.g. imidazolinone tolerance developed through mutation breeding), separate analyses will be performed comparing the effects on weed populations of the GMHT and non-GMHT herbicide programmes.
Assessment of statistical power of included studies
For each measurement parameter in a study the statistical power of the treatment effect will be assessed either as an ex-post calculation of the power of the study for the effect, or as a prospective power of the study to detect pre-established significant effects magnitudes (e.g. the power of the study to detect a significant 10%, 20% and 50% impact effect on the outcome).
Data which is linked or consequential or sequential will be assessed separately and relationships studied. Interactions with other external factors such as management, season and environment will be considered.
Unit of analysis issues
A data issue that is likely to occur is the multiple use of the same dataset for different comparisons (for example the Farm Scale studies conducted in UK, [21,25]. Data on higher taxonomic levels or groupings may include data derived from other studies on individual species and so might result in multiple use of the same data. In addition data on a species or effect may be used from a study designed to study another different species or effect. The design may therefore not be appropriate or optimal. Secondary datasets will be identified and related to primary (original) data in the database during the data extraction process. The number of datasets reused in analyses will be recorded and discussed. Analyses might be conducted including and excluding multiple use datasets to determine their impacts as described below.
Dealing with missing data, and non-interpretable data
Where the nature of the data is not apparent or the article has introduced methods for allowing for missing data or there are missing or unusual/unlikely values, then attempts will be made to contact the authors or others who may be able to comprehend and interpret the data sets. Adjustments for missing data will only be made when the reasons for the absence are clear.
The reviewers will note data that is missing or has been adjusted or substituted.
The outcomes of the systematic review and meta analyses, in terms of the sizes of effects on weed abundance in comparisons of GMHT systems with control crop systems, including 95% confidence intervals, will be presented in tabular and graphic form. Total numbers of observations and effect sizes that are significantly different greater or less than 0 will be shown. Negative effect sizes are associated with lower abundances of weeds in GMHT crops compared with non-GMHT controls.
The nature and quality of the results will be discussed and conclusions drawn where appropriate.
Assessment of heterogeneity
The heterogeneity of the data across studies will be examined using appropriate statistical methods. The effects of studying different weed taxa, different regions, crop managements (minimum tillage, herbicide application methods, rotation), commercial and experimental scales, and experimental designs will be examined for their influence on variation.
Sub sets of studies containing similar parameters such as tillage, herbicide timing will be separately assessed to see how these influence outcomes. Where it is identified that certain parameters cause high heterogeneity, separate analyses of data subsets will be conducted.
Sensitivity analyses will be conducted to determine whether the outcome of the systematic review and meta-analyses might be influenced disproportionally by:
Studies that provide a high proportion of the datasets used in the meta-analyses.
Studies with high levels of precision, statistical power and/or replication which will tend to have higher weightings.
Studies with low levels of statistical power, poor replication or high variability which diverge from weighted means to a greater extent than studies with high precision, power or low variability.
The funding sources for the overall results.
studies where final weed numbers were measured in comparison to studies where weed abundance was measured at different times.
Data on particular common, rare or unevenly distributed weed taxa.
Assessment of publication bias
Effect sizes from different studies will be compared for articles produced by authors or journals with different funding types. Consistent differences may indicate a publication bias or a bias depending on funding source of a study (see 3.3) or a Journal.
Appendix to “What are the effects of the cultivation of GM herbicide tolerant crops on botanical diversity? A systematic review protocol”
By Jeremy Sweet and Kaloyan Kostov
List of participants to the stakeholder consultation workshop on good review practice in GMO impact assessment
GRACE team members
Gloria Adduci, IFZ- Inter-University Research Centre for Technology, Work and Culture, Austria
Wendy Craig, International Centre for Genetic Engineering and Biotechnology (ICGEB), Italy
Jaqueline Garcia-Yi, Centre of Life and Food Sciences Weihenstephan, Technical University of Munich (TUM), Germany
Maria Garrone, Centre for European Policy Studies (CEPS), Belgium
Achim Gathmann, Federal Office of Consumer Protection and Food Safety (BVL), Germany
Paul Kenning Krogh, Department of Bioscience, Aarhus University, Denmark
Steffen Kecke, Julius Kühn Institut (JKI), Germany
Gijs Kleter, RIKILT-Institute of Food Safety, Wageningen University and Research Centre, The Netherlands
Christian Kohl, Julius Kühn Institut (JKI), Germany
Kaloyan Kostov, AgroBioInstitute (ABI), Bulgaria
Klaus Minol, Genius, Germany
Kai Priesnitz, Federal Office of Consumer Protection and Food Safety (BVL), Germany
Patrick Rüdelsheim, PERSEUS, Belgium
Joachim Schiemann, Julius Kühn Institut (JKI), Germany
Greet Smets, PERSEUS, Belgium
Armin Spök, IFZ- Inter-University Research Centre for Technology, Work and Culture, Austria
Jeremy Sweet, Sweet Environmental Consultants, UK
Stefan Unger, Julius Kühn Institut (JKI), Germany
Justus Wesseler, Centre of Life and Food Sciences Weihenstephan, Technical University of Munich (TUM), Germany
Ralf Wilhelm, Julius Kühn Institut (JKI), Germany
Rita, Andorkó, Hungarian Ministry of Rural Development, Hungary
Zsuzsanna Bardócz, Hungarian Ministry of Rural Development, Hungary
Martin Bencko, Ministry of Agriculture and Rural Development of the Slovak Republic, Slovakia
Doris Bühler, Federal Office for Agriculture (BLW), Switzerland
Victoria Colombo Rodríguez, Ministerio de Agricultura, Alimentación y Medio Ambiente, Spain
Omar del Río Fernández, Ministerio de Agricultura, Alimentación y Medio Ambiente, Spain
Yann Devos, European Food Safety Authority (EFSA), Italy
Lyubina Donkova, Ministry of Agriculture and Food, Bulgaria
Catherine Gerard, French Embassy in Germany, Germany
Niall Gerlitz, European Commission, DG SANCO, Belgium
Petra Heinze, Federal Office of Consumer Protection and Food Safety (BVL), Germany
Rangel Krastanov, Ministry of Agriculture and Food, Bulgaria
Louise Lundstrøm Nielsen, Danish Ministry of the Environment, Environmental Protection Agency, Denmark
Odeta Pivoriene, Ministry of Environment of Lithuania, Lithuania
Annette Pöting, Federal Institute for Risk Assessment (BfR), Germany
Eline Rademakers, Federal Public Service (FPS) Health, Food Chain Safety and Environment, Belgium
Leif Erik Rehder, U.S. Embassy in Berlin, Germany
Jane Richardson, European Food Safety Authority (EFSA), Italy
Neringa Sarkauskiene, Ministry of Environment of Lithuania, Lithuania
Andrea Scheepers, Federal Office of Consumer Protection and Food Safety (BVL), Germany
Paul Spencer, U.S. Embassy in Berlin, Germany
Beatrix Tappeser, Federal Agency for Nature Conservation (BfN), Germany
Valentina Zoretic-Rubes, Ministry of Health, Croatia
Civil Society Organisations
Rosa Binimelis, GenOk – Centre for Biosafety, Norway
Walter Haefeker, European Professional Beekeepers Association, Germany
Lise Nordgård, GenOk – Centre for Biosafety, Norway
Christoph Then, Testbiotech - Institute for Independent Impact Assessment in Biotechnology, Germany
Kristina Wagner, Eurogroup for Animals, Belgium
Dirk Zimmermann, Greenpeace e. V., Germany
Manuel Gómez-Barbero, EuropaBio, Belgium
Andrew Tommey, Pioneer Overseas Corporation, Belgium
Didier Breyer, Institut Scientifique de Santé Publique (WIV-ISP), Belgium
Sylvia Burssens, Ghent University, Belgium
Barbara De Santis, National Institute of Health (ISS), Italy
Geoff Frampton, School of Medicine University of Southampton, UK
Hrvoje Fulgosi, Ruđer Bošković Institute, Croatia
Boet Glandorf, National Institute for Public Health and the Environment, The Netherlands
Richard Goodman, University of Nebraska—Lincoln, USA
Linde Inghelbrecht, Department of Agricultural Economics, University of Ghent, Belgium
Harry Kuiper, Wageningen University and Research Centre, The Netherlands
Annalisa Paternò, Istituto Zooprofilattico Sperimentale delle Regioni Lazio e Toscana, Italy
Monica Racovita, International Centre for Genetic Engineering and Biotechnology (ICGEB), Italy
List of organisations submitting written comments
German Federal Agency for Nature Conservation (BfN).
Dutch National Institute for Public Health and the Environment (RIVM).
Richard E. Goodman
University of Nebraska, Lincoln.
European Professional Beekeepers Association.
List of variables extracted for the systematic review on GMHT crops
Unique identification number assigned to each publication
Author(s) of the listed publication.
Common and species name
Year of publication of study
Citation, e.g. journal name, volume and page numbers
Title of the publication
Type(s) of institutions/organisations that the author(s) are affiliated with
Institution of corresponding author (or of first author if no corresponding author was listed).
Indicates whether study was published in a peer reviewed journal.
Geoposition or region/country where field study was performed
Information on previous cropping, rotation, soil type, irrigation etc.
Number of experiments within a study (e.g. different locations, years, etc.)
Figure number, table number or page number where means and variation were found.
Indicates whether figures were scanned to obtain data values.
Herbicide tolerance gene engineered into the transgenic crop.
Conventional herbicide comparator treatment
Transgenic event of the crop tested.
Transgenic hybrid or variety name.
Non-transgenic comparator variety name.
Monocotyledon or dicotyledon
Finest level of taxonomic resolution reported for the plant(s).
Emergence/establishment E= early, M= midseason, L=late
Codes flag for non-independence to be considered for analyses: TGLE=taxonomic group lumped elsewhere; EMUE= experimental means used elsewhere; CMUE=control means used elsewhere;
Location of field(s) to the level of specificity provided by the author
Number of fields as described by the author.
Cultivation practices used within the fields (notes on tillage, herbicides, etc.)
Size of replicate plots (in hectares)
Explanations for any calculations done to obtain plot size
Indicates whether the listed plot size is an average or an estimate
Indicates whether the authors indicated that they randomly assigned replicates to treatments.
Date on which field plots were planted.
Date on which first sample was taken
Date on which last sample was taken
More detailed description on study duration
Brand name of herbicides used.
Active ingredient for herbicides used.
Amount of active ingredient per spray.
Indicates if herbicide was applied as spray, granule, LV,ULV
Number of herbicide applications.
Indicates if the number of herbicide applications is an average
Abbreviated description of sampling method
Detailed description of sampling method.
Frequency of repeated samples per replicate field or plot.
The number of times that each replicate field or plot was repeatedly sampled over the duration of the experiment.
Number of subsamples per true replicate.
Major category of response variable.
Detailed description of response variable.
True sample size for control treatment.
True sample size for experimental treatment.
Sample size for the control treatment as stated by the author.
Sample size for the experimental treatment as stated by the author.
Indicates whether values represent seasonal means across multiple sample days or means from peak days
Indicates whether we calculated the seasonal mean or peak days.
Explains how we calculated the seasonal mean or peak days
Indicates whether the comparison is with conventional treated HT or conventional treated control, minimum tillage vs conventional tillage
Mean for the control treatment.
Mean for the experimental treatment.
Standard error for the control treatment.
Standard error for the experimental treatment.
Standard deviation for the control treatment.
Standard deviation for the experimental treatment.
Unit of measurement for the response variable. E.g. weed/seed numbers, biomass, ground cover.
Statistical test used by author.
Indicates whether a significant effect was detected by the author.
Space for remarks for this record
This review is funded within the GRACE consortium, a EU-FP7 programme project, Grant Agreement KBBE-2011-6-311957. We acknowledge the support of Ramon Albajes with interpretation of Spanish and Portugese articles. We thank Christian Kohl of JKI for his support and guidance in developing this protocol. We are grateful to Michael Meissle, Steven E. Naranjo, Judith Riedel and Jörg Romeis for allowing us to use their manuscript and data base in formulating this protocol and to all collaborators within GRACE who provided valuable comments on drafts of this review protocol.
- Boatman ND, Parry HR, Bishop JD, Cuthbertson AGS: Impacts of Agricultural Change on Farmland Biodiversity in the UK. Issues in Environmental Science and Technology, Volume 25. In Biodiversity Under Threat. Edited by: Hester RE, Harrison RM. The Royal Society of Chemistry; 2007. Chapter 1, 1–32Google Scholar
- Sweet J, Bartsch D: Synthesis and overview studies to evaluate existing research and knowledge on biological issues on GM plants of relevance to Swiss environments. SNSF Nationales Forschungsprogramm “Nutzen und Risiken der Freisetzung gentechnisch veränderter Pflanzen” (NFP 59) 2012. vdf Hochschulverlag, 194 Seiten ISBN 978–3-7281–3498–1Google Scholar
- Sweet JB, Lutman PJW: A commentary on the BRIGHT programme on herbicide tolerant crops and the implications of the BRIGHT and farm scale evaluation programmes for the development of herbicide tolerant crops in Europe. Outlooks on Pest Management 2006, 2006: 249–254.View ArticleGoogle Scholar
- Brookes G: The farm-level impact of herbicide-tolerant soybeans in Romania. Agbioforum 2005,8(4):235–241. Available on the worldwide web at http://www.agbioforum.org Google Scholar
- Carpenter J: GM crops and patterns of pesticide use. Science 2001,2001(292):637–638.View ArticleGoogle Scholar
- EFSA: European Food Safety Authority: Application of systematic review methodology to food and feed safety assessments to support decision making. EFSA Journal 2010,8(6):1637. Available online: http://www.efsa.europa.eu Google Scholar
- Geisy JP, Dobson S, Solomon KR: Ecotoxicological risk assessment for Roundup herbicide. Rev Environ Contam Toxicol 2000, 167: 35–120.Google Scholar
- Kleter GA, Harris C, Stephenson G, Unsworth J: Comparison of herbicide regimes and the associated potential environmental effects of glyphosate-resistant crops versus what they replace in Europe. Pest Manag Sci 2008, 64: 479–488. 10.1002/ps.1513View ArticleGoogle Scholar
- Kovach J, Petzoldt C, Degni J, Tette J New York’s Food and Life Sciences Bulletin, Volume 139. In A Method to Measure the Environmental Impact of Pesticides. Geneva, NY: NYS Agricul. Exp. Sta. Cornell University; 1992:8. Annually updated http://www.nysipm.cornell.edu/publications/EIQ.html Google Scholar
- Pleasants JM, Oberhauser KS: Milkweed loss in agricultural fields because of herbicide use: effects on the monarch butterfly population. Insect Conservation and Diversity 2012., 2012: doi: 10.1111/j.1752–4598–2012.00196.x.14Google Scholar
- Sankula S, Blumenthal E: Impacts on US Agriculture of Biotechnology-Derived Crops Planted in 2005- an Update of Eleven Case Studies. Washington: NCFAP; 2006. http://www.ncfap.org Google Scholar
- Squire GR, Hawes C, Begg GS, Young MW: Cumulative impact of GM herbicide-tolerant cropping on arable plants assessed through species-based and functional taxonomies. Environ Sci Pollut Res 2009,16(1):85–94. doi:10.1007/s11356–008–0072–6 10.1007/s11356-008-0072-6View ArticleGoogle Scholar
- Sweet J, Bartsch D: Guidance on risk assessment of herbicide tolerant GM plants by the European Food Safety Authority. Journal für Verbraucherschutz und Lebensmittelsicherheit (Journal of Consumer Protection and Food Safety) 2011,6(supplement 1):65–72. DOI:10.1007/s00003–011–0686–3View ArticleGoogle Scholar
- Hurley TM, Mitchell PD, Frisvold GB: Effects of weed resistance concerns and resistance management practices on the value of Roundup Ready crops. Agbioforum 2009,12(3 & 4):291–302.Google Scholar
- Benbrook C: Impacts of genetically engineered crops on pesticide use in the U.S. – the first sixteen years. Environ Sci Eur 2012, 24: 1–24. 10.1186/2190-4715-24-1View ArticleGoogle Scholar
- Hartzler RG: Reduction in common milkweed (Asclepias syriaca) occurrence in Iowa cropland from 1999 to 2009. Crop Prot 2009, 29: 1542–1544. (2010)View ArticleGoogle Scholar
- Council NR: Impact of Genetically Engineered Crops on Farm Sustainability in the US. Washington DC: National Acadamies Press; 2010.Google Scholar
- Beckie HJ, Harker KN, Hall LM, Warwick SI, Légère A, Sikkema PH, Clayton GW, Thomas AG, Leeson JY, Séguin-Swartz G, Simard M-J: A decade of herbicide-resistant crops in Canada. Can J Plant Sci 2006, 86: 1243–1264. 10.4141/P05-193View ArticleGoogle Scholar
- Bonny S: Genetically modified glyphosate-tolerant soybean in the USA: adoption factors, impacts and prospects. A review. Agron Sustain Dev 2008, 28: 21–32. 10.1051/agro:2007044View ArticleGoogle Scholar
- Carpenter J: Impact of GM crops on biodiversity. GM Crops, Landes Bioscience 2011,2(1):7–23. 10.4161/gmcr.2.1.15086View ArticleGoogle Scholar
- Johnson WG, Davis VM, Kruger GR, Weller SC: Influence of glyphosate-resistant cropping systems on weed species shifts and glyphosate-resistant weed populations. Europ J Agronomy 2009, 31: 162–172. 10.1016/j.eja.2009.03.008View ArticleGoogle Scholar
- Graef F: Agro-environmental effects due to altered cultivation practices with genetically modified herbicide-tolerant oilseed rape and implications for monitoring: a review. Sustainable Agriculture 2009, 2: 229–242. Part 2, 229–242, DOI: 10.1007/978–90–481–2666–8_15View ArticleGoogle Scholar
- Monquero PA: Plantas transgênicas resistentes aos herbicidas: situação e perspectivas. Bragantia, Campinas 2005,64(4):517–531. 10.1590/S0006-87052005000400002View ArticleGoogle Scholar
- Heap, I: The international survey of herbicide resistant weeds. 2014.Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.