We will perform a rigorous review of studies of plant populations in fields and field margins of GMHT crops by complying with CEE requirements for Systematic Reviews (SR) and the EFSA Guidance on Systematic Reviewing
The aim of the literature search is to be as comprehensive as possible in revealing all available scientific and technical publications and reports which compare the environmental effects of GMHT and conventionally managed crops, with a specific focus on the effects on the botanical diversity and weed populations in farmland receiving the two comparable managements. The search will be indiscriminate and non-selective and include all the main GMHT crops being grown and studied including maize, soya, oilseed rape, sugar beet, cotton and rice. Commercialised GMHT crops such as maize, soya and rapeseed are being extensively cultivated in N and S America
 with more recent introductions of cotton, sugar beet and alfalfa into USA and rapeseed into Australia. Field studies have been conducted extensively in these countries and experimental studies have been conducted in many European and other countries. From existing literature reviews
[1–3, 12, 18–23] it can be anticipated that there is considerable literature on this topic.
The literature search strategy will capture the relevant literature for the systematic review by identifying as many relevant datasets as possible. The keywords will be broad and have a high-sensitivity and low-specificity in order to capture a high proportion of existing datasets. The search will include abstracting databases and reviews, as well as full text databases.
Terminology for the effects of the management of genetically modified herbicide tolerant crops will consider the main herbicides used in GMHT systems (glyphosate and glufosinate). Other herbicides such as dicamba 2,4-D, and other sulphonyl ureas will also be captured in the search but not specifically searched for as these GMHT systems have only been recently been developed and it is not likely that there will be sufficient data to review separately. The search terms will be selected from the following relevant terms.
The search strings for the literature databases consist of 3 parts (linked with AND). Terms in each part are linked with “OR”. Asterisks are used to include terms which are plural or have additional letters. Quotation marks denote multi-word terms (e.g. “herbicide toleran*”).
Part 1: (“herbicide toleran*” OR “herbicide resistan*” OR “glyphosate toleran*” OR “glufosinate toleran*” OR “glyphosate resistan*” OR “glufosinate resistan*” OR HT).
Limits the query to the HT crops for which this review is conducted.
Part 2: (field* OR plot* OR trial* OR farm* OR sampl* OR quadrat*).
This provides data from field studies as the terms refer to methods used in field studies.
Part 3: (weed* OR “weed species” OR “weed population” OR “weed abundance” OR “weed biomass” OR “seed bank” OR “seed return”).
This provides the entry for the following full search string:
(“herbicide toleran*” OR “herbicide resistan*” OR “glyphosate toleran*” OR “glufosinate toleran*” OR “glyphosate resistan*” OR “glufosinate resistan*” OR HT) (field* OR plot* OR location* OR trial* OR farm* OR sampl* OR quadrat*) (weed* OR “weed species” OR “weed population” OR “weed biomass” OR “weed cover” OR “seed bank” OR “seed return”).
Search terms for full text literature databases (e.g. Google Scholar, Scirus), are more restricted in order to avoid large amounts of non-relevant data.
Part 1: (herbicide* OR tolerant OR resistan* OR GM OR GE).
Part 2: (field* OR plot* OR location* OR trial* OR farm*).
Part 3: (weed* OR seed OR “seed bank”).
Part 4: (sample OR “weed abundance”).
The primary searches will be in English. Searches in other languages (e.g. Spanish and Portugese as these are relevant for S America where there is extensive cultivation of GMHT crops) will be conducted using search engines that contain non-English information (e.g. Google Scholar). One example might be to use international terms for GMOs and products, such as.
OGM OR glyphosate OR glufosinate
The precise format of the search strings will be adapted to the requirements of each database and other search words listed above will be used to broaden the search or in case some relevant papers contain none of the words in the main search strings. The search strings used will be recorded and descriptions of how each database was searched will be provided.
Data bases for the searches
Google scholar (search engine) and ISI Web of Knowledge were used for the original scoping exercise and were cross referenced against some recent literature reviews on HT crops. For the results of the scoping see below.
The systematic review will search the following abstracting literature databases:
Web of Science (ISI Web of Knowledge 1900 – 2013) (Thomson Reuters, New York, USA), contains peer reviewed scientific publications in English.
CAB Abstracts (1984–2013) (CABI, Wallingford, UK), database that includes local and non-English publications.
AGRICOLA (1970–2013) (National Agricultural Library, U.S. Department of Agriculture, Beltsville, USA): contains records of materials in agricultural and related sciences in the National Agricultural Library and cooperating institutes.
AGRIS (1975–2013) (Food and Agriculture Organization of the United Nations FAO, Rome, Italy): International System for Agricultural Sciences and Technology provides an international bibliographic database of national, and international centers.
Full text search engines:
GOOGLE SCHOLAR: online search engine of scientific and other scholarly works.
JSTOR: Digital library of academic journals, reports, books, and other primary sources.
NTIS (ProQuest), Cochrane Library, SCOPUS (for scientific reports), Medline & EMBASE may also be considered.
The following webpages containing information on effects of GMOs will also be searched:
Gmo-safety.eu: webpage established by the German Federal Ministry of Education and Research describing research projects and with links to reports etc.
EFSA.europa.eu: webpage of the European Food Safety Authority containing information related to GMO applications in EU.
EU GMO Register (http://gmoinfo.jrc.ec.europa.eu/). Lists all deliberate releases of GM plants for any experimental purposes.
International Survey of Herbicide resistant weeds
. Available at http://www.weedscience.org.
Previously published review papers on GMHT crops will be listed and screened for any references that were not picked up in literature searches.
Scientists or organisations that have conducted field studies of GMHT crops will be contacted directly to obtain any relevant data. All contacts and responses will be recorded.
Full-text databases, such as Google Scholar and Scientific Electronic Library Online (http://www.scielo.org) are also suitable for non-English searches (e.g. Spanish, Portugese).
Field studies of GMHT crops in South America and Spain published in English often contain references to Spanish and Portuguese language reports of field studies. These will also be examined for relevance to the systematic review. Any papers that appear to be relevant will be sent to: Professor Ramon Albajes, Universitat de Lleida, Department of Plant Production and Forest Sciences
Ramon has conducted studies of GMHT crops and is in a good position to assess the quality of studies in Spanish and Portuguese against the inclusion criteria.
Field studies of GMHT crops have mostly occurred in N and S America, Australasia and Europe and so very few reports in Oriental languages are anticipated. However if they are revealed by the search and appear relevant steps will be taken to obtain translations.
Data base of publications
All publications discovered in the searches will be collated by Endnote. Subsets will be listed according main topic. Publications meeting the selection criteria (see below) will be filed separately and subjected to more detailed analysis.
Scoping the literature
Preliminary searches made between February and October 2012 using the search terms listed above in Google scholar have revealed 318 articles on GMHT crops of which 51 scientific papers and reports contain comparative data, including 21 papers studying effects on biodiversity, 15 papers on ecological impacts, and 23 papers on weed resistance development. The papers and reports identified were compared with those listed in recent review papers of GMHT crops e.g.
[2, 15, 18]. The published reviews did not contain papers on GMHT crops that were not also identified in the scoping exercise. It is anticipated that a more comprehensive and systematic search using the full range of search terms and the data bases listed above will reveal more publications particularly as there are several recent comparative studies of GMHT crops commercialised in N and S America.
The quality of the studies has not been assessed but the volume of existing studies indicates that there is sufficient published information to justify a systematic review of the literature. The review will also reveal where there are gaps in the type of data collected, which in turn will identify where conclusions cannot be reached or conclusions are based on data subsets which might relate more to particular forms of bias.
Study inclusion criteria
The inclusion criteria are as follows:
Population: non-crop plant populations within fields/plots including field margins and cropped areas.
Intervention: the cultivation management of a GMHT crop which includes the use of the herbicide to which it is tolerant and tillage.
Comparator: a conventional crop (i.e. not GM HT) of a similar variety grown using conventional cultivation and weed management techniques in the same or immediately adjacent plots/fields.
Outcome: changes/differences in the non-crop plant and/or seed bank populations occurring in the GMHT and conventionally managed crops.
The studies must be comparisons between conventional weed management and GMHT management systems preferably in studies with split field or paired field systems or plots in fully randomised and replicated systems with sufficient replication or statistical power to show differences at <50% level. The compared fields should be receiving other management inputs which are similar and in similar rotational cropping systems.
The reviewed articles contain original data that has not been published elsewhere. Reviews, summaries, abstracts, comments papers, etc. will be listed separately and examined for their sources of information, data and bibliography (see Search strategy).
Data includes sample sizes, plot/field means and variation (SE, SD), for each treatment.
the study provides original data in the form of tables, figures or directly from the authors if requested.
Data are presented for each sampling time and not pooled through the season or year.
Weed data from studies of non-GM HT systems (e.g. Clearfield) will also be compared with weed data from GMHT crops and conventional crops.
The Comparisons should be of measurement end points (outcomes) such as the following:
Measurement end points: Populations and abundance measured in the field
Broad leaved weeds species and numbers
Graminaceous weeds species and numbers
Identified plant species totals
Weed seed bank, species populations and total numbers
feed Seed rain, species numbers and total numbers.
Plants assessed as plant numbers/m2 counted after herbicide applications and prior to harvest (earlier season data could also be looked at but the later timing is correlated with weed seed production and return).
Seed assessed as seed rain data collected prior to or at crop harvest, and/or seed numbers/m3 soil or germinated seedlings/m3 soil collected from soil auger sampling, or germinated seedlings/m2 soil in field after harvest of HT crop but before any weed management of the next crop.
If continuous HT cropping systems are studied then annual and accumulated data of these parameters will also be assessed.
The GMHT systems may consist of:
Glufosinate (glu) or glyphosate (gly) only used at Manufacturers Recommended rate
glu or gly only used at other rates and application timings,
glu or gly at various rates and timings used in programmes with other herbicides or weed management practices (eg hoeing)
glu or gly used in conjunction with minimum or no tillage systems.
Each treatment difference will be taken into account when analysing comparative effects.
The conventional (comparator) systems should be any conventional management that reflects routine normal or local farm practise. Conventional weed management will also include non-spraying methods such as mechanical hoeing, tillage etc., and the management practices used in each study will be documented.
Where the effects of the comparable weed management systems could be influenced by changes in other management factors such as crop rotation, irrigation, fertilizer treatments etc., data will be analyse separately.
Screening references: applying inclusion criteria
The references revealed by the searches described above will be entered into to Endnote X5 (Thomson Reuters). Separate Endnote files will be established for each search engine for a record of each search. The files will then be combined into one database which will eliminate duplicates automatically, though this will be checked manually. Some search engines do not allow transfer of articles to Endnote so that the search results will be copied or transfered to another file and checked manually. The inclusion criteria will be applied to titles and abstracts to remove unwanted references. The result will be an Endnote X5 database containing the potentially useful articles. These data will also be transferred to CADIMA (http://www.cadima.info/index.php?r=area/centralAccessPoint) for storage, evaluation and future availability.
The selected articles will have full texts extracted and examined further according to the inclusion criteria. Any articles excluded at this stage of the study for the systematic review will be documented in an Excel table and reasons for exclusion provided.
Screening references: quality assurance process
In this screening process, a random subset of 10% of the articles identified by the search procedure will be independently assessed by a second team member, applying the same inclusion criteria. This will be done at the abstract phase and full text phase as the abstracts often do not describe the type of comparative study conducted and the nature of the data collected. The results of the independent assessments will be analyzed and documented using Kappa statistics (http://www.vassarstats.net/kappa.html). Articles excluded in one assessment, but included by the other reviewer will be recorded and the reasons for the differences will be discussed by the review team. If the kappa-value falls below 0.6 indicating significant differences, a review will be undertaken so that inclusion criteria can be revised and tested for improved reviewer agreement and to optimise the screening process.
Study quality assessment
In order to assess the reliability of the presented results, each study will be assessed for the following possible sources of bias:
Selection bias. This can occur in field study design whereby the design of the study eg the plot size, herbicide treatment timings, sampling timing and method may favour one treatment.
Performance bias. Systematic differences between groups e.g. exposure to factors other than the intended intervention, such as the change in other management inputs (e.g. fertilizer, pest control, irrigation) associated with one treatment but not the other.
Detection bias (the way outcomes are measured differs between treatment groups, application of statistical analysis).
Attrition bias may occur in differences in sampling times following herbicide treatment, frequency, sizes between study groups, selective choice of parameters, and in missing data.
Reporting bias (e.g. preferential reporting of positive or desired outcomes).
Studies will be examined for a clear hypothesis and designs which are appropriate to achieve outcomes which test the hypothesis by testing both for similarities (equivalences) and differences.
The following sources of bias and influences on the studies are identified:
Research experimental studies conducted on farm will tend to have farmers using normal managed weed control on conventional crops which give acceptable levels of weed control (to the farmer) while optimising inputs and costs. By contrast the management of the GMHT crops will tend to follow a rather prescriptive or formulaic protocol for the herbicide applications as requested by the researcher and/or biotech company. Thus there will be performance bias in these studies and they will contrast with studies conducted on farms where GMHT systems are well established. In this latter case farmers will be familiar with the requirements of GMHT crops and will thus optimise their inputs. Thus the effects on weed populations may be different between experimental studies and field studies of commercial GMHT crops.
In areas where there are concerns about weed resistance development (eg where there is intensive and repeated use of GMHT crops), farmers will tend to go for maximum weed control to prevent survival of resistant weeds and so this performance bias may result in different GMHT treatments from those where resistance is not perceived as a problem.
Sequential cultivation of GMHT crops is likely to have different consequences than rotational cultivation and so this should be evaluated separately in the analysis.
Experimental and Reporting bias is likely as some studies will be sponsored or conducted by interested organisations (eg: agrochem Cos, biotech Cos) or by organisations antagonistic to GMOs or modern agricultural practices, and therefore there may be some selection of recorded parameters, data, results or reports to show a particular effect or show a “desired” effect or result.
For example v.low or zero weeds would be considered a desirable result by an Agchem company but an adverse effect (reduction in botanical diversity) by an environmental organisation.
The study quality assessments will be independently assessed, as in the references screening process, by a second team member, applying the same quality criteria. The results of the independent assessments will be analyzed and documented using Kappa statistics (http://www.vassarstats.net/kappa.html). Studies assessed differently by reviewers will be recorded and the reasons for the differences will be discussed by the review team. If the kappa-value falls below 0.6 indicating significant differences, a review will be undertaken so that quality criteria can be revised and tested for improved reviewer agreement and to optimise the quality assessment process.
The levels of bias in studies will be characterized and studies which contain bias which has clearly influenced results will be assessed separately and recorded in the systematic review. This will be documented to ensure transparency and reconstructability.
Data extraction strategy
Original data will be described in each article, the method of extraction (including contacts with authors) will also be provided. Any estimated or calculated values will be described or explained. Where presented data are insufficient or inadequate, authors will be directly contacted and requested to provide the appropriate data.
Data will be stored in CADIMA that will be built for this purpose. Tables in Excel format can be easily extracted from the database.
Random samples of 20% of the extracted data will be checked by the second member of the review team.
The variables extracted and recorded are shown in Table 3 in Appendix and will be similar to those proposed by Meissle M, Naranjo SE, Riedel J, Romeis J: Does the growing of Bt maize change populations or ecological functions of non-target animals compared to the growing of conventional non-GM maize? 2014: Submitted to Environmental Evidence. This is in order to have conformity and similarity of data bases for later extraction and interrogation purposes.
The extracted datasets will be collated and analyzed and a narrative summary of the extracted outcomes will be produced. Tables and figures illustrating and summarizing the evidence will be produced to show the products and outcomes of the literature analysis. The data will be grouped and presented in different ways in order to allow a range of analyses and demonstrate a range of outcomes. For example:
Number of years of study.
Location of study.
Spatial scale of experiment.
Commercial field studies.
In crop or field margin.
Taxonomic groups of weeds (e.g. graminae, broadleaved).
Types of weeds (e.g. annual, perennial).
HT system studied (glufosinate or glyphosate).
Studies comparing GM HT crops with minimal tillage vs conventionally cultivated and treated crops.
Weed herbicide resistance.
Suitable quantitative datasets will be analyzed statistically as described below.
For each measurement endpoint the effect size and direction in relation to comparator (ie + or – ve) will be assessed.
Statistical meta-analyses will be conducted for data sets of populations of weed taxonomic groups occurring frequently in studies with similar designs and measured parameters. The scoping exercise indicates that there may be sufficient similar data sets for conducting meta-analyses, but this will not be confirmed until a qualitative evaluation of these data sets has been conducted. The range of weed types and parameters analysed will depend on the quality and amount of appropriate data to allow quantitative assessments.
For meta-analyses, the following inclusion criteria will be applied:
Clearly defined and similar parameters (e.g. abundance of weed taxa) were measured in the field under similar circumstance of time in the season
The taxonomic group of the weed measured was clearly described.
Sampling strategy (e.g. size, frequency, timing) is given for each treatment.
Raw data as well as plot, treatment and site means are presented together with a measure of variation (SE, SD).
Convention crops and their normal routine herbicide treatments are compared with GMHT crops and their specific herbicide treatments (e.g. gly and glu).
Mean measures of effect and variance are reported in the article or can be acquired from the author.
Data are recorded on an annual/seasonal basis (pooled and cumulative data sets from more than one year may be separately analysed).
Measures of treatment effect
The weed response variable to herbicide treatments is abundance. If enough data are available, response variables of individual weed species and weeds of certain taxonomic groups will be analysed. The mean effect size is calculated as the difference between the effects on weed abundance of herbicide treatments applied to GMHT crops and the herbicide treatments applied to conventional crops. This is reported is Hedges’ d weighted mean response, and is calculated as the mean response divided by a pooled standard deviation and multiplied by a correction term to allow for small sample size bias.
GMHT and conventional crop fields will be compared for
Different crop species
different weed species and groups
herbicide resistance development
different herbicide treatments
differences in commercial crop studies and experimental studies
Where there are comparisons of the effects of GMHT systems with those of HT systems for crops developed through non-GM techniques (e.g. imidazolinone tolerance developed through mutation breeding), separate analyses will be performed comparing the effects on weed populations of the GMHT and non-GMHT herbicide programmes.
Assessment of statistical power of included studies
For each measurement parameter in a study the statistical power of the treatment effect will be assessed either as an ex-post calculation of the power of the study for the effect, or as a prospective power of the study to detect pre-established significant effects magnitudes (e.g. the power of the study to detect a significant 10%, 20% and 50% impact effect on the outcome).
Data which is linked or consequential or sequential will be assessed separately and relationships studied. Interactions with other external factors such as management, season and environment will be considered.
Unit of analysis issues
A data issue that is likely to occur is the multiple use of the same dataset for different comparisons (for example the Farm Scale studies conducted in UK, [21,25]. Data on higher taxonomic levels or groupings may include data derived from other studies on individual species and so might result in multiple use of the same data. In addition data on a species or effect may be used from a study designed to study another different species or effect. The design may therefore not be appropriate or optimal. Secondary datasets will be identified and related to primary (original) data in the database during the data extraction process. The number of datasets reused in analyses will be recorded and discussed. Analyses might be conducted including and excluding multiple use datasets to determine their impacts as described below.
Dealing with missing data, and non-interpretable data
Where the nature of the data is not apparent or the article has introduced methods for allowing for missing data or there are missing or unusual/unlikely values, then attempts will be made to contact the authors or others who may be able to comprehend and interpret the data sets. Adjustments for missing data will only be made when the reasons for the absence are clear.
The reviewers will note data that is missing or has been adjusted or substituted.
The outcomes of the systematic review and meta analyses, in terms of the sizes of effects on weed abundance in comparisons of GMHT systems with control crop systems, including 95% confidence intervals, will be presented in tabular and graphic form. Total numbers of observations and effect sizes that are significantly different greater or less than 0 will be shown. Negative effect sizes are associated with lower abundances of weeds in GMHT crops compared with non-GMHT controls.
The nature and quality of the results will be discussed and conclusions drawn where appropriate.
Assessment of heterogeneity
The heterogeneity of the data across studies will be examined using appropriate statistical methods. The effects of studying different weed taxa, different regions, crop managements (minimum tillage, herbicide application methods, rotation), commercial and experimental scales, and experimental designs will be examined for their influence on variation.
Sub sets of studies containing similar parameters such as tillage, herbicide timing will be separately assessed to see how these influence outcomes. Where it is identified that certain parameters cause high heterogeneity, separate analyses of data subsets will be conducted.
Sensitivity analyses will be conducted to determine whether the outcome of the systematic review and meta-analyses might be influenced disproportionally by:
Studies that provide a high proportion of the datasets used in the meta-analyses.
Studies with high levels of precision, statistical power and/or replication which will tend to have higher weightings.
Studies with low levels of statistical power, poor replication or high variability which diverge from weighted means to a greater extent than studies with high precision, power or low variability.
The funding sources for the overall results.
studies where final weed numbers were measured in comparison to studies where weed abundance was measured at different times.
Data on particular common, rare or unevenly distributed weed taxa.
Assessment of publication bias
Effect sizes from different studies will be compared for articles produced by authors or journals with different funding types. Consistent differences may indicate a publication bias or a bias depending on funding source of a study (see 3.3) or a Journal.