What are the effects of the cultivation of GM herbicide tolerant crops on botanical diversity? A systematic review protocol

Abstract

Background

There are concerns that the cultivation of genetically modified herbicide tolerant (GMHT) crops treated with broad spectrum herbicides will cause declines in botanical diversity and hence loss of biodiversity. Cultivation systems of these have different levels of inputs and management interventions and some incorporate the use of minimal/no tillage. Research results show a range of effects and the priority is to determine whether research studies show shifts in botanical diversity and/or declines in plant populations in GMHT compared with conventionally managed crops.

Methods

We will perform a rigorous review of studies of plant populations in fields and field margins of GMHT crops by complying with CEE requirements for Systematic Reviews (SR) and the EFSA Guidance on Systematic Reviewing. A Review Protocol (RP) is presented for the SR of data from field studies of GMHT crops, comparing the effects of GM crop, herbicide regimes and associated management applied to HT crops with conventional crops and their weed management for impacts on plant populations in fields and field margins as assessment end points or indicators of impacts on botanical diversity and associated food chain and ecosystem services effects. The literature search will include all the main GMHT crops including maize, soya, oilseed rape, sugar beet, cotton and rice. The keywords will be broad and the search strategy is developed to capture all literature relevant to the primary objective of the review. The range of data bases for the searches is described and all articles discovered in the searches will be collated by Endnote. The criteria against which studies will be included in the review and how they will be assessed are described. They include appropriate study designs, statistical power and comparators. The RP outlines the type of analyses that will be performed to assess bias of the selected studies and if covariables describing the heterogeneity of the studies introduce biases. Publications meeting the selection criteria will be filed separately and subjected to more detailed analysis and data on different plant species and types will be analysed separately, to determine outcomes.

Background

Impact of GM Herbicide Tolerant (GMHT) crops

HT crops are widely grown in N and S America and are being developed for many other regions, including Europe. Since use of herbicides has been associated with declines in farmland biodiversity in some regions, there are concerns that GMHT crops treated with broad spectrum herbicides will also cause declines in biodiversity [1–3]. However HT systems have different levels of inputs and management interventions and some incorporate the use of minimal or no tillage. Research results show a range of effects depending on crop type, the specific herbicide tolerance, ai applied, dose, timing and number of herbicide treatments, tillage system and cropping system [4–12]. Consequently it is not clear how GMHT crop management is affecting botanical diversity or affecting diversity of other biota, either through food chain effects or directly, in agricultural land. In order to determine GMHT cropping effects on farmland biodiversity a starting point is to determine effects on botanical diversity since this will indicate likely food chain effects [13]. There are also concerns that repeated use of the herbicides used on HT crops will promote herbicide resistance development [14] and further inappropriate use of herbicides leading to reductions in biodiversity [15].

In agroecosystems, sustainable agricultural production, integrated crop production and integrated pest management are broad protection goals associated with sustainable food production, which embrace the exploitation of a range of ecosystem services which are considered desirable to protect. These include pollination, predation, nutrient cycling, etc.… These protection goals are relevant in relation to herbicide usage in conventional and GMHT crops.

Concept

The main concern is that the cultivation of GMHT crops will reduce biodiversity and adversely affect ecosystem services in farmland regions [16]. Conceptual models and logic maps for the most significant environmental issues associated with GMHT crops were distributed to stakeholders. At the stakeholder workshop on 23–24.4.2013 presentations were made explaining the conceptual models and the rationale for the systematic review of GMHT crops. Stakeholders (Appendix listed in Table 1) were invited to comment on our proposed approach and the review topics and questions. In addition comments were received by email. The questions were modified in response to these comments and clarifications provided for some aspects of our approaches to the topics. The updated review questions were returned to stakeholders for prioritisation by them.

Stakeholder concern

At the Stakeholder meeting of 23 & 24 April 2013 and subsequent consultation with stakeholders it was agreed that this topic was a high priority, raised public concern and was scientifically controversial because of the mixed results from field studies and cropping experience in different crops in different regions. The EC is currently considering applications to commercialise GMHT crops in Europe and this study will inform them of the current state of knowledge on this topic. The SR questions on GMHT crop effects were scored above 3.5 (out of 5) on average with the highest scores for scientific disagreement. The SR will initially focus on the question receiving the higher score for scientific importance, with the question on weed resistance to herbicides used in GMHT systems considered according to time availability.

Objective of the review

The conceptual model shows that the Intervention is the introduction of GMHT crops, the main stressors are the changes in the herbicide applications and management practices, particularly cultivation and tillage. The populations directly affected are graminaceous and broadleaved weed populations and the species diversity in these populations (Figure  1). The concerns are that these will change and/or decline leading to effects through the food chain and reductions in biodiversity of a range of species including those with important functional roles in crop protection and fertilisation, and also species of conservational interest.

Figure 1

Conceptual Model describing potential impacts on biodiversity caused by the herbicides and management of GMHT crops (compared with conventional crops).

Several narrative reviews of GMHT crops have been conducted and it is considered that a systematic review (SR) would bring some added-value in comparison with a narrative review. The SR would determine whether research studies and field data from commercialised GMHT crops show shifts in botanical diversity or declines in plant populations in GMHT compared with conventionally managed crops.

Primary review question

Are populations of plant species changed by management regimes applied to GM HT crops compared with conventional crop management?

The SR will review data from field studies of GMHT crops and compare effects of GM crop, herbicide regimes and associated management applied to HT crops and conventional crops for impacts on plant species populations in fields, as assessment end points or indicators of impacts on botanical diversity and associated food chain and ecosystem services effects.

Secondary question

Arising from this primary question is the secondary review question of whether there is evidence that weeds are developing resistance to broad spectrum herbicides as a consequence GMHT crop management.

Methods

We will perform a rigorous review of studies of plant populations in fields and field margins of GMHT crops by complying with CEE requirements for Systematic Reviews (SR) and the EFSA Guidance on Systematic Reviewing [6].

Search strategy

The aim of the literature search is to be as comprehensive as possible in revealing all available scientific and technical publications and reports which compare the environmental effects of GMHT and conventionally managed crops, with a specific focus on the effects on the botanical diversity and weed populations in farmland receiving the two comparable managements. The search will be indiscriminate and non-selective and include all the main GMHT crops being grown and studied including maize, soya, oilseed rape, sugar beet, cotton and rice. Commercialised GMHT crops such as maize, soya and rapeseed are being extensively cultivated in N and S America [17] with more recent introductions of cotton, sugar beet and alfalfa into USA and rapeseed into Australia. Field studies have been conducted extensively in these countries and experimental studies have been conducted in many European and other countries. From existing literature reviews [1–3, 12, 18–23] it can be anticipated that there is considerable literature on this topic.

The literature search strategy will capture the relevant literature for the systematic review by identifying as many relevant datasets as possible. The keywords will be broad and have a high-sensitivity and low-specificity in order to capture a high proportion of existing datasets. The search will include abstracting databases and reviews, as well as full text databases.

Search terms

Terminology for the effects of the management of genetically modified herbicide tolerant crops will consider the main herbicides used in GMHT systems (glyphosate and glufosinate). Other herbicides such as dicamba 2,4-D, and other sulphonyl ureas will also be captured in the search but not specifically searched for as these GMHT systems have only been recently been developed and it is not likely that there will be sufficient data to review separately. The search terms will be selected from the following relevant terms.

Population terms

• 1. Weed*

• 2. “Weed species”

• 3. “Weed control”

• 4. “Weed population*”

• 5. “weed cover”

• 6. “weed biomass”

• 7. “Weed diversity”

• 8. “Weed community”

• 9. “Weed resistan*”

• 10. diversity

• 11. Biodiversity

• 12. Botanic*

• 13. “herbicide resistan*”

• 14. “herbicide toleran*”

• 15. “Weed seed”

• 16. “seed bank”

• 17. “Seed rain”

• 18. HT

Intervention terms

• 19. “Herbicide management”

• 20. Roundup*

• 21. “Liberty Link”

• 22. “Glyphosate toleran*”

• 23. “glyphosate resistan*”

• 24. “Glufosinate toleran*”

• 25. “glufosinate resistan*”

• 26. Till*

• 27. Cultivation*

• 28. “Rotation”

Outcome terms

• 29. Sample

• 30. Quadrat

• 31. Plot

• 32. “weed abundance”

• 33. “weed densit”

Others

• 34. trial

• 35. Field

• 36. Farm

Search strings

The search strings for the literature databases consist of 3 parts (linked with AND). Terms in each part are linked with “OR”. Asterisks are used to include terms which are plural or have additional letters. Quotation marks denote multi-word terms (e.g. “herbicide toleran*”).

Part 1: (“herbicide toleran*” OR “herbicide resistan*” OR “glyphosate toleran*” OR “glufosinate toleran*” OR “glyphosate resistan*” OR “glufosinate resistan*” OR HT).

Limits the query to the HT crops for which this review is conducted.

Part 2: (field* OR plot* OR trial* OR farm* OR sampl* OR quadrat*).

This provides data from field studies as the terms refer to methods used in field studies.

Part 3: (weed* OR “weed species” OR “weed population” OR “weed abundance” OR “weed biomass” OR “seed bank” OR “seed return”).

This provides the entry for the following full search string:

(“herbicide toleran*” OR “herbicide resistan*” OR “glyphosate toleran*” OR “glufosinate toleran*” OR “glyphosate resistan*” OR “glufosinate resistan*” OR HT) (field* OR plot* OR location* OR trial* OR farm* OR sampl* OR quadrat*) (weed* OR “weed species” OR “weed population” OR “weed biomass” OR “weed cover” OR “seed bank” OR “seed return”).

Search terms for full text literature databases (e.g. Google Scholar, Scirus), are more restricted in order to avoid large amounts of non-relevant data.

Part 1: (herbicide* OR tolerant OR resistan* OR GM OR GE).

Part 2: (field* OR plot* OR location* OR trial* OR farm*).

Part 3: (weed* OR seed OR “seed bank”).

Part 4: (sample OR “weed abundance”).

The primary searches will be in English. Searches in other languages (e.g. Spanish and Portugese as these are relevant for S America where there is extensive cultivation of GMHT crops) will be conducted using search engines that contain non-English information (e.g. Google Scholar). One example might be to use international terms for GMOs and products, such as.

OGM OR glyphosate OR glufosinate

The precise format of the search strings will be adapted to the requirements of each database and other search words listed above will be used to broaden the search or in case some relevant papers contain none of the words in the main search strings. The search strings used will be recorded and descriptions of how each database was searched will be provided.

Data bases for the searches

Google scholar (search engine) and ISI Web of Knowledge were used for the original scoping exercise and were cross referenced against some recent literature reviews on HT crops. For the results of the scoping see below.

The systematic review will search the following abstracting literature databases:

• Web of Science (ISI Web of Knowledge 1900 – 2013) (Thomson Reuters, New York, USA), contains peer reviewed scientific publications in English.

• CAB Abstracts (1984–2013) (CABI, Wallingford, UK), database that includes local and non-English publications.

• AGRICOLA (1970–2013) (National Agricultural Library, U.S. Department of Agriculture, Beltsville, USA): contains records of materials in agricultural and related sciences in the National Agricultural Library and cooperating institutes.

• AGRIS (1975–2013) (Food and Agriculture Organization of the United Nations FAO, Rome, Italy): International System for Agricultural Sciences and Technology provides an international bibliographic database of national, and international centers.

Full text search engines:

• GOOGLE SCHOLAR: online search engine of scientific and other scholarly works.

• JSTOR: Digital library of academic journals, reports, books, and other primary sources.

NTIS (ProQuest), Cochrane Library, SCOPUS (for scientific reports), Medline & EMBASE may also be considered.

The following webpages containing information on effects of GMOs will also be searched:

• Gmo-safety.eu: webpage established by the German Federal Ministry of Education and Research describing research projects and with links to reports etc.

• EFSA.europa.eu: webpage of the European Food Safety Authority containing information related to GMO applications in EU.

• EU GMO Register (http://gmoinfo.jrc.ec.europa.eu/). Lists all deliberate releases of GM plants for any experimental purposes.

• International Survey of Herbicide resistant weeds [24]. Available at http://www.weedscience.org.

Review papers

Previously published review papers on GMHT crops will be listed and screened for any references that were not picked up in literature searches.

Direct contacts

Scientists or organisations that have conducted field studies of GMHT crops will be contacted directly to obtain any relevant data. All contacts and responses will be recorded.

Non-English searches

Full-text databases, such as Google Scholar and Scientific Electronic Library Online (http://www.scielo.org) are also suitable for non-English searches (e.g. Spanish, Portugese).

Field studies of GMHT crops in South America and Spain published in English often contain references to Spanish and Portuguese language reports of field studies. These will also be examined for relevance to the systematic review. Any papers that appear to be relevant will be sent to: Professor Ramon Albajes, Universitat de Lleida, Department of Plant Production and Forest Sciences

Ramon has conducted studies of GMHT crops and is in a good position to assess the quality of studies in Spanish and Portuguese against the inclusion criteria.

Field studies of GMHT crops have mostly occurred in N and S America, Australasia and Europe and so very few reports in Oriental languages are anticipated. However if they are revealed by the search and appear relevant steps will be taken to obtain translations.

Data base of publications

All publications discovered in the searches will be collated by Endnote. Subsets will be listed according main topic. Publications meeting the selection criteria (see below) will be filed separately and subjected to more detailed analysis.

Scoping the literature

Preliminary searches made between February and October 2012 using the search terms listed above in Google scholar have revealed 318 articles on GMHT crops of which 51 scientific papers and reports contain comparative data, including 21 papers studying effects on biodiversity, 15 papers on ecological impacts, and 23 papers on weed resistance development. The papers and reports identified were compared with those listed in recent review papers of GMHT crops e.g. [2, 15, 18]. The published reviews did not contain papers on GMHT crops that were not also identified in the scoping exercise. It is anticipated that a more comprehensive and systematic search using the full range of search terms and the data bases listed above will reveal more publications particularly as there are several recent comparative studies of GMHT crops commercialised in N and S America.

The quality of the studies has not been assessed but the volume of existing studies indicates that there is sufficient published information to justify a systematic review of the literature. The review will also reveal where there are gaps in the type of data collected, which in turn will identify where conclusions cannot be reached or conclusions are based on data subsets which might relate more to particular forms of bias.

Study inclusion criteria

The inclusion criteria are as follows:

Population: non-crop plant populations within fields/plots including field margins and cropped areas.

Intervention: the cultivation management of a GMHT crop which includes the use of the herbicide to which it is tolerant and tillage.

Comparator: a conventional crop (i.e. not GM HT) of a similar variety grown using conventional cultivation and weed management techniques in the same or immediately adjacent plots/fields.

Outcome: changes/differences in the non-crop plant and/or seed bank populations occurring in the GMHT and conventionally managed crops.

Study design

• The studies must be comparisons between conventional weed management and GMHT management systems preferably in studies with split field or paired field systems or plots in fully randomised and replicated systems with sufficient replication or statistical power to show differences at <50% level. The compared fields should be receiving other management inputs which are similar and in similar rotational cropping systems.

• The reviewed articles contain original data that has not been published elsewhere. Reviews, summaries, abstracts, comments papers, etc. will be listed separately and examined for their sources of information, data and bibliography (see Search strategy).

• Data includes sample sizes, plot/field means and variation (SE, SD), for each treatment.

• the study provides original data in the form of tables, figures or directly from the authors if requested.

• Data are presented for each sampling time and not pooled through the season or year.

• Weed data from studies of non-GM HT systems (e.g. Clearfield) will also be compared with weed data from GMHT crops and conventional crops.

The Comparisons should be of measurement end points (outcomes) such as the following:

Measurement end points: Populations and abundance measured in the field

• a. Weed Totals,

• b. Broad leaved weeds species and numbers

• c. Graminaceous weeds species and numbers

• d. Identified plant species totals

• e. Weed seed bank, species populations and total numbers

• f. feed Seed rain, species numbers and total numbers.

Methods

• Plants assessed as plant numbers/m2 counted after herbicide applications and prior to harvest (earlier season data could also be looked at but the later timing is correlated with weed seed production and return).

• Seed assessed as seed rain data collected prior to or at crop harvest, and/or seed numbers/m3 soil or germinated seedlings/m3 soil collected from soil auger sampling, or germinated seedlings/m2 soil in field after harvest of HT crop but before any weed management of the next crop.

• If continuous HT cropping systems are studied then annual and accumulated data of these parameters will also be assessed.

The GMHT systems may consist of:

• Glufosinate (glu) or glyphosate (gly) only used at Manufacturers Recommended rate

• glu or gly only used at other rates and application timings,

• glu or gly at various rates and timings used in programmes with other herbicides or weed management practices (eg hoeing)

• glu or gly used in conjunction with minimum or no tillage systems.

Each treatment difference will be taken into account when analysing comparative effects.

The conventional (comparator) systems should be any conventional management that reflects routine normal or local farm practise. Conventional weed management will also include non-spraying methods such as mechanical hoeing, tillage etc., and the management practices used in each study will be documented.

Where the effects of the comparable weed management systems could be influenced by changes in other management factors such as crop rotation, irrigation, fertilizer treatments etc., data will be analyse separately.

Screening references: applying inclusion criteria

The references revealed by the searches described above will be entered into to Endnote X5 (Thomson Reuters). Separate Endnote files will be established for each search engine for a record of each search. The files will then be combined into one database which will eliminate duplicates automatically, though this will be checked manually. Some search engines do not allow transfer of articles to Endnote so that the search results will be copied or transfered to another file and checked manually. The inclusion criteria will be applied to titles and abstracts to remove unwanted references. The result will be an Endnote X5 database containing the potentially useful articles. These data will also be transferred to CADIMA (http://www.cadima.info/index.php?r=area/centralAccessPoint) for storage, evaluation and future availability.

The selected articles will have full texts extracted and examined further according to the inclusion criteria. Any articles excluded at this stage of the study for the systematic review will be documented in an Excel table and reasons for exclusion provided.

Screening references: quality assurance process

In this screening process, a random subset of 10% of the articles identified by the search procedure will be independently assessed by a second team member, applying the same inclusion criteria. This will be done at the abstract phase and full text phase as the abstracts often do not describe the type of comparative study conducted and the nature of the data collected. The results of the independent assessments will be analyzed and documented using Kappa statistics (http://www.vassarstats.net/kappa.html). Articles excluded in one assessment, but included by the other reviewer will be recorded and the reasons for the differences will be discussed by the review team. If the kappa-value falls below 0.6 indicating significant differences, a review will be undertaken so that inclusion criteria can be revised and tested for improved reviewer agreement and to optimise the screening process.

Study quality assessment

In order to assess the reliability of the presented results, each study will be assessed for the following possible sources of bias:

• Selection bias. This can occur in field study design whereby the design of the study eg the plot size, herbicide treatment timings, sampling timing and method may favour one treatment.

• Performance bias. Systematic differences between groups e.g. exposure to factors other than the intended intervention, such as the change in other management inputs (e.g. fertilizer, pest control, irrigation) associated with one treatment but not the other.

• Detection bias (the way outcomes are measured differs between treatment groups, application of statistical analysis).

• Attrition bias may occur in differences in sampling times following herbicide treatment, frequency, sizes between study groups, selective choice of parameters, and in missing data.

• Reporting bias (e.g. preferential reporting of positive or desired outcomes).

Studies will be examined for a clear hypothesis and designs which are appropriate to achieve outcomes which test the hypothesis by testing both for similarities (equivalences) and differences.

The following sources of bias and influences on the studies are identified:

• 1. Research experimental studies conducted on farm will tend to have farmers using normal managed weed control on conventional crops which give acceptable levels of weed control (to the farmer) while optimising inputs and costs. By contrast the management of the GMHT crops will tend to follow a rather prescriptive or formulaic protocol for the herbicide applications as requested by the researcher and/or biotech company. Thus there will be performance bias in these studies and they will contrast with studies conducted on farms where GMHT systems are well established. In this latter case farmers will be familiar with the requirements of GMHT crops and will thus optimise their inputs. Thus the effects on weed populations may be different between experimental studies and field studies of commercial GMHT crops.

• 2. In areas where there are concerns about weed resistance development (eg where there is intensive and repeated use of GMHT crops), farmers will tend to go for maximum weed control to prevent survival of resistant weeds and so this performance bias may result in different GMHT treatments from those where resistance is not perceived as a problem.

• 3. Sequential cultivation of GMHT crops is likely to have different consequences than rotational cultivation and so this should be evaluated separately in the analysis.

• 4. Experimental and Reporting bias is likely as some studies will be sponsored or conducted by interested organisations (eg: agrochem Cos, biotech Cos) or by organisations antagonistic to GMOs or modern agricultural practices, and therefore there may be some selection of recorded parameters, data, results or reports to show a particular effect or show a “desired” effect or result.

For example v.low or zero weeds would be considered a desirable result by an Agchem company but an adverse effect (reduction in botanical diversity) by an environmental organisation.

The study quality assessments will be independently assessed, as in the references screening process, by a second team member, applying the same quality criteria. The results of the independent assessments will be analyzed and documented using Kappa statistics (http://www.vassarstats.net/kappa.html). Studies assessed differently by reviewers will be recorded and the reasons for the differences will be discussed by the review team. If the kappa-value falls below 0.6 indicating significant differences, a review will be undertaken so that quality criteria can be revised and tested for improved reviewer agreement and to optimise the quality assessment process.

The levels of bias in studies will be characterized and studies which contain bias which has clearly influenced results will be assessed separately and recorded in the systematic review. This will be documented to ensure transparency and reconstructability.

Data extraction strategy

Original data will be described in each article, the method of extraction (including contacts with authors) will also be provided. Any estimated or calculated values will be described or explained. Where presented data are insufficient or inadequate, authors will be directly contacted and requested to provide the appropriate data.

Data will be stored in CADIMA that will be built for this purpose. Tables in Excel format can be easily extracted from the database.

Random samples of 20% of the extracted data will be checked by the second member of the review team.

The variables extracted and recorded are shown in Table 3 in Appendix and will be similar to those proposed by Meissle M, Naranjo SE, Riedel J, Romeis J: Does the growing of Bt maize change populations or ecological functions of non-target animals compared to the growing of conventional non-GM maize? 2014: Submitted to Environmental Evidence. This is in order to have conformity and similarity of data bases for later extraction and interrogation purposes.

Data analysis

The extracted datasets will be collated and analyzed and a narrative summary of the extracted outcomes will be produced. Tables and figures illustrating and summarizing the evidence will be produced to show the products and outcomes of the literature analysis. The data will be grouped and presented in different ways in order to allow a range of analyses and demonstrate a range of outcomes. For example:

• Number of years of study.

• Study year.

• Location of study.

• Crop type.

• Spatial scale of experiment.

• Experimental design.

• Commercial field studies.

• In crop or field margin.

• Taxonomic groups of weeds (e.g. graminae, broadleaved).

• Types of weeds (e.g. annual, perennial).

• Parameters measured.

• HT system studied (glufosinate or glyphosate).

• Studies comparing GM HT crops with minimal tillage vs conventionally cultivated and treated crops.

• Weed herbicide resistance.

Suitable quantitative datasets will be analyzed statistically as described below.

For each measurement endpoint the effect size and direction in relation to comparator (ie + or – ve) will be assessed.

Statistical meta-analyses will be conducted for data sets of populations of weed taxonomic groups occurring frequently in studies with similar designs and measured parameters. The scoping exercise indicates that there may be sufficient similar data sets for conducting meta-analyses, but this will not be confirmed until a qualitative evaluation of these data sets has been conducted. The range of weed types and parameters analysed will depend on the quality and amount of appropriate data to allow quantitative assessments.

For meta-analyses, the following inclusion criteria will be applied:

• Clearly defined and similar parameters (e.g. abundance of weed taxa) were measured in the field under similar circumstance of time in the season

• The taxonomic group of the weed measured was clearly described.

• Sampling strategy (e.g. size, frequency, timing) is given for each treatment.

• Raw data as well as plot, treatment and site means are presented together with a measure of variation (SE, SD).

• Convention crops and their normal routine herbicide treatments are compared with GMHT crops and their specific herbicide treatments (e.g. gly and glu).

• Mean measures of effect and variance are reported in the article or can be acquired from the author.

• Data are recorded on an annual/seasonal basis (pooled and cumulative data sets from more than one year may be separately analysed).

Measures of treatment effect

The weed response variable to herbicide treatments is abundance. If enough data are available, response variables of individual weed species and weeds of certain taxonomic groups will be analysed. The mean effect size is calculated as the difference between the effects on weed abundance of herbicide treatments applied to GMHT crops and the herbicide treatments applied to conventional crops. This is reported is Hedges’ d weighted mean response, and is calculated as the mean response divided by a pooled standard deviation and multiplied by a correction term to allow for small sample size bias.

GMHT and conventional crop fields will be compared for

• Different crop species

• different weed species and groups

• herbicide resistance development

• geographical regions

• different herbicide treatments

• differences in commercial crop studies and experimental studies

Where there are comparisons of the effects of GMHT systems with those of HT systems for crops developed through non-GM techniques (e.g. imidazolinone tolerance developed through mutation breeding), separate analyses will be performed comparing the effects on weed populations of the GMHT and non-GMHT herbicide programmes.

Assessment of statistical power of included studies

For each measurement parameter in a study the statistical power of the treatment effect will be assessed either as an ex-post calculation of the power of the study for the effect, or as a prospective power of the study to detect pre-established significant effects magnitudes (e.g. the power of the study to detect a significant 10%, 20% and 50% impact effect on the outcome).

Data which is linked or consequential or sequential will be assessed separately and relationships studied. Interactions with other external factors such as management, season and environment will be considered.

Unit of analysis issues

A data issue that is likely to occur is the multiple use of the same dataset for different comparisons (for example the Farm Scale studies conducted in UK, [21,25]. Data on higher taxonomic levels or groupings may include data derived from other studies on individual species and so might result in multiple use of the same data. In addition data on a species or effect may be used from a study designed to study another different species or effect. The design may therefore not be appropriate or optimal. Secondary datasets will be identified and related to primary (original) data in the database during the data extraction process. The number of datasets reused in analyses will be recorded and discussed. Analyses might be conducted including and excluding multiple use datasets to determine their impacts as described below.

Dealing with missing data, and non-interpretable data

Where the nature of the data is not apparent or the article has introduced methods for allowing for missing data or there are missing or unusual/unlikely values, then attempts will be made to contact the authors or others who may be able to comprehend and interpret the data sets. Adjustments for missing data will only be made when the reasons for the absence are clear.

The reviewers will note data that is missing or has been adjusted or substituted.

Synthesis

Quantitative synthesis

The outcomes of the systematic review and meta analyses, in terms of the sizes of effects on weed abundance in comparisons of GMHT systems with control crop systems, including 95% confidence intervals, will be presented in tabular and graphic form. Total numbers of observations and effect sizes that are significantly different greater or less than 0 will be shown. Negative effect sizes are associated with lower abundances of weeds in GMHT crops compared with non-GMHT controls.

The nature and quality of the results will be discussed and conclusions drawn where appropriate.

Assessment of heterogeneity

The heterogeneity of the data across studies will be examined using appropriate statistical methods. The effects of studying different weed taxa, different regions, crop managements (minimum tillage, herbicide application methods, rotation), commercial and experimental scales, and experimental designs will be examined for their influence on variation.

Sub sets of studies containing similar parameters such as tillage, herbicide timing will be separately assessed to see how these influence outcomes. Where it is identified that certain parameters cause high heterogeneity, separate analyses of data subsets will be conducted.

Sensitivity analyses will be conducted to determine whether the outcome of the systematic review and meta-analyses might be influenced disproportionally by:

Studies that provide a high proportion of the datasets used in the meta-analyses.

Studies with high levels of precision, statistical power and/or replication which will tend to have higher weightings.

Studies with low levels of statistical power, poor replication or high variability which diverge from weighted means to a greater extent than studies with high precision, power or low variability.

The funding sources for the overall results.

studies where final weed numbers were measured in comparison to studies where weed abundance was measured at different times.

Data on particular common, rare or unevenly distributed weed taxa.

Assessment of publication bias

Effect sizes from different studies will be compared for articles produced by authors or journals with different funding types. Consistent differences may indicate a publication bias or a bias depending on funding source of a study (see 3.3) or a Journal.

Appendix to “What are the effects of the cultivation of GM herbicide tolerant crops on botanical diversity? A systematic review protocol”

By Jeremy Sweet and Kaloyan Kostov

This appendix lists the people and organizations (Tables  1 and 2) involved in the consultations when developing the conceptual model, the prioritization of the topic and this protocol for a systematic review of the available information on the effects of GM herbicide tolerant crops on botanical diversity in farmland.

Table 1 List of participants to the stakeholder consultation workshop on good review practice in GMO impact assessment

Table 2 List of organisations submitting written comments

This appendix lists the variables extracted from scientific reports and papers for the systematic review on GMHT crops (Table  3) and which are used as the primary data base for analyzing the effects GMHT crops and their management on botanical diversity. The variable name in the database, the definition of the variable, the type, and whether the variable content is restricted to closed (predefined) terms are presented.

Table 3 List of variables extracted for the systematic review on GMHT crops