Ladybird beetles (Coleoptera: Coccinellidae) are abundant predatory species in many agroecosystems, are valued for their biological pest control functions, and have been recommended as test species for studies supporting the assessment of non-target effects of insect-resistant Bt crops. Although insecticidal Bt proteins are known to be highly specific against target pests, some recent laboratory studies reported putative toxic effects of Bt proteins on ladybird species. While such studies have been criticised because of methodological shortcomings or inconsistencies, they cast doubt on the insecticidal spectrum of activity of some Bt proteins. Performing a systematic review that synthesises all existing evidence on this controversial topic may help to resolve the remaining scientific uncertainties. The review question to be addressed by the systematic review is the following: Are ladybird beetles (Coleoptera: Coccinellidae) affected by Bt proteins expressed in genetically modified insect-resistant crops? The systematic review will focus on studies performed under controlled environmental conditions.
An extensive literature search will be conducted to identify the articles relevant to the review question. A wide range of electronic bibliographic databases, the internet search engine Google Scholar, and websites of specialized organizations will be searched. Citation searching, reference list-checking and searching of key journals will also be performed. The relevance of the identified articles will be assessed against a set of pre-defined eligibility criteria, following a two-step approach. In the first step, title and abstract (or summary) will be screened, whilst in the second step the full text of all remaining articles will be assessed by two members of the review team. All relevant studies will be subjected to an appraisal of external (generalisability) and internal (risk of bias) validity. Data from the selected studies will be extracted and synthesised in a narrative report. If a sufficient number of datasets generated with comparable experimental setup is available, statistical meta-analyses will be conducted on a range of comparisons and including sensitivity analyses.
Genetically modified (GM) crops have been cultivated around the world since 1996. In 2017, the area devoted to GM crops reached 189.8 million ha . More than half of this production involved crops that are engineered to express one or more insecticidal proteins (Cry or Vip proteins) from the bacterium Bacillus thuringiensis (Bt) for controlling lepidopteran and/or coleopteran pests. Before their commercial release, GM crops undergo an environmental risk assessment (ERA) to ensure that they do not cause unacceptable detrimental effects to the environment. One area of concern addressed in this risk assessment is the potential adverse impact on non-target arthropods (NTAs) and the ecosystem services they provide, including biological pest control by predators and parasitoids .
Ladybird beetles (Coleoptera: Coccinellidae) are important NTAs: they are abundant predatory species in many agroecosystems [3, 4], are valued for their biological pest control functions  and have been recommended as surrogate species for studies to support the ERA of Bt crops [6,7,8]. Most ladybirds feed preferably on aphids, which do not ingest significant amounts of Bt proteins . However, they also consume other available prey and, occasionally, pollen when prey is scarce , and therefore they can be exposed to Bt proteins under field conditions [7, 10]. Some species, such as Stethorus punctillum, are specialist predators of spider mites, which are known to contain high amounts of Bt protein when feeding on Bt plants, and are thus highly exposed to Bt proteins [11, 12]. Figure 1 describes a conceptual model on how insecticidal Bt proteins could result in harm to ladybird beetles when feeding directly on plant parts or indirectly when preying on herbivores that have ingested the Bt protein.
Insecticidal Bt proteins expressed in field-grown GM crops are known to be highly specific against target pests, and the scientific evidence accumulated with the assessment of these plants over the last three decades has demonstrated their environmental safety towards NTAs . However, some recent laboratory studies reported putative toxic effects of Bt proteins on some ladybird species [14,15,16]. Although such studies have been criticised because of methodological shortcomings or inconsistencies [17,18,19,20], they cast doubt on the insecticidal spectrum of activity of some Bt proteins and pointed to uncertainties that may be resolved by synthesising all existing evidence on this controversial topic.
Systematic reviews are evidence synthesis approaches that can be used to support the risk assessment of GM crops , especially when the available evidence shows different, or even, contradicting results. A common approach for synthesising and analysing the data collected from multiple individual studies included in a systematic review is to conduct a meta-analysis. Meta-analyses combine data from multiple studies, increasing the statistical power and overcoming the low level of replication of individual studies [22,23,24]. Although several meta-analyses on the effects of GM insect-resistant crops on different groups of NTAs have been published to date [25,26,27,28,29,30,31,32], none of them has focused particularly on ladybird beetles.
We intend to perform a systematic review on the effects of Bt proteins expressed in GM insect-resistant crops on ladybird beetles. Since the studies reporting putative toxic effects of Bt proteins on some ladybird species were conducted in the laboratory, the systematic review will focus on studies performed under controlled environmental conditions. The methodological quality of each relevant study will be assessed. If the number of datasets generated with comparable experimental setup is sufficient, statistical meta-analyses will be conducted.
Objective of the review
The objective of the review is to systematically search, critically appraise and synthesise all scientific literature on the effects of Bt proteins, i.e. Cry and Vip proteins, expressed in GM insect-resistant crops on ladybirds.
We plan to answer the following review question: Are ladybird beetles (Coleoptera: Coccinellidae) affected by Bt proteins expressed in GM insect-resistant crops?
Components of the review question
The review question has a PECO structure with the key elements consisting of:
Population All species belonging to the coleopteran family Coccinellidae, commonly known as ladybirds, ladybird beetles, or ladybugs;
Exposure Cry and Vip proteins (Bt proteins) expressed in GM insect-resistant crops for which there has been a regulatory approval for cultivation. This includes the Bt proteins as expressed in the GM plant and microbially-produced and purified Bt proteins (hereafter referred as to purified proteins);
Comparator Closely-related (non-Bt) plants for those studies using Bt plant material/tissue (e.g. leaves, pollen), or a negative control for those studies using purified Bt proteins (e.g. untreated artificial diet);
Outcome Effects of the Bt protein on life-history traits of the ladybird beetle (e.g. development, survival, reproduction) measured under controlled environmental conditions (i.e. laboratory and glasshouse studies).
Searching for articles
The search will be structured to reflect the population and exposure elements of the review question because both elements can be clearly defined and translated into broad search terms. On the contrary, the comparator and outcome components of the review question will remain open and not be present in the search string to avoid narrowing the search too much and thus missing relevant articles.
To enhance the sensitivity of the search, a wide range of search terms will be used (Table 1). Search terms will cover possible synonyms, related terms, acronyms, spelling variants, lay and scientific terminology and translation issues, and specific search functions, such as truncation, wildcards, proximity operators and quotation marks for multi-word terms. The Thesaurus search tools in Biosis Citation Index (Clarivate Analytics, Philadelphia, USA), CAB Abstracts (CABI, Wallingford, UK) and Zoological Record (Clarivate Analytics) were used to ensure that terms from the controlled vocabulary are included (e.g. coccinellidae, ladybirds, ladybugs). Only English search terms will be used.
The search strings will be composed of two parts, corresponding to the population and exposure elements of the review question, combined with the Boolean operator AND. Within each set, search terms will be linked with the Boolean operator OR (Tables 2, 3). This string will retrieve articles that contain at least one search term from each element.
The search strategy will be adapted to the different information sources, i.e. Web of Science (WoS) , Agricola , Open Access Theses and Dissertations , and Google Scholar (https://www.scholar.google.com). The search strings that will be used in the electronic bibliographic databases hosted in Web of Science and Agricola, AGRIS, Open Access Theses and Dissertations and Google Scholar are presented in Tables 2 and 3, respectively. Searches in Google Scholar will be performed using the open source software Publish or Perish (https://harzing.com/resources/publish-or-perish). We selected Publish or Perish because it eases the process to export the results retrieved in Google Scholar and import them into EndNote. Since nested Boolean operators do not work in Publish or Perish, several searches will be performed using the all the words command. The first 200 results from each search will be considered, as suggested by Haddaway et al. .
The searches are expected to retrieve articles written in English and articles written in other languages with at least a title, abstract/summary or keywords in English. Identified articles written in languages other than English will be translated as needed for further assessment of relevance and validity.
The search will be limited to those articles published from 1990 onwards. Since the commercial cultivation of GM crops started in 1996, the likelihood that relevant articles were published before 1990 is considered very low.
Estimating the comprehensiveness of the search
We performed a scoping exercise on 29 September 2017 with the aim of estimating the comprehensiveness of the search and validating the selected search string with the electronic bibliographic databases hosted in the WoS platform. For this purpose, 18 already-known relevant articles were selected: twelve articles that were listed in two reviews [20, 37] and six articles (published between 2014 and 2017) that had been previously identified by citation alert (see Additional file 1).
Individual searches were performed in five of the electronic databases that will be searched in the systematic review (i.e. WoS Core Collection, CAB Abstracts, Chinese Science Citation Database, BIOSIS Citation Index and Current Contents Connect), using the search string provided in Table 2. All retrieved lists of records were checked for the presence of the 18 relevant articles. The results of the scoping exercise are provided in the Additional file 2. WoS Core Collection and CAB Abstracts delivered the best results and were able to retrieve 16 articles each. All databases combined returned all 18 relevant articles. Therefore, the selected search string was considered valid. None of the 18 relevant articles were retrieved by the Chinese Science Citation Database. This was expected since none of them were published in journals indexed in the database. However, the Chinese Science Citation Database will still be searched for the systematic review since it might contain some other relevant articles which are not indexed in the other databases.
Electronic bibliographic databases
A broad range of relevant multi-disciplinary and subject-specific electronic bibliographic databases will be searched:
Web of Science (WoS) Core Collection (Clarivate Analytics). Contains most peer-reviewed scientific articles in English language. The following databases will be included: Science Citation Index Expanded, Conference Proceedings Citation Index—Science, Book Citation Index—Science, and Emerging Sources Citation Index;
BIOSIS Citation Index (Clarivate Analytics). Comprehensive reference database for life sciences;
CABI: CAB Abstracts (CABI). Comprehensive database that also includes more local and non-English articles;
Chinese Science Citation Database (Chinese Academy of Sciences). It provides scientific articles published in Chinese language (title and abstract in English);
Current Contents Connect (Clarivate Analytics). It contains peer-reviewed scientific articles in English language from different disciplines;
Data Citation Index (Clarivate Analytics). It contains data sets from a wide range of data repositories and connects them with the scientific literature to track data citation;
SciELO Citation Index (Sao Paulo Research Foundation for the cooperative publishing of open access journals on the internet). It provides access to scholarly literature in natural sciences, social sciences, arts and humanities published in leading open access journals from Latin American, Portugal, Spain and South Africa;
Zoological Records (Clarivate Analytics). It covers all aspects of animal research;
Agricola. It contains bibliographic records of materials acquired by the National Agricultural Library and cooperating institutions in agricultural and related sciences;
AGRIS (Food and Agriculture Organization of the United Nations). It facilitates access to articles, monographs, book chapters, and grey literature in the area of agriculture and related sciences;
Open Access Theses and Dissertations. Contains a collection of open access graduate theses and dissertations published around the world.
All databases, except Agricola, AGRIS, and Open Access Theses and Dissertations, are hosted in WoS.
Google Scholar, which is a web search engine that indexes peer-reviewed journals and other scholarly literature, such as books, conference papers, theses and dissertations, and technical reports, will be searched.
The websites of specialist organisations (regulatory agencies, industry organisations and civil society organisations) and web-based databases containing information on environmental effects of GMOs listed below will be searched.
Supplementary searches will be conducted to identify potential additional relevant articles not retrieved by other methods.
Citation searching will be conducted with those articles meeting the eligibility criteria and categorised as relevant after the screening process. All references citing each relevant article will be identified in WoS using the cited reference search tool (all eight WoS-hosted databases cited above will be considered) and screened.
Checking reference lists
The reference lists of those articles meeting the eligibility criteria and therefore categorised as relevant after the screening process will be checked.
Electronic-searching of key journals
The ten journals that yielded the highest number of records when performing the scoping exercise on 29 September 2017 will be checked electronically to identify very recent articles that have not yet been indexed in the electronic bibliographic databases. The journals are, in alphabetical order, Biological Control, Entomologia Experimentalis et Applicata, Environmental Entomology, IOBC/WPRS Bulletin, Journal of Applied Entomology, Journal of Economic Entomology, Pest Management Science, Plos One, Scientific Reports and Transgenic Research. The searches will be limited to articles published within the 12 months before the date of the search of the systematic review, and to articles published ahead of print.
A search update will be conducted if searches were performed more than 2 years prior to review completion. Search updates will be thoroughly documented and reported as described below.
Documenting and reporting the search process
The search will be fully documented and reported following the recommendations outlined in the European Food Safety Authority (EFSA) guidance on systematic review methodology and food/feed safety risk assessment , the EFSA explanatory note on literature searching  and the ROSES RepOrting standards for Systematic Evidence Syntheses . The ROSES form is included as Additional file 3. The following details will be recorded and reported in the systematic review:
List of all information sources (e.g. electronic bibliographic databases, organisational websites, hand searched journals) searched;
The full search strategy for each information source (copied and pasted exactly as run) and details of how each information source was searched will be recorded and provided in the systematic review;
The date on which the search was conducted;
The date of the most recent update of the database that was searched;
The date span of the search;
Any limits applied to the search (e.g. article types, dates);
The journal name, the journal URL (internet address) or publisher; the dates, volumes and issues searched; the method of searching, e.g. scanning tables of contents for each issue, or using a search engine; the search terms used (if any) (for electronic-searching key journals);
The number of articles identified through each information source and the final number of articles remaining after removing duplicates (automatically and manually). This information will be part of the flow diagram that includes information on the screening and critical appraisal and synthesis steps ;
Any deviations from the protocol in the search strategy, and their impact in the systematic review.
Article screening and study eligibility criteria
The articles retrieved by the literature searches will be transferred to EndNoteX9 (Clarivate Analytics). Retrieved articles will be tagged with the database provider and stored in one single EndNote file. Then, the articles will be combined into one file that will be exported to the systematic review software Distiller SR (Evidence Partners, Ottawa, Canada). Duplicates will be eliminated automatically. Remaining duplicates, if any, will be removed manually.
The relevance of the resulting articles after deduplication will be assessed manually following a two-step approach and applying the eligibility criteria listed in Table 4. In the first step, title and abstract (or summary) will be screened, whilst in the second step, the full text of all remaining articles will be assessed. At this second step, each excluded article will be documented with the reason for exclusion. Any articles for which their relevance still remains unclear after assessing the full text will be discussed by the review team to reach a consensus decision.
Screening consistency checking
Articles will be screened independently by at least two reviewers at the title, abstract, and full text level. To minimise any influence on eligibility decisions , reviewers will not be allowed to assess the relevance of any article they have co-authored. Any articles considered relevant by one reviewer and non-relevant by the second one will be documented and discussed to reach a consensus decision. If no consensus is reached, the opinion of another member of the review team will be sought.
From the articles retrieved by the different sources of scientific literature described above, only those articles fulfilling all the eligibility criteria listed in Table 4 will be considered relevant for the systematic review.
Pilot testing eligibility criteria
The validity of the eligibility criteria was pilot-tested on 6 June 2018, following the recommendations of Frampton et al. . For this purpose, a sample of 20 articles identified in the scoping exercise performed on 29 September 2017 was selected (see Additional file 4). This sample size is bigger than the size suggested by Higgins and Green  (i.e. 10–12 articles). The sample included articles which were thought by one reviewer to be definitely relevant (10 articles), definitely irrelevant (seven articles), and doubtful (three articles). Articles were selected randomly but ensuring that all categories were represented. A second member of the review team assessed all articles independently. Both reviewers agreed on the relevance of all 20 articles assessed. Therefore, the defined eligibility criteria were considered valid.
Documenting and reporting article screening and eligibility criteria
The relevance assessment will be fully documented and reported following the recommendations in Frampton et al. , the ROSES RepOrting standards for Systematic Evidence Syntheses , and the Collaboration for Environmental Evidence Guidelines and Standards for Evidence Synthesis in Environmental Management .
The following details will be recorded and reported in the systematic review:
Summary of the finally-agreed eligibility criteria, including the instructions given to the reviewers;
A flow diagram showing the results of the screening: number of articles screened, number of articles excluded upon screening titles and abstracts (for both reviewers); number of articles excluded at full-text screening; number of articles used in the systematic review;
Number of reviewers involved; whether screening decisions were independent; expertise of the reviewers;
Results of the assessment of reviewer agreement, and how disagreements were resolved;
A list of articles that were excluded at full-text (with reasons for exclusion);
A list of articles which had unclear eligibility status after completion of full-text screening (with explanation why they could not be classified).
A list of articles that could not be obtained for full-text screening;
The final list of articles eligible for the systematic review;
Any deviations from the protocol in the eligibility criteria or the screening process, and their impact in the systematic review.
Study validity assessment
Study validity criteria and process
Relevant studies will be assessed for both external validity (the degree to which the studies are appropriate or applicable for answering the review question) and internal validity (risk of bias). The validity of each study fulfilling the eligibility criteria will be evaluated against a set of criteria. The study validity criteria are divided into three groups according to whether they pertain to: (1) test substance; (2) test organism, and (3) study design. A list of potential criteria that will be considered and the conditions that are needed to fulfil each one of them are provided in Table 5. Further details about the scientific rationale followed to underpin the selection of the proposed criteria can be retrieved from Romeis et al. [20, 43].
For each study, all quality criteria will be assessed individually and categorised (e.g. fulfilled/low risk of bias; partially fulfilled/moderate risk of bias; not fulfilled/high risk of bias; not relevant; not assignable, in case that information is not provided or reported). An overall validity descriptor will be assigned to each study based on the results of the independent assessment of all criteria by two reviewers: (1) high validity/low risk of bias; (2) medium validity/medium risk of bias; (3) low validity/high risk of bias; or (4) not assignable.
In case that insufficient evidence is provided in the study to assess each of the validity criteria, corresponding authors will be contacted and asked for clarifications.
Details about the criteria assessed to determine the validity of the studies, the rationale followed to categorise each of criteria as well as the results of the assessment of each study will be provided in the systematic review. Information from the validity assessment of the eligible studies will be used in data synthesis; sensitivity analyses will be performed by comparing results with and without exclusion of studies with low and medium validity.
A similar quality assurance system to that established for the article screening process will be put in place. All studies will be assessed independently by two members of the review team. Reviewers will not be allowed to assess the study validity of studies they have co-authored. Inconsistencies or uncertainties between reviewers will be discussed to reach a consensus decision. If no consensus is reached, the opinion of another member of the review team will be sought.
Documenting and reporting study quality
The rationale behind the decision for each validity criterium and the overall external and internal validity of each study will be documented in the systematic review, as well as any deviation from the protocol.
Data coding and extraction strategy
The variables that will be extracted from relevant studies and recorded in an MS-Excel spreadsheet are listed in Table 6. The extracted data will be available as an additional file of the systematic review.
A random sample of entered data (at least 20% of the remaining studies after relevance assessment) will be checked by a second member of the review team.
The corresponding authors of those studies in which relevant data are not properly reported might be contacted; alternatively, data figures will be scanned using the open source software Plot Digitizer (http://plotdigitizer.sourceforge.net) and means and measures of within treatment variance will be estimated.
Potential effect modifiers and reasons for heterogeneity
Several potential effect modifiers and sources of heterogeneity were identified:
Protein type: Cry/Vip protein
Protein specificity: coleopteran-/lepidopteran-specific
Purified protein/plant-expressed protein
Ladybird beetle species
Stage tested: larvae, pupae, adults
Study type: laboratory/glasshouse; direct feeding study/indirect (prey-mediated) feeding study; artificial diet/plant study
Those effect modifiers will be part of the data extracted from the studies and recorded in the MS Excel spreadsheet. Additional effect modifiers may be added during the course of the systematic review. The influence of the potential effect modifiers causing heterogeneity will be investigated.
Data synthesis and presentation
The data extracted from all identified studies will be synthesised in a narrative report and in different tables, including information on the ladybird species tested, type of study (e.g. direct/indirect feeding assay with plant material/purified protein in artificial diet), characterisation of the test substance, exposure to the test substance, endpoints analysed, negative and positive controls and results of the statistical analysis. The data extracted will describe the body of evidence and will also serve to identify potential knowledge/data gaps.
If a sufficient number of datasets generated with comparable experimental setup is available, statistical meta-analysis will be conducted using the effect size estimator Hedge’s d for continuous variables (e.g., developmental time, pupal/adult weight) and odds ratio and/or risk ratio for dichotomous variables (e.g., survival/mortality).
Separate analyses will be conducted for different study types, type of Bt protein, spectrum of activity of the Bt protein, measurement endpoint, etc.
Sensitivity analysis will be conducted to explore the influence of individual studies, funding sources or affiliation of investigators for the overall result and study validity (i.e., by comparing results with and without exclusion of studies with low and medium validity), etc.
Publication bias will be estimated; effect sizes will be compared for articles with different funding and author affiliation types (see sensitivity analyses described above). Systematic differences would indicate a publication bias depending on funding source of a study or authors affiliation.
Availability of data and materials
ISAAA. Global status of commercialized biotech/GM crops: 2017. ISAAA Brief 53. 2017.
Devos Y, Romeis J, Luttik R, Maggiore A, Perry JN, Schoonjans R, Streissl F, Tarazona JV, Brock TCM. Optimising environmental risk assessments accounting for ecosystem services helps to translate broad policy protection goals into specific operational ones for environmental risk assessments. EMBO Rep. 2016;16:1060–3.
Meissle M, Álvarez-Alfageme F, Malone LA, Romeis J. Establishing a database of bio-ecological information on non-target arthropod species to support the environmental risk assessment of genetically modified crops in the EU. EFSA Support Publ. 2012;EN-334:1–170.
Riedel J, Romeis J, Meissle M. Update and expansion of the database of bio-ecological information on non-target arthropod species established to support the environmental risk assessment of genetically modified crops in the EU. EFSA Support Publ. 2016;EN-956:1–109.
Carstens K, Cayabyab B, De Schrijver A, Gadaleta PG, Hellmich RL, Romeis J, Storer N, Valicente FH, Wach M. Surrogate species selection for assessing potential adverse environmental impacts of genetically engineered insect-resistant plants on non-target organisms. GM Crops Food. 2014;5:11–5.
Romeis J, Meissle M, Álvarez-Alfageme F, Bigler F, Bohan D, Devos Y, Malone LA, Pons X, Rauschen S. Potential use of an arthropod database to support the nontarget risk assessment and monitoring of transgenic plants. Transgenic Res. 2014;23:995–1013.
Álvarez-Alfageme F, Ferry N, Castañera P, Ortego F, Gatehouse AMR. Prey-mediated effects of Bt maize on fitness and digestive physiology of the red spider mite predator Stethorus punctillum Weise (Coleoptera: Coccinellidae). Transgenic Res. 2008;17:943–54.
Dhillon MK, Sharma HC. Effects of Bacillus thuringiensis δ-endotoxins Cry1Ab and Cry1Ac on the coccinellid beetle, Cheilomenes sexmaculatus (Coleoptera, Coccinellidae) under direct and indirect exposure conditions. Biocontrol Sci Technol. 2009;19:407–20.
Schmidt JEU, Braun CU, Whitehouse LP, Hilbeck A. Effects of activated Bt transgene products (Cry1Ab, Cry3Bb) on immature stages of the ladybird Adalia bipunctata in laboratory ecotoxicity testing. Arch Environ Contam Toxicol. 2009;56:221–8.
Álvarez-Alfageme F, Bigler F, Romeis J. Laboratory toxicity studies demonstrate no adverse effects of Cry1Ab and Cry3Bb1 to larvae of Adalia bipunctata (Coleoptera: Coccinellidae): the importance of study design. Transgenic Res. 2011;20:467–79.
Rauschen S. A case of “pseudo science”? A study claiming effects of the Cry1Ab protein on larvae of the two-spotted ladybird is reminiscent of the case of the green lacewing. Transgenic Res. 2010;19:13–6.
Comas C, Lumbierres B, Pons X, Albajes R. No effects of Bacillus thuringiensis maize on nontarget organisms in the field in southern Europe: a meta-analysis of 26 arthropod taxa. Transgenic Res. 2014;23:135–43.
Naranjo SE. Impacts of Bt crops on non-target organisms and insecticide use patterns. In: CAB reviews. Perspectives in agriculture, veterinary science, nutrition and natural resources, vol. 4. 2009. p. 11.
Pellegrino E, Bedin S, Nuti M, Ercoli L. Impact of genetically engineered maize on agronomic, environmental and toxicological traits: a meta-analysis of 21 years of field data. Sci Rep. 2018;8:3113.
EFSA (European Food Safety Authority), Devos Y, Guajardo IM, Álvarez F, Glanville J. Explanatory note on literature searching conducted in the context of GMO applications for (renewed) market authorisation and annual post-market environmental monitoring reports on GMOs authorised in the EU market. EFSA Support Publ. 2019;EN-1614:1–62.
Haddaway NR, Macura B, Whaley P, Pullin AS. ROSES RepOrting standards for Systematic Evidence Syntheses: pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps. Environ Evid. 2018;7:7.
Romeis J, Hellmich RL, Candolfi MP, Carstens K, De Schrijver A, Gatehouse AMR, Herman RA, Huesing JE, McLean MA, Raybould A, Shelton AM, Waggoner A. Recommendations for the design of laboratory studies on non-target arthropods for risk assessment of genetically engineered plants. Transgenic Res. 2011;20:1–22.
Meissle M, Romeis J. Insecticidal activity of Cry3Bb1 expressed in Bt maize on larvae of the Colorado potato beetle, Leptinotarsa decemlineata (Coleoptera: Chrysomelidae). Entomol Exp Appl. 2009;131:308–19.
Raybould A, Kilby P, Graser G. Characterising microbial protein test substances and establishing their equivalence with plant-produced proteins for use in risk assessments of transgenic crops. Transgenic Res. 2013;22:445–60.
Duan JJ, Teixeira D, Huesing JE, Jiang C. Assessing the risk to nontarget organisms from Bt corn resistant to corn rootworms (Coleoptera: Chrysomelidae): tier-I testing with Orius insidiosus (Heteroptera: Anthocoridae). Environ Entomol. 2008;37:838–44.
Scoping exercise for validating the search string. This file contains the results of the scoping exercise that was performed with six electronic bibliographic databases to test and validate the search string.
Articles used to pilot test the eligibility criteria. This file lists the articles that were used to test the eligibility criteria.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Álvarez-Alfageme, F., Devos, Y., Muñoz-Guajardo, I. et al. Are ladybird beetles (Coleoptera: Coccinellidae) affected by Bt proteins expressed in genetically modified insect-resistant crops? A systematic review protocol.
Environ Evid8, 25 (2019). https://doi.org/10.1186/s13750-019-0169-z