Skip to main content


Table 5 Proposed criteria used to assess the validity of relevant studies

From: Are ladybird beetles (Coleoptera: Coccinellidae) affected by Bt proteins expressed in genetically modified insect-resistant crops? A systematic review protocol

Group Criterion Conditions to be met
Test substance Characterisationa The test substance is properly characterised and described. For studies with purified Bt proteins information on the source, purity, nominal concentration, batch/lot number, solvent used (vehicle), etc. should be provided. For studies with Bt plant material, information on the transformation event, the hybrid/variety, Bt protein expression levels, etc. should be given
Biological activitya There is sufficient evidence that the test substance remains biologically active when provided to the test species. Biological activity can be demonstrated, for instance, by performing sensitive insect bioassays (e.g. Meissle and Romeis [44])
Equivalencea,b The test substance is biochemically and functionally equivalent to the Bt protein expressed in the GM plant. An overview of suitable methods to determine the equivalence between microbe- and plant-produced insecticidal proteins is given in Raybould et al. [45]
Stabilitya,b The test substance remains stable once incorporated into the artificial diet to ensure consistent exposure over the course of the study. Stability can be monitored by measuring Bt protein concentration throughout the duration of the study. Stability of the test substance can also be ensured by replacing the diet at regular intervals (e.g. Raybould and Vlachos [46])
Homogeneitya,b The test substance is homogeneously distributed in the artificial diet to ensure that test organisms are not able to avoid the test substance altogether or are exposed to lower than expected Bt protein levels. Homogeneity of the diet can be determined, for instance, by analysing subsamples of the diet (e.g. Duan et al. [47])
Test organism Life-stage testedc Life-stages need to be selected that are most likely to be susceptible to the Bt protein protein and thus are most likely to detect an adverse effect. In general, neonates are more sensitive than later instars (Glare and O’Callaghan [48])
Study design Negative controlc The study includes a suitable negative control. This is essential to separate any background effects of the test system from effects due to the test substance (e.g. a diet identical to the test diet in all respects except the test substance in studies with purified Bt proteins, or a near-isogenic line in studies with Bt plant material)
Concentration/dose selectiona The test organism is continuously exposed to the test substance throughout the duration of the study under worst-case conditions (i.e. ≥ 1 × EEC)
Test substance ingestiona There is sufficient direct or indirect evidence that the test species has ingested the test substance. Ingestion can be confirmed directly by immune-assays such as ELISA, or indirectly, with the inclusion of a suitable positive control (e.g. Li and Romeis [12], Álvarez-Alfageme et al. [17]) or weighing the test organisms or food before and after feeding
Measurement endpointsc Measurement endpoints are suitable to be evaluated in the laboratory/glasshouse and likely to indicate the possibility of adverse effects (e.g. mortality, fecundity, development duration, body mass, or the percentage of individuals that reach a certain life-stage)
Test durationc The duration of the test considers the measurement endpoints, the biology and the life-stage tested of the test organism, and the characteristics and mode of action of the test substance
Experimental conditionsc The experimental conditions (e.g. temperature, humidity, light:dark conditions) are appropriate for the test organisms and similar between the control and the treatment groups
Sample sizec The sample size and the amount of missing data is similar between treatments
Statistical designc The study employs a sufficient number of samples and replicates (e.g. based on power analyses), randomises treatments, ensures independence of observations and uses appropriate statistical methods
  1. Bt: Bacillus thuringiensis; GM: genetically modified; EEC: expected environmental concentration; ELISA: enzyme-linked immunosorbent assay. Transformation event: insertion of DNA into the plant genome as a result of a single transformation process
  2. aCriterion to assess external validity (generalisability)
  3. bCriterion mostly relevant for studies with purified Bt proteins
  4. cCriterion to assess internal validity (risk of bias)