Strategies for managing spring frost risks in orchards: effectiveness and conditionality—A systematic review protocol
Environmental Evidence volume 10, Article number: 32 (2021)
Spring frosts pose an important threat to orchard productivity in temperate zones and predictions of future occurrences do not exclude damaging events. However, there is no up-to-date and systematic comparison of the effectiveness and conditionality of the existing passive and active damage prevention strategies. This review seeks to answer the questions “How do the performances of spring frost damage reduction strategies in temperate fruit orchards compare?” and “How do environmental conditions affect frost damage reduction strategies in temperate fruit orchards?”.
Methods and output
The review covers on-site frost damage prevention mechanisms and interventions for the most important temperate pome, stone and citrus fruit trees as well as grapevine. Searches include the core collection and regional databases on the Web of Science platform, Scopus, as well as specialized libraries like Agris, Agricola, CAB Abstracts, Groenekennis and selected institutional websites. Included studies report on the effectiveness of at least one intervention in reducing spring frost damage or increasing temperatures in the field. The validity of the studies will be evaluated based on their risk of general research bias and on topic-specific, stakeholder co-designed validity criteria. Data will be extracted regarding the study setup (study design, location, characteristics of frosts) and the resulting temperature increase, crop development alteration and/or damage reductions. The effectiveness of interventions will be evaluated in terms of yields, damage to flowers, ambient temperature and/or crop development. An Evidence Atlas will identify general and geographic research gaps, as well as research trends. Meta-regression of effectiveness on environmental conditions will be attempted.
Spring frosts are among the most important causes of yield loss in fruit orchards in temperate regions . Frost damage occurs when the ambient temperature drops below a crop and development stage-specific critical threshold . A single frost night on 19th April 2017 led to apple yield losses of up to 78% in Europe compared to the preceding seven years (Fig. 1) .
Since the 1950s several fruit research stations across Europe report an earlier start of flowering  which could potentially lead to higher probabilities for frosts during the flowering period. However, in a warming climate, also the last frost day is occurring earlier in the year and regional studies arrive at contrasting conclusions over the evolution of the frost hazard during flowering. While some studies report on unchanged or increased frequencies [5,6,7,8,9,10,11,12] for certain regions, decreased risks were reported for other regions or by other studies [8,9,10,11,12,13,14,15]. Although studies differ substantially in the employed phenological models, climate models and assessment methodologies, most of them confirm that frost during flowering will remain a major risk factor for fruit production in warming temperate regions.
Research and interests
Similar reviews were conducted in the past, whereby the most prominent reference is a handbook for frost protection, published by the FAO and edited by Snyder and de Melo-Abreu . With the majority of the referenced studies dating prior to 1995, an updated and systematic review seems needed to summarise new research findings and identify gaps. Specialised reviews have recently been published on protection against frost by means of wind turbines  and on innovative sensor networks for early frost detection . Another recent and comprehensive summary on frost prevention techniques was published by ‘Les instituts techniques agricoles’ , which is geared towards practitioners and lacks academic referencing. In 2018 and 2019 the EIP-AGRI Focus Group collected knowledge of stakeholders and experts from across Europe on fruit protection against frost . Besides a characterisation of strengths and weaknesses of common passive and active interventions, a list of research needs was compiled. First and foremost it mentioned the need for “Studying and comparing the effectiveness of methods under different conditions” (p. 35) followed by “More biology to the models” and “Establishing a database on potential yields for different species/varieties and critical temperatures on species/variety level”.
The interests of stakeholders in this topic converge in a better assessment and understanding of the effectiveness of techniques. The targeted end-users of the review are the fruit producers and their advisors. Other major stakeholders are insurance businesses and governmental agricultural services at the regional, national, and international level as well as academic, national or regional fruit research centres. The Belgian ‘Proefcentrum fruitteelt vzw’, 3800 Sint-Truiden, Kerkom, (PCFruit vzw), a major experimental and applied research station for fruit production, was directly involved in the design of the present review protocol, regarding the selection of databases, search strings, population definition, data extraction and validity assessment criteria. An ultimate group of stakeholders are the consumers of fresh fruit who prefer fruits with low prices and high quality, implying no damage suffered from frost or frost damage remediation treatments.
Objective of the review
Farmers can choose between or combine different kinds of frost risk management strategies: financial (insurances, income diversification) or physical (windbreaks, foliar applications, sprinklers, wind machines, …). This review will focus on the latter kind, provided, that they affect either ambient temperatures or crop development.
This review seeks to answer two research questions. The primary one is “How do the performances of spring frost damage reduction strategies in temperate fruit orchards compare?” while the second is “'How do environmental conditions affect the effectiveness of frost damage reduction strategies in temperate fruit orchards?”.
Thereby the main objective is to compare the effectiveness of the different measures that target at least partially the reduction of yield losses due to spring frost. The identified environmental preconditions shall be used for a classification of the techniques as a basis for informed decision making in the fruit sector. Only tested interventions described in the published literature are considered, thereby excluding patents.
The review will aim to produce a comprehensive inventory of the available evidence of frost protection measures for commercially important temperate fruit tree orchards and vineyards. The studied fruits were selected from a list of the major temperate fruits  making use of expert knowledge (Table 1). The review will include all geographical locations that are under a temperate climate according to the Köppen-Geiger climate classification system , corresponding to the main climate type ‘C’, see Fig. 2.
We will limit the review to preventing frost damage during early development stages (after bud break in spring) and not during the harvest period (autumn) nor storage. We consider damages during dormancy as well as the temporal evolution or spatial distribution of frost events and associated losses beyond the scope of this review.
Several kinds of sources will be drawn upon for this review: academic literature from peer-reviewed journals as well as ‘practitioners’, ‘specialist’ or ‘grey’ literature comprising theses, institutional and governmental reports, which will be identified via dedicated search engines. The selection of relevant databases and journals was based on consultations with librarians of the University of Leuven and fruit research experts, and on the recommendations of the Collaboration for Environmental Evidence (CEE) .
Peer-reviewed articles will be searched for based on Title, Abstract and Keywords (Author based and generated) using the advanced search facility of the databases “Scopus” and the following collections in “Web of Science” (WoS): Core collection and regional collections covering publications from Korea (KJD), Russia (RSCI), China (CSCD) and Latin America (SCIELO), as detailed in Table 2, where the temporal coverage of every index is given. No period, document type or language will be excluded at this stage.
Libraries and databases specialised in agriculture including Agricola (United States Department of Agriculture), CAB Abstracts (Centre for Agriculture and Bioscience International), FAO Agris (Food and Agriculture Organisation), Groenekennis (University of Wageningen) will be searched for with time and index settings as detailed in Table 2.
For an article to be returned, at least one term referring to the review population (common name, Latin name or type of fruit) is needed and either a combination of highly specific terms like “frost” and a term of context or action—or a predefined association including generic terms like “cold” or “low temperature”. A successful combination could thus be “apple” AND “protection” AND “frost”. The search string was initially developed for the Web of Science (Table 3) and was then adjusted to the specific syntax of the other databases/search engines (Additional file 3). The Groene Kennis database includes (Dutch) classification tags, although only until 2015, including “fruitteelt” (Fruit production) and “beschadigingen door vorst” (frost damage), which will be searched for separately. By default, the search widget of the Groene Kennis database searches full text, leading to significant noise. With the URLs given in Additional file 3, only the Abstract, Title and Keywords are considered (Rob Genderen, Wageningen University, personal communication on 9/2/2021). Also the CAB Abstracts are searched using the descriptors "frost injury" and "frost protection" while the search in AGRICOLA is amended by searching by the more generic Code 4520 for “Miscellaneous plant diseases, injuries and control”.
Since the Groene Kennis and FAO Agris search results cannot be exported to a .RIS file at once, the bibliometric data will be scraped and written into .RIS files. Entries from FAO Agris will be filtered by the eligible languages at this step.
Estimations of the comprehensiveness of the searches
It became evident during preliminary scoping exercises that an important share of studies was published before 1990. This is confirmed by a recent review on wind-based protection against frost damage . Therefore, no date filters will be applied. The earliest records listed in the employed databases range from 1903 (Agricola) to 1955 (Web of Science databases) (Table 2).
Prior to the protocol definition, 149 references to articles dealing with frost protection at large were extracted from past reviews [1, 16,17,18]. Based on the title only, around two thirds potentially satisfy the eligibility criteria for the review (listed in Table 5). They served as a reference for the formulation of the search strings. A substantial part was retrieved through the Web of Science and the remainder -mostly grey literature- was traced back to their sources or libraries. These were added to the lists of additional databases and websites to be searched (Table 4).
Automated alerts for the defined searches will be activated in Scopus and Web of Science, in order to include papers published during the review period. The other databases will be queried again, including a filter for the publication year 2021 if possible, at the end of the data extraction phase and before the analysis phase.
In the international databases, searches are exclusively conducted in English, since we found that searching with German or French translations of the search strings did not have an added value, due to abstracts translated in English. Publications with English abstracts and full texts in Dutch, German, French and Spanish are thus considered as well. Including five languages in the review process reduces the language bias and the risk of missing out from key literature . While the most prominent publications stem from northern America and Europe, within the available resources we will include publications from fruit growing regions in temperate Asia and Latin America , considering their importance for fruit production (Fig. 2).
Google Scholar and Baidu Scholar are queried in English (see Table 1 in the Additional file 1), as a complementary source, but are not used as a primary source due to the lack of transparency of the sorting algorithm . The results will be restricted to the first 200 results.
National fruit research institutes in Germany (INRES), France (CTIFL), Italy (Laimburg Research Centre) and Spain (IRTA) were contacted to inquire about their relevant publications. The provided papers will undergo an eligibility check and will be listed accordingly as additional material.
Further, selected relevant institutional websites are searched for using simplified keywords in English, German, French, Dutch and Spanish (Table 4). This list may increase during the review process.
Article screening process
The screening process will be conducted by the authors of the protocol. In accordance with the CEE guidelines , several screening steps can be distinguished and the number of eliminated studies at every step of the screening process will be reported in a ROSES flow diagram using a dedicated tool .
Secondly, the title and abstract of every remaining article will be read. Articles will be removed if they do not fulfil one of the eligibility criteria (see Table 5). The online tool ‘Rayyan’ (https://rayyan.qcri.org/) is preferred for this step because it enables multiple users to assess papers and conflict solving. Every abstract will be read individually, but papers can be filtered by potential exclusion keywords (e.g. “mango”) which supports an efficient inclusion or exclusion of articles while recording the reason for exclusion, e.g. “wrong population” .
For consistency checking, a random selection of 10% of the articles will be reviewed by more than one reviewer and a Kappa score  will be computed. If a lower agreement than 0.6 is reached, the criteria definition and examples in Table 5 will be rediscussed and adjusted. In case no consensus can be found, a third reviewer will be consulted for mediation. The resulting list of excluded articles and conflict resolution documentation will be made accessible online and added as supplementary material to the review.
Thirdly, the included references will be gathered by means of a reference manager software  and full texts will be retrieved. Corresponding authors will be asked for their copy if there is no institutional access (incl. interlibrary loan). Articles without full text access will be listed separately and attempts made to retrieve the full text will be documented.
Fourthly, publications will be read at full text level. In case its abstract led to the inclusion of a publication, but the full text reveals ineligibility, the exclusion and reason thereof, will be documented in a separate table as additional material. At this stage 10% of the full texts will be reviewed by more than one author and conflicts solved with the mediation of a third reviewer. The results will equally be reported.
Articles will be retained or excluded based on the criteria listed in Table 5. Studies will be retained when they compare the effect of a given measure on the local temperature, the plant development (e.g. budburst delay), the condition/count of buds or flowers or yields against a control population. Articles that fail to meet one or more criteria will be excluded.
Study validity assessment
Studies in the environmental sciences rarely satisfy gold standards of internal and external validity simultaneously. Studies that do not satisfy certain criteria (e.g. min. sample size, random selection of experimental units) are considered of low validity for this study. This does not necessarily mean that they generally lack quality. Hence, existing critical appraisal tools for systematic reviews need to be adjusted for the context of environmental sciences . Studies not reporting the necessary details have an uncertain validity which in this review will be treated as low validity.
To avoid a false sense of hierarchy, no ranking of study set-ups will be undertaken, unlike systematic reviews in the medical field, where pre-defined rankings exist . Instead, the design of each included study will be assessed based on predefined quality criteria (Table 6). The first set of criteria is based on Bilotta et al.  where typical biases are listed regarding general aspects of the study design.
The second list of specific criteria was added in consultation with experts. Distinctive sections were designed for the judgement of sample sizes. In this review, the spatial application scale (experimental unit from which random sampling is possible) of an intervention can vary from an entire field (e.g. wind turbines) to a row (e.g. water sprinklers) or individual trees (e.g. foliar application), of which then trees or branches are sampled for analysis (observations). Considering the outcome measures, temporal replication at daily/nightly units within a field is possible if the outcomes are temperature measurements during or following an intervention, while in the case of analysing flower damage or yields repetitions of several days cannot be considered as independent treatments. Several crop cycles (years) are considered to be independent replicates. Considering the scale of the interventions, we differentiate between the application and assessment level, since not always all manipulated trees are assessed for damage.
All criteria formulations result from an interactive testing process with a subset of 20 papers which were independently assessed by two authors. This subset consisted of papers that were identified as likely eligible and directly accessible in the Web of Science Core Collection during the search string finetuning process. Ambiguous criteria were discussed and refined iteratively until there was a full consensus. The final criteria have been presented to the fruit research experts (PCFruit) and were approved with minor edits.
The validity assessment will be done on full texts of the eligible selection in a public project on the SysRev platform (https://sysrev.com). This software is preferred due to its efficient data extraction capabilities and is also suited for transparent conflict solving between multiple users . For copyright reasons, full texts will not be uploaded.
Based on the criteria in Table 6, a final judgement of the study validity as low, middle or high will be made, without subjectively judging some criteria more important than others. If the study reports on methods and results in a rigorous way, at most one criteria may be classified as having a high risk of bias. These studies will be classified as overall low risk of bias. On the other hand, overall high risk of bias is assigned if the study has major shortcomings in conducting and/or presenting the research. This is defined as having a risk for bias for at least 3 criteria. All other studies will be classified as overall medium risk of bias. A detailed assessment of individual criteria will be reported in the supplementary materials of the review.
Data coding and extraction strategy
Within the project environment on sysrev.com, a set of pre-coded fields, as well as several open fields, will be established to extract details on the study in a systematic manner. Four categories of data will be extracted. The first collects metadata on the study including the setting of the experiment. Secondly, details on the population and the intervention will be recorded. Thirdly, the conditions for effectiveness will be retained when they are provided. Lastly, the results will be recorded with any given measure of variance. Table 7 shows the pre-coded answers or specifications for the open text fields. If applicable, several options can be selected at once, or written as free text (separated by a semicolon).
The definition of codes results from an iterative process. The repeatability of this process was tested before the finalisation of the protocol. Two authors extracted the data for the subset of 20 papers used for testing the validity assessment. When there was substantial disagreement, codes were adjusted until a Kappa score > 0.6 was reached. Furthermore, the codes were reviewed by experts from the research centre PCFruit.
During the data extraction phase, a random selection of studies (at least 10%) will be cross-checked by another author, with special attention to the outcome variables and human errors. The extracted data (tidy extraction sheets) will be available as additional material.
No Data/ Not applicable/Other is always an optional answer. Missing information will be reported during the synthesis stage. The extracted data records will be added as supplementary information to the review.
Potential effect modifiers/reasons for heterogeneity
To our best knowledge, no protocol exists for studies of the effectiveness of frost protection measures. Every fruit type has a specific sensitivity to temperatures which varies throughout the development stages .
Further, the variation in environmental conditions, the scale of the fields, the distance to the operated technology, the resolution, accuracy and position of the temperature measurement devices and setup in open fields or controlled environments (e.g. greenhouse or lab) make the comparison between studies difficult. Table 8 provides a list of the most important factors that may influence the effects. The list has been compiled based on experiences of the Frost Inno project by the Belgian research centre PCFruit .
As additional information to the review, the dominant soil textural class will be identified based on a global soil database (http://www.soilgrids.org) using the soilDB package in R . When no data is available (urban areas), the nearest possible gridpoint will be retrieved. Data from the global Shuttle Radar Topography Mission  will be used for filling gaps regarding elevation.
Data synthesis and presentation
The research outputs will primarily be descriptive statistics of the identified publication and secondly a comparison of the performance of spring frost damage reduction strategies in temperate fruit orchards. The conditions for their effectiveness will be explained, to the extent that they are reported on in the selected literature.
A ROSES flow diagram will expose the body of literature found with the described search strings. The retained studies will be presented in an ‘evidence atlas’, as an interactive world map with datapoints comprising the collected metadata per study . This ‘atlas’ will indicate the number of included studies, the studied fruit types over time etc. These findings will be contrasted with maps of global fruit production based on FAO statistics. Timelines on the level of detail in reporting and bar-plots of studies by set-up or environmental conditions will complement the ‘atlas’.
The narrative synthesis will be supported by limited quantitative analyses of the identified intervention effectiveness by outcome category. Accordingly, effect sizes will be computed separately for measures of (i) temperature, (ii) crop yields, (iii) damage to flowers or buds or (iv) delay/advance in phenology.
Based on extensive scoping exercises we anticipate very low levels of detail regarding the reported variability of the results and foresee working with Raw Mean Differences (RMD) for temperature, flower damage and budding delays.
The convention of the RMD is defined as Xintervention − Xcontrol, with X being the mean outcome measures per experiment. A positive RMD is an indication of an effective intervention in the case of temperature and bud break dates and of an ineffective/counterproductive one for the damages to flowers and buds. Comparisons between studies will be possible without standardisation since in all cases the scales are inherently meaningful (temperature in °C, and flower survival in %).
Studies reporting on final harvests (yields) as outcome measures (kg per tree or unit area) need to be pre-processed. First, all results will be converted to kg/tree, using the reported planting density or row distances. Still, these yields per tree cannot be compared directly between studies on different fruit types, given their various normal yield values. The Response Ratio (RR) will be computed instead as a relative measure of change independent of the order of magnitude . The RR is computed as the Xintervention/Xcontrol, with X being the mean outcome measure per experiment. Values between 0 and 1 indicate a negative impact of the intervention on yields, a 1 indicates no effect and values > 1 indicate a positive effect of the intervention. Due to the asymmetry of the RR, a natural logarithmic transformation is used to make the scale symmetric .
Studies that fail to provide mean differences between populations in any of the outcome categories will be excluded from quantitative data synthesis.
We will further conduct linear mixed model regression of the effectiveness against reported characteristics of the fields and frost events. Since information on the precision is not available, this regression is an exploratory analysis only. Hypothesized contributing fixed factors include continuous variables like the recorded minimum temperature during the intervention, the elevation and latitude of the test sites and soil texture (given in % sand, clay or silt). Random factors include the study identifier, the fruit type and the development stage during which frost occurred.
Analysis of effectiveness and reported costs will be attempted, as this information is of high relevance for fruit farmers.
Sensitivity analysis to the study validity will be conducted, leaving out low validity studies, in order to test if low validity studies report higher or lower effect sizes.
All described analyses will be conducted in R language. The script will be provided as additional material.
As in all scientific domains, there is a higher likelihood to publish positive results than negative ones and studies with statistically significant findings are also more likely to be cited and by consequence to be found. Commonly funnel plots are used as means to express the publication bias. Without an estimation of variances only the sample sizes can be used for constructing these plots, acknowledging the limitations of this approach  .
Availability of data and materials
This article will be published open access, under the terms of the Creative Commons Attribution 4.0 International License http://creativecommons.org/licenses/by/4.0/). Unrestricted use, distribution, and reproduction in any medium are permitted provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Snyder RL, de Melo-Abreu JP. Frost protection: fundamentals, practice and economics vol. 1. Rome: Food and Agriculture Organization of the United Nations; 2005 (Environment and natural resources series).
Miranda C, Bilavcik A, Chaloupka R, Dreisiebner-Lanz S, Gastol M, Luedeling E, et al. Phenology and critical temperatures. 2019. (EIP-AGRI Focus Group Protecting fruit production from frost damage). Report No.: 5. https://ec.europa.eu/eip/agriculture/sites/agri-eip/files/fg30_mp5_phenology_critical_temperatures.pdf. Accessed 15 Feb 2021.
WAPA. European apple and pear crop forecast. Brussels: World Appel and Pear Organisation; 2018.
Legave J-M, Guédon Y, Malagi G, El Yaacoubi A, Bonhomme M. Differentiated responses of apple tree floral phenology to global warming in contrasting climatic regions. Front Plant Sci. 2015;6:1054. https://doi.org/10.3389/fpls.2015.01054/abstract.
Kunz A, Blanke MM. Effects of climate change on fruit tree physiology—based on 55 years of meteorological and phenological data at Klein-Altendorf. Acta Hortic. 2016;1130:49–54.
Unterberger C, Brunner L, Nabernegg S, Steininger KW, Steiner AK, Stabentheiner E, et al. Spring frost risk for regional apple production under a warmer climate. PLoS ONE. 2018;13(7):e0200201.
Vitasse Y, Schneider L, Rixen C, Christen D, Rebetez M. Increase in the risk of exposure of forest and fruit trees to spring frosts at higher elevations in Switzerland over the last four decades. Agric For Meteorol. 2018;248:60–9.
Chmielewski F, Müller A, Küchler W. Climate changes and frost hazard for fruit trees. Ann Meteorol. 2005;41(2):488–91.
Pfleiderer P, Menke I, Schleussner C-F. Increasing risks of apple tree frost damage under climate change. Clim Change. 2019;157(3):515–25.
Sgubin G, Swingedouw D, Dayon G, García de Cortázar-Atauri I, Ollat N, Pagé C, et al. The risk of tardive frost damage in French vineyards in a changing climate. Agric For Meteorol. 2018;250–251:226–42.
Chmielewski FM, Götz K-P, Weber KC, Moryson S. Climate change and spring frost damages for sweet cherries in Germany. Int J Biometeorol. 2018;62(2):217–28.
Meier M, Fuhrer J, Holzkämper A. Changing risk of spring frost damage in grapevines due to climate change? A case study in the Swiss Rhone Valley. Int J Biometeorol. 2018. https://doi.org/10.1007/s00484-018-1501-y.
Eccel E, Rea R, Caffarra A, Crisci A. Risk of spring frost to apple production under future climate scenarios: the role of phenological acclimation. Int J Biometeorol. 2009;53(3):273–86.
Hoffmann H, Rath T. Future bloom and blossom frost risk for malus domestica considering climate model and impact model uncertainties. PLoS ONE. 2013;8(10):e75033.
Molitor D, Caffarra A, Sinigoj P, Pertot I, Hoffmann L, Junk J. Late frost damage risk for viticulture under future climate conditions: a case study for the Luxembourgish winegrowing region. Aust J Grape Wine Res. 2014;20(1):160–8.
Hu Y, Asante EA, Lu Y, Mahmood A, Buttar NA, Yuan S. A review of air disturbance technology for plant frost protection. Int J Agric Biol Eng. 2018;11(3):21–8.
Diedrichs A, Bromberg F, Dujovne D, Brun K, Watteyne T. Prediction of frost events using machine learning and IoT sensing devices. IEEE Internet Things J. 2018;5(6):4589–97.
Hirschy M. Gel et grêle en viticulture et arboriculture - état des lieux des dispositifs de protection contre les aléas climatiques. Paris: Acta – les instituts techniques agricoles; 2020. p. 56. https://hal.archives-ouvertes.fr/hal-02769435/document. Accessed 18 Aug 2020.
EIP-Agri Focus Group. Protecting fruit production from frost damage. 2019.
Smith LG, Somerset SM. Fruits of temperate climates|Commercial and Dietary Importance. In: Caballero B, editor. Encyclopedia of food sciences and nutrition. 2nd ed. Oxford: Academic Press; 2003. p. 2753–61. http://www.sciencedirect.com/science/article/pii/B012227055X005307. Accessed 20 Nov 2020.
Kottek M, Grieser J, Beck C, Rudolf B, Rubel F. World Map of the Köppen-Geiger climate classification updated. Meteorol Z. 2006;15(3):259–63.
FAO. FAOSTAT. 2021. http://www.fao.org/faostat. Accessed 16 June 2021.
Haddaway NR, Macura B, Whaley P, Pullin AS. ROSES RepOrting standards for Systematic Evidence Syntheses: pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps. Environ Evid. 2018;7(1):7.
Collaboration for Environmental Evidence. Guidelines and standards for evidence synthesis in environmental management. In: Pullin AS, Frampton GK, Livoreil B, Petrokofsky G, editors. 2018. (Version 5.0). https://environmentalevidence.org/information-for-authors/. Accessed 4 June 2021.
Neimann Rasmussen L, Montgomery P. The prevalence of and factors associated with inclusion of non-English language studies in Campbell systematic reviews: a survey and meta-epidemiological study. Syst Rev. 2018;7(1):129.
Gusenbauer M, Haddaway NR. Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Res Synth Methods. 2020;11(2):181–217.
Haddaway NR. ROSES_flowchart: R package and Shiny app for generating systematic review and map flow diagrams. Zenodo; 2020. https://zenodo.org/record/4294810. Accessed 11 Mar 2021.
The EndNote Team. EndNote. Philadelphia: Clarivate; 2013.
Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplication of database search results for systematic reviews in EndNote. J Med Libr Assoc JMLA. 2016;104(3):240–3.
van der Mierden S, Tsaioun K, Bleich A, Leenaars CHC. Software tools for literature screening in systematic reviews in biomedical research. ALTEX Altern Anim Exp. 2019;36(3):508–17.
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.
Bilotta GS, Milner AM, Boyd IL. Quality assessment tools for evidence from environmental science. Environ Evid. 2014;3(1):14.
OECD. Test No. 509: Crop Field Trial. OECD; 2009 (OECD Guidelines for the Testing of Chemicals, Section 5). https://www.oecd-ilibrary.org/environment/test-no-509-crop-field-trial_9789264076457-en. Accessed 13 Feb 2021.
Vanhoutte B, Remy S. FROSTinno: Innovative and energy efficient frost control in fruit growing. pcfruit. https://www.pcfruit.be/en/frostinno-innovative-and-energy-efficient-frost-control-fruit-growing. Accessed 15 Feb 2021.
Skovlin JM, Beaudette DE, Roecker SM, Brown AG. Soil database interface. 2021.
Earth Resources Observation and Science (EROS) Center. Shuttle radar topography mission (SRTM) 1 arc-second global. U.S. Geological Survey; 2017. https://www.usgs.gov/centers/eros/science/usgs-eros-archive-digital-elevation-shuttle-radar-topography-mission-srtm-1-arc?qt-science_center_objects=0#qt-science_center_objects. Accessed 14 June 2021.
Haddaway NR, Feierman A, Grainger MJ, Gray CT, Tanriver-Ayder E, Dhaubanjar S, et al. EviAtlas: a tool for visualising evidence synthesis databases. Environ Evid. 2019;8(1):22.
Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to meta-analysis. Chichester: Wiley; 2009. https://doi.org/10.1002/9780470743386.
Higgins JPT, Tianjing L, Deeks JJ, editors. Chapter 6: Choosing effect measures and computing estimates of effect. In: version 6.2 (updated February 2021). Cochrane; 2021. (Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page M, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.2 (updated February 2021)). https://training.cochrane.org/handbook/current/chapter-06. Accessed 14 Aug 2021.
Sterne JA, Egger M. Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. J Clin Epidemiol. 2001;54(10):1046–55.
The authors wish to thank Thomas Vandendriessche, Sylvette Vanderstraeten and Krizia Tuand, the biomedical reference librarians of the KU Leuven Libraries–2Bergen (Leuven, Belgium), for their help in setting up and conducting the systematic literature search; as well as Wim Van den Noortgate, Faculty of Psychology and Educational Sciences, KU Leuven and Koenraad Van Meerbeek, Department Earth and Environmental Sciences, KU Leuven, for advice regarding the data synthesis design.
The protocol benefitted substantially from input by the Research Centre for Fruit (PCFruit vzw), 3800 Sint-Truiden (Kerkom), Belgium.
BD was funded by Fonds Wetenschappelijk Onderzoek Vlaanderen (Belgium), grant number 1S34920N. BB was funded by a predoctoral scholarship KU Leuven with internal reference number 3E200801.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests. Authors of research studies included in this review will not be involved in any decisions regarding their own work. Procedural independence is guaranteed as none of the authors has (co-)authored studies which could be included in this review.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Drepper, B., Bamps, B., Gobin, A. et al. Strategies for managing spring frost risks in orchards: effectiveness and conditionality—A systematic review protocol. Environ Evid 10, 32 (2021). https://doi.org/10.1186/s13750-021-00247-7