Skip to main content

What evidence exists on the effectiveness of the techniques and management approaches used to improve the productivity of field-grown tomatoes under conditions of water-, nitrogen- and/or phosphorus-deficit? A systematic map

A Systematic Map Protocol to this article was published on 12 July 2019



Agriculture is facing an unprecedented challenge in having to reduce its environmental footprint whilst ensuring food security to an ever-growing global population. Towards this end, several strategies have been investigated and implemented to help maintain or improve crop yield under reduced water and/or nutrient provision for key commercial commodities such as tomatoes. Despite the high commercial, nutritional, and food-cultural value, there is no synthesis of evidence regarding yield maintenance of tomato (as a model crop) under resource-deficit. This systematic map therefore provides an overview of the evidence that exists on the effectiveness of techniques and management approaches aimed at improving the productivity of field-grown tomatoes under conditions of water-, nitrogen- (N) and/or phosphorus (P)-deficit.


Following the published map protocol, systematic searches of peer reviewed- and grey-literature were conducted using research publication databases, and specialist websites. A total of 14,377 unique articles were identified as potentially relevant to our research question, of which 927 were screened at the full-text level. Of that subset, 291 articles met all the pre-defined eligibility criteria. Basic information and meta-data on the interventions reported were recorded for these articles and a systematic map was compiled with the extracted data.


The articles included in the systematic map database were used to identify several significant points including: (1) from the year 2000, the number of articles investigating strategies to improve field-grown tomato yield under conditions of water and/or nutrient deficit follows an upward trend; (2) large evidence bases (> 50%) originated from the United States, India, and Italy; (3) most studies addressed water alone as a resource (49%), with only 18% of studies focussing on N and 4% on P alone. Only 4% of records assessed all three resources simultaneously; (4) most evidence (77%) aims to improve resource use-efficiency via either irrigation, fertilisation, or crop and soil management strategies; and (5) different geographical regions appear to focus on different groups of interventions.


This systematic map identifies a range of interventions that have been successfully implemented in fields to improve the yield of commercial tomatoes under conditions of water, N and/or P deficit. However, only half of the relevant literature reported evidence on more than one intervention, which highlights the need for more integrated approaches to assess multiple interventions to adapt to deficits of key-resources simultaneously. In addition, the use of ‘techno-chemical’, ‘breeding and genetic’ and ‘computational’ interventions are only reported in a small number of records (< 8% of the gathered evidence). Hence, these interventions may also be considered as subjects to prioritise in future funding strategies.


One of the greatest challenges of the twenty-first century lies in the need for the farming sector to adopt new measures to help address the consequence of intensive agriculture. The impacts of intensive cropped systems are categorised as major contributors to greenhouse gas (GHG) emissions, soil quality degradation, and biodiversity loss. This scenario emerges at a time when food demand is increasing due to population growth that is expected to peak at ca. 9.8 billion by 2050 [1, 2]; and must be satisfied during a period when the effects of climate change are manifesting as stochastic weather effects that threaten crop yields. More efficient use of water and key nutrients such as synthetic fertiliser nitrogen (N) and phosphorous (P) is coming under special focus as improved crop types and agronomic management practices are sought.

Large quantities of water (from rain or irrigation systems) are lost in-field by percolation, run-off, and evaporation before they can be acquired by crop plants [3]. In addition, due to global climate change, average rainfall is predicted to decrease, and temperatures expected to rise. This leads to an increase in plant evapotranspiration rates, and consequently a loss of water for biomass accumulation and yield [4]. Water is already a scarce resource in many parts of the world, and the water shortages are expected to intensify in the coming years. When these factors are allied to increased water demand, this resource will become increasingly sought, or even fought after and expensive [5]. Irrigation is also the source of several environmental concerns, such as the excessive depletion of water from subterranean aquifers, irrigation-driven erosion, and increased soil salinity. The demand for N and P fertilisers is also projected to increase steadily worldwide, despite fertiliser production being energetically costly, and resources being limited and unevenly distributed. On average, only 30–50% of applied N fertiliser and ca. 45% of P fertiliser is taken up by crops [6,7,8]. In European agriculture, soil leaching of N and P can reach 60% of applied amounts and activates microbial and algal growth in water reservoirs with negative impacts on water quality and biodiversity. Consequently, EU legislation has been adopted to strictly control the management and quality of agricultural water (Directive 2000/60/EC) [9] and the application of nitrate fertilisers (Directive 1991/676/CEE) [10]. Despite efforts by farmers to comply with these regulations, there is still a significant proportion of applied-water and -nutrients being lost. In general, the potential loss of irrigation water and fertilisers is greater in open field production than in protected cultivation as the latter allow for a better control of environmental factors. In a cropped system comparison study carried out in Spain, the irrigation water per ton of tomatoes was almost doubled in the open field system compared to that from protected cultivation [11]. Thus, there is an urgent need to identify alternative strategies to secure sustainable food production through resilient cropped systems that can maintain yields under conditions of low usage and/or multiple deficit-stresses.

Tomato is considered one of the most important vegetable crops worldwide due to its large cultivation area and economic value [12, 13]. It is grown in different cropping systems adapted to the different pedo-climatic conditions encountered in the EU. The tomato plant has many agronomically important traits such as fleshy fruit, a sympodial shoot, and compound leaves, which other model plants (e.g., rice and Arabidopsis) do not have [14]. In addition, the tomato belongs to the extremely large family, Solanaceae, and is closely related to many commercially important plants such as potato, eggplant, peppers, tobacco, and petunias. Hence, knowledge obtained from studies conducted on tomato can be easily applied to these plants [14]. Tomato serves as a genetic-model organism for the family Solanaceae and consequently, interventions applicable to tomato, such as the use of crop breeding, living manures, biostimulants, grafting, and variable rate technologies, may be easily transferred to other crops.

This systematic map focusses on field-grown tomato as a model field crop with management interventions focussing on water (due to resource scarcity), N (need for restricted use) and P (availability decline). In open field production, numerous strategies have already been adopted by farmers to improve the use-efficiency of these resources, such as drip-irrigation and the use of mulching [12, 15]. Others, however, remain under-studied and have rarely been put into practice on commercial farms either due to the difficulty in implementation and/or to economic uncertainties [16,17,18,19].

To the best of our knowledge, no efforts have been made to assemble and collate the large body of evidence describing approaches that enhance tomato production in the field with reduced use of fertiliser N or P and/or water. In this context, systematic maps offer a robust methodological approach to better understand the evidence that exists on the strategies tested, discriminating those which are used commonly, or neglected. The question addressed by our systematic map is already a main focus of the international scientific community as well as commercial entities and non-governmental organisations as exemplified by the EU funded projects (Rootopower, TRADITOM, TomGEM, and TOMRES), which use different approaches to improve the response of tomato plant as model crop to abiotic stressors resulting from climate change.

This systematic map provides a clear description of the available evidence on techniques and management approaches aimed at improving the productivity of field-grown tomatoes under conditions of water-, nitrogen- and/or phosphorus-deficit. It also identifies the topics that have drawn the most attention of researchers, highlighting not only knowledge-clusters, but also knowledge-gaps that should help inform future research efforts.

Stakeholder engagement

The focus of this systematic map was assisted through a specialist workshop held in Mallorca (Spain) in October 2018. Participants consisted of EU H2020 TOMRES project partners and stakeholders, which included experts in tomatoes from academia, research institutes and small and medium-sized enterprises (SMEs) from across Europe. This workshop helped to identify preliminary questions to initiate the synthesis of evidence. It also provided an opportunity to gauge the interest of the research question amongst potential users and research specialists. The discussion and feedback from this workshop helped to refine the aim and scope of the systematic map. Further details on this workshop are given in the systematic map protocol [20].

Objective of the review

The purpose of this systematic map is to provide a description of available evidence on techniques and management approaches addressing the effects of key resource limitations (water, N and P) on the yield of field-grown tomatoes. The question was characterised by specific criteria defined according to four key elements, known as the PICO elements, as follows.

  • Population: all/any tomato types (commercial crop),

  • Intervention: strategies addressing water-, nitrogen- and phosphorus- use-efficiency in open field conditions,

  • Comparator: current or standard practices or no water-, nitrogen- nor phosphorus- deficit stress,

  • Outcome: productivity effects in terms of yields and expressed either as fruit number or fruit weight.


This systematic map followed detailed methods described in the a priori systematic map protocol of Quesada et al. [20]. The selected mapping method conformed to ROSES reporting standards (Additional file 1).

Deviation from the protocol

Some changes to our original screening strategy were necessary. In particular, the title and abstract screening, and meta-data extraction and coding was performed by at least one reviewer instead of two due to resource limitation and time constraints. Instead, one assessor from the review team reviewed 10% of the studies being assessed by the other reviewers at each screening stage and meta-data extraction to ensure further criteria consistency and the quality of the meta-data extraction process.

In addition, although it was anticipated that secondary studies would be accepted, these were excluded and used as a source of additional eligible articles. This change was necessary to reflect the nature of secondary studies which summarise findings of primary studies, that may already be included in the map. Hence to avoid double-counting these were excluded (see Section “Bibliographies from relevant reviews section” below).

Search for articles

Search strategy

The search strategy was developed to maximise the coverage of the search and to ensure that the evidence captured was sufficient, comprehensive, and relevant to meet the guidelines provided by the Collaboration for Environmental Evidence [21] and address the research question objectively. The search strategy followed two processes:

  1. 1.

    Determining the most appropriate search terms to use for online literature searches; and,

  2. 2.

    Choosing key sources of literature for both published and unpublished studies.

Search terms and search strings

Key literature was obtained from experts in the field of tomatoes. A word frequency analysis within the key literature was carried out manually and using the Centre for Research in Evidence-Based Practice (CREBP). The search terms identified were then tested against a ‘test-list’ library comprising a set of 47 published articles and other literature known to be relevant to our research question. The following search string was used in bibliographic databases. The five elements of our questions were combined using the Boolean operators AND (terms must be found) and OR (at least one term must appear).

((TS = (tomato* OR lycoper*) AND TS = ((field* OR ground* OR land* OR soil$) AND (water OR drought$ OR nitr* OR "N" OR phosph* OR "P" OR nutrient$ OR "abiotic" OR “climate chang*”) AND (use$ OR uptake$ OR effici* OR optim* OR stress* OR defici* OR resistan* OR toleran* OR "arid" OR adapt* OR availab* OR "content" OR "amount")) AND TS = (yield* OR "production" OR productiv* OR weight$ OR "kg" OR "t" OR "biomass"))) AND LANGUAGE: (English) [shown as formatted for Web of Science Core Collection].

This search string was adapted to consider the specific requirements of each databases, grey literature and specialist websites used. All searches were recorded so the searches can be easily repeated in the future. Full details of the exact search terms and search strings including the string structure, date of the searches and the word frequency analysis (CREBP) are given in Additional file 2.

The search used title, abstract and keyword levels and was restricted to those articles available in the English language due to constraints of the review team. When the databases did not allow searches at title, abstract or keywords levels, searches were performed at full-text level. No time or document type restriction was applied.

Bibliographical databases and search platforms

This systematic map was based on literature searches conducted using five publication databases and search platforms: (1) CAB Abstracts (via Web of Science); (2) Web of Science Core Collection (consisting of the following indexes: SCI-EXPANDED, SSCI, A&HCI, CPCI-S, CPCI-SSH and ESCI. See Additional file 2 for the complete list of citation indexes); (3) Zetoc (British Library); (4) PubAg (USDA—National Agricultural Library); and (5) AGRIS (Agricultural Science and Technology Information Systems).

Search engines

Additional searches were undertaken using common internet search engines Google and Google scholar. For these searches, the software Harzing’s ‘Publish or Perish’ was used to retrieve and analyse literature [22]. For both search engines, the first 100 results were organised by relevance, selected, and assessed during the screening process. This was chosen as the cut-off point as it was considered likely to capture the most relevant article to our research question. The dates at which the searches were conducted for all databases and grey literature are given in Additional file 2.

Specialist searches including grey literature

Several specialist databases have been searched to identify non-peer-reviewed- or grey-literature, namely Open Grey, Worldwide Science, and Bielefeld Academic Search Engine. In addition, 14 specialist websites including those of relevant organisations were searched. A smaller selection of key terms and search stings were used and simplified as these sources were limited by their search functions (Additional file 2).

Bibliographies from relevant reviews and books

After having collected the literature from the different sources described above, we identified 37 relevant reviews (Additional file 3). We searched their bibliographic references manually, resulting in the identification of 30 articles of potential relevance to answer our research questions. We retrieved the pdf file of the selected titles and then screened their full-texts.

Testing comprehensiveness

Thirty published articles, provided by stakeholders, of potential relevance to our research question (i.e., they address abiotic stress and tomato crops) were screened against scoping search results to assess whether searches were able to identify these records (Additional file 2). Comprehensiveness was checked by how many articles were identified from the test library (26 out of 30; four missing articles were not available in the databases/search platform used).

Assembling a library of search results

After all searches were completed and the list of identified references collated, the results were transferred to the reference management software Endnote [23] and duplicates removed. The remaining articles were imported into the CADIMA software [24] as one database.

Article screening and study eligibility criteria

Consistency checks

On importing the references to the platform, a common consistency check was performed by the four reviewers using a random subset of articles at both the title and abstract levels to avoid bias and ensure criteria consistency between the reviewers. This subset of 100 articles were independently screened by all reviewers and results compared to determine whether the reviewers accepted, rejected or were unsure about the same titles/abstracts. At the end of the process, a kappa value of 0.85 was achieved [25] indicating an acceptable level of consistency among the reviewers. When disagreement arose, uncertainties were clarified and resolved, and the articles reassessed. This process was repeated at the full-text screening stage, with a random subset of 50 articles independently screened by the four reviewers to ensure continued objectivity when extracting meta-data from relevant articles. During this process, issues regarding the lack of clarity in which the coding categories should be applied and whether an article should be included or excluded were identified. Any disagreements were discussed, and additional, more detailed guidance was added to the extraction codebook to improve clarity. The reviewers did not author any of the articles screened or assessed for this systematic map.

Screening process

Subsequent assessment of the entire set of records (14,377) was undertaken independently by one of the four reviewers at the title stage, followed by an abstract screening of those considered ‘relevant’ or ‘unclear’. In addition, one assessor from the review team reviewed 10% of the articles being assessed by the other reviewers to ensure further criteria consistency. Articles which did not have an abstract were rejected. This process led to 1,618 articles being accepted for the full-text evaluation. Reasons for exclusion of articles at the various levels of screening are described in Fig. 1. The full-texts of each included article were then sourced and underwent full-text screening individually. The irretrievable full-text records, that may still be relevant in addressing the research question, are listed in Additional file 4. Articles found by means other than database or search engine searches (i.e., searches of reviews) were entered at the full-text stage of the screening process. Assessment of the potentially relevant records (n = 927) led to 291 deemed suitable to address the research question of this systematic map (Fig. 1). The reasons for excluding any articles at this stage were recorded (Additional file 5).

Fig. 1
figure 1

An overview of the systematic mapping process. The design and content of this figure follows ROSES guidance [26]

Full-text screening and meta-data extraction were performed independently by one reviewer. In addition, one assessor from the review team reviewed 10% of the articles being assessed by the other reviewers to assure consistency of eligibility decisions and the meta-data extraction process. Any disagreements were discussed, and to improve clarity more-detailed guidance was added to the extraction codebook. The finalised extraction form and codebook, that collate the variables and define the coding structure of the meta-data extraction, for the map, is shown in Additional file 6.

Eligibility criteria

To ensure consistent assessment and provide a reproducible methodology, inclusion and exclusion criteria were defined. Different eligibility criteria were applied at title, abstract, and full-text stages as detailed in Table 1.

Table 1 Eligibility criteria for each PICO elements applied at the title/abstract and/or full-text
Eligible population

The systematic map focusses on commercial tomato production worldwide i.e., without any geographically restriction. All non-commercial tomato production and other vegetables or crops were excluded.

Eligible intervention

All studies providing evidence of the effects of a technique, treatment, or management strategy on use-efficiency of either water, N, P, or combinations of these in open field production were included. These can be strategies that influence tomato production under conditions of water- or/and N- and/or P-deficit or use-efficiency of these resources. Studies in which the measure of exposure or occurrence was not directly related to water, N or P use, were excluded (e.g., salinity tolerance or waterlogging, where resource deficiency is brought about by other stressors). Studies that reported any water or nutrient use-efficiency strategies under controlled or protected conditions, such as in greenhouses or polytunnels, hydroponics or other forms of controlled-environment conditions were also excluded.

Eligible comparator

Usual or current practices before the introduction of the intervention were considered valid comparators. These include old, conventional, or standard practices that differ from the intervention of study in that they do not aim to improve tomato productivity under water- or/and N- or/and P-deficit stress or by enhancing the use-efficiency of these resources. Absence of the intervention studied was also considered a relevant comparator.

Eligible outcome

Only outcomes on productivity i.e., yield were included. These were expressed either by fruit number or weight. Studies reporting only on the effects on other crop outcomes, such as quality or nutritional properties were excluded.

Eligible type of study design

Primary experimental, quasi-experimental or observational studies were considered relevant. Studies that described a specific research method or reported on new empirical or observational evidence of an intervention effect were included. Secondary studies, that collate evidence/data, such as reviews and books, were excluded.

Eligible language

Only articles in English are included although we acknowledge that the decision to restrict the searches to texts written in English may introduce a bias by missing relevant articles that have been published in a different language, such as Chinese or Spanish (see Tower of Babel bias in the Limitations of the systematic map section).

Eligible date

The map includes articles from 1973 to 2019, although no time restrictions were applied.

Study validity assessment

No critical appraisals of the study validity were performed beyond coding the study design and data sources.

Data coding strategy

The meta-data were extracted by populating options (variables) within a specifically designed form and are included in the final database (Additional file 7). They describe the nature of the literature, variables related to the studied interventions that could influence their outcomes (e.g., temperature or precipitation) as well as variables that were reported systematically (e.g., geographical statistics). Data coding is divided into four categories: (1) general information i.e., literature database source, search string used, assessment date, and reviewer for each record; (2) bibliographic information i.e., publication type, authorship, year, journal, country of authorship; (3) basic information of the study i.e., region or country of assessment, geographic coordinates, farm type, study design, soil type, temperature and precipitation during the vegetation period, duration of the assessment; and (4) properties of the interventions assessed i.e., group and type of intervention, methodology of the study, stressor(s) assessed, impact on the resources assessed and outcome. Meta-data extraction was performed by four reviewers. During this process, it was not considered necessary to contact authors since all eligible articles reported all the required information regarding the interventions used.

Data mapping method

A systematic map database was compiled that describes the existing literature base on the techniques and management approaches used to improve the productivity of field-grown tomatoes under conditions of water-, N- and/or P-deficit (Additional file 7). The searchable and accessible database was created in Microsoft Excel and provides key characteristics of the research, including bibliographic information as well as all data coded as part of the meta-data extraction process (such as trials design, geographic location, intervention characteristic). The data extracted were analysed in Excel. The frequency of general characteristics of the articles (e.g., geographical location and year of publication), resource studied (e.g., water, N and /or P), and groups/types of interventions were evaluated in table forms using conditional formatting and heat maps to identify knowledge clusters and gaps. The systematic map results are visualised in a series of figures. For example, to show the geographic distribution of studies, a colour-coded world map is presented. A three-component Venn type figure was selected to show the number of articles focussing on water, and/or N and/or P as well as any combined stress studied. The interventions identified have been categorised in eight different groups (e.g. irrigation, fertilisation and crop and soil management) and represented in a pie chart figure. Each intervention has been further classified into sub-groups within each of the main eight groups. For example, interventions in the irrigation group have been further divided in 11 further sub-groups (i.e., types) such as drip irrigation, deficit irrigation or irrigation rate. These are shown in pie and bar chart figures as well as tables. The mapping of the results was performed to present the results obtained in a clear and simple manner and to facilitate the identification of knowledge clusters or gaps which directly relate to the primary research question.


Descriptive statistics

Figure 1 details the step-by-step results of the systematic mapping process according to ROSES guidance [26]. Overall, following removal of any duplicates, a total of 14,377 potentially relevant articles were identified through searches of specialist databases that curated peer-reviewed and grey literature. The results of all searches, including search strings, exact date of the searches performed between April and October 2019, subscription information is reported in Additional file 2. At the title screening stage, 9483 articles were excluded. A further 3276 articles were excluded following abstract screening, leaving 1618 for full-text screening, 691 of which were irretrievable (see Additional file 4 for the full list of irretrievable articles). At the full-text screening stage, 636 records were excluded for not meeting the eligibility criteria. A full list of these, with their reasons for exclusion are available in Additional file 5. A total of 291 studies were included in the systematic map database for descriptive narrative (Fig. 1). A list of all the studies extracted and the associated meta-data extracted (i.e., general, and bibliographic information) is included in the systematic map database (Additional file 7).

Studies per decade

Study publication dates ranged from 1973 and 2019, with the majority (72%) published after year 2007 (Fig. 2). From around the year 1993, relevant records began to appear consistently. From the year 2000, there is a clear upwards trend in the number of articles published investigating strategies to improve tomato productivity under conditions of water-, N- and/or P-deficit.

Fig. 2
figure 2

Periods of publication (decadal) for all relevant studies

Literature type

The vast majority (96%) of the studies in this systematic map were articles published in scientific peer-reviewed journals. There was a small number (< 4% each) from other types of publications such as book chapters, conference papers, university theses or technical reports.

Mapping the quantity of studies relevant to the question

Geographical location

The number of relevant studies from each country globally is highlighted in Fig. 3, and a large proportion of the relevant studies (> 71%, n = 208) were undertaken by only eight countries. The United States and India accounted for the highest proportion of studies (28 and 24%, respectively). Among the countries of Europe, Italy produced the highest number of articles (13%, n = 30). Several countries (i.e., Egypt, China, Nigeria, Turkey, and Saudi Arabia) each contributed between 5 and 6%. There were 16 countries for which there was only one relevant record. Overall, there were studies from 41 different countries across the world covering tropical, temperate, and arid biogeographical, or pedoclimatic, zones.

Fig. 3
figure 3

World map indicating the geographical distribution of evidence. The relative ‘number’ of records across countries are represented using a (blue) ‘colour density’ scale (0–60). Values are identified according to country of manuscript source, as determined by the correspondence address provided for the first- or lead-author. It should be noted that the country of authorship and the country where the studies were conducted were the same in 287 out of the 291 articles included in this systematic map

Study designs

Several pieces of information were recorded to capture the basic details of each study. Examples are listed below.

  • The duration of studies was mostly short. Thirty-five percent of studies only lasted for one season, while 49% of studies were either two or three seasons in length. There was only 5% of studies that lasted 4–5 seasons and just one study was more than five seasons long. Ten percent of the studies did not specify the length of the trial.

  • The experimental design of the relevant studies was mainly randomised block design (60%), followed by split plot design (19%) and factorial design (4%).

  • The main planting method was transplantation of plantlets, which was performed in 79% of studies, and only 3% of studies used direct seeding.

  • The soil texture information gathered showed that 16% of studies were performed on clay dominated soils, 42% on sandy soils, and 7% on soils with a significant silt composition. In 23% of the studies the texture was either not listed or described in an alternative manner. The pH was given as pH < 6 for 6% of studies, between pH 6 and 7 for 13% of studies and pH > 7 for 29% of studies, but for more than half of the studies the soil pH was not specified. Due to variations in the details provided, such as differences in units and soil classification systems, it was not possible to summarise or compare soil attributes easily.

Mapping the resource studied and impact on the resource

The research question of the systematic map focussed on the deficit of three resources: water, N and P (Fig. 4a). There was a small number of studies (n = 12), where all three resources were addressed. These mainly involved a combination of two interventions (n = 7, 58%) predominantly focussed on the use of irrigation and fertilisation type of interventions (42%). Other combinations reported included the implementation of irrigation strategies in conjunction with the use of biostimulants and biofertilisers (n = 1), mulching (n = 1) and soil amendment (n = 1). Only one study tested the use of three strategies (irrigation, fertilisation and crop and soil management). Overall, the greatest number of studies tested water use alone (49%, n = 144). In comparison, there were 52 studies on N use alone and 13 studies on P use alone. The number of records that included both N- and P-use was 18% (n = 54). Few studies reported on use-efficiency interrelationship of water and nutrients (5%, n = 15). These mainly focussed on the use of irrigation and fertilisation type of interventions (n = 8, 53%). Twenty-seven percent of these studies used a single intervention, 50% of which focussed on soil amendments in the form of biochar and manure.

Fig. 4
figure 4

a Number of relevant studies addressing field-grown tomato yield in response to conditions of water, nitrogen (N), and/or phosphorus (P) deficit. b. Number of interventions reporting an impact on either resource- ‘Availability’,-‘Uptake’, and/or -‘Use-Efficiency’

The interventions studied mostly assessed resource use-efficiency (50%, n = 147), followed by resource uptake efficiency (30%, n = 88), and resource availability (19%, n = 56) (Fig. 4b).

Mapping the interventions studied

Categorisation of relevant studies was undertaken by assigning each into one of eight intervention groups (Fig. 5). The largest intervention group was irrigation (~ 49% of all studies), followed by fertilisation (22%), and crop and soil management (16%). The five other minor intervention groups, each accounting for less than 10% of the evidence collected, included: (i) soil amendments; (ii) biostimulants and biofertilisers; (iii) techno-chemical; (iv) breeding and genetic and (v) computational approaches.

Fig. 5
figure 5

Number of relevant studies reported for each of the eight distinct intervention groups that aim to improve the productivity of field-grown tomatoes under conditions of resource deficit

More than 50% of the studies assessed only a single intervention. Two interventions were reported in 41% (mainly N and P based studies), and in only 8% of studies were there three interventions reported (Table 2).

Table 2 Number of studies and their relative proportion of relevant records that reported on the use of either 1, 2 or 3 interventions to improve the productivity of field-grown tomatoes under conditions of water-, nitrogen- and/or phosphorus-deficit

Mapping the irrigation interventions

In total irrigation was used as an intervention to study resource deficit 181 times (Fig. 5). Each of these interventions were divided in 11 different types (Fig. 6). Drip irrigation was the predominant type of irrigation intervention (n = 52), followed by deficit irrigation (n = 36), which together account for 50% of the relevant irrigation interventions. In contrast, limited evidence exists on the use of partial root-zone drying, alternative irrigation source/type, furrow irrigation, and sprinkler systems. Other types of irrigation interventions (n = 8) included studies on drain pitch depth, irrigation location, scheduling and lay-out.

Fig. 6
figure 6

Number of studies for each of the 11 different types of irrigation intervention identified

More than half of the studies (53%) were conducted in four of the main tomato producers that are the United States, India, and Italy, and Saudi Arabia. Most of the studies (93%) were conducted over one or 2–3 seasons.

Mapping the fertilisation interventions

Fertilisation interventions were the second most frequent group after irrigation, with 101 relevant studies identified. Five different types of fertilisation interventions were reported (Table 3). Fertiliser rate was the most common (n = 44) followed by the type of fertiliser (inorganic/ organic) used (n = 27). Ten studies reported on the application location (e.g., soil or foliar) as an intervention, while 11 articles studied the mode of fertiliser application. Other types of interventions (n = 4) refer to alternative fertilisation/fertigation systems and schedule of fertiliser application.

Table 3 Number of studies for each of the five different types of fertilisation intervention identified

Fifty-three percent of these studies were conducted in India and the USA, with another 20% carried out in Egypt, Italy, and Nigeria. The duration of the studies lasted mostly one season (36%) or 2–3 seasons (44%) and 48% of the experimental designs were randomised block design.

Mapping the crop and soil management, soil amendments and biostimulants and biofertilisers interventions

The use of crop and soil management (n = 72), soil amendments (n = 42) and biostimulants and biofertilisers interventions (n = 28) have been reported in 31% of the relevant studies identified. These are presented together for simplicity (Fig. 7).

  • Five different types of crop and soil management interventions were reported (n = 72). Cover crops and mulching were the two most frequent crop and soil management studies (n = 48 in total) reported. Limited evidence exists on manipulating planting density (n = 9) and the use of no-till/reduced tillage intervention (n = 4). Other crop and soil management interventions (n = 11) were reported, which included integrated nutrient management, different cultivation sites, transplanting date, and shading. The United States (n = 16), Italy (n = 13) and India (n = 11) accounted for more than 55% of these studies.

  • Soil amendment interventions were reported in 42 studies. These include the use of compost (n = 16), manure (n = 13), vermiculite/perlite (n = 2), biochar (n = 6), peat amendment (n = 1) and clay amendment (n = 3). Fifty percent of the evidence were published over the past 10 years and 79% were conducted in Asia or Africa (especially India and Nigeria) on mainly sand or a sandy loam type of soil. Only a few numbers of papers studied a soil amendment intervention over 4–5 seasons (8%), with most reporting their data after one season (36%).

  • There were three different types of biostimulants and biofertilisers interventions reported (n = 28): (i) plant-growth promoting rhizobacteria (PGPR); (ii) arbuscular mycorrhizal fungi (AMF); and (iii) acid. These were mainly studied in Europe and Asia (70%) over 1–3 seasons (76% of the studies).

Fig. 7
figure 7

Number of studies for each of the five types of crop and soil management, four type of soil amendments and three types of biostimulants and biofertilisers intervention identified

Mapping the techno-chemical, breeding and genetic and computational interventions

Due to the limited number of records collected, which reported on the techno-chemical (n = 9), breeding and genetic (n = 22), and computational interventions (n = 6), these are presented together for simplicity (Fig. 8). Most of these interventions (80%) are still relatively new as evidence only started to be regularly published over the past 10 years.

  • The techno-chemical groups of interventions included the use of anti-transpirants (n = 4), controlled/slow nutrient release (n = 2), pan evaporation and tensiometer readings (n = 2) and grafting techniques (n = 1). All these interventions used a randomised block design and the duration of the studies lasted between 2 and 3 seasons (67% of all studies). Seventy-eight percent of these studies were conducted in either the USA or Egypt.

  • Twenty-two studies were reported in the breeding and genetic group of intervention, which included 20 breeding studies and two genetic modification studies. The duration of the studies was between one and three seasons (72%), whilst 11% of the experiments were carried out over a longer period of 4–5 seasons.

  • Only a small number of computational or precision agriculture studies were reported (n = 6). These are concentrated in regions where high tomato productions is recorded, that are the USA, India, Italy, and Saudi Arabia.

Fig. 8
figure 8

Number of studies for each of the three type of techno-chemical, two types breeding and genetic and one type of computational interventions identified

Mapping the impact of all interventions on fruit quality traits

The impact of interventions on fruit quality traits was reported in 87 studies which represents approximately 30% of the total (291). Eight different fruit quality traits (pH, total soluble solids, sugar, vitamin C, lycopene, fruit length, fruit diameter, acidity, pericarp thickness) were reported in these studies (Fig. 9). The fruit quality trait reported most frequently was total soluble solids (n = 73). A smaller, yet appreciable, number of studies reported on pH (n = 28), acidity (n = 43) and vitamin C (n = 22). In comparison, few studies reported on quality traits, such as sugar, lycopene, fruit length, fruit diameter and pericarp thickness.

Fig. 9
figure 9

Range of tomato fruit quality traits assessed by the relevant studies (in addition to their impact on field-grown tomato yield)

Limitations of the map

Limitations of the map due to the search strategy

Firstly, the searches were limited to the English language. This may introduce the ‘Tower of Babel’ bias wherein authors generally choose to publish significant results in English [27]. However, a substantial volume of literature exists in other languages, such as Chinese, Spanish, and Portuguese. Therefore, more work to include other languages could be done in future syntheses. For example, a total of 99 non-English articles were identified by our search strategy (i.e., had English abstracts) but were excluded. Although, it is unclear how many of these articles would have met all the inclusion criteria; the ability to include these untranslated articles would add strength to the accuracy of the map.

Secondly, articles were limited to those available open-access or through authors’ institutional paid journal subscriptions. As a result, many potentially relevant studies were irretrievable (43%, n = 691 of which 76 do not appear to be in English), and we acknowledge that access to this body of evidence would add further strength to the accuracy of the map and could identify additional relevant intervention. More specifically, for 53% of these irretrievable articles, full-text links could not be found on google scholar. In addition, 47% were not accessible without paid subscriptions or purchase of books of conference proceedings. We therefore anticipate that our systematic map could be upgraded in the future to include these potentially relevant articles. To alleviate such a limitation, we would recommend that the team involved benefit from different types of institutional accesses and that it commits resources to making personal contact with the authors or request full-text prints through e.g., ResearchGate. This should also include the articles that were rejected at the abstract screening stage, due to the lack of abstracts.

Limitations in coding and synthesis

Data extraction aimed to capture general characteristics of the interventions. The need to categorise each intervention into groups and, subsequently, types, means that potentially significant nuances may have been lost due to the subjectivity in assigning studies to groups or types of intervention. To avoid this potential bias, it may be that interventions are recorded as free text in the codebook rather than by selecting pre-defined lists of groups and types. Details and nuances of each intervention would then be recorded and could be used to identify additional knowledge gaps and knowledge clusters.


This systematic map provides a description of the evidence available on the effectiveness of the techniques and management approaches used to improve the productivity of field-grown tomatoes under conditions of water-, nitrogen- and/or phosphorus-deficit. Tomato has been chosen as a genetic model, and hence the findings presented here have potential relevance to other crops which require improvement in the multiple stress or deficit tolerances. We identified 291 relevant articles describing a total of 461 interventions. This map reveals several knowledge gaps in the current evidence base, including the lack of integrated approaches that assess multiple interventions and resource deficit simultaneously. Additionally, it highlights the lack of evidence reporting upon relatively novel strategies such as the use of techno-chemical interventions and breeding cultivar, which started to be published on a regular basis about 10 years ago. There are also clear concentrations of research efforts by geographical location and evidence from different countries/regions focuses on different approaches. Hence, this systematic map can serve to direct future research and development efforts for more sustainable production of field-grown tomato. As such, these insights may also help direct the foci of policy makers wishing to establish an improved evidence-base for more informed decision-making regarding interventions, which may enhance tomato resource use-efficiency, with respect to water, nitrogen, and phosphorus. In addition, this systematic map could form the basis of a full systematic review to allow a critical assessment, appraisal, and synthesis of the available evidence presented here. For instance, several intervention groups highlighted here could be built upon in separate systematic reviews such as for the irrigation interventions, which are well-represented and hence amenable to full synthesis and meta-analysis by a systematic review with the prospect of determining the effectiveness of the interventions described in this systematic map regardless of crops species concerned. This critical assessment could enable decision makers, including policymakers, to make definitive recommendations applicable to cropped systems more-generally.

Implication for policy/management

This systematic map can be used by policymakers, who want to gauge the extent of available evidence on the effectiveness of the techniques and management approaches used to improve the productivity under conditions of water-, nitrogen- and/or phosphorus-deficit for any field-grown crops. It is likely that knowledge gaps shown here for a popular and frequently studied crop such as tomato would exist to a greater extent for other less commercially important crops. In addition, inefficient irrigation practices cause excessive water usage, and excessive application of (N and P) fertilisers can limit yield in the longer term, while also causing environmental damage. In these contexts, the type and quality of evidence mapped therein highlights possible specific approaches that could improve water, N and P use without compromising yield. To confirm this, critical appraisal and meta-analysis should be conducted in the form of one or several systematic reviews, in order to assess the strength and extent of the evidence. This agroecological potential is of particular significance given current pressures to meet the challenges of climate change and food security, whilst avoiding eutrophication and loss of soil function caused by suboptimal agronomy and/or variety choice. More stringent governance may therefore be implemented to improve crop resource use-efficiency and help reduce the overall impact of agriculture on the environment without compromising productivity. More specifically, this map could help improve current legislation already adopted within the European Union, such as the Nitrates Directive (91/676/EEC) [9] and the Water Framework Directive (2000/60/EC) [10], which strictly control the application of nitrate fertilisers and the management and quality of agricultural water respectively. The map could also help inform policies in European Union member states or elsewhere who seek to shift towards more sustainable food production system without compromising on productivity.

To better guide policy making, further work should be conducted to include economic indicators such as: (i) the financial value of tomato production; and (ii) the number of jobs/ businesses active in tomato production. Additionally, further studies are needed on the ‘water footprint’ concept to explore the impact of tomatoes on water consumption. Life cycle assessment could also facilitate a comparative analysis of different strategies.

Implication for research

This systematic map is a powerful “springboard” for the development of future research and development programmes on field-grown tomatoes, and other field-grown crops more generally, all of which face the same requirement for multiple stress (or deficit) tolerances.

Knowledge gaps

This map identifies several understudied interventions that may be highlighted as ‘knowledge gaps’, and hence could benefit from more intense research effort. These gaps are listed below in no specific order.

  1. 1.

    There is a lack of evidence studying more than one resource at a time (28%) and only 9% of those addressed water and nutrient deficit at the same time. This highlights the need for more integrated approaches that assess multiple interventions and resource deficits simultaneously to determine the interactions or the relationships between these critical resources for tomato crops. Future research direction should therefore focus on studies investigating multiple resource deficits.

  2. 2.

    Further research into specific interventions could increase our knowledge in resource use-efficiency in tomato production. For instance, there have been significant improvements achieved in the field of breeding and N use-efficiency in other crops, such as barley [28]. Consequently, there is a great need for an increase in studies which employ breeding or genetics to address current concerns regarding resource use-efficiency in tomato production. Likewise, despite the well-known issues with the strong reliance of crop production on inorganic fertilisers to fulfil nutrient supply requirements and the evidence that alternatives exist, only 28 articles studied the use of biostimulants or biofertilisers. Hence, future research should give greater focus to alternative strategies when attempting to address issues related to resource deficit/ use.

  3. 3.

    There are also a limited number of reports studying a techno-chemical intervention (n = 9) or a computational one (n = 6). Such a small number of studies that utilised what are described as emerging or new technologies is important to acknowledge. The lack of evidence on the use of precision agriculture is particularly surprising as the development of new technologies for agriculture offers significant opportunity for improved crop management, especially in the case of irrigated crops requiring applications of fertilisers, such as field-grown tomatoes. There is therefore plenty of scope for research on precision agriculture to be undertaken as the site-specific supply of inputs can increase efficiency, improve sustainability and act to tackle food security [29].

  4. 4.

    There are clear concentrations of research efforts from specific geographical (or socio-economic) locations such as the United States, India, and Italy. Whilst this is expected since these countries are among the top 10 tomato producers in the world, it raises the question as to how well this effort address the productivity challenges in other (less developed) countries. This map was not restricted to a single specific region or climatic zone. Therefore, future research also needs to focus on those continents and countries where there were either few or no relevant studies reported despite high tomato production being recorded there (e.g., Spain, Iran, Brazil, and Mexico). An increased number of relevant studies within a specific region would enable evidence to be gathered and synthesised for the impact of resource deficit on tomato yields in each pedoclimatic zone (e.g., Mediterranean).

  5. 5.

    In addition, it is interesting to note that different geographical regions focussed on different group of interventions. For instance, the techno-chemical group of intervention has been studied mainly in Egypt and the USA (78%), whilst the use of biostimulants and biofertilisers was predominantly reported in Europe and Asia (70%). Similarly, soil amendment strategies have largely been implemented in Asia and Africa (79% of cases). Future research could therefore focus on ensuring a greater diversity of interventions being used in any given geographical region to improve the productivity of field-grown tomatoes under conditions of resource deficit.

  6. 6.

    The duration of the studies, when reported was relatively short (1 or 2–3 seasons, 84%), whilst the benefits of some interventions may take longer to fully reap the rewards of the strategy applied e.g., soil amendment group of interventions and in particular the use of cover crops, manure, and biochar. Future research funding should therefore allow for studies to be conducted over longer periods.

This map highlights the need for a more systematic approach to reporting the details of experimental sites, with special respect to soil chemical and functional attributes such as: nutritional status, especially with respect to carbon content, N and P, though also important micronutrients (minerals); bulk density; structure and type; and soil pH. A standard set of parameters for all field trials of resource use-efficiency would enable the impact from any test intervention(s) to be placed into more informative functional, applied, and comparative contexts.

Knowledge clusters

This map suggests several interventions that may warrant future evidence synthesis and meta-analysis. For instance, irrigation accounts for over 39% of all the interventions studied. Similarly, it may be beneficial to conduct a full systematic review of the ‘fertilisation group’ of interventions geared to improve resource use-efficiency. A critical assessment, appraisal, and synthesis of the available evidence could determine the effectiveness of the techniques and management approaches described in this systematic map to improve the productivity of field-grown tomatoes, and other field-grown crops more generally, under conditions of water-, nitrogen- and/or phosphorus-deficit. These details could provide important agronomic advice for farmers, and ex-farmgate stakeholders with an interest in more resilient and multiple stress tolerant (tomato) production, whilst reducing their environmental impact.

Availability of data and materials

All data generated or analysed during this study are included in this published article [and its Additional files 17].


  1. Roberts L. 9 billion? Science. 2011;333(6042):540–3.

    Article  CAS  PubMed  ADS  Google Scholar 

  2. United Nations. Department of economic and social affairs, population division. World population prospects: The 2017 revision, key findings and advance tables; Working paper No. ESA/P/WP/248. 2017.

  3. Morison JIL, Baker NR, Mullineaux PM, Davies WJ. Improving water use in crop production. Philos Trans R Soc Lond B Biol Sci. 2007;363(1491):639–58.

    Article  PubMed Central  Google Scholar 

  4. European Environment Agency (2021). Water-limited crop yield. Accessed 30 Sept 2020.

  5. Lavalle C, Micale F, Houston TD, Camia A, Hiederer R, Lazar C, Conte C, Amatulli G, Genovese G. Climate change in Europe. 3. Impact on agriculture and forestry. A review. Agron Sustain Dev. 2009;29(3):433–46.

    Article  Google Scholar 

  6. Keeney DR. Sources of nitrate to ground water. In: Keeney DR, editor. Developments in agricultural and managed forest ecology. Amsterdam: Elsevier; 1989.

    Google Scholar 

  7. Sainju UM. Determination of nitrogen balance in agroecosystems. MethodsX. 2017;4:199–208.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Tilman, et al. Agricultural sustainability and intensive production practices. Nature. 2002;418:671–7.

    Article  CAS  PubMed  ADS  Google Scholar 

  9. Communities CotE. Directive 91/676/EEC Concerning the Protection of Waters Against Pollution Caused by Nitrates from Agricultural Sources. Brussels: Council of the European Communities 1991, 12 December 1991.

  10. Directive WF. Directive 2000/60/EC of the European Parliament and of the Council of 23 October 2000 establishing a framework for Community action in the field of water policy. Off J Eur Communities. 2000;22(12):2000.

    Google Scholar 

  11. Martínez-Blanco J, Muñoz P, Antón A, Rieradevall J. Assessment of tomato Mediterranean production in open–field and standard multi-tunnel greenhouse, with compost or mineral fertilizers, from an agricultural and environmental standpoint. J Clean Prod. 2011;19(9–10):985–97.

    Article  Google Scholar 

  12. Heuvelink E. Tomatoes. Wallingford: CABI Publishing; 2005.

    Book  Google Scholar 

  13. Eurostat (2019) The fruit and vegetable sector in the EU - a statistical overview. Eurostat: Statistics Explained. Accessed 25 Mar 2019.

  14. Kimura S, Sinha N. Tomato (Solanum lycopersicum): a model fruit-bearing crop. Cold Spring Harb Protoc. 2008.

    Article  Google Scholar 

  15. Atherton J, Rudich J. The tomato crop: a scientific basis for improvement. New York: Chapman and Hall; 1986.

    Book  Google Scholar 

  16. Kubota C, McClure MA, Kokalis-Burelle N, Bausher MG, Rosskopf EN. Vegetable grafting: history, use, and current technology status in North America. HortScience. 2008;43(6):1664–9.

    Article  Google Scholar 

  17. Lee JM, Kubota C, Tsao SJ, Bie Z, Echevarria PH, Morra L, Oda M. Current status of vegetable grafting: diffusion, grafting techniques, automation. Sci Hortic. 2010;127(2):93–105.

    Article  Google Scholar 

  18. Djidonou D, Gao Z, Zhao X. Economic analysis of grafted tomato production in sandy soils in northern Florida. HortTechnology. 2013;23(5):613–21.

    Article  Google Scholar 

  19. Chauhan H, Bagyaraj DJ, Selvakumar G, Sundaram SP. Novel plant growth promoting rhizobacteria—prospects and potential. Appl Soil Ecol. 2015;95:38–53.

    Article  Google Scholar 

  20. Quesada N, Iannetta PP, White PJ, Tran F, Begg GS. What evidence exists on the effectiveness of the techniques and management approaches used to improve the productivity of field grown tomatoes under conditions of water-, nitrogen-and/or phosphorus-deficit? A systematic map protocol. Environ Evidence. 2019;8(1):26.

    Article  Google Scholar 

  21. CEE Collaboration for Environmental Evidence (2018). Guidelines and Standards for Evidence synthesis in Environmental Management. Version 5.0 (AS Pullin, GK Frampton, B Livoreil and G Petrokofsky, Eds). Accessed 30 Oct 2018.

  22. Harzing AW. Publish or Perish (2007). Accessed 10 Aug 2019.

  23. EndNote. EndNote X8 for Windows and Mac. Clarivate analytics. 2016.

  24. Kohl C, McIntosh EJ, Unger S, Haddaway NR, Kecke S, Schiemann J, Wilhelm R. Online tools supporting the conduct and reporting of systematic reviews and systematic maps: a case study on CADIMA and review of existing tools. Environ Evid. 2018;7:8.

    Article  Google Scholar 

  25. Higgins JP, Green S. Cochrane handbook for systematic reviews of interventions. Hoboken: Wiley; 2011.

    Google Scholar 

  26. Haddaway NR, Macura B, Whaley P, Pullin AS. (2017) ROSES flow diagram for systematic maps. Version 1.0.

  27. Grégoire G, Derderian F, Le Lorier J. Selecting the language of the articles included in a meta-analysis: is there a tower of babel bias? J Clin Epidemiol. 1995;48(1):159–63.

    Article  PubMed  Google Scholar 

  28. Bingham IJ, Karley AJ, White PJ, Thomas WTB, Russell JR. Analysis of improvements in nitrogen use efficiency associated with 75 years of spring barley breeding. Eur J Agron. 2012;42:49–58.

    Article  Google Scholar 

  29. Gebbers R, Adamchuk VI. Precision agriculture and food security. Science. 2010;327(5967):828–31.

    Article  CAS  PubMed  ADS  Google Scholar 

Download references


The authors would like to thank all the participants of the expert workshop held in October 2018 hosted by the University of the Balearic Islands (UIB, Spain) and Paola Colla (University of Turin, UNITO) for logistical support. The authors also would like to thank Dr Christopher Lancefield at the University of St Andrews for his help accessing irretrievable papers.


The work presented in this paper was conducted as part of the EU TOMRES project, which has received funding from the European’s Horizon-2020 Research and Innovation Action programme under Grant Agreement number 727929. The James Hutton Institute is supported by the ‘Rural and Environmental Science and Analytical Services’ (RESAS), a Division of the Scottish Government.

Author information

Authors and Affiliations



NQ jointly with PW, GB, AS, DS, GN and PI conceived the study. PI and GB secured the financial support. NQ and GB presented the mapping methodology. NQ developed the search strategy and built the test library. NQ and FT implemented the search and screened the articles with the contribution of the rest of the review team (MY, DB, JH, PI). FT coordinated the mapping process, analysis, and presentation of the results. FT and JH drafted and revised the manuscript. All authors read and contributed to the final manuscript. All authors have read and approved the final manuscript.

Corresponding author

Correspondence to Pietro P. M. Iannetta.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1:

ROSES form (excel file).

Additional file 2:

Search terms and search strings (excel file).

Additional file 3:

Relevant reviews (excel file).

Additional file 4:

Database of irretrievable papers (excel file).

Additional file 5:

Database of excluded studies at the full-text screening stage (excel file).

Additional file 6:

Codebook of variables (excel file).

Additional file 7:

Database of all accepted studies in the Systematic Map (excel file).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tran, F., Holland, J.E., Quesada, N. et al. What evidence exists on the effectiveness of the techniques and management approaches used to improve the productivity of field-grown tomatoes under conditions of water-, nitrogen- and/or phosphorus-deficit? A systematic map. Environ Evid 10, 14 (2021).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: