- Systematic Review Protocol
- Open Access
A systematic map protocol: What evidence exists to link agricultural practices with ecological impacts for Irish waterbodies?
Environmental Evidence volume 4, Article number: 14 (2015)
Increasing world population, changing consumption patterns, and the agri-food industry ambition for expansion will drive an increase in outputs from future Irish agriculture. This must be achieved within the context of the targets established within E.U. environmental directives, such as the Nitrates (91/676/EEC) and Water Framework Directives (2000/60/EC). How and if sustainable intensification can be achieved is unclear, with climate change and competition for land (e.g. bioenergy crops) increasing pressures on agriculture and the environment. The Environmental Protection Agency’s report, Ireland’s Environment, investigated the status of water quality and concluded agricultural diffuse and small point sources of pollution were the main cause of 50 % and 13 % of river and groundwater pollution, respectively (EPA, 2012). As a result of significant investment in research, there is now an understanding of the factors controlling the impact of agriculture on water quality in Ireland. However, in facing the challenges of sustainable intensification, climate change, and increasing demand for delivery of ecosystem services from rural environments, there is a need to determine to what extent existing research can provide answers to these challenges and what further research is required to balance water quality protection and agricultural production in the future.
The objective of this systematic map protocol is to plan development of an evidence-base to inform recommendations for further research on mitigating the impact of agriculture on water quality in Ireland under future climate change and sustainable intensification. The primary question for this systematic map is: What evidence exists to link agricultural practices with ecological impacts in Irish waterbodies? This involves coding studies based on variables such as scale, methodology, chemical and ecological impacts, and location. Following an extensive search for relevant research documents, documents selected for inclusion in the systematic map will be implemented on a hierarchical basis, with documents first screened at title and abstract level and subsequently at full text level using predetermined selection criteria. Coding will include elements of critical appraisal (e.g. study length, study scale, and experimental design). A geographical map indicating where each study occurred will be linked to the database describing all studies included in the systematic map.
The Environmental Protection Agency’s report, Ireland’s Environment , investigated the status of water quality and concluded that agricultural diffuse and small point sources of pollution were the main cause of 50 % and 13 % of river water and groundwater pollution, respectively. The factors and processes controlling export of diffuse pollution from agriculture to waterbodies are well understood as a result of significant investment in research on these topics in recent decades. Four large-scale research projects have, to date, given rise to three seminal reports on agriculture and water quality [2–4], with a fourth report arising out of the STRIVE Pathway project due in the near future . Research reported by Tunney et al.  and Carton et al.  have been instrumental in defining the strategy for the implementation of the Water Framework and Nitrates Directives in Ireland, while the Agricultural Catchment Programme (ACP)  has played a central role in testing the effectiveness of measures implemented through the Nitrates Action Progamme . With the STRIVE Pathways project focused on developing a catchment management tool to aid in targeting mitigation measures within catchments , the outputs of that project are set to have a significant role in managing the impact of agriculture on water quality in the future. In addition to these reports, there have been a significant number of other published documents arising out of research on agriculture and water quality in Ireland. There is a need to synthesize this knowledge and place it in the wider international research knowledge base to determine the extent which existing research can provide answers to the challenges posed by the objectives of the Water Framework Directive (WFD) and the need to balance water quality protection and agriculture production in the future against a backdrop of climate change and the drive for sustainable intensification.
Much of the research in Ireland over the past 20 years has focused on either quantifying or mitigating the contribution that agriculture has made to the decline in water quality. Prior to implementation of the WFD, research mainly investigated the impact of agricultural practices on chemical water quality. Subsequently, research increasingly focused on determining how changes in chemical water quality due to agricultural practices linked with ecological water quality . Accurate source apportionment of pollutant inputs to waterbodies for targeting mitigation has posed a significant challenge . This is because of the multiple sources contributing these pollutants, lag times in pollutant delivery through the catchment, differences in the bioavailability of sources and their timing of delivery, and the large number of physical and biological factors that influence chemical water quality [9, 10]. The difficulty of linking these inputs to ecological impact has presented a far more complex and holistic problem for research [11, 12].
The focus of the WFD is on achieving good or maintaining high ecological status, with changes in ecological status inextricably linked with a multitude of impacts by agriculture on aquatic ecosystems (e.g. chemical water quality, sediment, hydrological regime). Ecological water quality will only improve if these impacts are mitigated. The focus of this review on the ecological impact of agriculture on water quality is supported by four interrelated issues:
Determining the variables that constrain/maintain ecological status
With the requirement to achieve good ecological status in all waterbodies by 2027, research has attempted to disentangle the biophysical and chemical factors that constrain/maintain the status of waterbodies. These factors include inter alia: geomorphology, environmental drivers (temperature, light), chemical thresholds, recolonisation, and hydrology -all of which vary on a site specific basis [10, 13]. Although mitigation measures may reduce, for example, nutrient inputs below the required threshold values for recovery, other factors (e.g. hydrology) may constrain recovery and/or the direction in which it takes [11, 14]. In such a scenario, further reductions in nutrient inputs from agriculture may not improve ecological status and may potentially have unnecessary impacts on agricultural production within the catchment. The issue of determining the variables that constrain/maintain ecological status is addressed in the search criteria methodology of this study under ecological impacts (Study inclusion criteria).
Compliance gaps for agricultural pollutants
Identifying the agricultural pollutant threshold values required to achieve/maintain ecological status is a key step in achieving water quality targets . If current chemical water quality standards are achieved but there is no corresponding improvement in ecological water quality, this may be due to a lag time in recovery; a hysteresis effect; other factors constraining recovery; and/or a site specific threshold value . If these factors can be identified, then a more accurate assessment of the compliance gap between current nutrient loads from agriculture and the reductions required to achieve these threshold values can be determined. In determining the threshold values for nutrients, consideration needs to be given to the bioavailability of different sources so measures can be targeted at those areas contributing the greatest load of bioavailable nutrients to waterbodies . The issue of assessing the compliance gap between agricultural pollutants and the required reductions in those pollutants is dealt with in the search criteria of this study under exposures and outcomes (Study inclusion criteria).
Identifying critical source areas within catchments
Currently, across Ireland, mitigation measures to control agricultural pollution are predominately implemented at farm-scale with very little consideration given to intra- and inter-farm variation. Within a catchment, the contribution of agricultural land to changes in water quality is heterogeneous with some areas posing a more significant threat to both chemical and biological water quality . Currently critical source areas (CSA) are defined as areas where high risk sources of pollutants overlap with areas that are hydrologically connected to waterbodies [17, 18]. These are the areas that deliver pollutants to waterbodies and will largely determine chemical water quality. However, it is the timing of this delivery that will be a key factor in determining when and where pollutants impact on aquatic ecology. The ability to identify CSAs based on ecological and chemical impacts will facilitate increased accuracy in targeting measures and significantly increase the likelihood of achieving water quality targets . To date, there has been no incentive or policy drive in Ireland for the farmer to change practice within these areas, largely because of a poor evidence base for policy development. In the search criteria of this research, any study relating to CSA will be included if an agricultural practice causes a threat. Therefore, all CSAs in Ireland are included in this study within the exposure and outcome terms in Study inclusion criteria.
Maximising the cost effectiveness of mitigation measures
The generic approach to the implementation of mitigation measure, as outlined above, has reduced their cost-effectiveness with no consideration given to the impact of inter alia: soil type, topography, farming system, hydrology and past agricultural management that has left a legacy of stored nutrients in the catchment (e.g. groundwater nitrogen and soil total phosphorus). Basing the implementation of measures on ecological impact would increase cost-effectiveness and the likelihood of achieving the targets of the WFD . This would help ensure that farmer buy-in and the best ‘value for money’ approach is achieved with the limited financial resources available for the implementation of mitigation measures . If a practical approach to targeting measures at CSA is developed, then the relative merits of targeted and generic approaches to the implementation of mitigation measures can be consider by policy maker. The issue of cost effectiveness of mitigation measures is not dealt with directly in this research, but ultimately the map could inform this issue by identifying relevant mitigation measures investigated to date.
In addition to these four issues, better understanding of the ecological impact of agricultural practices is also required to determine if the objectives of the WFD can be achieved within the context of climate change and the predicted growth of the agricultural sector in Ireland (Food Harvest 2020) . Climate change is likely to impact directly on aquatic ecology and indirectly on the sources and transport pressures related to contaminant export from agriculture to water, adding an additional layer of complexity to the relationship between agricultural practice and ecological impact . Recently, there has also been an increased focus on the delivery of ecosystem services from agricultural land, with the protection of biodiversity as a key driver . The equity of payments for the delivery of aquatic ecosystem services will be significantly enhanced if the link between land use activity and ecological impact can be established.
In order to inform future research needs, a systematic map of the existing research will be carried out to help identify evidence gaps. This will involve coding studies based on a range of variables such as scale, methodology, impact and location. Following an extensive search for relevant research documents, selection of documents for inclusion in the systematic map will be implemented on a hierarchical basis, with the documents first screened at title and abstract level and subsequently at full text level using a an number of predetermined selection criteria.
The objective of this systematic map is to develop an evidence-base to inform recommendations for further research on mitigating the impact of agriculture on water quality in Ireland, in the context of achieving the targets of inter alia EU WFD and Food Harvest 2020 report. The EU WFD aims to protect and improve water quality using a river basin approach. Ireland’s Food Harvest 2020 report aims to increase volumetric production in the dairy sector by 50 %; in the value of beef output by 40 %; and provides additional production targets for the sheep, pig, food and energy, forestry, and marine agricultural sectors. The challenge of trying to achieve production targets within Food Harvest 2020 while complying with obligations under the EU WFD in Ireland has not been addressed to date. As the objective of this research is broad in scope, the systematic map is restricted to the island of Ireland to allow full investigation of all studies related to the impact of agriculture on water quality within the country.
Primary research question
Based on the rationale detailed above, the question identified for this systematic map is:
What evidence exists to link agricultural practices with ecological impacts in Irish waterbodies?
Irish waterbodies include all surface, groundwater and estuarine waterbodies, including wetlands, coastal bodies and turloughs on the island of Ireland. Agricultural practices includes all farming activities and agricultural land use related to the pig, poultry, beef, dairy, sheep and arable farming sectors that potentially impact on water quality. In addition, studies that focus on agricultural mitigation measures to reduce the impact of agriculture on aquatic ecosystems will be included. Ecological impacts include all outcomes related to changes in the ecological, chemistry, and hydromorphology of waterbodies including inter alia nutrients, sediment, microorganisms, hydrology and macroinvertebrates.
There are two related sub-questions which will focus specifically on the delivery of pollutants and the effectiveness of mitigation measures:
Has the delivery of agricultural pollutants to groundwater and surface waters been demonstrated?
Has the impact of mitigation measures on ecological and chemical water quality been demonstrated?
Sub-question 1 has been included because it is implicit in the primary review question that there is a link between chemical and ecological water quality with both needing to be addressed when determining the impact of agriculture on waterbodies. As such, the first step in evaluating the evidence between agricultural practice and ecological status is to determine if research has demonstrated agricultural pollutants are being delivered to waterbodies. The second sub-question has been included because mitigation measures will play a central role in achieving the objectives of the WFD in the context of growth in the agricultural industry.
Data mapping and presentation
The key aim in conducting this systematic mapping process is to aid in the identification of future research needs relating to agriculture and water quality in Ireland. The systematic map outputs will be in the form of a database of studies that will describe the nature and location of evidence on the review topic. This database will be easily searchable and freely accessible via Ireland’s Environmental Protection Agency website (http://erc.epa.ie/safer/reports) and on the Centre for Environmental Evidence (CEE) website. Following the completion of the systematic map, the findings of the mapping process will be summarized and placed within the context of research in other countries, particularly the United Kingdom. A summary document of the findings will then be circulated to key experts in the areas of water quality and agriculture in Ireland and the United Kingdom. They will be asked to review the findings and to reply to an online survey related to future research requirements. Subsequently, the experts will be invited to a workshop to further discuss and prioritise areas of research. The systematic map, online survey, and workshop outputs will then form the basis of a final list of recommendation of future research needs related to agriculture and water quality in Ireland.
Prior to the selection of search terms, review articles covering the impact of agriculture on water quality were consulted. Search terms were identified based on the Population, Exposure and Outcomes components of the primary question. In addition, the search term “Ireland” was included in the search criteria in order to geographically restrict the search to just the island of Ireland. Due to the broad extent of the research question, the geographic scope of the systematic map was limited to the island of Ireland to ensure the number of studies included in the map was manageable. The Comparator component of primary question was not included in the search terms, with the inclusion of spatial and temporal replication forming a key part of the subsequent selection process. A preliminary list of search terms were circulated within the project team. Following a number of iterations, the list of search terms was then circulated to members of a steering committee overseeing this project. The final list of search terms used in this systematic mapping process are detailed in Table 1. The language is restricted to English, as the primary language of scientific publication in Ireland.
Databases and other sources
When conducting the searches a wide range of databases and sources will be utilised. The selection of online databases detailed below was based on a review of previous systematic maps/reviews that covered similar topics as this study. Eight online databases were identified
Web of Science (All databases) which covers
Web of Science Core Collection
CABI –CAB Abstracts
FSTA –Food Science Resource
SciELO Citation Index
Directory of Open Access Journals (including Plus One)
The search string presented in Table 1 was the optimum format for Web of Science but was altered to suit other systems (e.g. Google Scholar) when required In the production of the systematic map, an additional file will be provided showing the search string used in each of databases for repeatability. Consistent with guidance from the CEE, the first 100 hits from Google Scholar will be selected for further review.
A selection of specialist websites from Republic of Ireland and Northern Ireland will also be searched for relevant publications that may not have been identified in searches of online database. These websites are:
Searchable Database of Environmental Protection Agency (EPA) Research Reports: http://erc.epa.ie/safer/reports
Searchable Database of EPA Research Projects: http://erc.epa.ie/smartsimple/
Department of Agriculture Food and the Marine http://agriculture.gov.ie/
Agricultural and Food Development Authority www.teagasc.ie
Department of the Environment http://www.doeni.gov.uk/niea/
Department of Agriculture and Rural Development http://www.dardni.gov.uk/
Sustainable Water Network http://www.swanireland.ie/
Department of Environment, Community and Local Government http://www.environ.ie/en/
Engineering Village http://www.engineeringvillage.com
http://erc.epa.ie/safer (i.e. not just the reports Section)
http://www.waterjpi.eu (which contain a list of projects – you can filter it down to IE – there are also other projects than the EPA-funded included into Database
http://rian.ie/ which is the new Irish portal for Open Access
Search comprehensiveness assessment
The number of hits in Web of Science for each set of search terms is reported in Table 1. Several approaches were taken to ensure that a comprehensive list of articles was identified when using these search terms. A list of 32 key documents, collated based on the bibliographies of existing reviews of Irish research, were used to test the comprehensiveness of the search terms used in Web of Science (Additional file 1). The search terms were adjusted until all 32 key documents were found. These documents were selected based on knowledge of existing research in agriculture and water quality in Ireland and included studies on source, mobilisation impact pathways, the nutrient transfer continuum, and key contaminants. A range of publication dates from 1974–2013 and a variety of authors were also included in the selection. The reference lists of previous Irish review articles and the project participants personal archives of the Irish articles related to this area of research were also compared against the articles identified using the search criteria in Additional file 1. Following removal of erroneous articles, the full list of references will be sent to a range of researchers across Ireland who has published regularly on this topic, and they will be asked to identify if any relevant articles have been omitted. In addition, authors will be emailed if an identified article was unattainable through normal channels.
Study inclusion criteria
This systematic map protocol will identify what evidence exists to link agricultural practices with ecological impacts in Irish waterbodies. The core inclusion criteria are:
Relevant population: Irish waterbodies including all surface, groundwater and estuarine waterbodies, including wetlands, coastal bodies, and turloughs within the island of Ireland.
Types of exposure/interventions: All farming activities and agricultural land use related to the pig, poultry, beef, dairy, sheep and arable farming sectors that potentially impact on water quality. In addition, studies that focus on agricultural mitigation measures to reduce the impact of agriculture on aquatic ecosystems will be included.
Types of comparators: Water quality before and after changes in farming practices and/or implementation of mitigation measures (temporal comparator) or comparisons with similar areas of zero/lower intensity agriculture or no mitigation measure implemented (spatial comparator)
Types of outcomes: All ecological impacts related to changes in the ecological, chemical and hydromorphology of waterbodies including inter alia nutrients, sediment, microorganisms, hydrology, and macroinvertebrates
Types of study: As this review is focused on ecological impact of agriculture, only studies that have been carried out at greater than field scale and have multiple spatial or temporal replication will be included in the systematic map. Although studies carried out below field scale are invaluable in understanding natural process, they do not directly evaluate the impact the delivery of pollutants or their impact in waterbodies. However, where an included document makes reference to smaller scale studies, this paper will be reviewed if necessary to elucidate large scale processes occurring at field or catchment scale.
It was hypothesized that searching for studies conducted using the “Ireland” to restrict the geographical scope may exclude those studies conducted in Northern Ireland if they used the term “United Kingdom” or “Great Britain” to define their geographic scope. The extent of this problem has been investigated and found to be insignificant. Results indicate that only 1 article in a 1000 examined, include Northern Ireland as part of larger investigations in the United Kingdom or Great Britain but was not picked up using the search criteria in Table 1.
In order to identify a core list of articles for inclusion in the systematic map, a hierarchical screening process will be undertaken. This hierarchical process will cover the title, abstract and the full text with screening based on the predetermined criteria detailed above. Where doubt over the validity of a document arises during the screening process, it will be retained for further assessment. The title of the paper will be screened first to remove erroneous articles unrelated to water quality. Subsequently, the abstracts will be reviewed to determine if the article dealt with water quality and agriculture in Ireland. General review articles will be excluded at this stage of the screening process. As some reports do not contain abstracts, they will automatically progress to the full text screen stage which will be based on the criteria in Table 1.
To ensure that individual reviewers are interpreting the inclusion criteria similarly, each reviewer will review the same 50 documents and the results will be compared using a Kappa test of agreement, with a score of greater than 0.6 indicating strong agreement in the articles selected. Where difference in the interpretation of the criteria occur, this will be discussed and a common interpretation agreed.
Data extraction strategy
Table 2 details the coding criteria that will be used to extract information from individual studies. The coding criteria were initially identified during a review of similar systematic map studies and subsequently revised through an iterative process involving the project team and steering committee. The coding criteria will first be applied to the papers detailed in Additional file 1 and, if necessary, revised again based on the outcome of that scoping study. The data extracted from each document will be recorded in a Microsoft Access database.
The coding criteria 1–19 in Table 2 relate to the characteristics of the study and the study site and will be used to evaluate the validity of each study in relation to assessing the impact of agriculture on ecological water quality. In addition, these criteria will also provide future end-users of the database with the ability to search the database based on these specific criteria. The remaining coding criteria detailed in Table 2 are designed to extract information from each document on the impact of agriculture on ecological water quality. In addition, the criteria will also extract information on the impact of mitigation measures and whether the delivery of agricultural pollutants to waterbodies has been demonstrated. All the information extracted and input into the database will then be used to make a final assessment on the strength of the evidence.
Critical appraisal of study internal validity
Unlike a systematic review, a full critical appraisal is not necessary for systematic mapping. However, coding will be used to allow users to evaluate the internal validity of each study using the following coding variables; study length, study scale, experimental design, comparator type, replication, and methodological detail.
Environmental Protection Agency. Ireland’s Environment: An Assessment. Environmental Protection Agency; 2012. (www.epa.ie).
Tunney H, Coulter B, Daly K, Kurz I, Coxon C, Jeffery, et al. Quantification of phosphorus losses from soil to water: Final Report and Literature Review. Environmental Protection Agency Ireland; 2000. (www.epa.ie).
Carton OT, Tunney H, Daly K, Ryan M, Kurz I, Doody DG, et al. Eutrophication from Agricultural Sources-Integration Report. Ireland: Environmental Protection Agency; 2008 (www.epa.ie).
Teagasc. Agricultural Catchments Programme: Phase 1 Report. Johnstown Castle, Wexford: Teagasc; 2013 (www.teagasc.ie).
Archbold M, Bruen M, Deakin D, Doody D, Flynn R, Kelly-Quinn M, et al. Contaminant Movement and Attenuation along Pathways from the Land Surface to Aquatic Receptors – A Review. STRIVE Report. Environmental Protection Agency Ireland; 2010. (www.epa.ie).
Wall D, Jordan P, Melland A, Mellander P, Buckley C, Reaney SM, et al. Using the nutrient transfer continuum concept to evaluate the European Union Nitrates Directive National Action Programme. Environ Sci Policy. 2011;14:664–74.
Donohue I, McGarrigle ML, Paul M. Linking catchment characteristics and water chemistry with the ecological status of Irish rivers. Water Res. 2006;40:91–8.
Doody D, Withers PJA, Dils RM. Prioritizing Waterbodies to Balance Agricultural Production and Environmental Outcomes. Environ Sci Technol. 2014;48(14):7697–9.
Hamilton SK. Biogeochemical time lags may delay responses of streams to ecological restoration. Freshw Biol. 2012;57 Suppl 1:43–57.
Jarvie HP, Sharpley AN, Withers PJA, Scott JT, Haggard BE, Neal C. Phosphorus Mitigation to Control River Eutrophication: Murky Waters, Inconvenient Truths, and “Postnormal” Science. J Environ Qual. 2013;42(2):295–304.
Page E, Heathwaite T, Moss AL, Reynolds B, Beven C, Pope KJL, et al. Managing the impacts of nutrient enrichment on river systems: dealing with complex uncertainties in risk analyses. Freshw Biol. 2012;57:108–23.
Moss B. Water Pollution by Agriculture. Philos Trans R Soc B. 2008;363:659–66.
Smith VH, Schindler DW. Eutrophication science: Where do we go from here? Trends Ecol Evol. 2009;24:201–7. 18.
Palmer M, Allan JD, Meyer J, Bernhardt ES. River restoration in the twenty-first century: Data and experiential knowledge to inform future efforts. Restor Ecol. 2007;15:472–81.
Harris GP, Heathwaite L. Why is achieving good ecological outcomes in rivers so difficult? Freshw Biol. 2013;57 Suppl 1:97–107.
Withers PJA, Jarvie HP. Delivery and cycling of phosphorus in UK rivers: Implications for catchment management. Sci Total Environ. 2008;400:379–95.
Jordan P, Melland AR, Mellander P, Shortle G, Wall D. The seasonality of phosphorus transfers from land to water: implications for trophic impacts and policy evaluation. Sci Total Environ. 2012;434:101–9.
Thompson J, Cassidy R, Doody DG, Flynn R. Predicting critical source areas of sediment in headwater catchments. Agr Ecosyst Environ. 2013;179:41–52.
Schulte RPO, Doody DG, Byrne P, Cockerill C, Carton OT. Lough Melvin: developing cost effective measures to prevent phosphorus enrichment of a unique aquatic habitat. Tearmann: Irish J Agri-environ Res. 2009;7:211–28.
Jeppesen E, Kronvang B, Meerhoff M, Søndergaard M, Hansen KM, Andersen HE, et al. Climate change effects on runoff, phosphorus and lake ecological state, and potential adaptations. J Environ Qual. 2009;38:1930–41.
Dodds WK, Perkin JS, Gerken JE. Human impact on freshwater ecosystem services: a global perspective. Environ Sci Technol. 2013;47:9061–8.
This work was s funded through the Environmental Protection Agency (EPA) STRIVE funding program 2013–2020. The authors would like to acknowledge the input of the EPA steering committee: Donal Daly, Patricia Torpey, Ray Spain, Chris Stoate, Patrick Byrne, Jenny Deakin, Tom Stafford and Alice Wemaere. The authors would also like to acknowledge the technical support of Colm McKeaney at AFBI-NI and Prof. Andrew Pullen at the Center for Evidence-based Collaboration, University of Bangor.
The authors declare that they have no competing interests.
DD and CA carried out systematic review protocol design and testing. PW and SC participated in project planning and drafting of this paper. All authors read and approved the final manuscript.