Search strategy
Literature specific to developing countries can often be difficult to retrieve, or of varying quality and/or suitability in answer to a research question. This has been noted particularly in the health field, where reliable information is often sparse and difficult to locate [8, 9].
The search approach proposed for this review, detailed below, aims to address these issues with a highly sensitive strategy applied in a variety of mixed-topic (for example, medical, geographical, and social science databases) interfaces [10]. This approach allows us to retrieve a broad evidence base of both quantitative and qualitative research and minimise the effect of bias.
Scoping
Preliminary scoping has been undertaken to assess the size and quality of the literature base for this review, and to test various approaches to defining the search syntax. Sample abstracts have confirmed the suitability of the search strategy and highlighted heterogeneity of the data.
Searching
A syntax laden intervention/population search would be inappropriate for this review given the variety of outcomes under consideration. Instead, using a simple yet sensitive approach allows a similar search to be used and consistency maintained between the many search interfaces, as well as allowing for cohesion in the overall search protocol.
See Additional File 1 for a sample search strategy written for Web of Science (ISI). This will be translated as appropriate for use in other search interfaces.
Search syntax
We have truncated Arsenic (arsen*) which gives us a variety of related expressions such as; arsenic, arsenates, arsenides, and arsenites, and will address the prevalent species found in groundwater [5]. It is not appropriate to specify terms such as 'arsenic contamination' or 'pollution' as this would prohibit a sensitive retrieval.
Population Terms
We propose searching with a broad sensitivity in regard to the population. Using water as a free-text term has, in scoping, captured all of the benchmarked papers with a sensitivity that precludes a need for a lengthy definition of synonyms.
Where it is necessary, we have defined terms which might not be captured on water alone. These terms, such as the noun 'groundwater', have been included as search terms in their own right.
Additional Terms
In mixed topic resources with no controlled indexing (such as Web of Science), we may require an additional developing countries filter to focus our results to the country populations under review. This filter will only be used in large mixed topic resources without formal indexing as a way to retain specificity. Its use and effect will be extensively checked prior to use by comparing the results obtained with and without using the filter.
The rationale for this filter is that by specifying the developing countries we know to exist, it minimises the risk of blocking outcomes which might be useful to the review, if for example we were to use a NOT cluster.
The country population filter has been developed using AusAID's list of developing countries (http://www.ausaid.gov.au/ngos/devel_list.cfm) which has been cross-checked with the International Human Development Indicators (http://hdr.undp.org/en/statistics/). The International Monetary Fund's (IMF) list of Emerging and Developing Countries was referenced in testing the strategy but the metric was thought to be largely based upon economic outputs and therefore did not correlate with the aims of the filter and expression of the project (http://www.imf.org/external/pubs/ft/weo/2010/01/weodata/groups.htm#oem).
To retain sensitivity, the filter uses a mixture of generic terms such as, 'Developing Countries,' coupled with naming and referencing developing countries and their inhabitants. We have also included countries such as Taiwan, which is considered a newly emerged country, as the date parameters of search pre-date their change of status.
It is likely that this filter will only be used in Web of Science to help control the balance of retrieval. Where it is used, extensive testing to cross-check the retrieval will be conducted. A narrative will be recorded with each search which uses this filter to explain its use.
Limits
All searching will be limited: 1990-Current (the period in which arsenic contamination of groundwater became widely recognised and interventions to address the problem were introduced). Searches will be limited to the English language, as scoping searches indicate that this is the language of publication for scientific studies in countries where arsenic contamination of groundwater is an issue.
Search Resources
The literature-base required to sufficiently answer all aspects of the question needs to be drawn from a variety of topical areas, including; health, engineering, geography and sociology, as well as databases specifically focused on developing countries [11]. Therefore we propose a mixed-topical approach for this review in online databases and catalogues (Box 1).
As not all relevant data are indexed within databases, searching web-resources and web-sites ensures a complete approach and minimises the risk of bias [12]. The resources shown in Box 2, along with conference abstracts and Google, [13] will help us locate grey literature [14]. The search terms used will be arsen* and water and the date of searching will be recorded.
Search Sources: Other Sources
In addition to searching the resources above, we propose to employ 'pearl-growing' techniques [15], such as: Citation chasing from included references, "Forwards" citation chasing on included references using citation databases (Science Citation Index/Social Science Citation Index) to check if any new papers have cited the paper in question since it was published, and contacting authors of included studies [16, 17].
Search Results
The exported files from the searching will be uploaded and de-duplicated in Endnote X4 (Thompson Reuters). Where an export is not possible, for example from a resource without RIS functionality, the data will be exported to a word file and saved.
The searches will be recorded using PRISMA guidelines [18]. This will include the list of databases searched (with their data parameters), recording of the date searched and the strategy as run. Limits applied, the results yielded and an accurate recording of the de-duplication process will be annotated in a search annex.
Study inclusion criteria
Titles and abstracts of all studies located by the search strategy will be screened for inclusion by a reviewer (with a random sample of 10% being checked by a second reviewer) according to the criteria below. Where it is not clear from the title and abstract alone if the study meets the inclusion criteria, the full-text of the study will be obtained to enable assessment (with a random sample of 10% being checked by a second reviewer). Any disagreements about whether a study meets the inclusion criteria will be resolved by discussion, with the involvement of a third reviewer if necessary. All studies to be included must have been published from 1990 onwards and in the English language. The PICO format (population, intervention, comparators, and outcomes) will be used to define study inclusion criteria:
Population (or 'subject')
People living in developing countries (as defined by AusAID: http://www.ausaid.gov.au/ngos/devel_list.cfm)
Intervention
Any technology (see Table 1) designed to remove or lower arsenic concentration in water intended for human consumption. This does NOT include national or local policy or specific educational interventions.
Comparators
Pre/post comparison, comparison with another technology intended to remove or lower arsenic concentration in groundwater, or comparison with no intervention.
Outcomes (must be measured in field-based studies)
-
1)
Arsenic concentration in groundwater intended for human consumption
-
2)
Arsenic concentration in human tissue or body fluids.
-
3)
Observation of people's behaviour relating to technologies intended to remove or lower arsenic concentration in groundwater intended for human consumption
-
4)
People's knowledge or attitudes relating to technologies intended to remove or lower arsenic concentration in groundwater intended for human consumption
The above outcomes are listed hierarchically according to the accuracy with which they are likely to measure the effectiveness of interventions; for example, arsenic concentration in groundwater is considered a more accurate measure of an intervention's effectiveness than people's knowledge or attitudes. Nevertheless, all of the above outcomes may provide useful measures of interventions' effectiveness. However, there is a possibility that the number of outcome measures in included studies is such that the production of a rigorous review that includes all of these outcomes, within the time and resources available, would be unrealistic. In this eventuality, in consultation with AusAID and with a view to optimising the review's potential to inform policy making, we shall either exclude those outcomes that are more distant measures of interventions' effectiveness or exclude studies rated as 'weak' by the quality appraisal tool.
Types of study
For the synthesis of effectiveness data, only comparative study designs will be included (including, but not limited to, randomised controlled trials, before and after studies, and cross-sectional studies). If a recent and high quality systematic review that substantively answers an aspect of the review questions is located, we shall (in consultation with AusAID) utilise the review as a source of potentially includable studies, and update and extend it if it is considered feasible to do so.
Potential effect modifiers and reasons for heterogeneity
The contexts in which technologies designed to remove or lower arsenic concentration in groundwater are implemented may have a significant impact on their effectiveness [19, 20]. Education about technologies, level of community development, the acceptability of interventions, and/or national policy may act as important effect modifiers. Additional effect modifiers identified in included studies will be recorded so that their impact on the effectiveness of interventions can be assessed.
Study quality assessment
Comparative study designs reporting quantitative data will be assessed using the McMaster University Effective Public Health Practice Project quality appraisal tool (http://www.ephpp.ca/tools.html). This tool is suitable for use across a wide range of quantitative study designs and has explicit criteria for rating studies as 'strong', 'moderate', or 'weak'. The tool also includes criteria for assessing implementation fidelity, which may be of particular importance when considering the implementation of arsenic removal (or mitigation) technologies or water management practices in the developing world.
Study quality appraisal will be conducted by one reviewer, with a random sample of 25% checked by a second reviewer. Disagreements will be resolved by discussion, with the involvement of a third reviewer if necessary.
Data extraction strategy
Data will be extracted from included studies by one reviewer using a modified version of a data extraction form designed for use with complex public health studies [21] - see Additional File 2. A random sample of at least 10% of the completed data extraction forms will be checked by a second reviewer. Discrepancies will be resolved by discussion, with the involvement of a third reviewer if necessary. The data to be extracted will follow the outcomes described above and will also include details of the population, type of intervention, and context in which implemented.
Data synthesis and presentation
The quality of the included studies will be assessed using, where available, appropriate checklists for the studies (e.g. EPHPP quality assessment tool for quantitative studies).
The extracted data are likely to be heterogeneous in terms of the interventions assessed, the outcomes reported, the study design, and population, among others. These differences will have to be evaluated qualitatively to determine whether quantitative synthesis using meta-analysis methods will produce meaningful results (e.g. are processes similar between countries to allow synthesis of results across different countries? Or would it be preferable to limit analyses by country?). If study results are deemed to be comparable but are reported using different outcomes measures (e.g. odds ratios or mean differences), methods are available to translate outcome measures on the same scale to assist meta-analysis [22].
If groups of results are deemed to be comparable, an assessment of their statistical heterogeneity will be made using the I2 statistic [23]. This will help to inform whether the data are too heterogeneous (statistically) to provide useful results when combined. The possibility of reporting/publication biases will be assessed where available. However, we are aware that such techniques (funnel plot, regression tests [24]) perform poorly when the number of comparable study results to be assessed is small, as could be the case with this review.
Depending on the heterogeneity between study results either fixed effects or random effects meta-analysis methods will be used to analyse the extracted data. The fixed effects method assumes that the only variation between study results is that due to sampling error and so will be appropriate where the I2 statistic indicates mild heterogeneity. On the other hand, the random effects method assumes that there is variation between studies beyond that due to sampling error, and so will be appropriate where greater heterogeneity is present. Note that the random effects model accounts for heterogeneity, but does not help to explain potential sources of heterogeneity. To explore possible sources of heterogeneity we will undertake subgroup analyses and, where the number of study results allows, meta-regression.
Sensitivity analyses will be undertaken to explore the impact of study quality, possible reporting/publication bias and factors related to study design on the meta-analysis estimates.
To enable the synthesised quantitative data about the effectiveness of interventions to be understood in the light of relevant contextual factors, narrative synthesis tools and techniques such as grouping, tabulation, conceptual mapping, and sub-group analyses [25] will be used.
