Updating reviews: commitments and opportunities

Unlike primary research, all reviews, whether systematic or not, have a limited ‘shelf-life’. As new findings of primary research are reported, dated review findings become less reliable as an assessment of the best available evidence. This can be a particular issue when reviews report a quantitative synthesis that provides combined means of effects of interventions or impacts of exposures. Combined means can be changed substantially by the addition of findings from a new, large, well designed study. This is particularly true when the evidence base is weak as is often the case in environmental management. Equally, new findings may enable a quantitative synthesis, when the previously published review may have concluded that no quantitative synthesis was possible. For these reasons a key component of systematic review (SR) methodology is the commitment to keep SRs updated on an appropriate timescale so that they track the development of the evidence base. Collaboration for Environmental Evidence (CEE) guidelines note that out of date SRs can be misleading and suggest an average time period of five years for updates [1]. However, science in different areas of environmental management advances at different rates and so five years is very much a guideline and not a rule.

Registration of an update, as with a new SR, is through the submission of a protocol. The protocol should cite the original protocol and be clear about how the new one differs from (and possibly improves on) the old. Updating is also an opportunity to learn from previous efforts and to improve methodology. It is therefore not expected that an update will be a faithful repetition of the original. However, changes should be highlighted and explained.

The CEE Library of completed reviews (http://www.environmentalevidence.org/Reviews.html) includes over 20 SRs that are more than five years old and none have so far been updated. The majority were conducted as part of the process of developing systematic review methodology for environmental management and could no doubt be improved in many ways. Updates can thus be in the form of methodology, as reflected in the development of CEE Guidelines (now at version 4.2), as well as adding new research findings. Some reviews can be even more dated than their publication date suggests as the searches may have been conducted years earlier. The first example of an update in progress has recently been published as a protocol [2] and relates to a CEE SR originally published in 2010 in which the search was conducted in 2008 [3].

Beside updating CEE SRs, an exciting opportunity exists to update other reviews and meta-analyses to meet CEE standards. In general, published reviews and meta-analyses are of very variable standard [4] and the raising of these standards is a key objective of CEE. It seems sensible then to have dual objectives of updating old reviews by adding new findings and updating methodology and conduct of the review to CEE standards. We call on authors of reviews and meta-analyses to consider if it is the right time to update their review and to register their protocol with CEE. Again, registration would be the same process except that the protocol would likely be entirely new and refer only to the former review article as a basis for the CEE SR.

Of course many original authors may not be motivated to periodically update a review and so I would like to encourage anyone interested in conducting a SR and developing their skills in the methodology to consider conducting updates of old reviews (systematic or not). If you are interested I am happy for you to contact me in the first instance to discuss the process.