Primary question

What are the comparative biological effects of SMMs in protecting marine biodiversity? Here, we examine whether SMMs worldwide deliver effective conservation of marine biodiversity. To address this question, studies that examine the effects on an outcome comparing at least one spatial management measure (intervention) vs no protection (open access area, OA) or between interventions (as OA could be not always available) will be considered (see Additional file 2: Table S1 for more details on PICO elements of the question).

The biological effect of SMMs will be examined including both the protection parameters (latitude, age, size, distance between interventions) and habitat and species responses (more details provided in section: “potential reason for heterogeneity”).

Search strategy and terms

A comprehensive search of peer-reviewed scientific literature and grey literature will be undertaken in order to compile a database of studies that documents and compares the biological effects of the considered spatial management measure (SMMs): marine reserves (MRs), marine protected areas (MPAs), partial permanent protection (PPP) and partial temporal protection (TP) measures and open access areas (OA). Searching will be carried out across a range of resources in order to minimise the possibility of publication and related biases [17]. In addition relevant review articles will be subjected to bibliography checks for relevant references. At the search stage, the comparison between interventions (SMMs) may be observed through several approaches (study designs): before-and-after intervention, or no-interventions (control) versus intervention (impact), or both (i.e. before after control impact, BACI), or intervention versus alternative intervention. Studies will be treated the same, but differently coded for easy identification and subsequent separation during the analysis stage.

To guarantee a good balance between sensitivity and specificity of the search, a scoping search phase was performed. A scoping search process, using several bibliographic databases, estimated the volume of relevant literature and set the search strategy by determining the most appropriate search terms (see Additional file 2 for a full list of the search term combinations used).

A complex search string was created to be used in databases and websites that allow complex search through advanced search options (Additional file 2A–C). The Boolean operators “AND/OR” were used where appropriate and the search was undertaken across title, keywords and abstract. Search terms were based on the following phrases (where * denotes a wildcard to search for alternate endings):

“marine protected area*” OR “fishery exclusion zone*” OR “economic exclusion zone*” OR (“no take zone*” OR “no take area*” OR “no-take zone” OR “no-take area”) OR (“special protection zone*” OR “special protection area” OR “SPA”) OR (“site of community importance” OR “site of community interest”) OR “partial protection” OR “temporal protection” OR “permanent protection” OR “marine reserve*” OR “buffer zone*” OR “closed area*” OR “marine park” OR “marine sanctuary” OR “restricted area*” OR “nursery area*” OR “fishing gear restriction*” OR “integrated coastal zone management” OR “ICZM” OR (“special area* of conservation” OR “SAC”) OR (“site of special scientific interest” OR “SSSI” OR “marine conservation zone” OR “MCZ”) OR “mari* spatial planning” OR “marine directive” OR “mari* spatial management” OR “vulnerable” OR “protection effect*” OR “restoration effect” OR “harvest refug*”

AND

“density” OR “abundance” OR “biomass” OR “*diversity” OR “richness” OR “evenness” OR “Shannon” OR “species number” OR “size” OR “length” OR “life history trait*” OR “LH trait*” OR “maturity” OR “longevity” OR “reproduction” OR “sex” OR “mobility” OR “Biological trait*”

AND

“sea” OR “marine” OR “ocean” OR “marine ecosystem*”

The complex search string will be replaced by simple strings, modified according to the search functionality of the databases and websites that do not allow advanced search. In this case search terms will be limited as website search engines generally only accept simple search terms (Additional file 2B).

Search sources

Article screening and study inclusion criteria

The results of each search will be screened for relevance. A bibliographic software package (endnote) will be used to organise all references retrieved from computerised databases. Duplicates will be removed and articles not relevant or that do not contain relevant information or data removed. Through a three stage screening process (title–abstract–full text) eligible studies to include in the review will be identified. The decision of inclusion criteria is one of the most influential decisions in the review process; at each stage studies will be selected to meet specific inclusion criteria chosen to minimise biases and human error, if there is insufficient information to exclude an article it will be retained until the next stage. The adoption of inclusion criteria, mainly based on the presence of PICO elements components (population, intervention, comparator, outcomes, to see Additional file 2: Table S1 for more details), will guarantee a transparent screening process based upon congruency with the review questions, relevance and methodology quality [19]. In the first instance, the title of articles will be assessed in order to remove spurious citations and then abstracts will be assessed according to these inclusion criteria.

Articles remaining after this filter will be considered at full text.

If needed, authors of relevant articles and subject experts will be contacted to clarify study eligibility (to request further advice and information, such as missing data and additional references). Articles with no data or characterised by: pseudoreplication, significant flaw in the method or analytical techniques, incorrect estimated baseline of protection or management will be excluded. An additional file of articles excluded at full text will be provided with reasons for exclusion.

To measure the effect of between-reviewer variance in assessing relevance the kappa statistic will be calculated [20]. Two reviewers will apply the above inclusion criteria to a subsample of 10 % of articles, or 100 articles (whichever is greater), at the start of the abstract screening stage. The kappa statistic estimates the level of agreement between reviewers, a rating of “substantial” (0.5 or above) is recommended to pass the assessment [18]. If comparability is not achieved the same reviewers will discuss the discrepancies and the scope and interpretation of the question elements will be redefined with potential modification of the criteria specification. After this agreement appraisal and related discussion the inclusion criteria will be applied to the rest of the citations by one reviewer.

Critical appraisal of study quality

Articles accepted at full text will be included in the review and reported studies subject to critical appraisal according to their design and quality. Reviewers will assess the methodologies used in all studies reported in articles accepted at full text. Two reviewers will examine a random subset of at least 25 % of the included studies to assess repeatability of study quality assessment. Variation in methodological and analytical quality of scientific studies will be checked [11, 21–23]. For each study design elements that reduced susceptibility to bias will be recorded. Following the same approach applied by Sciberras et al. [24] the articles will be categorised according to the study design, respectively into: before after control impact (BACI) studies, control impact (CI) studies and before after (BA) studies; variation to spatial and temporal scale and habitat heterogeneity will be take into account as the two main sources of bias. BACI studies, accounting both spatial and temporal variability in the environment [12, 24, 25], will be selected as the best design to allow the detection of the effectiveness of interventions on marine biodiversity without any inferences (less risk from bias). Subsequently, the influence of sampling design on the magnitude of the response to interventions will be explored by running a sensitivity analysis using all studies and those with BACI design only.

Studies containing information of poor or deteriorated enforcement of intervention will be excluded from the analyses. Additionally, the level of replication (sampling effort and number of included SMMs) will be recorded to check for study reliability.

Potential reasons for heterogeneity

To account for potential bias due to differences in habitat among the interventions (SMMs) and control sites (baseline differences) in the comparison between interventions, a sensitivity analysis will be conducted in parallel to the main analysis to examine the influence of including studies where habitat variation (e.g. substratum type, substratum composition and complexity, rugosity and exposure) affects the overall magnitude and direction of the intervention effect [26, 27].

Data extraction strategy

Articles accepted at full text will be recorded in a database (Additional file 3). Data extraction will be undertaken using a review-specific data extraction form (sensu Lipsey and Wilson [28]). Density, abundance, biomass, length and diversity metrics will be treated as continuous variables. Sample sizes, means and variance (or other variance measures including standard deviation, standard error, 95 % confidence interval) values will be extracted as presented from tables or within the text. Different taxa were disaggregated as far as possible. Multiple non-independent data-sets may be extracted from each article, for example where different depths or habitats within an intervention and adjacent open access area are surveyed and data are also aggregated at the intervention level to maintain independence. Data from figures will be extracted using the data-extraction software PlotDigitizer.

Data synthesis

The review will include multiple comparisons and meta-analysis between all possible pairs of interventions. Previous reviews [12, 15, 16, 29] illustrate that sufficient data with comparators are available for meta-analysis, but that investigations of heterogeneity are limited by data availability. Our synthesis will therefore consist of meta-analyses to address the primary question with meta-regression and subgroup analyses used to investigate reasons for heterogeneity between studies.

Variables that affect the effectiveness of intervention measures will be classified. Responses of specific taxa will be treated as independent observations so as to investigate the effectiveness of different interventions on the response of the population, regardless of taxa. The response will be measured by the percentage difference of the population before and after (or impact/control) the establishment of the intervention, or between different sites of a known gradient of interventions.