Comparing groups versus individuals in decision making: a systematic review protocol

Abstract

Background

Biodiversity management requires effective decision making at various stages. However decision making in the real world is complex, driven by multiple factors and involves a range of stakeholders. Understanding the factors that influence decision making is crucial to addressing the conflicts that arise in conservation. Decisions can be made either by individuals or by groups. This precise context has been studied extensively for several decades by behavioural economists, social psychologists and intelligence analysts. The observations from these disciplines can offer useful insights for biodiversity conservation. A systematic review on group versus individual decision making is currently lacking. This systematic review would enable us to synthesize the key insights from these disciplines for a range of scenarios useful for conservation.

Methods

The review will document studies that have investigated differences between group and individual decision making. The focus will be on empirical studies; the comparators in this case are decisions made by individuals while the intervention is group decision making. Outcomes include level of bias in decision outcomes or group performance. The search terms will include various combinations of the words “group”, “individual” and “decision-making”. The searches will be conducted in major publication databases, google scholar and specialist databases. Articles will be screened at the title and abstract and full text level by two reviewers. After checking for internal validity, the articles will be synthesized into subsets of decision contexts in which decision making by groups and individuals have been compared. The review process, all extracted data, original studies identified in the systematic review process and inclusion and exclusion decisions will be freely available as Additional file 1 in the final review.

Background

A key question in decision making is whether groups make better decisions than individuals [1–3]. This question is highly relevant to biodiversity conservation. For instance, individual experts may be consulted on management decisions or policy issues whereas boards of directors may make decisions on conservation donations and local communities may use group-based decision making for natural resource management. In the past two decades there has been a pronounced shift towards using groups instead of individuals for decision making (e.g., in key financial institutions such as the Bank of England [4]). Recently, it has also been suggested that groups should be preferred over individual experts for providing science advice to governments [5]. These suggestions are often based on individual attempts to investigate the demerits and merits of group versus individual decision making. However, we are unaware of any studies that have compared group versus individual decision making in conservation. Moreover, there are no systematic reviews in any discipline on decision making by groups versus individuals (but see [1, 2, 6] for narrative reviews).

Thus far there has been very limited research on cognitive aspects of decision making in conservation [but see [7], though it has been studied extensively in other disciplines (e.g. social psychology). Conservation biologists typically approach decision making (and particularly conflict management) as biologists and not from a cognitive approach [8]. Hence, it is not surprising that the literature on decision making in conservation revolves around short term economic incentives (e.g. payment for crop losses due to wildlife, payment for carbon [9, 10]) or technical solutions (e.g., changes in agri-environment schemes, prioritization aspects [11]) or even legal measures (e.g., laws on hunting in protected areas). There is also a considerable body of literature on structured decision making and adaptive management [12], and the cost-effectiveness of conservation decisions [13]. Psychological biases or other cognitive aspects of the decision making process are rarely considered.

However, a review of 1048 business decisions, found that the process of decision making (e.g., level of participation) was more important (by a factor of six) than the analysis (e.g., detailed financial modelling) in determining the quality of decisions [14]. Thus, psychological factors play a pivotal role in decision making and there is ample evidence to suggest that decisions are often influenced by the context in which judgements are made [15]. In particular, decision making is fraught with psychological biases, as shown by the seminal work of Kahneman and Taversky [16]. The impact of such biases in decision making is considerable and has been the subject of extensive research [15]. Understanding these factors can have significant impact on improving the efficiency of the judgement process [15] and on furthering conservation goals [17] since efficient management interventions routinely require effective and rational decision making. Psychological aspects are particularly relevant for biodiversity decisions as they usually involve multiple stakeholders and often opposing viewpoints [17]. The complexity in decision making is often exacerbated by the political nature of the decision itself (e.g., conservation of hen harriers [17]). Furthermore, the urgency within which decisions need to be made (e.g., in the face of a natural calamity) adds to this complexity.

Different facets of decision making (e.g., gender, type of task, effect of biases etc.) have been compared between groups and individuals over the years but the merits of each are unresolved. Attempts at collating and analysing the evidence on decision making by groups versus individuals are limited to narrative reviews which suffer from lack of documentation of the review process itself (see [1, 2, 6, 18]). They are thereby prone to being biased, subject to inter-reviewer differences and hence not repeatable. A systematic review can provide unbiased insights that cannot be obtained otherwise from individual empirical studies. This leads to the motivation for the present systematic review.

In this systematic review, we will attempt to assess when, where, and to what extent, are group decisions different from individual decisions. Are groups more rational or are they more error prone? Rationality refers to the rational choice theory in economics. Error refers to deviation of responses from the “true” value for questions where a “true” value is already known or can be easily calculated (e.g., intellective tasks).

The expected outcomes of the present review could be useful to a range of decision makers or to those who facilitate decision making in the real world. For example, both the International Union for the Conservation of Nature (IUCN) and the New Zealand Department of Conservation have structured their delivery of conservation advice through a series of specialist groups or teams, who collectively develop advice and inform decisions on particular taxa [19]. Often these groups meet in person to define their advice. If the research reviewed here indicated that decisions made by individuals are actually more rational, or more likely to be correct than decisions made by groups, then these organisations might want to reconsider the group-based structure. Conversely, government policymakers often seek advice from independent experts individually [5]. If our review finds that decisions made by groups are consistently more rational, or more likely to be correct, than those made by individuals, then such processes could be redesigned. Our results will be actively communicated to conservation NGOs and policymakers, through our direct involvement in a range of conservation-related science-policy interfaces and networks (for example, the Cambridge Conservation Initiative, and the Eklipse project [20]). In addition, the review might be useful to the theory and practice of decision making as traditionally studied in disciplines such as economics or social psychology.

Key concepts from other disciplines

While individual behaviour in decision making has received substantial attention [15], decision making in groups is relatively understudied. Some of the empirical research on groups tends to suggest that groups perform better than individuals in decision outcomes for selected biases, while others suggest that groups exacerbate selfishness and myopic loss aversion compared to individuals [1]. Decision making by groups without any interaction can be vastly different from those in interactive face-to-face settings due to “groupthink” [21] and “egocentrism” effects [22, 23]. Groupthink refers to the phenomenon whereby individuals in a group tend to seek agreement, harmony and unity among the group members at the expense of independent critical thinking and individual better judgement. The tendency to be accepted as a good group member leads to acceptance of the majority solution that may not be evidence based or rational [21]. Egocentrism refers to the situation where individuals falsely assume that their views are commonly agreed upon by the majority and they overestimate the consensus level [23]. Investigating the differences between group versus individual decision making under a range of scenarios can improve our current understanding of the decision making process [18] and help reduce some of these biases.

Group versus individual decision making has been investigated for over four decades in several disciplines other than conservation (e.g., economics and social psychology), leading to significant advancements in these fields. For instance, in economics decision making has been extensively investigated under the paradigm of game theory. Typically these involve decisions in an interactive context where the outcome of a decision depends to a certain extent on the decisions of others. Game theory usually has some interesting assumptions, namely: (1) participants in a game have perfect knowledge of their interests and preferences, (2) participants have the ability to perfectly calculate the actions necessary to serve these interests, (3) participants are self-centred and care only about their own interests, (4) each participant has perfect knowledge of the rules of the game and knows that others are aware of 1–3 above.

If all these assumptions are valid, then each participant’s behaviour in most games (e.g., Prisoner’s Dilemma) can be predicted by a “rational benchmark” (e.g., Nash Equilibrium) [15]. This also makes it easier to calculate deviations from such rational benchmarks. However a sizeable volume of research has questioned these assumptions [15]. Thus, though game theory can act as a normative theory to estimate rational decision making (or deviations thereof), it falls short of being a descriptive tool. The deviations are often influenced by biases and form the fodder for investigation of social psychologists. In addition, research on collective action may be a good predictor of deviations from rational theory. Rational theory traditionally uses self-interested decisions and thresholds, such as Nash equilibrium, to judge individual decisions. These assumptions may not hold in the context of collective action where the objective is to maximise the group goal or group objective (which may be different from individual goals or objectives).

In social psychology, group research (particularly on small groups) has focussed on group performance for a variety of tasks (e.g., letters to numbers problem, mathematical tasks) and the processes by which the decision is arrived at (e.g., discussion versus no discussion, aggregation of responses, decision rules used, information processing) (see review [2, 24]).

Both intellective and judgemental tasks are interesting from a conservation perspective as practitioners routinely engage in making decisions that cognitively rely on either task (or both). For instance, consider the decision to invest in a guided busway as opposed to a bike lane in a city. Both might contribute to reducing carbon emissions by providing alternate modes of traffic compared to cars. While deciding on the basis of quantitative evidence available with respect to costs and feasibility may be intellective task, the final decision may rely on the “buy in” and prioritisation by different stakeholders engaged in making the decision (a judgemental task).

Research on conservation decision making is relatively in its nascent stage in conservation and much remains unknown. The advances in other disciplines could provide critical transferable knowledge for conservation. To avoid past mistakes, and to facilitate better decision making in conservation, an interdisciplinary comparison of decision making in different disciplines could be critical.

Objective of the review

This systematic review aims to summarize the best available evidence for group and individual based decision making across a range of disciplines. The interdisciplinary nature of the review makes it vulnerable to intellectual turf wars between disciplinary boundaries. However, we have strived to make the aims, objectives and methodology of the review as transparent and explicit as possible. In doing so, we have made an effort to look beyond the disciplinary silos and search for patterns and processes that shape decision making in groups versus individuals.

Primary question

In what contexts do groups differ from individuals in decision making?

Components of primary question

Population/subject a population of decisions made by humans either as groups or as individuals.

Intervention group decisions.

Comparator individual decisions.

Outcome task outcome for groups and individuals (e.g., greater accuracy, disposition to bias).

Secondary-questions

The sub-questions are:

• 1. Under what context(s) in decision making have groups and individuals been compared so far?

• 2. What proportion of studies has used both qualitative and quantitative analysis?

• 3. Where have the studies been conducted?

• 4. Which behavioural biases have been compared between groups and individuals?

• 5. What proportion of the empirical studies used hypothetical versus real choice in their experiments?

• 6. Do groups perform better/worse in judgemental tasks compared to individuals?

• 7. Do groups perform better/worse in intellective tasks compared to individuals?

• 8. Does being in an anonymous/vs interactive setting impact decision outcomes?

Methods

Search strategy

The search strings have been modified to ensure an optimum level of both sensitivity and specificity. Sensitivity refers to the ability of a search to capture all relevant articles. Specificity refers to the ability of a search to capture only relevant articles and minimize irrelevant hits. In order to account for specificity of the search, several iterations of the keywords have been used in the scoping search (see Additional file 1) until a desirable level of irrelevant hits was excluded. Briefly, we tested a range of biases in decision making research based on published reviews on the topic [1, 6]. However, the search terms were narrowly defined and resulted in few hits. To increase the sensitivity of the search it was later decided to broaden the scope of the search and not use any of the biases as search terms. We also made a preliminary list of games/procedures that have been used to test differences between groups and individuals.

Search terms

The main objective of the review is to test differences between groups and individuals (comparator) in decision making (outcome). In keeping with this over-arching framework the terms “group and individual” and “decision making” will be used in all the searches. The search terms have already been tested in the scoping search (see Additional file 1 for scoping exercise). Search engines differ in their functionality. While some e.g., Scopus, allow references to be sorted based on “relevance”, others do not. In case there is a possibility to search for terms in different locations in a text (e.g., title or abstract) we will limit the search to title, abstract and keywords (e.g., the field ‘Topic’ in Web of Science). Searching for terms in the full text was found to be low in specificity in a preliminary search (see Additional file 1). Wildcards e.g., group* will also be used. An asterisk is a ‘wildcard’ that represents all possible characters including no character. Where Boolean operators (e.g., AND/OR/NOT) and wildcards are permissible, the following search terms will be used:

• 1. TOPIC (group* and individual*) AND TOPIC: (“decision making”) AND TOPIC: (interact*).

• 2. TOPIC (“group* and individual*”) AND TOPIC: (“decision making”) AND TOPIC: (interact*).

• 3. TOPIC (group* + individual*) AND TOPIC: (interact*) AND TOPIC: (“decision making”).

• 4. TOPIC (group* + individual*) AND TOPIC: (“decision-making”) AND TOPIC: (interact*).

• 5. TOPIC (group* + individual*) AND TOPIC: (“decisio*”) AND TOPIC: (interact*).

• 6. TOPIC (group* + individual*) AND TOPIC: (“decision making”) AND TOPIC: (anonym*).

• 7. TOPIC (group* + individual*) AND TOPIC: (“decision making”) AND TOPIC: (“intellective task”).

In the scoping exercise, related search terms have been experimented with by a team of two authors. In case Boolean operators are not allowed, the input will be modified to include iterative variations of “group*, “individual*” and *decision making*”. Since search engines vary in their use of search strings or keywords, the search process will be modified accordingly. For instance in the scoping exercise it was found that DOAJ does not support complex strings. Hence, search strings will be tailored for optimization.

The search will not be limited to any document type restrictions or geographic restrictions. However, only English language terms will be used and only peer-reviewed journal articles published in English will be included in the review. We acknowledge that there may be articles in other languages, however the resources required for translating these documents or for including grey literature are beyond the scope of this review.

Databases

The search will be limited to the years 1995–2015. The rationale is that in most disciplines, there has been an exponential boom in publications post 1995 due to technological advancements. In addition, the studies prior to 1995 which are relevant for decision making will be covered in the narrative reviews on the topic. Search strings used for each database will be provided, as an Additional file 1 in the final review, to enable repetition. Given the financial and other resource limitations of this study, it was also considered most optimal to limit the database search to the past 20 years.

The following online databases will be searched.

• 1. ISI Web of Science.

• 2. Scopus.

• 3. Science direct.

• 4. DOAJ.

• 5. JSTOR.

Web search engines

Each search engine has its own algorithm for returning search results (usually based on centrality indices e.g., eigenvector centrality). Though the sensitivity of web engines is relatively high, the specificity may be low compared to databases such as Web of Science. In addition some websites do not allow all search hits to be downloaded at once. Hence, to optimise search effort, the first 50 hits will be saved and searched for relevant articles. The total number of hits will also be recorded.

• 6. Google: http://www.google.com.

• 7. Google scholar http://scholar.google.com/

Specialist sources

• 8. Repec.

In the scoping search, the specialist database Repec [25] was found to be particularly useful. The search results could not be automatically downloaded. Hence the hits will be manually searched for the top 250 articles (100–27 % relevance). In the scoping exercise it was found that several of the working papers in Repec database had been subsequently published in peer-reviewed journals. To avoid overlap, the peer-reviewed published versions of the working papers will be used in the database.

• 9. PsycINFO.

PsycINFO [26] was suggested during the expert consultation phase and found to be an excellent repository of social psychology articles on decision-making. The top 250 hits will be manually searched for articles on decision making. Only peer-reviewed articles will be searched.

In addition, one expert each from a range of disciplines (economics, management and social psychology) will be consulted after the preliminary selection of articles and asked to contribute their insights into the literature database.

Bibliographies and citations of key references

The search is intended to be comprehensive and unbiased. The bibliography of articles selected for the final review and bibliographies of narrative reviews will be searched to cross-check any references missed in the initial screening. Missing articles identified through bibliographies will be subjected to the same inclusion/exclusion protocol for original studies and included in the final review if relevant. All relevant articles from key narrative reviews will be used as a benchmark to check the sensitivity of the search in the scoping process.

The bibliographies of all narrative reviews will be searched manually by a team of two researchers. Citations to all articles identified in the final screening process will be searched online.

All references will be exported to an Endnote library that will be added as an appendix as a text file to the final review. Duplicates will be removed before selecting relevant articles. The search will be limited to pdf documents.

Article screening and study inclusion criteria

An endnote database will be created for all the documents. Initially, separate lists will be compiled for each combination of search terms for each of the databases searched. Later these will be compiled into a combined list and sorted for relevance. The articles will be screened based on their titles and abstracts and then the full text in a two-step process. A subset of 200 articles from two different databases will be screened by two independent reviewers for title, abstract and full text screening. The aim is to assess the neutrality of the screening process and reduce between-reviewer differences in all three stages. The Kappa statistic will be used to measure the level of agreement. If the Kappa is less than 0.6 then the screening will be redone after discussing the differences in interpretation.

A note will be made of all hits for which the pdfs could not be found but were found to be relevant at the title and abstract level. In case the abstract is not available, the article will pass on to the next stage of full text screening. Figure 1 gives a schematic representation of the steps involved in the screening process. The number of articles at each step will be noted. If required the process will be iterated to refine the decision rules used for inclusion/exclusion. List of studies that are included at each stage will be added as Additional file 1.

Fig. 1

Schematic decision tree for the systematic review screening process

As outlined in Fig. 1, there are several steps in the article screening phase. In the first phase all duplicates will be removed. In subsequent phases, the following criteria will be used to decide inclusion/exclusion of an article:

• Population the review will include all categories of decision making where groups and individual decisions have been compared since the main aim of this review is to understand decision making. The articles should thereby be specifically focussed on decision making and measure performance of groups and individuals in decision making. The study must specifically be carried out on humans. In the scoping exercise, some articles were found on decision making in non-human primates or other animals which could be eliminated using the “NOT” operator. These studies will not be included in the review. In the review scoping, the most common subjects were found to be students though this review is ideally targeted at understanding decision making in real world situations (often with experts). A note will be made for both these categories (students and experts).

• Interventions the intervention being studied is group decision making.

• Comparator the comparator is individual decision making. Only those studies which have compared groups to individuals in decision making will be included. Studies looking only at individuals or only at groups will not be included. We will make a note of such studies but they will not be included in the analyses. Such articles may be referred to in the discussion section.

• Type of study studies that have empirically tested and quantified the differences between groups and individuals in a decision making context will be included. Articles which have merely mentioned decision making in their discussion or conclusion section, without empirically measuring/investigating the decision process, will not be included. Narrative articles will not be included. Only primary sources of information will be included in the analyses. Conference proceedings, opinion pieces, editorials and patents will not be used in the analyses. Reviews will also not be included in the systematic review. Reviews will be used only as sources of primary information particularly for papers published before 1995 and for cross-checking the sensitivity of the search terms. Within each study, a note will be made of the decision rules used in the experiments (e.g., decision by unanimity vs majority). Intra-group factors such as group size, age differences etc., will be recorded, but will not form a part of the analysis. Gender composition of groups will be noted wherever provided. For the purpose of this study, we will consider only the effect of being in a group irrespective of these intra-group factors. In the case of studies on rational choice, where rationality is defined by a particular Nash equilibrium, only those studies will be included that clearly mention the Nash equilibrium or the rational threshold. In such studies, the threshold forms the baseline for comparison between groups and individuals. Disciplinary bias (i.e. in some disciplines decision making may be studied more than others) unfortunately may not be avoided. This might result in fewer studies to compare across studies using the same procedure.

Study quality assessment

After the initial selection of articles based on the title and abstract, the articles will be critically appraised for internal validity. Studies where the methodology has not been clearly described (e.g., not provided sample size) will be excluded. Studies where there was a bias in the study sample or have poor study design will be excluded. If the sample size is too low, then the study will be made a note of but not included in subsequent meta-analysis. Studies where the same data has been published twice in different journals by the same group of authors will be treated as only one study.

In the scoping exercise, we came across studies that have provided experimental results for ‘before/after’ (BA) the treatment (being in a group). However, for the purpose of this study we will focus on only ‘comparator/intervention’ (CI) part of the studies and not on the ‘BACI’ design. Firstly the main aim of this exercise is to compare group versus individuals. Hence whether a participant was in a group before or after is not of much consequence. Secondly, the BA comparison may not have been tested in majority of the studies. In order to be consistent we will adhere to the ‘comparator/intervention’ design throughout the analysis.

The studies will be screened for quality by one of the reviewers using the critical appraisal tool designed by the Critical Appraisal Skills Programme [27]. In the case of ambiguous studies, the team will take a decision by unanimity. A subset of 50 studies will be screened by two independent reviewers to assess inter reviewer variability. The kappa analysis will be repeated to test for inter-reviewer consensus of at least 0.6. In the case of disagreement, the study quality issues will be rechecked and differences resolved through consensus.

Rationale for exclusion of each article will be noted in a spreadsheet in a transparent way. This spreadsheet will be included as an appendix in the final review.

Data extraction strategy

The final selected articles will be categorised based on the methodology used to test differences between groups and individuals. For instance, the articles will be primarily classified into two categories: intellective tasks and judgemental tasks. All data will be extracted into a spreadsheet and included as an Additional file 1 in the final review. The test library of the preliminary scoping search has been used to identify and finalise the data categories that will be extracted from the final set. The following data will be screened for:

• Place of study (location and country).

• Time period (year of publishing and year when study was conducted, if mentioned).

• Type of task (intellective, judgemental).

• Target group (students, experts, gender).

• Group size.

• Type of setting (anonymous, interactive, mixed).

• Sample size.

• Outcome measures (qualitative, quantitative).

• Psychological biases tested (if any).

• Procedure used (e.g., lottery game, beauty contest game).

• Biases in conducting the study (if any).

• Accuracy of response (if tested or not).

• Type of test/game used.

• Type of decision rule used to arrive at group decision (e.g., majority, unanimity).

• Group mean.

• Group standard deviation.

• Group sample size.

• Individual mean.

• Individual standard deviation.

• Individual sample size.

• Baseline for comparison in the case of game theory tasks (e.g., number of safe choices for lottery games).

If group and individual mean and standard deviation values are not available then the following will be noted:

• p value.

• t test value.

• Any other statistic used to measure group and individual differences.

Within each category the decision rules used to arrive at the group decision will be used as subgroups (e.g., unanimity versus majority rule for group decisions, trustor versus trustee for trust games). Data extraction will be piloted for a subsample of the articles. Based on this initial analysis the data extraction strategy will be modified appropriately. In case there is missing data, the author of the original sources will be contacted by email requesting information for the data gaps. The full list of articles selected for the final review will be added as an Additional file 1.

Potential effect modifiers and reasons for heterogeneity

We speculate that the potential reasons for heterogeneity in the dataset could be due to the following:

• Limited number of studies, hence less statistical power.

• Variability in the participants (e.g. gender).

• Variability in the methodology used to compare group decisions with individual decisions.

• Subgroup types e.g., the decision rule used to arrive at a group decision is more powerful than being in a nominal group.

This list was compiled based on the scoping review and consultation with experts.

Data synthesis and presentation

The studies will be grouped according to the descriptor data. Data will be synthesized in the form of a narrative text with a table summarizing the findings from each paper based on the criteria outlined in the previous section. If adequate reliable data are available then we will proceed towards a meta-analysis. In such a scenario, both fixed effects and random effects models will be used to investigate the differences between groups and individuals within each procedure category (e.g., lottery game). Depending on the availability of the data, subgroup analysis will be carried out. The difference between groups and individuals (i.e. effect size) will be calculated using Hedges d and/or Odds ratio. The data will be presented as forest plots with accompanying information on heterogeneity (I² values) and p value. In addition, sensitivity analysis and publication bias will also be presented.

Definitions

• 1. Anonymous decision setting where members in a group interact in an anonymous setting via a facilitator (or computer). Individual identities are not known and this reduces several biases (e.g., dominance effect, halo effect).

• 2. Interactive decision setting where group members can freely discuss their ideas to reach a consensus.

• 3. Mixed setting decision setting that would involve a mixture of anonymous and interactive decisions.

• 4. Intellective task tasks for which a “correct” answer/solution exists (or can be arrived at) and clearly demonstrated. Examples include estimating the population size of a country or time taken to solve a mathematical task.

• 5. Judgemental task tasks for which there is no defined correct solution. Judgemental tasks include estimating the price of an item or deciding how much to reciprocate in a gift exchange scenario.

• 6. Hypothetical choice experiments experiments where no real money is offered as an incentive for better performance/decision outcome.

• 7. Real choice experiments decision experiments where real money is offered as an incentive for better performance/decision outcome.