Table 1 Types of bias relevant to the environmental sector in the alphabetical order
| Â | Types of bias | General interpretations | Bias occurring in* | Bias relating to** | Relevant domains of bias*** | Source**** |
|---|---|---|---|---|---|---|
1 | All’s well literature bias | Occurs when review papers deliberately omit or inadequately address studies that present contrasting results compared to studies yielding the results desired by certain individuals or groups of individuals | Secondary research | Screening Data synthesis | Screening biases Data synthesis biases | |
2 | Allocation bias (Allocation of intervention bias) | Occurs when non-random assignment of participants to study groups (i.e., allocation) creates an imbalance between the groups (i.e., exchangeability does not hold) | Primary research | Exchangeability | Confounding biases | |
|  | Allocation of intervention bias | See 2 ‘Allocation bias’ |  |  |  |  |
3 | Allocation sequence bias | Occurs when the allocation sequence is not concealed to researchers and the selection creates an imbalance between the study groups (i.e., exchangeability does not hold) | Primary research | Exchangeability | Confounding biases | [34] |
4 | Analysis bias | Occurs when the knowledge of the results of a study causes inappropriate analyses and selective reporting | Primary research Secondary research | Reporting Statistical methods Data synthesis | Outcome reporting biases Outcome assessment biases Data synthesis biases | |
5 | Ascertainment bias | Occurs when the ascertainment of subjects or areas included in a study or the ascertainment of measurements or collected data for analysis differs between the groups after the intervention/exposure | Primary research | Exchangeability Measurement | Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases | |
6 | Attention bias (Hawthorne effect) | Occurs when human or animal subjects change their behaviour due to the awareness of being observed by researchers | Primary research | Exchangeability Measurement | Confounding biases Performance biases | |
7 | Attrition bias | Occurs when missing data between groups are unequal, and exchangeability does not hold (cannot be assumed) | Primary research | Exchangeability | Post-intervention/exposure selection biases | |
8 | Authorization bias | Occurs when there is inability to obtain authorization for use of certain data or documents, and that causes deviation from a true effect | Primary research Secondary research | Exchangeability Searching | Confounding biases Post-intervention/exposure selection biases Searching biases | [36] |
9 | Availability bias | Occurs when the information that is readily available differs systematically from the information obtained by aggregating all pertinent data to answer the review question | Secondary research | Searching | Searching biases | |
10 | Bigwig bias | Occurs when information in documents written by high profile authors differs systematically from the wider evidence base relevant to the review question and is more likely to be cited as evidence in a review | Secondary research | Searching Screening Data synthesis | Searching biases Screening biases Data synthesis biases | [29] |
11 | Bogus control bias | Occurs when subjects or areas initially assigned to the intervention/exposure group are taken out from that group or moved to the control (no intervention/exposure) group and the assumption of exchangeability no longer holds | Primary research | Exchangeability | Confounding biases Post-intervention/exposure selection biases | |
12 | Case definition bias | Occurs when the intervention/exposure is vaguely described or lacks relevant information, and the vagueness or the lack of information leads to mismeasurement of the intervention/exposure by researchers in an observational study | Primary research | Measurement | Misclassified/mismeasured comparison biases | [34] |
13 | Channelling bias | Occurs when researchers assign subjects or areas, that are more likely to respond to the intervention/exposure, to the intervention/exposure group, creating an imbalance between the groups (i.e., exchangeability does not hold) | Primary research | Exchangeability | Confounding biases | |
14 | Chronological bias | Occurs when the timing of allocation of subjects or areas to a group (or groups) is different within or between groups and the time lag influences both the intervention/exposure and the outcome, or the subjects or areas are subject to a deviated intervention/exposure due to the time lag | Primary research | Exchangeability Measurement | Confounding biases Performance biases | |
15 | Citation bias | Occurs when studies with statistically significant results are more likely to be cited as pieces of evidence in a review | Secondary research | Searching Screening Data synthesis | Searching biases Screening biases Data synthesis biases | |
|  | Classification bias | See 52 ‘Information bias’ |  |  |  |  |
16 | Cognitive dissonance bias | Occurs when an investigator or a reviewer’s claim about an effect contradicts their obtained evidence due to the investigator or reviewer’s cognitive bias | Primary research Secondary research | Statistical methods Data synthesis | Outcome assessment biases Data synthesis biases | |
17 | Collider bias | Occurs when conditioning on a variable (i.e., filtering by a certain value of that variable) that is a common effect of the exposure and the outcome | Primary research | Exchangeability | Confounding biases Post-intervention/exposure selection biases | [55] |
18 | Compliance bias | Occurs when subjects compliant with the intervention differ from those not compliant in some characteristic and the characteristic is also associated with the outcome, or when non-compliance causes deviated initiation, implementation or discontinuation of the intervention between groups and the deviation is not taken into account | Primary research | Exchangeability Measurement | Confounding biases Post-intervention/exposure selection biases Performance biases | |
19 | Confidentiality bias | Occurs when unrestricted research data, that systematically differs from restricted research data, are exclusively used due to commercial or privacy considerations | Primary research Secondary research | Exchangeability Searching | Confounding biases Searching biases | [29] |
20 | Confirmation bias | Occurs when the information to confirm ideas, beliefs or hypotheses is systematically different from the truth | Primary research Secondary research | Exchangeability Measurement Reporting Statistical methods Searching Screening Study appraisal and data coding/extraction Data synthesis | Confounding biases Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases Outcome reporting biases Outcome assessment biases Searching biases Screening biases Study appraisal and data coding/extraction biases Data synthesis biases | |
21 | Conflict of interest | Occurs when a conflict of interest causes a systematic difference in estimation of effect | Primary research Secondary research | Exchangeability Measurement Reporting Statistical methods Searching Screening Study appraisal and data coding/extraction Data synthesis | Confounding biases Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases Outcome reporting biases Outcome assessment biases Searching biases Screening biases Study appraisal and data coding/extraction biases Data synthesis biases | [33] |
22 | Confounding bias | Occurs when an uncontrolled (or inappropriately controlled) variable (confounder) that influences both the intervention/exposure and the outcome exists | Primary research | Exchangeability | Confounding biases | |
23 | Confounding bias by group | Occurs when there is a variable describing communities (groups) and that influences both the intervention or exposure and the outcome | Primary research | Exchangeability | Confounding biases | [35] |
24 | Confounding bias by indication | Occurs when there exists an indication for the intervention/exposure (a preceding phenomenon that affects the intervention/exposure) or a variable influenced by this indication that affects both the intervention/exposure and the eventual outcome | Primary research | Exchangeability | Confounding biases | |
25 | Contamination bias | Occurs when subjects or areas in the comparator group (no or alternative intervention/exposure) accidentally receive the intervention/exposure (and vice versa), and this contamination leads to the violation of the assumption of exchangeability | Primary research | Exchangeability | Confounding biases | |
26 | Correlation bias | Occurs when equating non-causal association with causal association | Primary research Secondary research | Exchangeability Study appraisal and data coding/extraction Data synthesis | Confounding biases Study appraisal and data coding/extraction biases Data synthesis biases | |
|  | Cost bias | See 100 ‘Resource bias’ |  |  |  |  |
27 | Co-treatment bias | Occurs when inappropriate or unplanned co-treatment(s) is/are used in an experiment that results in confounding or altered treatment(s) | Primary research | Exchangeability Measurement | Confounding biases Performance biases | |
28 | Data capture bias (Data capture error) | Occurs when data capture (data collection and recording) errors favour or disfavour the outcome for the intervention/exposure group or the comparator group | Primary research Secondary research | Exchangeability Measurement Study appraisal and data coding/extraction | Confounding biases Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases Study appraisal and data coding/extraction biases | [34] |
|  | Data capture error | See 28 ‘Data capture bias’ |  |  |  |  |
29 | Data collection bias | Occurs when data collectors' beliefs influence the way relevant data are collected and that results in inaccurate estimation of an effect | Primary research Secondary research | Exchangeability Measurement Study appraisal and data coding/extraction | Confounding biases Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases Study appraisal and data coding/extraction biases | [31] |
|  | Data completeness bias | See 68 ‘Missing data bias’ |  |  |  |  |
|  | Data entering bias | See 30 ‘Data entry bias’ |  |  |  |  |
30 | Data entry bias (Data entering bias) | Occurs when omitting data entry, inaccurately entering data, or inaccurately converting hand-written data into digital form | Primary research Secondary research | Exchangeability Measurement Statistical methods Study appraisal and data coding/extraction | Confounding biases Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases Outcome assessment biases Study appraisal and data coding/extraction biases | |
31 | Data merging bias | Occurs when multiple data sets are merged incorrectly and the merged data produce an intentionally or unintentionally inaccurate estimate of an effect | Primary research Secondary research | Reporting Statistical methods Study appraisal and data coding/extraction | Outcome reporting biases Outcome assessment biases Study appraisal and data coding/extraction biases | [34] |
32 | Data-dredging bias (‘Looking for the pony’) | Occurs when the results of inappropriate statistical analyses or synthesis methods, that are not pre-specified, are presented, or statistical or synthesis methods are used inappropriately to support claims | Primary research Secondary research | Reporting Statistical methods Data synthesis | Outcome reporting biases Outcome assessment biases Data synthesis biases | |
33 | Definition bias | Occurs when definitions of important words used in a study or a review are not accurate or precise so that misinterpretation affects estimation of an effect | Primary research Secondary research | Exchangeability Measurement Searching Screening Study appraisal and data coding/extraction | Confounding biases Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases Searching biases Screening biases Study appraisal and data coding/extraction biases | [32] |
34 | Design bias | Occurs when inferior experiments or treatments, reflecting a desire for certain results, are employed | Primary research | Exchangeability Measurement | Confounding biases Post-intervention/exposure selection biases Performance biases Detection biases | |
35 | Detection bias | Occurs when there are systematic differences in measurement of outcomes between groups | Primary research | Measurement | Detection biases | |
36 | Differential maturing bias | Occurs when there are uneven secular trends (long-term and sustained changes or patterns) between the groups due to differential maturing, and that creates an imbalance between the groups before the intervention or exposure (time-varying confounding) | Primary research | Exchangeability | Confounding biases | [35] |
37 | Dissemination bias | Occurs when retrieval of information does not result in a representative sample of studies due to the way study results are disseminated, or when study results affect reporting or dissemination | Primary research Secondary research | Reporting Searching | Outcome reporting biases Searching biases | [35] |
38 | Distribution assumption bias | Occurs when a data analyst conducts a statistical test under a false assumption of the data distribution or the approach is not justifiable so that the inference is invalid | Primary research | Statistical methods | Outcome assessment biases | [34] |
|  | Dropout bias | See 120 ‘Withdrawal bias’ |  |  |  |  |
39 | Duplication bias (Multiple publication bias) | Occurs when duplicated research findings are included and synthesised in a review | Secondary research | Screening | Screening biases | |
40 | End-digit preference bias | Occurs when there are systematic differences in quantitative measurements due to a preference for a certain ending digit (e.g., 0 by rounding up/down rather than retaining the original digits), typically ending with an unusual frequency | Primary research Secondary research | Statistical methods Study appraisal and data coding/extraction | Outcome assessment biases Study appraisal and data coding/extraction biases | |
41 | Estimator bias | Occurs when an inappropriate estimator (a statistic used to estimate the parameter) is used for estimating the true causal effect (parameter) | Primary research Secondary research | Statistical methods Data synthesis | Outcome assessment biases Data synthesis biases | [34] |
42 | Exclusion bias | Occurs when the criteria for inclusion of subjects or areas into a study or analysis are applied differently to the intervention/exposure group and the comparator group | Primary research | Exchangeability | Confounding biases Post-intervention/exposure selection biases | |
43 | Expectation bias | Occurs when observers err in measurement, recording, or data collection due to prior expectations | Primary research Secondary research | Exchangeability Measurement Searching Screening Study appraisal and data coding/extraction | Confounding biases Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases Searching biases Screening biases Study appraisal and data coding/extraction biases | |
44 | Exposure suspicion bias | Occurs when a knowledge or awareness of the subject's or area's exposure status influences selection, measurement, reporting, or analysis | Primary research | Exchangeability Measurement Reporting Statistical methods | Confounding biases Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases Outcome reporting biases Outcome assessment biases | |
45 | Familiarity bias | Occurs when reviewers limit searching to documents relevant to their own discipline | Secondary research | Searching | Searching biases | [29] |
46 | Foreign language exclusion bias | Occurs when language is restricted in a review and studies with certain direction of results or certain topics are more or less likely to be disseminated in the eligible language(s) | Secondary research | Searching Screening | Searching biases Screening biases | [34] |
47 | Friend control bias | Occurs when there is an association of the actual intervention/exposure status between human subjects in the intervention/exposure group and their friends in the comparator group due to mismeasurements, misclassifications of or the lack of adherence to the intervention/exposure | Primary research | Measurement | Performance biases | [35] |
|  | Handling data bias | See 68 ‘Missing data bias’ |  |  |  |  |
|  | Hawthorne effect | See 6 ‘Attention bias’ |  |  |  |  |
48 | Hot stuff bias | Occurs when research results on hot topics are published even though they are not valid | Primary research Secondary research | Exchangeability Measurement Reporting Statistical methods Searching Screening Study appraisal and data coding/extraction Data synthesis | Confounding biases Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases Outcome reporting biases Outcome assessment biases Searching biases Screening biases Study appraisal and data coding/extraction biases Data synthesis biases | |
49 | Inclusion bias | Occurs when inclusion of subjects or areas into a study creates an imbalance between the groups (i.e., exchangeability does not hold) | Primary research | Exchangeability | Confounding biases Post-intervention/exposure selection biases | |
50 | Inclusion control bias | Occurs when some subjects or areas of the comparator group also receive the intervention/exposure to some extent (biased towards null when there is an effect) | Primary research | Exchangeability | Confounding biases Post-intervention/exposure selection biases | [34] |
51 | Industry sponsorship bias | Occurs when research outcomes are intentionally aligned with the interests of the funding commercial manufacturer, undermining the validity of the inference | Primary research Secondary research | Exchangeability Measurement Reporting Statistical methods Searching Screening Study appraisal and data coding/extraction Data synthesis | Confounding biases Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases Outcome reporting biases Outcome assessment biases Searching biases Screening biases Study appraisal and data coding/extraction biases Data synthesis biases | |
52 | Information bias (Observation bias, Classification bias) | Occurs when the information used in a study deviates from the truth | Primary research | Exchangeability Measurement | Confounding biases Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases | |
53 | Insensitive measure bias | Occurs when changes or differences in the outcome of interest are not detected accurately due to insufficient methods or inappropriate choice of an outcome variable | Primary research | Measurement | Detection biases | |
54 | Instrument bias | Occurs when instruments used for measurement are inaccurate, too imprecise or different between groups (hence not comparable between groups) | Primary research | Exchangeability Measurement | Confounding biases Misclassified/mismeasured comparison biases Performance biases Detection biases | |
55 | Interobserver variability bias | Occurs when multiple observers' measurements of the intervention/exposure or the outcome are different, and the differences influence the estimate of an effect | Primary research | Measurement | Misclassified/mismeasured comparison biases Performance biases Detection biases | [34] |
56 | Interpretation bias | Occurs when researchers misinterpret the obtained data and results | Primary research Secondary research | Statistical methods Study appraisal and data coding/extraction Data synthesis | Outcome assessment biases Study appraisal and data coding/extraction biases Data synthesis biases | |
57 | Intra-observer variability bias | Occurs when an observer measures the intervention/exposure or the outcome differently and the differences influence the estimate of an effect | Primary research | Measurement | Misclassified/mismeasured comparison biases Performance biases Detection biases | [34] |
58 | Investigator bias | Occurs when study investigators (measurers) are not blinded and the lack of blinding influences estimation of an effect | Primary research | Exchangeability Measurement | Confounding biases Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases | [37] |
59 | Laboratory data bias | Occurs when procedures implemented at a laboratory produce a systematic error in the estimate of an effect (may be confirmed by comparing data from multiple laboratories) | Primary research | Exchangeability Measurement | Confounding biases Post-intervention/exposure selection biases Performance biases Detection biases | [34] |
60 | Lack of blinding bias | Occurs when the lack of concealment of the intervention, exposure or control received by subjects or areas affects selection of subjects, areas or data, measurement of the treatment or the outcome, and/or assessment of the effectiveness or the impact | Primary research | Exchangeability Measurement Statistical methods | Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases Outcome assessment biases | [65] |
|  | Lack of intention to treat analysis bias | See 117 ‘Treatment analysis bias’ |  |  |  |  |
61 | Language bias | Occurs when studies with certain results or topics are more likely to be published in a certain language | Secondary research | Searching Screening | Searching biases Screening biases | |
62 | Latency bias | Occurs when the outcome is assessed in an inappropriate time interval (i.e., the time interval is too short or long) for the intended assessment of causal effect | Primary research Secondary research | Statistical methods Data synthesis | Outcome assessment biases Data synthesis biases | [34] |
63 | Literature search bias | Occurs when searching of literature does not capture a representative sample of all existing studies relative to the review question | Secondary research | Searching | Searching biases | [34] |
64 | Location bias | Occurs when some journals are not indexed in all databases so that the choice of databases influences results of literature searching | Secondary research | Searching | Searching biases | [29] |
|  | ‘Looking for the pony’ | See 32 ‘Data-dredging bias’ |  |  |  |  |
|  | Loss to follow-up bias | See 120 ‘Withdrawal bias’ |  |  |  |  |
65 | Measurement bias | Occurs when measurements of relevant variables are not accurate or precise enough | Primary research | Measurement | Misclassified/mismeasured comparison biases Performance biases Detection biases | [31] |
66 | Meta-analysis bias | Occurs when meta-analyses based on a non-representative sample of studies are conducted | Primary research Secondary research | Searching Screening Study appraisal and data coding/extraction Data synthesis | Searching biases Screening biases Study appraisal and data coding/extraction biases Data synthesis biases | [36] |
67 | Misclassification bias | Occurs when subjects or areas, or the exposure or intervention is misclassified, and that distorts the association between the intervention/exposure and the outcome | Primary research | Exchangeability Measurement | Confounding biases Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases | |
68 | Missing data bias (Handling data bias, Data completeness bias) | Occurs when there is an imbalance in missing data between the comparison groups | Primary research | Exchangeability | Post-intervention/exposure selection biases | [34] |
69 | Misuse of statistics | Occurs when descriptive or inferential statistical analysis is inappropriately conducted, resulting in unjustified and invalid conclusions about the estimate of an effect | Primary research Secondary research | Statistical methods Data synthesis | Outcome assessment biases Data synthesis biases | [37] |
70 | Multiple exposure bias | Occurs when another exposure acts as a confounder affecting both the exposure of interest and the outcome | Primary research | Exchangeability | Confounding biases | [34] |
|  | Multiple publication bias | See 39 ‘Duplication bias’ |  |  |  |  |
|  | Neyman bias | See 89 ‘Prevalence-incidence bias’ |  |  |  |  |
71 | Non-contemporaneous control bias (Non-simultaneous comparison bias) | Occurs when there is a systematic difference in the timing of observation or investigation between the intervention/exposure and the control (or alternative intervention/exposure), and exchangeability between groups cannot be assumed due to the time difference | Primary research | Exchangeability | Confounding biases | |
72 | Non-random sampling bias | Occurs when non-random sampling results in an imbalance between the intervention/exposure and the comparator groups (i.e., exchangeability does not hold) | Primary research | Exchangeability | Confounding biases Post-intervention/exposure selection biases | |
73 | Non-respondent bias | Occurs when there is unequal loss of human subjects between the groups due to non-responses, or when there is a systematic difference in outcome measurement between the groups due to late and/or early responses | Primary research | Exchangeability Measurement | Post-intervention/exposure selection biases Detection biases | |
74 | Non-specification bias | Occurs when the intended intervention or exposure is not clearly defined, and an unintentional intervention or exposure occurs that influences both the intended intervention or exposure and the outcome | Primary research | Exchangeability | Confounding biases | [36] |
|  | Non-simultaneous comparison bias | See 71 ‘Non-contemporaneous control bias’ |  |  |  |  |
75 | Novelty bias | Occurs when the interventions or exposures are reported to have a greater effect simply because they are novel | Primary research Secondary research | Exchangeability Measurement Reporting Statistical methods Searching Screening Study appraisal and data coding/extraction Data synthesis | Confounding biases Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases Outcome reporting biases Outcome assessment biases Searching biases Screening biases Study appraisal and data coding/extraction biases Data synthesis biases | [70] |
76 | Obsequiousness bias | Occurs when human subjects alter questionnaire answers due to perception of the desired direction of the outcome held by investigators | Primary research | Measurement | Detection biases | |
|  | Observation bias | See 52 ‘Information bias’ |  |  |  |  |
77 | Observer bias | Occurs when there are systematic differences in the process of observing and recording information between the groups in primary research | Primary research | Exchangeability Measurement | Confounding biases Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases | |
78 | Omitted-variable bias | Occurs when one or more important explanatory variables are not included in a regression model where omission of such variables is not appropriate for estimating an effect | Primary research | Statistical methods | Outcome assessment biases | |
79 | One-sided reference bias | Occurs when citing references to only those supporting one side of available evidence and hence systematically deviating from the truth | Secondary research | Searching Screening | Searching biases Screening biases | |
80 | Optimism bias | Occurs when researchers' or study subjects' beliefs that new interventions are better than established ones influence a study or review | Primary research Secondary research | Exchangeability Measurement Reporting Statistical methods Searching Screening Study appraisal and data coding/extraction Data synthesis | Confounding biases Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases Outcome reporting biases Outcome assessment biases Searching biases Screening biases Study appraisal and data coding/extraction biases Data synthesis biases | [33] |
81 | Outcome reporting bias | Occurs when outcomes are selectively reported in a published document | Primary research | Reporting | Outcome reporting biases | [72] |
|  | Outlier handling bias | See 115 ‘Tidying-up bias’ |  |  |  |  |
82 | Overmatching bias | Occurs when researchers match by (also ‘block on’, meaning filter by a certain value of that variable) a non-confounding variable that is associated with the intervention/exposure but not to the outcome | Primary research | Exchangeability | Confounding biases | |
83 | Perception bias | Occurs when relevant information or subjects, areas, interventions, exposures, controls, alternative interventions, alternative exposures or outcomes are misinterpreted in a study or review | Primary research Secondary research | Exchangeability Measurement Reporting Statistical methods Searching Screening Study appraisal and data coding/extraction Data synthesis | Confounding biases Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases Outcome reporting biases Outcome assessment biases Searching biases Screening biases Study appraisal and data coding/extraction biases Data synthesis biases | [73] |
84 | Performance bias (Procedure bias) | Occurs when the treatment procedure of interest is altered without changing the inferential goal in an experimental study | Primary research | Measurement | Performance biases | |
85 | Positive results bias | Occurs when positive results are more likely to be disseminated or submitted and accepted for publication than non-significant or negative results | Primary research Secondary research | Reporting Searching Screening | Outcome reporting biases Searching biases Screening biases | |
|  | Post hoc analysis bias | See 86 ‘Post hoc analysis bias’ |  |  |  |  |
86 | Post-hoc significant bias (Post hoc analysis bias) | Occurs when researchers choose a critical significance level (α, 0.05 is commonly employed), and/or set a non-directional (two-tailed) or directional (one-tailed) hypothesis after some examination of data in an attempt to yield significant results | Primary research Secondary research | Statistical methods Data synthesis | Outcome assessment biases Data synthesis biases | |
87 | Pre-publication bias | Occurs when previously published research, that is errant, is used to support a particular outcome in primary research | Primary research | Exchangeability Measurement Reporting Statistical methods | Confounding biases Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases Outcome reporting biases Outcome assessment biases | [30] |
88 | Prevailing paradigm bias | Occurs when studies that are relevant to or support prevailing paradigms are more likely to be published in academic journals | Primary research Secondary research | Reporting Searching Screening | Outcome reporting biases Searching biases Screening biases | [29] |
89 | Prevalence-incidence bias (Neyman bias, Selective survival bias) | Occurs when uneven exclusion or attrition of subjects or areas with severe or mild responses occurs between the groups, and the assumption of exchangeability no longer holds | Primary research | Exchangeability | Post-intervention/exposure selection biases | |
|  | Procedure bias | See 84 ‘Performance bias’ |  |  |  |  |
90 | Proficiency bias | Occurs when the intervention/exposure under study is unequally applied to individual subjects or areas due to interpersonal or intrapersonal differences | Primary research | Measurement | Performance biases | |
91 | Publication bias | Occurs when the likelihood of a study being accepted for publication is influenced by the results of the study | Primary research Secondary research | Reporting Searching Screening | Outcome reporting biases Searching biases Screening biases | |
92 | Recall bias | Occurs when recall of relevant events or experiences by human subjects is inaccurate or incomplete | Primary research | Measurement | Misclassified/mismeasured comparison biases Detection biases | |
93 | Reference bias | Occurs when studies referenced as evidence in a review are not a representative sample of all existing studies relative to the review question | Secondary research | Searching Screening Data synthesis | Searching biases Screening biases Data synthesis biases | [29] |
94 | Regression dilution bias | Occurs when imprecise measurements of the intervention/exposure are used to estimate the effect of the intervention/exposure in an observational study | Primary research | Measurement | Misclassified/mismeasured comparison biases | [35] |
95 | Relative control bias | Occurs when there is an association of the actual intervention/exposure status between the intervention/exposure group and the comparator group due to mismeasurements, misclassifications of or the lack of adherence to the intervention/exposure | Primary research | Measurement | Misclassified/mismeasured comparison biases Performance biases | [35] |
96 | Repeated peeks bias | Occurs when repeated peeks at accumulating data in a study or review leads to inappropriate termination of data collection | Primary research Secondary research | Exchangeability Measurement Searching Screening Study appraisal and data coding/extraction | Confounding biases Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases Searching biases Screening biases Study appraisal and data coding/extraction biases | |
97 | Reporting bias | Occurs when certain results of a study or certain studies are more likely to be reported or unreported | Primary research Secondary research | Reporting Searching Screening | Outcome reporting biases Searching biases Screening biases | |
98 | Research bias | Occurs when studies are carried out and reported on specific organisms or systems, or in particular conditions, because of varying levels of practicality or the anticipation of specific outcomes | Primary research Secondary research | Reporting Searching Screening | Outcome reporting biases Searching biases Screening biases | [29] |
99 | Researcher bias (Sponsor bias) | Occurs when fraud or manipulation of research design, data or results occurs due to vested interests of researchers and organisations | Primary research Secondary research | Exchangeability Measurement Reporting Statistical methods Searching Screening Study appraisal and data coding/extraction Data synthesis | Confounding biases Post-intervention/exposure selection biases Misclassified/mismeasured comparison biases Performance biases Detection biases Outcome reporting biases Outcome assessment biases Searching biases Screening biases Study appraisal and data coding/extraction biases Data synthesis biases | [30] |
100 | Resource bias (Cost bias) | Occurs when available budget or resources limit searching for evidence, resulting in a systematic difference between the identified and missed studies | Secondary research | Searching | Searching biases | [29] |
101 | Response fatigue bias | Occurs when study subjects suffer from fatigue due to implementation of the intervention/exposure and their responses are not obtained or they did not go through the entire process of the intended intervention/exposure | Primary research | Exchangeability Measurement | Post-intervention/exposure selection biases Performance biases | [34] |
102 | Review bias | Occurs when a researcher's prior knowledge or belief triggers misinterpretation of available data on the intervention/exposure or the outcome in primary research | Primary research | Measurement | Misclassified/mismeasured comparison biases Performance biases Detection biases | [34] |
103 | Reviewer bias | Occurs when the selection of studies from the available evidence is not a representative sample of all existing studies relative to the review question | Secondary research | Searching Screening Study appraisal and data coding/extraction Data synthesis | Searching biases Screening biases Study appraisal and data coding/extraction biases Data synthesis biases | [30] |
|  | Rumination bias | See 118 ‘Underlying cause bias’ |  |  |  |  |
|  | Sample size bias | See 121 ‘Wrong sample size bias’ |  |  |  |  |
104 | Sampling bias | Occurs when exchangeability between groups does not hold due to employed sampling techniques | Primary research | Exchangeability | Confounding biases Post-intervention/exposure selection biases | [34] |
105 | Scale degradation bias | Occurs when researchers make outcome measurement scales less precise or accurate to obscure differences between groups under comparison | Primary research Secondary research | Measurement Study appraisal and data coding/extraction | Detection biases Study appraisal and data coding/extraction biases | |
106 | Selection bias | Occurs when exchangeability between groups does not hold due to the way that selection of subjects or areas into the study or analysis is carried out | Primary research | Exchangeability | Confounding biases Post-intervention/exposure selection biases | |
|  | Selective survival bias | See 89 ‘Prevalence-incidence bias’ |  |  |  |  |
107 | Sick quitter bias | Occurs when the exposure or intervention that specific subjects receive is inaccurately measured or classified due to changes in the subjects' behaviour caused by sickness (e.g., lack of adherence, reclassification from intervention/exposure to no intervention/exposure by the researchers after the intervention/exposure is applied) | Primary research | Measurement | Misclassified/mismeasured comparison biases Performance biases | [35] |
108 | Significance bias | Occurs when statistical significance is confused with environmental, ecological, biological or agricultural significance | Secondary research | Study appraisal and data coding/extraction Data synthesis | Study appraisal and data coding/extraction biases Data synthesis biases | |
109 | Spatial bias | Occurs when populations that are spatially distinct are compared, and this spatial difference affects both the intervention/exposure and the outcome being studied | Primary research | Exchangeability | Confounding biases | [34] |
110 | Spin bias | Occurs when research findings or data are inappropriately used or interpreted to support or oppose effectiveness or impact | Primary research Secondary research | Reporting Statistical methods Screening Study appraisal and data coding/extraction Data synthesis | Outcome reporting biases Outcome assessment biases Screening biases Study appraisal and data coding/extraction biases Data synthesis biases | [81] |
|  | Sponsor bias | See 99 ‘Researcher bias’ |  |  |  |  |
111 | Starting time bias | Occurs when starting time for the intervention/exposure or outcome measurement is different within or between groups and the time lag (or variable influenced by the time lag) influences both the intervention/exposure and the outcome or causes inaccurate measurements of the intervention/exposure or the outcome | Primary research | Exchangeability Measurement | Confounding biases Misclassified/mismeasured comparison biases Performance biases Detection biases | |
112 | Substitution game bias | Occurs when substitution of the outcome or the intervention/exposure with a surrogate is imprecise or inaccurate | Primary research | Measurement | Misclassified/mismeasured comparison biases Performance biases Detection biases | |
113 | Susceptibility bias (Vulnerability bias) | Occurs when different study groups have different characteristics and the difference indicates that one group is more or less susceptible to the exposure (i.e., the characteristics are associated with the exposure and thus the difference creates an imbalance between the groups, meaning exchangeability does not hold) | Primary research | Exchangeability | Confounding biases Post-intervention/exposure selection biases | |
114 | Temporal bias | Occurs when studies with smaller p-values or larger effects are more likely to be published in academic journals more rapidly | Secondary research | Searching Screening | Searching biases Screening biases | [29] |
115 | Tidying-up bias (Outlier handling bias) | Occurs when outliers or other untidy measurements or results are inappropriately excluded or non-reported | Primary research Secondary research | Reporting Statistical methods Study appraisal and data coding/extraction Data synthesis | Outcome reporting biases Outcome assessment biases Study appraisal and data coding/extraction biases Data synthesis biases | |
116 | Transfer bias | Occurs when a certain aspect of a study leads to uneven losses in follow-up between the groups | Primary research | Exchangeability | Post-intervention/exposure selection biases | |
117 | Treatment analysis bias (Lack of intention to treat analysis bias) | Occurs when researchers fail to keep subjects or areas in the group they are assigned to, when there is deviation from the intended treatment, or when subjects do not adhere to the assigned treatment | Primary research | Exchangeability Measurement | Confounding biases Performance biases | |
118 | Underlying cause bias (Rumination bias) | Occurs when the intervention or exposure (group) is more carefully planned and implemented than the control (no intervention or exposure), and that results in imbalances between the groups or misclassification or mismeasurement of control | Primary research | Exchangeability Measurement | Confounding biases Misclassified/mismeasured comparison biases | |
119 | Volunteer bias | Occurs when human subjects involved in a study differ in some way among those in the intervention/exposure group or between groups due to willingness to participate as volunteers | Primary research | Exchangeability Measurement | Confounding biases Post-intervention/exposure selection biases Performance biases | |
|  | Vulnerability bias | See 113 ‘Susceptibility bias’ |  |  |  |  |
120 | Withdrawal bias (Dropout bias, Loss to follow-up bias) | Occurs when subjects who have dropped out of a study differ from remaining subjects, and the differences modify the effectiveness or impact | Primary research | Exchangeability | Confounding biases Post-intervention/exposure selection biases | |
121 | Wrong sample size bias (Sample size bias) | Occurs when used sample size is inappropriate in a study or analysis for assessing the effect of the intervention or exposure due to violation of the assumption that exchangeability holds or the hypothesis can be tested with the collected samples | Primary research | Exchangeability Statistical methods | Confounding biases Post-intervention/exposure selection biases Outcome assessment biases |