Stage of article assessment | Total number assessed | Number passing assessment | Agreement level, κ (interpretation according to [ 30 ]) | Subset assessed in duplicate (% of total) | ||
---|---|---|---|---|---|---|
RW vs. JE (2010 searches) | JE vs. LEN (2013 searches) | RW vs. JE (2010 searches) | JE vs. LEN (2013 searches) | |||
Title assessment | 11457 | 4244 | 1. 0.43 (moderate) | 0.61 (substantial) | 100 per trial (total 500; 5.75%) | 279 (9.84%) |
2. 0.62 (substantial) | ||||||
3. 0.74 (substantial) | ||||||
4. 0.56 (moderate) | ||||||
5. 0.74 (substantial) | ||||||
Abstract assessment | 4244 | 1119 | 1. 0.20 (slight) | 0.56 (moderate) | 1. 50 | 75 (7.76%) |
2. 0.43 (moderate) | 2. 49 | |||||
3. 0.66 (substantial) | 3. 50 | |||||
4. 0.72 (substantial) | 4. 50 | |||||
5. 0.59 (moderate) | 5. 330 | |||||
5b. 0.70 (substantial) | 5b. 330 | |||||
(total 859; 26.22%) | ||||||
N/A | 0.96 (almost perfect) | N/A | ||||
Full text assessment | 1119 | 614* | 50 (4.48%) |