Cochrane Crowd: Using citizen science to meet the challenge of information overload in evidence production

ID: 

2006

Session: 

Poster session 2 Thursday: Evidence synthesis - methods / improving conduct and reporting

Date: 

Thursday 14 September 2017 - 12:30 to 14:00

Location: 

Exhibition Halls 1 & 2

All authors in correct order:

Noel-Storr A1, Dooley G2, Steele E1, Mavergames C3, Turner T4, McDonald S4, Green S4, Tovey D5, Murano M4, Elliott J4, Thomas J6
1 University of Oxford, United Kingdom
2 Metaxis, United Kingdom
3 German Cochrane Centre, Germany
4 Monash University, Australia
5 Cochrane, United Kingdom
6 UCL, United Kingdom
Presenting author and contact person

Presenting author:

Anna Noel-Storr

Contact person:

Anna Noel-Storr
Abstract text
Background: At a time when research output is expanding exponentially, citizen science, the process of engaging willing volunteers in scientific research activities, has an important role to play in helping to manage the information overload. It also creates a model of contribution that enables anyone with an interest in health to contribute meaningfully and in a way that is flexible. Citizen science models have shown to be extremely effective in other domains such as astronomy and ecology.

Objectives: Cochrane Crowd (crowd.cochrane.org) is a citizen science platform that offers contributors a range of micro-tasks, designed to help identify and describe clinical trials and diagnostic studies.

Methods: The platform enables contributors to dive into needed tasks that capture and describe health evidence. Brief interactive training modules, and agreement algorithms help ensure accurate collective decisions. Contributors work online or offline; they can view their activity and performance in detail. They can choose to work in topic areas of interest. As contributors progress, they unlock new tasks.

Results: Cochrane Crowd was launched in February 2016. Three micro-tasks are available: RCT identification and diagnostic test accuracy (DTA) identification, and PICO (Population, Intervention, Comparator and Outcomes) extraction at citation level.

The Cochrane Crowd community comprises 5000 contributors from 117 countries. Over 1 million individual classifications have been made, and 32 000 reports of randomised trials have been identified for Cochrane’s Central Register of Controlled Trials. Evaluations to assess crowd accuracy have shown crowd sensitivity is 99.1%, and crowd specificity is 99%. Main motivations for involvement are that people want to help Cochrane, and people want to learn.

Conclusions:This model of contribution is becoming an established part of Cochrane’s effort to manage the deluge of information produced in a way that offers contributors a way to get involved, learn and play a crucial role in evidence production.