Optimising search filters for active literature surveillance: a concordance study

ID: 

18923

Session: 

Long oral session 18: Efficiency in searching

Date: 

Friday 15 September 2017 - 11:00 to 12:30

Location: 

All authors in correct order:

Alper BS1, Hertzman-Miller R1, Iorio A2
1 EBSCO Health DynaMed Plus, USA
2 McMaster University, Canada
Presenting author and contact person

Presenting author:

Brian Alper

Contact person:

Abstract text
Background: A major challenge in keeping clinical guidelines and structured recommendations current is to identify new, relevant evidence in a resource-efficient way. Prior research found that sensitivity (minimising the number of missed relevant references) or efficiency (low number needed to read, NNR) can be adjusted over a wide range, spanning from the high efficiency/low sensitivity of the McMaster Premium Literature Service (PLUS) to the high sensitivity/high NNR of the PubMed Clinical Queries.

Objectives: We explored the concordance of the Clinical Queries/PLUS approach with the systematic literature surveillance process used for systematically updating an evidence-based clinical reference (DynaMed Plus) and derived a search approach with an optimal balance of sensitivity and efficiency.

Methods: We identified all articles representing primary evidence with a publication date of 2015 that were included in either PLUS (clinically valuable evidence selected based on strict methodologic criteria) or DynaMed Plus (the best-available evidence to answer clinically relevant questions). We assessed concordance of different filtering strategies against this empirical set from a composite of 503 clinical journals. We assessed sensitivity and NNR of 3 main search strategies and several combinations.

Results: The reference standard included 6720 articles. A sensitive PCQ-based strategy had relative sensitivity 0.96 and NNR 11.5. A balanced strategy using free-text search to capture pre-publication record data developed with HEDGES technology from PCQ had relative sensitivity 0.86 and NNR 7.5. A DynaMed Plus-based strategy had relative sensitivity 0.95 and NNR 6. The different filters had variable performance within different subsets of journals.

Conclusions: A critical factor for an efficient filter strategy is the journal. A sensitive and more efficient surveillance strategy for clinically usable evidence can be achieved by developing journal-specific filtering approaches balancing sensitivity and efficiency. External validation of the optimal search strategy is under way.