Background: A major challenge in keeping clinical guidelines and structured recommendations current is to identify new, relevant evidence in a resource-efficient way. Prior research found that sensitivity (minimising the number of missed relevant references) or efficiency (low number needed to read, NNR) can be adjusted over a wide range, spanning from the high efficiency/low sensitivity of the McMaster Premium Literature Service (PLUS) to the high sensitivity/high NNR of the PubMed Clinical Queries.

Objectives: We explored the concordance of the Clinical Queries/PLUS approach with the systematic literature surveillance process used for systematically updating an evidence-based clinical reference (DynaMed Plus) and derived a search approach with an optimal balance of sensitivity and efficiency.

Methods: We identified all articles representing primary evidence with a publication date of 2015 that were included in either PLUS (clinically valuable evidence selected based on strict methodologic criteria) or DynaMed Plus (the best-available evidence to answer clinically relevant questions). We assessed concordance of different filtering strategies against this empirical set from a composite of 503 clinical journals. We assessed sensitivity and NNR of 3 main search strategies and several combinations.

Results: The reference standard included 6720 articles. A sensitive PCQ-based strategy had relative sensitivity 0.96 and NNR 11.5. A balanced strategy using free-text search to capture pre-publication record data developed with HEDGES technology from PCQ had relative sensitivity 0.86 and NNR 7.5. A DynaMed Plus-based strategy had relative sensitivity 0.95 and NNR 6. The different filters had variable performance within different subsets of journals.

Conclusions: A critical factor for an efficient filter strategy is the journal. A sensitive and more efficient surveillance strategy for clinically usable evidence can be achieved by developing journal-specific filtering approaches balancing sensitivity and efficiency. External validation of the optimal search strategy is under way.