Dataset Information

Optimal search strategies for retrieving scientifically strong studies of diagnosis from Medline: analytical survey.

ABSTRACT:

Objective

To develop optimal search strategies in Medline for retrieving sound clinical studies on the diagnosis of health disorders.

Design

Analytical survey.

Setting

Medline, 2000.

Participants

170 journals for 2000 of which 161 were indexed in Medline.

Main outcome measures

The sensitivity, specificity, precision ("positive predictive value"), and accuracy of 4862 unique terms in 17,287 combinations were determined by comparison with a hand search of all articles (the "gold standard") in 161 journals published during 2000 (49,028 articles).

Results

Only 147 (18.9%) of 778 articles about diagnostic tests met basic criteria for scientific merit. Combinations of search terms reached peak sensitivities of 98.6% at a specificity of 74.3%. Compared with best single terms, best multiple terms increased sensitivity for sound studies by 6.8% (absolute increase), while also increasing specificity (absolute increase 6.0%) when sensitivity was maximised. When terms were combined to maximise specificity, the single term, specificity.tw. (98.4%), outperformed combinations of terms. The strategies newly reported in this paper outperformed other validated search strategies except for one strategy that had slightly higher sensitivity (99.3% v 98.6%) but lower specificity (54.7% v 74.3%).

Conclusion

New empirical search strategies in Medline can optimise retrieval of articles reporting high quality clinical studies of diagnosis.

SUBMITTER: Haynes RB

PROVIDER: S-EPMC403841 | biostudies-literature | 2004 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Optimal search strategies for retrieving scientifically strong studies of diagnosis from Medline: analytical survey.

Haynes R Brian RB Wilczynski Nancy L NL

BMJ (Clinical research ed.) 20040408 7447

<h4>Objective</h4>To develop optimal search strategies in Medline for retrieving sound clinical studies on the diagnosis of health disorders.<h4>Design</h4>Analytical survey.<h4>Setting</h4>Medline, 2000.<h4>Participants</h4>170 journals for 2000 of which 161 were indexed in Medline.<h4>Main outcome measures</h4>The sensitivity, specificity, precision ("positive predictive value"), and accuracy of 4862 unique terms in 17,287 combinations were determined by comparison with a hand search of all ar ...[more]

PMID: 15073027

Similar Datasets

Project description:BackgroundKeyword searching through PubMed and other systems is the standard means of retrieving information from Medline. However, ad-hoc retrieval systems do not meet all of the needs of databases that curate information from literature, or of text miners developing a corpus on a topic that has many terms indicative of relevance. Several databases have developed supervised learning methods that operate on a filtered subset of Medline, to classify Medline records so that fewer articles have to be manually reviewed for relevance. A few studies have considered generalisation of Medline classification to operate on the entire Medline database in a non-domain-specific manner, but existing applications lack speed, available implementations, or a means to measure performance in new domains.ResultsMScanner is an implementation of a Bayesian classifier that provides a simple web interface for submitting a corpus of relevant training examples in the form of PubMed IDs and returning results ranked by decreasing probability of relevance. For maximum speed it uses the Medical Subject Headings (MeSH) and journal of publication as a concise document representation, and takes roughly 90 seconds to return results against the 16 million records in Medline. The web interface provides interactive exploration of the results, and cross validated performance evaluation on the relevant input against a random subset of Medline. We describe the classifier implementation, cross validate it on three domain-specific topics, and compare its performance to that of an expert PubMed query for a complex topic. In cross validation on the three sample topics against 100,000 random articles, the classifier achieved excellent separation of relevant and irrelevant article score distributions, ROC areas between 0.97 and 0.99, and averaged precision between 0.69 and 0.92.ConclusionMScanner is an effective non-domain-specific classifier that operates on the entire Medline database, and is suited to retrieving topics for which many features may indicate relevance. Its web interface simplifies the task of classifying Medline citations, compared to building a pre-filter and classifier specific to the topic. The data sets and open source code used to obtain the results in this paper are available on-line and as supplementary material, and the web interface may be accessed at http://mscanner.stanford.edu.

Project description:BackgroundSystematic reviews (SRs) are used to inform clinical practice guidelines and healthcare decision making by synthesising the results of primary studies. Efficiently retrieving as many relevant SRs as possible is challenging with a minimum number of databases, as there is currently no guidance on how to do this optimally. In a previous study, we determined which individual databases contain the most SRs, and which combination of databases retrieved the most SRs. In this study, we aimed to validate those previous results by using a different, larger, and more recent set of SRs.MethodsWe obtained a set of 100 Overviews of Reviews that included a total of 2276 SRs. SR inclusion was assessed in MEDLINE, Embase, and Epistemonikos. The mean inclusion rates (% of included SRs) and corresponding 95% confidence intervals were calculated for each database individually, as well as for combinations of MEDLINE with each other database and reference checking. Features of SRs not identified by the best database combination were reviewed qualitatively.ResultsInclusion rates of SRs were similar in all three databases (mean inclusion rates in % with 95% confidence intervals: 94.3 [93.9-94.8] for MEDLINE, 94.4 [94.0-94.9] for Embase, and 94.4 [93.9-94.9] for Epistemonikos). Adding reference checking to MEDLINE increased the inclusion rate to 95.5 [95.1-96.0]. The best combination of two databases plus reference checking consisted of MEDLINE and Epistemonikos (98.1 [97.7-98.5]). Among the 44/2276 SRs not identified by this combination, 34 were published in journals from China, four were other journal publications, three were health agency reports, two were dissertations, and one was a preprint. When discounting the journal publications from China, the SR inclusion rate in the recommended combination (MEDLINE, Epistemonikos and reference checking) was even higher than in the previous study (99.6 vs. 99.2%).ConclusionsA combination of databases and reference checking was the best approach to searching for biomedical SRs. MEDLINE and Epistemonikos, complemented by checking the references of the included studies, was the most efficient and produced the highest recall. However, our results point to the presence of geographical bias, because some publications in journals from China were not identified.Study registrationhttps://doi.org/10.17605/OSF.IO/R5EAS (Open Science Framework).

Dataset Information

Optimal search strategies for retrieving scientifically strong studies of diagnosis from Medline: analytical survey.

Objective

Design

Setting

Participants

Main outcome measures

Results

Conclusion

Publications

Optimal search strategies for retrieving scientifically strong studies of diagnosis from Medline: analytical survey.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets