Optimal search filters for detecting quality improvement studies in Medline.

BACKGROUND: As the knowledge translation and comparative effectiveness research agendas gain momentum, we can expect more evidence on which to base quality improvement (QI) programmes. Unaided searches for such content in the literature, however, are likely to be daunting, with searches missing key articles while mainly retrieving articles that are irrelevant to the question being asked. The objective of this study was to develop and validate optimal Medline search filters for retrieving original and review articles about clinical QI. METHODS: Analytical survey in the McMaster Clinical Hedges database and Health Knowledge Refinery (HKR) of 161 clinical journals to determine the operating characteristics of QI search filters developed by computerised combinations of terms selected to detect original QI studies and systematic reviews meeting basic methodological criteria for scientific merit. Results from a derivation random subset of articles were tested in a validation random subset. RESULTS: The Clinical Hedges QI database contained 49,233 citations of which 471 (0.96%) were original or review QI studies; of those, 282 (60%) were methodologically sound. Combinations of search terms reached peak sensitivities of 100% at a specificity of 89.3% for detecting methodologically sound original and review QI studies, and sensitivities of 97.6% at a specificity of 53.0% for detecting all original and review QI studies independent of rigour. Operating characteristics of the search filters derived in the development database worked similarly in the validation database, without statistical differences. CONCLUSION: New empirically derived Medline search filters have been validated to optimise retrieval of original and review QI articles.

Tackling the growth of the obesity literature: obesity evidence spreads across many journals.

OBJECTIVE: This study identified the journals with the highest yield of clinical obesity research articles and surveyed the scatter of such studies across journals. The study exemplifies an approach to establish a journal collection that is likely to contain most new knowledge about a field. DESIGN AND METHODS: All original studies that were cited in 40 systematic reviews about obesity topics ('included studies') were compiled and journal titles in which they were published were extracted. The journals were ranked by the number of included studies. The highest-yielding journals for clinical obesity and the scatter across journal titles were determined. A subset of these journals was created in MEDLINE (PubMed) to test search recall and precision for high-quality studies of obesity treatment (that is, articles that pass predetermined methodology criteria, including random allocation of participants to comparison groups, assessment of clinical outcomes, and at least 80% follow-up). RESULTS: Articles in 252 journals were cited in the systematic reviews. The three highest-yielding journals specialized in obesity, but they published only 19.2% of the research, leaving 80.8% scattered across 249 non-obesity journals. The MEDLINE journal subset comprised 241 journals (11 journals were not indexed in MEDLINE) and included 82% of the clinical obesity research articles retrieved by a search for high-quality treatment studies ('recall' of 82%). Of the articles retrieved, 11% were about clinical obesity care ('precision' of 11%), compared with precision of 6% for obesity treatment studies in the full MEDLINE database. CONCLUSION: Obesity journals captured only a small proportion of the literature on clinical obesity care. Those wishing to keep up in this field will need to develop more inclusive strategies than reading these specialty journals. A journal subset based on these findings may be useful when searching large electronic databases to increase search precision.

Lost in publication: Half of all renal practice evidence is published in non-renal journals.

Physicians often scan a select number of journals to keep up to date with practice evidence for patients with kidney conditions. This raises the question of where relevant studies are published. We performed a bibliometric analysis using 195 renal systematic reviews. Each review used a comprehensive method to identify all primary studies for a focused clinical question relevant to patient care. We compiled all the primary studies included in these reviews, and considered where each study was published. Of the 2779 studies, 1351 (49%) were published in the top 20 journals. Predictably, this list included Transplantation Proceedings (5.9% of studies), Kidney International (5.3%), American Journal of Kidney Diseases (4.7%), Nephrology Dialysis Transplantation (4.3%), Transplantation (4.2%), and Journal of the American Society of Nephrology (2.4%). Ten non-renal journals were also on this list, including New England Journal of Medicine (2.4%), Lancet (2.3%), and Diabetes Care (2.2%). The remaining 1428 (51%) studies were published across other 446 journals. When the disciplines of all journals were considered, 59 were classified as renal or transplant journals (42% of articles). Other specialties included general and internal medicine (16%), endocrinology (diabetes) and metabolism (6.5%), surgery (6.2%), cardiovascular diseases (6.1%), pediatrics (4.3%), and radiology (3.3%). About half of all renal practice evidence is published in non-renal journals. Browsing the top journals is important. However, relevant studies are also scattered across a large range of journals that may not be routinely scanned by busy physicians, and keeping up with this literature requires other continuing education strategies.

Enhancing retrieval of best evidence for health care from bibliographic databases: calibration of the hand search of the literature.

BACKGROUND: Medical practitioners have unmet information needs. Health care research dissemination suffers from both "supply" and "demand" problems. One possible solution is to develop methodologic search filters ("hedges") to improve the retrieval of clinically relevant and scientifically sound study reports from bibliographic databases. To develop and test such filters a hand search of the literature was required to determine directly which articles should be retrieved, and which not retrieved. OBJECTIVE: To determine the extent to which 6 research associates can agree on the classification of articles according to explicit research criteria when hand searching the literature. DESIGN: Blinded, inter-rater reliability study. SETTING: Health Information Research Unit, McMaster University, Hamilton, Ontario, Canada. PARTICIPANTS: 6 research associates with extensive training and experience in research methods for health care research. MAIN OUTCOME MEASURE: Inter-rater reliability measured using the kappa statistic for multiple raters. RESULTS: After one year of intensive calibration exercises research staff were able to attain a level of agreement at least 80% greater than that expected by chance (kappa statistic) for all classes of articles. CONCLUSION: With extensive training multiple raters are able to attain a high level of agreement when classifying articles in a hand search of the literature.

Reasons for the loss of sensitivity and specificity of methodologic MeSH terms and textwords in MEDLINE.

OBJECTIVE: To determine the reasons for the loss of sensitivity and specificity of methodologic MeSH terms and textwords in MEDLINE for identifying sound clinical studies of the etiology, prognosis, diagnosis, prevention, or treatment of disorders in adult general medicine. DESIGN: Analytic survey of the information retrieval properties of methodologic MeSH terms and textwords selected to detect studies meeting basic methodologic criteria for direct clinical use in general adult medicine. MEASURES: Frequency of non-use and misuse of relevant methodologic MeSH terms and textwords among studies meeting and not meeting the basic criteria for clinical practice as determined by the manual review (the gold standard) of all articles in 10 internal and general medicine journals for 1986 and 1991. RESULTS: Loss of sensitivity due to the non-use of relevant methodologic terms among articles meeting basic methodologic criteria was more pronounced in the areas of diagnosis, prognosis, and etiology than treatment in 1991 and 1986. The use of relevant methodologic terms has improved from 1986 to 1991 in all areas except prognosis. Loss of specificity due to the use of relevant methodologic terms among articles not meeting basic methodologic criteria occurred most frequently in the areas of treatment and etiology. CONCLUSIONS: Although the appropriate use of methodologic MeSH and textwords has improved from 1986 to 1991 among studies meeting basic methodologic criteria for direct clinical use in general adult medicine much improvement is still needed in the areas of diagnosis, prognosis, and etiology. Improvement is needed in assigning the relevant methodologic index terms to studies that meet the methods criteria and in having the authors use the relevant methodologic textwords in the title or abstract. Some improvement is also needed in not using methodologic terms when the study clearly does not meet the methods criteria.

Preliminary assessment of the effect of more informative (structured) abstracts on citation retrieval from MEDLINE.

OBJECTIVE: To compare the recall and precision of citation retrieval from MEDLINE when searching in journals containing traditional abstracts with journals containing abstracts prepared according to the guidelines of more informative abstracts (MIAs) for clinical journal articles. DESIGN: Analytic survey of operating characteristics of MeSH terms and textwords selected to detect studies meeting basic methodological criteria for direct clinical use in general adult medicine in priority journals with and without MIAs. MEASURES: The sensitivity, specificity, and precision of search terms were determined by comparison with a manual review of all articles (the gold standard) in 10 internal and general medicine journals for 1986 and 1991. The performance of each search term was compared before and after the implementation of MIAs and in two journal sets with and without MIAs in 1991. RESULTS: The sensitivity of some terms increased with the use of MIAs and also increased by the same amount in journals with usual abstracts in 1991 compared to 1986. The use of MIAs had the greatest impact in the areas of etiology and prognosis. Many MeSH and textwords had comparable performance before and after the implementation of MIAs and between journals with and without MIAs in 1991. CONCLUSIONS: More informative abstracts improved the retrieval properties of some, but not all, textwords and medical subject headings. Further work may be needed to enhance the effects of MIAs on bibliographic retrieval.

Developing optimal search strategies for detecting clinically sound studies in MEDLINE.

OBJECTIVE: To develop optimal MEDLINE search strategies for retrieving sound clinical studies of the etiology, prognosis, diagnosis, prevention, or treatment of disorders in adult general medicine. DESIGN: Analytic survey of operating characteristics of search strategies developed by computerized combinations of terms selected to detect studies meeting basic methodologic criteria for direct clinical use in adult general medicine. MEASURES: The sensitivities, specificities, precision, and accuracy of 134,264 unique combinations of search terms were determined by comparison with a manual review of all articles (the "gold standard") in ten internal medicine and general medicine journals for 1986 and 1991. RESULTS: Less than half of the studies of the topics of interest met basic criteria for scientific merit for testing clinical applications. Combinations of search terms reached peak sensitivities of 82% for sound studies of etiology, 92% for prognosis, 92% for diagnosis, and 99% for therapy in 1991. Compared with the best single terms, multiple terms increased sensitivity for sound studies by over 30% (absolute increase), but with some loss of specificity when sensitivity was maximized. For 1986, combinations reached peak sensitivities of 72% for etiology, 95% for prognosis, 86% for diagnosis, and 98% for therapy. When search terms were combined to maximize specificity, over 93% specificity was achieved for all purpose categories in both years. Compared with individual terms, combined terms achieved near-perfect specificity that was maintained with modest increases in sensitivity in all purpose categories except therapy. Increases in accuracy were achieved by combining terms for all purpose categories, with peak accuracies reaching over 90% for therapy in 1986 and 1991. CONCLUSIONS: The retrieval of studies of important clinical topics cited in MEDLINE can be substantially enhanced by selected combinations of indexing terms and textwords.

Quantitative comparison of pre-explosions and subheadings with methodologic search terms in MEDLINE.

OBJECTIVE: To compare the retrieval characteristics of subheadings with methodologic textwords and MeSH terms in MEDLINE for identifying sound clinical studies on the etiology, prognosis, diagnosis, prevention and treatment of disorders in general adult medicine. DESIGN: Analytic survey of the information retrieval properties of methodologic textwords, single methodologic MeSH terms, pre-explosions and subheadings selected to detect studies meeting basic methodologic criteria for direct clinical use in general adult medicine. MEASURES: The sensitivity, specificity, and precision of search terms were determined by comparing the citations retrieved by the search strategies in MEDLINE with that of a manual review (the gold standard) of all articles in 10 internal and general medicine journals for 1986 and 1991. RESULTS: For treatment and diagnosis in 1991, and treatment, diagnosis, and etiology in 1986, pre-explosions yielded the highest sensitivity, with typical absolute increases exceeding 15%. For etiology and prognosis in 1991, and prognosis in 1986, textwords or MeSH terms yielded the highest sensitivity. In all cases the increase in sensitivity was coupled with a loss in specificity and precision. CONCLUSIONS: Compared with searching with single methodologic textwords and subject headings, the detection of sound clinical studies on the diagnosis and treatment of disorders in general adult medicine was consistently enhanced by searching with pre-explosions, but at a price of decreased specificity and precision.

Assessment of methodologic search filters in MEDLINE.

OBJECTIVE: To determine the retrieval characteristics of methodologic textwords and MeSH terms in MEDLINE for identifying methodologically sound studies on the etiology, prognosis, diagnosis, and prevention and treatment of disorders in general adult medicine. DESIGN: Comparison of methodologic search terms and phrases for the retrieval of citations in MEDLINE with a manual hand search of the literature (the gold standard) for 10 internal and general medicine journals for 1986 and 1991. MEASURES: Sensitivity (proportion of methodologically sound and correct topic studies retrieved) and specificity (proportion of unsound or wrong topic articles not retrieved) of the search strategies. RESULTS: The individual terms yielding the best sensitivity for 1991 by purpose category were: risk (tw) for etiology; exp cohort studies for prognosis; sensitivity (tw) for diagnosis; and clinical trial (pt) for treatment. The corresponding terms for 1986 were: risk (tw) for etiology; prognosis: (tw) for prognosis; sensitivity (tw) for diagnosis; and random: (tw) for treatment. CONCLUSIONS: The performance of methodologic MeSH terms and textwords varied greatly in MEDLINE and changed from 1986 to 1991. More complex search strategies may be required to optimize retrieval.

Behavioral and physiological effects of a beta blocker and relaxation therapy on mild hypertensives.

In this industrial-based study we compared the blood pressure (BP)-lowering effectiveness of relaxation, a beta blocker, and the combined use of these two treatments in 47 untreated, mildly hypertensive blue collar steel workers. Using a randomized two by two factorial design, patients received either nadolol or placebo drug daily, and either a relaxation training or an education program, each lasting 8 weeks. A pre-intervention and post-intervention stress test measured response of heart rate and BP to mental and physical tasks. BP assessments were done at baseline, post-intervention, 1 month, and 3 month follow-up. Change in several self-report measures was determined. Results showed that beta blocker was more effective in lowering BP than placebo, but relaxation was not more effective in lowering BP than health education. The combined effect of beta blocker and relaxation was not superior to beta blocker alone. Compliance with relaxation practice was not superior to compliance with medication. We conclude that pharmacologic treatment is superior to the relaxation therapy tested.

The likelihood of remaining normotensive following antihypertensive drug withdrawal.

OBJECTIVE: To determine the proportion and characteristics of mild hypertensives who remain normotensive after withdrawal from drug treatment. DESIGN: Longitudinal descriptive study. SETTING: A university family practice unit (FPU) and a large steel company (DOFASCO) in Hamilton, Ontario. PATIENTS: Participants were 107 of 125 (86%) eligible hypertensives at the two sites; 103 (96%) subjects completed the study. INTERVENTION: Subjects discontinued antihypertensive medication and were followed until blood pressure (BP) became elevated or for 12 months, whichever was shorter. MEASUREMENTS AND MAIN RESULTS: BPs were monitored according to a predefined schedule and before, during, and after mental and physical stress tests. Thirty-eight (37%, 95% confidence interval [CI] 27-46%) subjects remained normotensive at 12 months. Predictors of remaining normotensive included lower medicated standing diastolic blood pressure (DBP) (87.6 versus 91.8 mm Hg, 95% CI for the difference 2.2-6.2, p less than 0.001) and longer duration of normotension on drugs (12.6 months versus 8.7, 95% CI for the difference 0.9-6.9, p = 0.012). There was no significant relationship between maintenance of normotension and age, medication potency, duration of hypertension, weight, lying BP, change in heart rate, or BP during mental or physical stress tests; the power to detect a clinically important difference in lying DBP was 99% but for other variables was lower (21-75%). CONCLUSIONS: The probability of hypertensives' remaining normotensive for one year after drug withdrawal increases as the medicated standing DBP decreases and as the number of months of BP control while on medication increases. Studies with larger sample sizes are warranted to determine whether other variables may be significant predictors.