ABSTRACT
Comparative diagnostic test accuracy studies assess and compare the accuracy of 2 or more tests in the same study. Although these studies have the potential to yield reliable evidence regarding comparative accuracy, shortcomings in the design, conduct, and analysis may bias their results. The currently recommended quality assessment tool for diagnostic test accuracy studies, QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies-2), is not designed for the assessment of test comparisons. The QUADAS-C (Quality Assessment of Diagnostic Accuracy Studies-Comparative) tool was developed as an extension of QUADAS-2 to assess the risk of bias in comparative diagnostic test accuracy studies. Through a 4-round Delphi study involving 24 international experts in test evaluation and a face-to-face consensus meeting, an initial version of the tool was developed that was revised and finalized following a pilot study among potential users. The QUADAS-C tool retains the same 4-domain structure of QUADAS-2 (Patient Selection, Index Test, Reference Standard, and Flow and Timing) and comprises additional questions to each QUADAS-2 domain. A risk-of-bias judgment for comparative accuracy requires a risk-of-bias judgment for the accuracy of each test (resulting from QUADAS-2) and additional criteria specific to test comparisons. Examples of such additional criteria include whether participants either received all index tests or were randomly assigned to index tests, and whether index tests were interpreted with blinding to the results of other index tests. The QUADAS-C tool will be useful for systematic reviews of diagnostic test accuracy addressing comparative questions. Furthermore, researchers may use this tool to identify and avoid risk of bias when designing a comparative diagnostic test accuracy study.
Subject(s)
Bias , Diagnosis , Quality Assurance, Health Care , Review Literature as Topic , Surveys and Questionnaires , Evidence-Based Medicine , HumansSubject(s)
Brain Ischemia/diagnosis , Diffusion Magnetic Resonance Imaging/methods , Diffusion Magnetic Resonance Imaging/standards , Evidence-Based Medicine/standards , Perfusion Imaging/methods , Perfusion Imaging/standards , Practice Guidelines as Topic/standards , Stroke/diagnosis , Brain Ischemia/drug therapy , Contraindications , Evidence-Based Medicine/methods , Humans , Neurology/methods , Neurology/standards , Societies, Medical/standards , Stroke/drug therapyABSTRACT
PURPOSE: To prospectively develop a search strategy in MEDLINE for identifying studies on the diagnostic performance of any imaging modality, with maximized and minimized retrieval of relevant and irrelevant studies, respectively. MATERIALS AND METHODS: Predefined inclusion criteria were used to conduct a hand search of two sets of radiologic journal articles for studies on assessment of the diagnostic performance of imaging techniques. These two sets of articles formed independent derivation and validation record sets for developing and evaluating the search strategy. The sensitivity and positive predictive values (PPVs) of search terms from the derivation reference-standard set of records were used to select terms and develop two components of the search strategy. The first component was used to identify any study (from the imaging literature) in which diagnostic test performance was assessed. The second component was used to identify studies of any imaging modality. The two components were combined in the final search strategy. The sensitivity, specificity, and PPV of the search strategy in the derivation and validation record sets were calculated. RESULTS: The final search strategy had a sensitivity of 92.8%, a specificity of 58.5%, and a PPV of 25.1% for retrieval of the derivation set of records. Validation with an independent set of records gave a sensitivity of 91.9% (95% confidence interval [CI]: 87.1%, 95.1%), a specificity of 52.2% (95% CI: 49.2%, 55.2%), and a PPV of 25.1% (95% CI: 22.0%, 28.5%). Removal of irrelevant publication types further improved specificity and PPV in the validation set: to 77.6% (95% CI: 75.0%, 80.0%) and 40.9% (95% CI: 36.2%, 45.7%), respectively. The volume of imaging literature retrieved from MEDLINE by using the described search strategy has tripled since 1975. CONCLUSION: A sensitive search strategy to identify studies of the diagnostic performance of any imaging test was developed and validated. The retrieval estimates of this strategy in MEDLINE are adequate to develop a register of studies.
