Linking clinical quality indicators to research evidence - a case study in asthma management for children.

BACKGROUND: Clinical quality indicators are used to monitor the performance of healthcare services and should wherever possible be based on research evidence. Little is known however about the extent to which indicators in common use are based on research. The objective of this study is to measure the extent to which clinical quality indicators used in asthma management in children with outcome measurements can be linked to results in randomised controlled clinical trial (RCT) reports. This work is part of a broader research program to trial methods that improve the efficiency and accuracy of indicator development. METHODS: National-level indicators for asthma management in children were extracted from the National Quality Measures Clearinghouse database and the National Institute for Health and Care Excellence quality standards by two independent appraisers. Outcome measures were extracted from all published English language RCT reports for asthma management in children below the age of 12 published between 2005 and 2014. The two sets were then linked by manually mapping both to a common set of Unified Medical Language System (UMLS) concepts. RESULTS: The analysis identified 39 indicators and 562 full text RCTs dealing with asthma management in children. About 95% (37/39) of the indicators could be linked to RCT outcome measures. CONCLUSIONS: It is possible to identify relevant RCT reports for the majority of indicators used to assess the quality of asthma management in childhood. The methods reported here could be automated to more generally support assessment of candidate indicators against the research evidence.

Linking quality indicators to clinical trials: an automated approach.

OBJECTIVE: Quality improvement of health care requires robust measurable indicators to track performance. However identifying which indicators are supported by strong clinical evidence, typically from clinical trials, is often laborious. This study tests a novel method for automatically linking indicators to clinical trial registrations. DESIGN: A set of 522 quality of care indicators for 22 common conditions drawn from the CareTrack study were automatically mapped to outcome measures reported in 13 971 trials from ClinicalTrials.gov. INTERVENTION: Text mining methods extracted phrases mentioning indicators and outcome phrases, and these were compared using the Levenshtein edit distance ratio to measure similarity. MAIN OUTCOME MEASURE: Number of care indicators that mapped to outcome measures in clinical trials. RESULTS: While only 13% of the 522 CareTrack indicators were thought to have Level I or II evidence behind them, 353 (68%) could be directly linked to randomized controlled trials. Within these 522, 50 of 70 (71%) Level I and II evidence-based indicators, and 268 of 370 (72%) Level V (consensus-based) indicators could be linked to evidence. Of the indicators known to have evidence behind them, only 5.7% (4 of 70) were mentioned in the trial reports but were missed by our method. CONCLUSIONS: We automatically linked indicators to clinical trial registrations with high precision. Whilst the majority of quality indicators studied could be directly linked to research evidence, a small portion could not and these require closer scrutiny. It is feasible to support the process of indicator development using automated methods to identify research evidence.

Comparing clinical quality indicators for asthma management in children with outcome measures used in randomised controlled trials: a protocol.

INTRODUCTION: Clinical quality indicators are necessary to monitor the performance of healthcare services. The development of indicators should, wherever possible, be based on research evidence to minimise the risk of bias which may be introduced during their development, because of logistic, ethical or financial constraints alone. The development of automated methods to identify the evidence base for candidate indicators should improve the process of indicator development. The objective of this study is to explore the relationship between clinical quality indicators for asthma management in children with outcome and process measurements extracted from randomised controlled clinical trial reports. METHODS AND ANALYSIS: National-level indicators for asthma management in children will be extracted from the National Quality Measures Clearinghouse (NQMC) database and the National Institute for Health and Care Excellence (NICE) quality standards. Outcome measures will be extracted from published English language randomised controlled trial (RCT) reports for asthma management in children aged below 12 years. The two sets of measures will be compared to assess any overlap. The study will provide insights into the relationship between clinical quality indicators and measurements in RCTs. This study will also yield a list of measurements used in RCTs for asthma management in children, and will find RCT evidence for indicators used in practice. ETHICS AND DISSEMINATION: Ethical approval is not necessary because this study will not include patient data. Findings will be disseminated through peer-reviewed publications.

Automatic evidence retrieval for systematic reviews.

BACKGROUND: Snowballing involves recursively pursuing relevant references cited in the retrieved literature and adding them to the search results. Snowballing is an alternative approach to discover additional evidence that was not retrieved through conventional search. Snowballing's effectiveness makes it best practice in systematic reviews despite being time-consuming and tedious. OBJECTIVE: Our goal was to evaluate an automatic method for citation snowballing's capacity to identify and retrieve the full text and/or abstracts of cited articles. METHODS: Using 20 review articles that contained 949 citations to journal or conference articles, we manually searched Microsoft Academic Search (MAS) and identified 78.0% (740/949) of the cited articles that were present in the database. We compared the performance of the automatic citation snowballing method against the results of this manual search, measuring precision, recall, and F1 score. RESULTS: The automatic method was able to correctly identify 633 (as proportion of included citations: recall=66.7%, F1 score=79.3%; as proportion of citations in MAS: recall=85.5%, F1 score=91.2%) of citations with high precision (97.7%), and retrieved the full text or abstract for 490 (recall=82.9%, precision=92.1%, F1 score=87.3%) of the 633 correctly retrieved citations. CONCLUSIONS: The proposed method for automatic citation snowballing is accurate and is capable of obtaining the full texts or abstracts for a substantial proportion of the scholarly citations in review articles. By automating the process of citation snowballing, it may be possible to reduce the time and effort of common evidence surveillance tasks such as keeping trial registries up to date and conducting systematic reviews.

Systematic review automation technologies.

Systematic reviews, a cornerstone of evidence-based medicine, are not produced quickly enough to support clinical practice. The cost of production, availability of the requisite expertise and timeliness are often quoted as major contributors for the delay. This detailed survey of the state of the art of information systems designed to support or automate individual tasks in the systematic review, and in particular systematic reviews of randomized controlled clinical trials, reveals trends that see the convergence of several parallel research projects.We surveyed literature describing informatics systems that support or automate the processes of systematic review or each of the tasks of the systematic review. Several projects focus on automating, simplifying and/or streamlining specific tasks of the systematic review. Some tasks are already fully automated while others are still largely manual. In this review, we describe each task and the effect that its automation would have on the entire systematic review process, summarize the existing information system support for each task, and highlight where further research is needed for realizing automation for the task. Integration of the systems that automate systematic review tasks may lead to a revised systematic review workflow. We envisage the optimized workflow will lead to system in which each systematic review is described as a computer program that automatically retrieves relevant trials, appraises them, extracts and synthesizes data, evaluates the risk of bias, performs meta-analysis calculations, and produces a report in real time.

Gene-disease association with literature based enrichment.

MOTIVATION: Gene set enrichment analysis (GSEA) annotates gene microarray data with functional information from the biomedical literature to improve gene-disease association prediction. We hypothesize that supplementing GSEA with comprehensive gene function catalogs built automatically using information extracted from the scientific literature will significantly enhance GSEA prediction quality. METHODS: Gold standard gene sets for breast cancer (BrCa) and colorectal cancer (CRC) were derived from the literature. Two gene function catalogs (CMeSH and CUMLS) were automatically generated. 1. By using Entrez Gene to associate all recorded human genes with PubMed article IDs. 2. Using the genes mentioned in each PubMed article and associating each with the article's MeSH terms (in CMeSH) and extracted UMLS concepts (in CUMLS). Microarray data from the Gene Expression Omnibus for BrCa and CRC was then annotated using CMeSH and CUMLS and for comparison, also with several pre-existing catalogs (C2, C4 and C5 from the Molecular Signatures Database). Ranking was done using, a standard GSEA implementation (GSEA-p). Gene function predictions for enriched array data were evaluated against the gold standard by measuring area under the receiver operating characteristic curve (AUC). RESULTS: Comparison of ranking using the literature enrichment catalogs, the pre-existing catalogs as well as five randomly generated catalogs show the literature derived enrichment catalogs are more effective. The AUC for BrCa using the unenriched gene expression dataset was 0.43, increasing to 0.89 after gene set enrichment with CUMLS. The AUC for CRC using the unenriched gene expression dataset was 0.54, increasing to 0.9 after enrichment with CMeSH. C2 increased AUC (BrCa 0.76, CRC 0.71) but C4 and C5 performed poorly (between 0.35 and 0.5). The randomly generated catalogs also performed poorly, equivalent to random guessing. DISCUSSION: Gene set enrichment significantly improved prediction of gene-disease association. Selection of enrichment catalog had a substantial effect on prediction accuracy. The literature based catalogs performed better than the MSigDB catalogs, possibly because they are more recent. Catalogs generated automatically from the literature can be kept up to date. CONCLUSION: Prediction of gene-disease association is a fundamental task in biomedical research. GSEA provides a promising method when using literature-based enrichment catalogs. AVAILABILITY: The literature based catalogs generated and used in this study are available from http://www2.chi.unsw.edu.au/literature-enrichment.

Role of citation tracking in updating of systematic reviews.

We proposed to use automatic citation tracking to enhance the retrieval of new evidence for updating Systematic Reviews (SR). We tested on a Cochrane review from 2003 (updated 2010) and retrieved 12 of the papers to be added (recall 85.7%). Citation tracking yields a high proportion of the required literature.

The implications of biomarker evidence for systematic reviews.

BACKGROUND: In Evidence-Based Medicine, clinical practice guidelines and systematic reviews are crucial devices for medical practitioners in making clinical decision. Clinical practice guidelines are systematically developed statements to support health care decisions for specific circumstances whereas systematic reviews are summaries of evidence on clearly formulated clinical questions. Biomarkers are biological measurements (primarily molecular) that are used to diagnose, predict treatment outcomes and prognosticate disease and are increasingly used in randomized controlled trials (RCT). METHODS: We search PubMed for systematic reviews, RCTs, case reports and non-systematic reviews with and without mentions of biomarkers between years 1990-2011. We compared the frequency and growth rate of biomarkers and non-biomarkers publications. We also compared the growth of the proportion of biomarker-based RCTs with the growth of the proportion of biomarker-based systematic reviews. RESULTS: With 147,774 systematic reviews indexed in PubMed from 1990 to 2011 (accessed on 18/10/2012), only 4,431 (3%) are dedicated to biomarkers. The annual growth rate of biomarkers publications is consistently higher than non-biomarkers publications, showing the growth in biomarkers research. From 20 years of systematic review publications indexed in PubMed, we identified a bias in systematic reviews against the inclusion of biomarker-based RCTs. CONCLUSIONS: With the realisation of genome-based personalised medicine, biomarkers are becoming important for clinical decision making. The bias against the inclusion of biomarkers in systematic reviews leads to medical practitioners deprive of important information they require to address clinical questions. Sparse or weak evidence and lack of genetic training for systematic reviewers may contribute to this trend.

Genetic and epigenetic biomarkers of colorectal cancer.

Cancer is a heterogeneous disease caused, in part, by genetic and epigenetic alterations. These changes have been explored in studies of the pathogenesis of colorectal cancer (CRC) and have led to the identification of many biomarkers of disease progression. However, the number of biomarkers that have been incorporated into clinical practice is surprisingly small. We review the genetic and epigenetic mechanisms of colorectal cancer and discuss molecular markers recommended for use in early detection, screening, diagnosis, determination of prognosis, and prediction of treatment outcomes. We also review important areas for future research.

Study of microarray time series data based on Forward-Backward Linear Prediction and Singular Value Decomposition.

We propose a method to analyse the periodicities of gene expression profiles based on the spectral domain approach. Our spectral reconstruction method outperforms three other recently proposed methods, which do not require any prior knowledge. It is proven that an alternative method for studying cell-cycle regulation is possible even where very little prior knowledge is available. We also investigate the potential of combining signals with similar frequency components to form an overdetermined system of equations, and use least squares solution to estimate the spectral frequency. Results show that this new method is able to estimate the peak frequency more accurately.

Autoregressive-model-based missing value estimation for DNA microarray time series data.

Missing value estimation is important in DNA microarray data analysis. A number of algorithms have been developed to solve this problem, but they have several limitations. Most existing algorithms are not able to deal with the situation where a particular time point (column) of the data is missing entirely. In this paper, we present an autoregressive-model-based missing value estimation method (ARLSimpute) that takes into account the dynamic property of microarray temporal data and the local similarity structures in the data. ARLSimpute is especially effective for the situation where a particular time point contains many missing values or where the entire time point is missing. Experiment results suggest that our proposed algorithm is an accurate missing value estimator in comparison with other imputation methods on simulated as well as real microarray time series datasets.

Multi-scale parametric spectral analysis for exon detection in DNA sequences based on forward-backward linear prediction and singular value decomposition of the double-base curves.

This paper presents a new method for exon detection in DNA sequences based on multi-scale parametric spectral analysis. A forward-backward linear prediction (FBLP) with the singular value decomposition (SVD) algorithm FBLP-SVD is applied to the double-base curves (DB-curves) of a DNA sequence using a variable moving window sizes to estimate the signal spectrum at multiple scales. Simulations are done on short human genes in the range of 11bp to 2032bp and the results show that our proposed method out-performs the classical Fourier transform method. The multi-scale approach is shown to be more effective than using a single scale with a fixed window size. In addition, our method is flexible as it requires no training data.

Cost-effective handling of digital medical images in the telemedicine environment.

BACKGROUND: This paper concentrates on strategies for less costly handling of medical images. Aspects of digitization using conventional digital cameras, lossy compression with good diagnostic quality, and visualization through less costly monitors are discussed. METHOD: For digitization of film-based media, subjective evaluation of the suitability of digital cameras as an alternative to the digitizer was undertaken. To save on storage, bandwidth and transmission time, the acceptable degree of compression with diagnostically no loss of important data was studied through randomized double-blind tests of the subjective image quality when compression noise was kept lower than the inherent noise. A diagnostic experiment was undertaken to evaluate normal low cost computer monitors as viable viewing displays for clinicians. RESULTS: The results show that conventional digital camera images of X-ray images were diagnostically similar to the expensive digitizer. Lossy compression, when used moderately with the imaging noise to compression noise ratio (ICR) greater than four, can bring about image improvement with better diagnostic quality than the original image. Statistical analysis shows that there is no diagnostic difference between expensive high quality monitors and conventional computer monitors. CONCLUSION: The results presented show good potential in implementing the proposed strategies to promote widespread cost-effective telemedicine and digital medical environments.

Improving diagnostic quality of MR images through controlled lossy compression using SPIHT.

This paper attempts to improve the diagnostic quality of magnetic resonance (MR) images through application of lossy compression as a noise-reducing filter. The amount of imaging noise present in MR images is compared with the amount of noise introduced by the compression, with particular attention given to the situation where the compression noise is a fraction of the imaging noise. A popular wavelet-based algorithm with good performance, Set Partitioning in Hierarchical Trees (SPIHT), was employed for the lossy compression. Tests were conducted with a number of MR patient images and corresponding phantom images. Different plausible ratios between imaging noise and compression noise (ICR) were considered, and the achievable compression gain through the controlled lossy compression was evaluated. Preliminary results show that at certain ICR's, it becomes virtually impossible to distinguish between the original and compressed-decompressed image. Radiologists presented with a blind test, in certain cases, showed preference to the compressed image rather than the original uncompressed ones, indicating that under controlled circumstances, lossy image compression can be used to improve the diagnostic quality of the MR images.