Outcome risk model development for heterogeneity of treatment effect analyses: a comparison of non-parametric machine learning methods and semi-parametric statistical methods.

BACKGROUND: In randomized clinical trials, treatment effects may vary, and this possibility is referred to as heterogeneity of treatment effect (HTE). One way to quantify HTE is to partition participants into subgroups based on individual's risk of experiencing an outcome, then measuring treatment effect by subgroup. Given the limited availability of externally validated outcome risk prediction models, internal models (created using the same dataset in which heterogeneity of treatment analyses also will be performed) are commonly developed for subgroup identification. We aim to compare different methods for generating internally developed outcome risk prediction models for subject partitioning in HTE analysis. METHODS: Three approaches were selected for generating subgroups for the 2,441 participants from the United States enrolled in the ASPirin in Reducing Events in the Elderly (ASPREE) randomized controlled trial. An extant proportional hazards-based outcomes predictive risk model developed on the overall ASPREE cohort of 19,114 participants was identified and was used to partition United States' participants by risk of experiencing a composite outcome of death, dementia, or persistent physical disability. Next, two supervised non-parametric machine learning outcome classifiers, decision trees and random forests, were used to develop multivariable risk prediction models and partition participants into subgroups with varied risks of experiencing the composite outcome. Then, we assessed how the partitioning from the proportional hazard model compared to those generated by the machine learning models in an HTE analysis of the 5-year absolute risk reduction (ARR) and hazard ratio for aspirin vs. placebo in each subgroup. Cochran's Q test was used to detect if ARR varied significantly by subgroup. RESULTS: The proportional hazard model was used to generate 5 subgroups using the quintiles of the estimated risk scores; the decision tree model was used to generate 6 subgroups (6 automatically determined tree leaves); and the random forest model was used to generate 5 subgroups using the quintiles of the prediction probability as risk scores. Using the semi-parametric proportional hazards model, the ARR at 5 years was 15.1% (95% CI 4.0-26.3%) for participants with the highest 20% of predicted risk. Using the random forest model, the ARR at 5 years was 13.7% (95% CI 3.1-24.4%) for participants with the highest 20% of predicted risk. The highest outcome risk group in the decision tree model also exhibited a risk reduction, but the confidence interval was wider (5-year ARR = 17.0%, 95% CI= -5.4-39.4%). Cochran's Q test indicated ARR varied significantly only by subgroups created using the proportional hazards model. The hazard ratio for aspirin vs. placebo therapy did not significantly vary by subgroup in any of the models. The highest risk groups for the proportional hazards model and random forest model contained 230 participants each, while the highest risk group in the decision tree model contained 41 participants. CONCLUSIONS: The choice of technique for internally developed models for outcome risk subgroups influences HTE analyses. The rationale for the use of a particular subgroup determination model in HTE analyses needs to be explicitly defined based on desired levels of explainability (with features importance), uncertainty of prediction, chances of overfitting, and assumptions regarding the underlying data structure. Replication of these analyses using data from other mid-size clinical trials may help to establish guidance for selecting an outcomes risk prediction modelling technique for HTE analyses.

Psychometric evaluation of the DePaul Symptom Questionnaire-Short Form (DSQ-SF) among adults with Long COVID, ME/CFS, and healthy controls: A machine learning approach.

Long COVID shares a number of clinical features with myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS), including post-exertional malaise, severe fatigue, and neurocognitive deficits. Utilizing validated assessment tools that accurately and efficiently screen for these conditions can facilitate diagnostic and treatment efforts, thereby improving patient outcomes. In this study, we generated a series of random forest machine learning algorithms to evaluate the psychometric properties of the DePaul Symptom Questionnaire-Short Form (DSQ-SF) in classifying large groups of adults with Long COVID, ME/CFS (without Long COVID), and healthy controls. We demonstrated that the DSQ-SF can accurately classify these populations with high degrees of sensitivity and specificity. In turn, we identified the particular DSQ-SF symptom items that best distinguish Long COVID from ME/CFS, as well as those that differentiate these illness groups from healthy controls.

Optimizing Computer-Aided Diagnosis with Cost-Aware Deep Learning Models.

Classical machine learning and deep learning models for Computer-Aided Diagnosis (CAD) commonly focus on overall classification performance, treating misclassification errors (false negatives and false positives) equally during training. This uniform treatment overlooks the distinct costs associated with each type of error, leading to suboptimal decision-making, particularly in the medical domain where it is important to improve the prediction sensitivity without significantly compromising overall accuracy. This study introduces a novel deep learning-based CAD system that incorporates a cost-sensitive parameter into the activation function. By applying our methodologies to two medical imaging datasets, our proposed study shows statistically significant increases of 3.84% and 5.4% in sensitivity while maintaining overall accuracy for Lung Image Database Consortium (LIDC) and Breast Cancer Histological Database (BreakHis), respectively. Our findings underscore the significance of integrating cost-sensitive parameters into future CAD systems to optimize performance and ultimately reduce costs and improve patient outcomes.

Evaluation of Eye Gaze Dynamics During Physician-Patient-Computer Interaction in Federally Qualified Health Centers: Systematic Analysis.

BACKGROUND: Understanding the communication between physicians and patients can identify areas where they can improve and build stronger relationships. This led to better patient outcomes including increased engagement, enhanced adherence to treatment plan, and a boost in trust. OBJECTIVE: This study investigates eye gaze directions of physicians, patients, and computers in naturalistic medical encounters at Federally Qualified Health Centers to understand communication patterns given different patients' diverse backgrounds. The aim is to support the building and designing of health information technologies, which will facilitate the improvement of patient outcomes. METHODS: Data were obtained from 77 videotaped medical encounters in 2014 from 3 Federally Qualified Health Centers in Chicago, Illinois, that included 11 physicians and 77 patients. Self-reported surveys were collected from physicians and patients. A systematic analysis approach was used to thoroughly examine and analyze the data. The dynamics of eye gazes during interactions between physicians, patients, and computers were evaluated using the lag sequential analysis method. The objective of the study was to identify significant behavior patterns from the 6 predefined patterns initiated by both physicians and patients. The association between eye gaze patterns was examined using the Pearson chi-square test and the Yule Q test. RESULTS: The results of the lag sequential method showed that 3 out of 6 doctor-initiated gaze patterns were followed by patient-response gaze patterns. Moreover, 4 out of 6 patient-initiated patterns were significantly followed by doctor-response gaze patterns. Unlike the findings in previous studies, doctor-initiated eye gaze behavior patterns were not leading patients' eye gaze. Moreover, patient-initiated eye gaze behavior patterns were significant in certain circumstances, particularly when interacting with physicians. CONCLUSIONS: This study examined several physician-patient-computer interaction patterns in naturalistic settings using lag sequential analysis. The data indicated a significant influence of the patients' gazes on physicians. The findings revealed that physicians demonstrated a higher tendency to engage with patients by reciprocating the patient's eye gaze when the patient looked at them. However, the reverse pattern was not observed, suggesting a lack of reciprocal gaze from patients toward physicians and a tendency to not direct their gaze toward a specific object. Furthermore, patients exhibited a preference for the computer when physicians directed their eye gaze toward it.

Biomedical heterogeneous data categorization and schema mapping toward data integration.

Data integration is a well-motivated problem in the clinical data science domain. Availability of patient data, reference clinical cases, and datasets for research have the potential to advance the healthcare industry. However, the unstructured (text, audio, or video data) and heterogeneous nature of the data, the variety of data standards and formats, and patient privacy constraint make data interoperability and integration a challenge. The clinical text is further categorized into different semantic groups and may be stored in different files and formats. Even the same organization may store cases in different data structures, making data integration more challenging. With such inherent complexity, domain experts and domain knowledge are often necessary to perform data integration. However, expert human labor is time and cost prohibitive. To overcome the variability in the structure, format, and content of the different data sources, we map the text into common categories and compute similarity within those. In this paper, we present a method to categorize and merge clinical data by considering the underlying semantics behind the cases and use reference information about the cases to perform data integration. Evaluation shows that we were able to merge 88% of clinical data from five different sources.

Cytokine network analysis in a community-based pediatric sample of patients with myalgic encephalomyelitis/chronic fatigue syndrome.

OBJECTIVES: Studies have demonstrated immune dysfunction in adolescents with Myalgic Encephalomyelitis/Chronic Fatigue Syndrome (ME/CFS); however, evidence is varied. The current study used network analysis to examine relationships between cytokines among a sample of pediatric participants with ME/CFS. METHODS: 10,119 youth aged 5-17 in the Chicagoland area were screened for ME/CFS; 111 subjects and controls were brought in for a physician examination and completed a blood draw. Youth were classified as controls (Cs, N = 43), ME/CFS (N = 23) or severe (S-ME/CFS, N = 45). Patterns of plasma cytokine networks were analyzed. RESULTS: All participant groups displayed a primary network of interconnected cytokines. In the ME/CFS group, inflammatory cytokines IL-12p70, IL-17A, and IFN-γ were connected and included in the primary membership, suggesting activation of inflammatory mechanisms. The S-ME/CFS group demonstrated a strong relationship between IL-17A and IL-23, a connection associated with chronic inflammation. The relationships of IL-6 and IL-8 in ME/CFS and S-ME/CFS participants also differed from Cs. Together, these results indicate pro-inflammatory responses in our illness populations. DISCUSSION: Our data imply biological differences between our three participant groups, with ME/CFS and S-ME/CFS participants demonstrating an inflammatory profile. Examining co-expression of cytokines may aid in the identification of a biomarker for pediatric ME/CFS.

Pre-illness data reveals differences in multiple metabolites and metabolic pathways in those who do and do not recover from infectious mononucleosis.

Metabolic pathways related to energy production, amino acids, nucleotides, nitrogen, lipids, and neurotransmitters in myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) may contribute to the pathophysiology of ME/CFS. 4501 Northwestern University college students were enrolled in a prospective, longitudinal study. We collected data before illness, during infectious mononucleosis (IM), and at a 6 month follow-up for those who recovered (N = 18) versus those who went on to develop ME/CFS 6 months later (N = 18). Examining pre-illness blood samples, we found significant detectable metabolite differences between participants fated to develop severe ME/CFS following IM versus recovered controls. We identified glutathione metabolism, nucleotide metabolism, and the TCA cycle (among others) as potentially dysregulated pathways. The pathways that differed between cases and controls are essential for proliferating cells, particularly during a pro-inflammatory immune response. Performing a series of binary logistic regressions using a leave-one-out cross-validation (LOOCV), our models correctly classified the severe ME/CFS group and recovered controls with an accuracy of 97.2%, sensitivity of 94.4%, and specificity of 100.0%. These changes are consistent with the elevations in pro-inflammatory cytokines that we have reported for patients fated to develop severe ME/CFS 6 months after IM.

Predictors for Developing Severe Myalgic Encephalomyelitis/Chronic Fatigue Syndrome Following Infectious Mononucleosis.

Background: About 10% of individuals who contract infectious mononucleosis (IM) have symptoms 6 months later that meet criteria for myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS). Our study for the first time examined whether it is possible to predict who will develop ME/CFS following IM. Methods: We have reported on a prospectively recruited cohort of 4,501 college students, of which 238 (5.3%) developed IM. Those who developed IM were followed-up at six months to determine whether they recovered or met criteria for ME/CFS. The present study focuses on 48 students who after six months had a diagnosis of ME/CFS, and a matched control group of 58 students who had no further symptoms after their IM. All of these 106 students had data at baseline (at least 6 weeks prior to the development of IM), when experiencing IM, and 6 months following IM. Of those who did not recover from IM, there were two groups: 30 were classified as ME/CFS and 18 were classified as severe ME/CFS. We measured the results of 7 questionnaires, physical examination findings, the severity of mononucleosis and cytokine analyses at baseline (pre-illness) and at the time of IM. We examined predictors (e.g., pre-illness variables as well as variables at onset of IM) of those who developed ME/CFS and severe ME/CFS following IM. Results: From analyses using receiver operating characteristic statistics, the students who had had severe gastrointestinal symptoms of stomach pain, bloating, and an irritable bowel at baseline and who also had abnormally low levels of the immune markers IL-13 and/or IL-5 at baseline, as well as severe gastrointestinal symptoms when then contracted IM, were found to have a nearly 80% chance of having severe ME/CFS persisting six months following IM. Conclusions: Our findings are consistent with emerging literature that gastrointestinal distress and autonomic symptoms, along with several immune markers, may be implicated in the development of severe ME/CFS.

Learning Latent Spiculated Features for Lung Nodule Characterization.

Computer-aided Diagnosis (CAD) systems have long aimed to be used in clinical practice to help doctors make decisions by providing a second opinion. However, most machine learning based CAD systems make predictions without explicitly showing how their predictions were generated. Since the cognitive process of the diagnostic imaging interpretation involves various visual characteristics of the region of interest, the explainability of the results should leverage those characteristics. We encode visual characteristics of the region of interest based on pairs of similar images rather than the image content by itself. Using a Siamese convolutional neural network (SCNN), we first learn the similarity among nodules, then encode image content using the SCNN similarity-based feature representation, and lastly, we apply the K-nearest neighbor (KNN) approach to make diagnostic characterizations using the Siamese-based image features. We demonstrate the feasibility of our approach on spiculation, a visual characteristic that radiologists consider when interpreting the degree of cancer malignancy, and the NIH/NCI Lung Image Database Consortium (LIDC) dataset that contains both spiculation and malignancy characteristics for lung nodules.Clinical Relevance - This establishes that spiculation can be quantified to automate the diagnostic characterization of lung nodules in Computed Tomography images.

Ontology-Based Radiology Teaching File Summarization, Coverage, and Integration.

Radiology teaching file repositories contain a large amount of information about patient health and radiologist interpretation of medical findings. Although valuable for radiology education, the use of teaching file repositories has been hindered by the ability to perform advanced searches on these repositories given the unstructured format of the data and the sparseness of the different repositories. Our term coverage analysis of two major medical ontologies, Radiology Lexicon (RadLex) and Unified Medical Language System (UMLS) Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT), and two teaching file repositories, Medical Imaging Resource Community (MIRC) and MyPacs, showed that both ontologies combined cover 56.3% of terms in the MIRC and only 17.9% of terms in MyPacs. Furthermore, the overlap between the two ontologies (i.e., terms included by both the RadLex and UMLS SNOMED CT) was a mere 5.6% for the MIRC and 2% for the RadLex. Clustering the content of the teaching file repositories showed that they focus on different diagnostic areas within radiology. The MIRC teaching file covers mostly pediatric cases; a few cases are female patients with heart-, chest-, and bone-related diseases. The MyPacs contains a range of different diseases with no focus on a particular disease category, gender, or age group. MyPacs also provides a wide variety of cases related to the neck, face, heart, chest, and breast. These findings provide valuable insights on what new cases should be added or how existent cases may be integrated to provide more comprehensive data repositories. Similarly, the low-term coverage by the ontologies shows the need to expand ontologies with new terminology such as new terms learned from these teaching file repositories and validated by experts. While our methodology to organize and index data using clustering approaches and medical ontologies is applied to teaching file repositories, it can be applied to any other medical clinical data.

Drusen diagnosis comparison between hyper-spectral and color retinal images.

Age-related macular degeneration (AMD) is a degenerative aging disorder, which can lead to irreversible vision loss in older individuals. The emergence of clinical applications of retinal hyper-spectral imaging provides a unique opportunity to capture important spectral signatures, with the potential to enhance the molecular diagnosis of retinal diseases. In this study, we use a machine learning classification approach to explore whether hyper-spectral images offer an improved outcome compared to standard RGB images. Our results show that the classifier performs better on hyper-spectral images with improved accuracy and sensitivity for drusen classification compared to standard imaging. By examining the most important features in the classification task, our data suggest that drusen are highly heterogeneous. Our work provides further evidence that hyper-spectral retinal image data are uniquely suited for computer-aided diagnosis and detection techniques.

A Run-Length Encoding Approach for Path Analysis of C. elegans Search Behavior.

The nematode Caenorhabditis elegans explores the environment using a combination of different movement patterns, which include straight movement, reversal, and turns. We propose to quantify C. elegans movement behavior using a computer vision approach based on run-length encoding of step-length data. In this approach, the path of C. elegans is encoded as a string of characters, where each character represents a path segment of a specific type of movement. With these encoded string data, we perform k-means cluster analysis to distinguish movement behaviors resulting from different genotypes and food availability. We found that shallow and sharp turns are the most critical factors in distinguishing the differences among the movement behaviors. To validate our approach, we examined the movement behavior of tph-1 mutants that lack an enzyme responsible for serotonin biosynthesis. A k-means cluster analysis with the path string-encoded data showed that tph-1 movement behavior on food is similar to that of wild-type animals off food. We suggest that this run-length encoding approach is applicable to trajectory data in animal or human mobility data.

Case definitions integrating empiric and consensus perspectives.

BACKGROUND: There has been considerable controversy regarding how to name and define the illnesses known as myalgic encephalomyelitis (ME) and chronic fatigue syndrome (CFS). The IOM report has proposed a new clinical criteria and name for this illness, but aspects of these recommendations have been scrutinized by patients and scientists. PURPOSE: It is possible that both empiric and consensus approaches could be used to help settle some of these diagnostic challenges. Using patient samples collected in the United States, Great Britain, and Norway (N=556), the current study attempted to categorize patients using more general as well as more restricted case definitions. RESULTS: Overall, the outcomes suggest that there might be four groupings of patients, with the broadest category involving those with chronic fatigue (N=62), defined by 6 or more months of fatigue which can be cannot be explained by medical or psychiatric conditions. A second category involves those patients that have chronic fatigue that can be explained by a medical or psychiatric condition (N=47). A third category involves more specific criteria that have been posited both by the IOM report, a Canadian Clinical Case criteria, a ME-ICC criteria and a more empiric approach. These efforts have specified domains of substantial reductions of activity, post-exertional malaise, neurocognitive impairment, and sleep dysfunction (N=346). Patients with these characteristics were more functionally impaired than those meeting just chronic fatigue criteria, p < .05. Finally, those meeting even more restrictive ME criteria proposed by Ramsay, identified a smaller and even more impaired group, p < .05. DISCUSSION: The advantages of using such empirical and consensus approaches to develop reliable classification and diagnostic efforts are discussed.

Identifying Key Symptoms Differentiating Myalgic Encephalomyelitis and Chronic Fatigue Syndrome from Multiple Sclerosis.

It is unclear what key symptoms differentiate Myalgic Encephalomyelitis (ME) and Chronic Fatigue syndrome (CFS) from Multiple Sclerosis (MS). The current study compared self-report symptom data of patients with ME or CFS with those with MS. The self-report data is from the DePaul Symptom Questionnaire, and participants were recruited to take the questionnaire online. Data were analyzed using a machine learning technique called decision trees. Five symptoms best differentiated the groups. The best discriminating symptoms were from the immune domain (i.e., flu-like symptoms and tender lymph nodes), and the trees correctly categorized MS from ME or CFS 81.2% of the time, with those with ME or CFS having more severe symptoms. Our findings support the use of machine learning to further explore the unique nature of these different chronic diseases.

Computational Methods for Tracking, Quantitative Assessment, and Visualization of C. elegans Locomotory Behavior.

The nematode Caenorhabditis elegans provides a unique opportunity to interrogate the neural basis of behavior at single neuron resolution. In C. elegans, neural circuits that control behaviors can be formulated based on its complete neural connection map, and easily assessed by applying advanced genetic tools that allow for modulation in the activity of specific neurons. Importantly, C. elegans exhibits several elaborate behaviors that can be empirically quantified and analyzed, thus providing a means to assess the contribution of specific neural circuits to behavioral output. Particularly, locomotory behavior can be recorded and analyzed with computational and mathematical tools. Here, we describe a robust single worm-tracking system, which is based on the open-source Python programming language, and an analysis system, which implements path-related algorithms. Our tracking system was designed to accommodate worms that explore a large area with frequent turns and reversals at high speeds. As a proof of principle, we used our tracker to record the movements of wild-type animals that were freshly removed from abundant bacterial food, and determined how wild-type animals change locomotory behavior over a long period of time. Consistent with previous findings, we observed that wild-type animals show a transition from area-restricted local search to global search over time. Intriguingly, we found that wild-type animals initially exhibit short, random movements interrupted by infrequent long trajectories. This movement pattern often coincides with local/global search behavior, and visually resembles Lévy flight search, a search behavior conserved across species. Our mathematical analysis showed that while most of the animals exhibited Brownian walks, approximately 20% of the animals exhibited Lévy flights, indicating that C. elegans can use Lévy flights for efficient food search. In summary, our tracker and analysis software will help analyze the neural basis of the alteration and transition of C. elegans locomotory behavior in a food-deprived condition.

Reflections on the Institute of Medicine's systemic exertion intolerance disease.

The Institute of Medicine (IOM) in the United States has recently proposed that the term systemic exertion intolerance disease (SEID) replace chronic fatigue syndrome. In addition, the IOM proposed a new case definition for SEID, which includes substantial reductions or impairments in the ability to engage in pre­illness activities, unrefreshing sleep, postexertional malaise, and either cognitive impairment or orthostatic intolerance. Unfortunately, these recommendations for a name change were not vetted with patient and professional audiences, and the new criteria were not evaluated with data sets of patients and controls. A recent poll suggests that the majority of patients reject this new name. In addition, studies have found that prevalence rates will dramatically increase with the new criteria, particularly due to the ambiguity revolving around exclusionary illnesses. Findings suggest that the new criteria select more patients who have less impairment and fewer symptoms than several other criteria. The implications of these findings are discussed in the current review.

Chronic Fatigue Syndrome and Myalgic Encephalomyelitis: Toward An Empirical Case Definition.

Current case definitions of Myalgic Encephalomyelitis (ME) and chronic fatigue syndrome (CFS) have been based on consensus methods, but empirical methods could be used to identify core symptoms and thereby improve the reliability. In the present study, several methods (i.e., continuous scores of symptoms, theoretically and empirically derived cut off scores of symptoms) were used to identify core symptoms best differentiating patients from controls. In addition, data mining with decision trees was conducted. Our study found a small number of core symptoms that have good sensitivity and specificity, and these included fatigue, post-exertional malaise, a neurocognitive symptom, and unrefreshing sleep. Outcomes from these analyses suggest that using empirically selected symptoms can help guide the creation of a more reliable case definition.

Presynaptic BK channel localization is dependent on the hierarchical organization of alpha-catulin and dystrobrevin and fine-tuned by CaV2 calcium channels.

BACKGROUND: Large conductance, calcium-activated BK channels regulate many important physiological processes, including smooth muscle excitation, hormone release and synaptic transmission. The biological roles of these channels hinge on their unique ability to respond synergistically to both voltage and cytosolic calcium elevations. Because calcium influx is meticulously regulated both spatially and temporally, the localization of BK channels near calcium channels is critical for their proper function. However, the mechanism underlying BK channel localization near calcium channels is not fully understood. RESULTS: We show here that in C. elegans the localization of SLO-1/BK channels to presynaptic terminals, where UNC-2/CaV2 calcium channels regulate neurotransmitter release, is controlled by the hierarchical organization of CTN-1/α-catulin and DYB-1/dystrobrevin, two proteins that interact with cortical cytoskeletal proteins. CTN-1 organizes a macromolecular SLO-1 channel complex at presynaptic terminals by direct physical interaction. DYB-1 contributes to the maintenance or stabilization of the complex at presynaptic terminals by interacting with CTN-1. We also show that SLO-1 channels are functionally coupled with UNC-2 calcium channels, and that normal localization of SLO-1 to presynaptic terminals requires UNC-2. In the absence of UNC-2, SLO-1 clusters lose the localization specificity, thus accumulating inside and outside of presynaptic terminals. Moreover, CTN-1 is also similarly localized in unc-2 mutants, consistent with the direct interaction between CTN-1 and SLO-1. However, localization of UNC-2 at the presynaptic terminals is not dependent on either CTN-1 or SLO-1. Taken together, our data strongly suggest that the absence of UNC-2 indirectly influences SLO-1 localization via the reorganization of cytoskeletal proteins. CONCLUSION: CTN-1 and DYB-1, which interact with cortical cytoskeletal proteins, are required for the presynaptic punctate localization of SLO-1 in a hierarchical manner. In addition, UNC-2 calcium channels indirectly control the fidelity of SLO-1 puncta localization at presynaptic terminals. We suggest that the absence of UNC-2 leads to the reorganization of the cytoskeletal structure that includes CTN-1, which in turn influences SLO-1 puncta localization.

Toward Understanding the Size Dependence of Shape Features for Predicting Spiculation in Lung Nodules for Computer-Aided Diagnosis.

We analyze the importance of shape features for predicting spiculation ratings assigned by radiologists to lung nodules in computed tomography (CT) scans. Using the Lung Image Database Consortium (LIDC) data and classification models based on decision trees, we demonstrate that the importance of several shape features increases disproportionately relative to other image features with increasing size of the nodule. Our shaped-based classification results show an area under the receiver operating characteristic (ROC) curve of 0.65 when classifying spiculation for small nodules and an area of 0.91 for large nodules, resulting in a 26% difference in classification performance using shape features. An analysis of the results illustrates that this change in performance is driven by features that measure boundary complexity, which perform well for large nodules but perform relatively poorly and do no better than other features for small nodules. For large nodules, the roughness of the segmented boundary maps well to the semantic concept of spiculation. For small nodules, measuring directly the complexity of hard segmentations does not yield good results for predicting spiculation due to limits imposed by spatial resolution and the uncertainty in boundary location. Therefore, a wider range of features, including shape, texture, and intensity features, are needed to predict spiculation ratings for small nodules. A further implication is that the efficacy of shape features for a particular classifier used to create computer-aided diagnosis systems depends on the distribution of nodule sizes in the training and testing sets, which may not be consistent across different research studies.

Assessing diagnostic complexity: An image feature-based strategy to reduce annotation costs.

Computer-aided diagnosis systems can play an important role in lowering the workload of clinical radiologists and reducing costs by automatically analyzing vast amounts of image data and providing meaningful and timely insights during the decision making process. In this paper, we present strategies on how to better manage the limited time of clinical radiologists in conjunction with predictive model diagnosis. We first introduce a metric for discriminating between the different categories of diagnostic complexity (such as easy versus hard) encountered when interpreting CT scans. Second, we propose to learn the diagnostic complexity using a classification approach based on low-level image features automatically extracted from pixel data. We then show how this classification can be used to decide how to best allocate additional radiologists to interpret a case based on its diagnosis category. Using a lung nodule image dataset, we determined that, by a simple division of cases into hard and easy to diagnose, the number of interpretations can be distributed to significantly lower the cost with limited loss in prediction accuracy. Furthermore, we show that with just a few low-level image features (18% of the original set) we are able to determine the easy from hard cases for a significant subset (66%) of the lung nodule image data.