An information-oriented paradigm in evaluating accuracy and agreement in radiology.

The goal of any radiological diagnostic process is to gain information about the patient's status. However, the mathematical notion of information is usually not adopted to measure the performance of a diagnostic test or the agreement among readers in providing a certain diagnosis. Indeed, commonly used metrics for assessing diagnostic accuracy (e.g., sensitivity and specificity) or inter-reader agreement (Cohen [Formula: see text] statistics) use confusion matrices containing the number of true- and false positives/negatives results of a test, or the number of concordant/discordant categorizations, respectively, thus lacking proper information content. We present a methodological paradigm, based on Shannon's information theory, aiming to measure both accuracy and agreement in diagnostic radiology. This approach models the information flow as a "diagnostic channel" connecting the state of the patient's disease and the radiologist or, in the case of agreement analysis, as an "agreement channel" linking two or more radiologists evaluating the same set of images. For both cases, we proposed some measures, derived from Shannon's mutual information, which can represent an alternative way to express diagnostic accuracy and agreement in radiology.Key points• Diagnostic processes can be modeled with information theory (IT).• IT metrics of diagnostic accuracy are independent from disease prevalence.• IT metrics of inter-reader agreements can overcome Cohen κ pitfalls.

Fifty years of Shannon information theory in assessing the accuracy and agreement of diagnostic tests.

Since 1948, Shannon theoretic methods for modeling information have found a wide range of applications in several areas where information plays a key role, which goes well beyond the original scopes for which they have been conceived, namely data compression and error correction over a noisy channel. Among other uses, these methods have been applied in the broad field of medical diagnostics since the 1970s, to quantify diagnostic information, to evaluate diagnostic test performance, but also to be used as technical tools in image processing and registration. This review illustrates the main contributions in assessing the accuracy of diagnostic tests and the agreement between raters, focusing on diagnostic test performance measurements and paired agreement evaluation. This work also presents a recent unified, coherent, and hopefully, final information-theoretical approach to deal with the flows of information involved among the patient, the diagnostic test performed to appraise the state of disease, and the raters who are checking the test results. The approach is assessed by considering two case studies: the first one is related to evaluating extra-prostatic cancers; the second concerns the quality of rapid tests for COVID-19 detection.

Pressure-flow breath representation eases asynchrony identification in mechanically ventilated patients.

Breathing asynchronies are mismatches between the requests of mechanically ventilated subjects and the support provided by mechanical ventilators. The most widespread technique in identifying these pathological conditions is the visual analysis of the intra-tracheal pressure and flow time-trends. This work considers a recently introduced pressure-flow representation technique and investigates whether it can help nurses in the early detection of anomalies that can represent asynchronies. Twenty subjects-ten Intensive Care Unit (ICU) nurses and ten persons inexperienced in medical practice-were asked to find asynchronies in 200 breaths pre-labeled by three experts. The new representation increases significantly the detection capability of the subjects-average sensitivity soared from 0.622 to 0.905-while decreasing the classification time-from 1107.0 to 567.1 s on average-at the price of a not statistically significant rise in the number of wrong identifications-specificity average descended from 0.589 to 0.52. Moreover, the differences in experience between the nurse group and the inexperienced group do not affect the sensitivity, specificity, or classification times. The pressure-flow diagram significantly increases sensitivity and decreases the response time of early asynchrony detection performed by nurses. Moreover, the data suggest that operator experience does not affect the identification results. This outcome leads us to believe that, in emergency contexts with a shortage of nurses, intensive care nurses can be supplemented, for the sole identification of possible respiratory asynchronies, by inexperienced staff.

A structured therapeutic education program for children and adolescents with type 1 diabetes: an analysis of the efficacy of the "Pediatric Education for Diabetes" project.

BACKGROUND: Therapeutic education for Type 1 Diabetes involves the process of transmitting knowledge and developing the skills and behavior required to treat the disease. guidelines agree on stressing the importance of therapeutic educational intervention in teaching self-management skills to children and adolescents with Type 1 Diabetes (T1D). This study presents the results of the "Pediatric Education for Type 1 Diabetes (T1D)" (PED) project, specifically designed for children and adolescents aged 6 to 16, and structured on guidelines indications, as part of a broader clinical-educational intervention for Type 1 diabetes. METHODS: Twenty-four patients with Type 1 diabetes (mean age: 12,13 y; SD=1.48 y; range 9-14) were studied in a 12-month PED structured project followed by an educational summer camp. All the activities were designed and organized by a multidisciplinary team (dietitian, pediatric diabetologist, nurse, psychologist and adult diabetologist). Glycated hemoglobin (HbA1C), knowledge about Type 1 Diabetes (T1D) (self-monitoring and nutrition), self-management (self-monitoring, nutrition and flexibility of medical treatment), and wellbeing were used as outcome measures. RESULTS: Data suggest that the PED had a positive impact on all the targeted levels indicated for recommended care. CONCLUSIONS: The results of this study seem to confirm the effectiveness in altering the three levels of "knowing," "know-how" and "wellbeing" required to optimize the quality of life of young patients with Type 1 diabetes. In addition, the proposed model, where a pediatric diabetologist always cooperates with an adult diabetologist, seems to be a permanent solution to the transitional gap widely discussed in the literature.

An effective pressure-flow characterization of respiratory asynchronies in mechanical ventilation.

Ineffective effort during expiration (IEE) occurs when there is a mismatch between the demand of a mechanically ventilated patient and the support delivered by a Mechanical ventilator during the expiration. This work presents a pressure-flow characterization for respiratory asynchronies and validates a machine-learning method, based on the presented characterization, to identify IEEs. 1500 breaths produced by 8 mechanically-ventilated patients were considered: 500 of them were included into the training set and the remaining 1000 into the test set. Each of them was evaluated by 3 experts and classified as either normal, artefact, or containing inspiratory, expiratory, or cycling-off asynchronies. A software implementing the proposed method was trained by using the experts' evaluations of the training set and used to identify IEEs in the test set. The outcomes were compared with a consensus of three expert evaluations. The software classified IEEs with sensitivity 0.904, specificity 0.995, accuracy 0.983, positive and negative predictive value 0.963 and 0.986, respectively. The Cohen's kappa is 0.983 (with 95% confidence interval (CI): [0.884, 0.962]). The pressure-flow characterization of respiratory cycles and the monitoring technique proposed in this work automatically identified IEEs in real-time in close agreement with the experts.

Beyond kappa: an informational index for diagnostic agreement in dichotomous and multivalue ordered-categorical ratings.

Agreement measures are useful tools to both compare different evaluations of the same diagnostic outcomes and validate new rating systems or devices. Cohen's kappa (κ) certainly is the most popular agreement method between two raters, and proved its effectiveness in the last sixty years. In spite of that, this method suffers from some alleged issues, which have been highlighted since the 1970s; moreover, its value is strongly dependent on the prevalence of the disease in the considered sample. This work introduces a new agreement index, the informational agreement (IA), which seems to avoid some of Cohen's kappa's flaws, and separates the contribution of the prevalence from the nucleus of agreement. These goals are achieved by modelling the agreement-in both dichotomous and multivalue ordered-categorical cases-as the information shared between two raters through the virtual diagnostic channel connecting them: the more information exchanged between the raters, the higher their agreement. In order to test its fair behaviour and the effectiveness of the method, IA has been tested on some cases known to be problematic for κ, in the machine learning context and in a clinical scenario to compare ultrasound (US) and automated breast volume scanner (ABVS) in the setting of breast cancer imaging. Graphical Abstract To evaluate the agreement between the two raters [Formula: see text] and [Formula: see text] we create an agreement channel, based on Shannon Information Theory, that directly connects the random variables X and Y, that express the raters outcomes. They are the terminals of the chain X⇔ diagnostic test performed by [Formula: see text] ⇔ patient condition[Formula: see text] ⇔ diagnostic test performed by [Formula: see text] ⇔ Y.

Informational analysis: a Shannon theoretic approach to measure the performance of a diagnostic test.

Diagnostic test accuracy, based on sensitivity, specificity, positive/negative predictive values (dichotomous case), and on ROC analysis (continuous case), should be expressed with a single, coherent index. We propose to modelize the diagnostic test as a flow of information between the disease, that is, a hidden state of the patient, and the physicians. We assume that (1) sensitivity, specificity, and false-positive/false-negative rates are the probabilities of a binary asymmetric channel; (2) the diagnostic channel information is measured by mutual information. We introduce two summary measures of accuracy, namely the information ratio (IR) for the dichotomous case, and the global information ratio (GIR) for the continuous case. We apply our model to a study by Pisano et al. (N Engl J Med 353(17):1773-1783, 2005), who compared digital versus film mammography, in diagnosing breast cancer in a screening population of 42,760 women. In film mammography, the maximum IR (0.178) corresponds to the standard cutoff of sensitivity and specificity provided by the ROC analysis (GIR 0.200). Maximum IR and GIR for digital mammography are higher (0.201 and 0.229, respectively), but IR corresponds to a cutoff with higher sensitivity but lower specificity, thus suggesting that larger information provided by digital mammography carries the risk of more false-positive cases.

Differentiating small (≤1 cm) focal liver lesions as metastases or cysts by means of computed tomography: a case-study to illustrate a fuzzy logic-based method to assess the impact of diagnostic confidence on radiological diagnosis.

PURPOSE: To quantify the impact of diagnostic confidence on radiological diagnosis with a fuzzy logic-based method. MATERIALS AND METHODS: Twenty-two oncologic patients with 20 cysts and 30 metastases ≤1 cm in size found at 64-row computed tomography were included. Two readers (R1/R2) expressed diagnoses as a subjective level of confidence P(d) in malignancy within the interval [0,1] rather than on a "crisp" basis (malignant/benign); confidence in benignancy was 1 - p(d). When cross-tabulating data according to the standard of reference, 2 × 2 table cells resulted from the aggregation between p(d)/1 - p(d) and final diagnosis. We then assessed (i) readers diagnostic performance on a fuzzy and crisp basis; (ii) the "divergence" δ(F, C) (%) as a measure of how confidence impacted on crisp diagnosis. RESULTS: Diagnoses expressed with lower confidence increased fuzzy false positives compared to crisp ones (from 0 to 0.2 for R1; from 1 to 2.4 for R2). Crisp/fuzzy accuracy was 94.0%/93.6% (R1) and 94.0/91.6% (R2). δ(F, C) (%) was larger in the case of the less experienced reader (R2) (up to +7.95% for specificity). According to simulations, δ(F, C) (%) was negative/positive depending on the level of confidence in incorrect diagnoses. CONCLUSION: Fuzzy evaluation shows a measurable effect of uncertainty on radiological diagnoses.

Family fingerprints: a global approach to structural classification.

Protein domain classification is a useful tool to deduce functional properties of proteins. Many software to classify domains according to available databases have been proposed so far. This paper introduces the notion of "fingerprint" as an easy and readable digest of the similarities between a protein fragment and an entire set of sequences. This concept offers us a rationale for building an automatic SCOP classifier which assigns a query sequence to the most likely family. Fingerprint-based analysis has been implemented in a software tool and we report some experimental validations for it.

Splitting the BLOSUM score into numbers of biological significance.

Mathematical tools developed in the context of Shannon information theory were used to analyze the meaning of the BLOSUM score, which was split into three components termed as the BLOSUM spectrum (or BLOSpectrum). These relate respectively to the sequence convergence (the stochastic similarity of the two protein sequences), to the background frequency divergence (typicality of the amino acid probability distribution in each sequence), and to the target frequency divergence (compliance of the amino acid variations between the two sequences to the protein model implicit in the BLOCKS database). This treatment sharpens the protein sequence comparison, providing a rationale for the biological significance of the obtained score, and helps to identify weakly related sequences. Moreover, the BLOSpectrum can guide the choice of the most appropriate scoring matrix, tailoring it to the evolutionary divergence associated with the two sequences, or indicate if a compositionally adjusted matrix could perform better.