Reliability and accuracy of individual Alberta Stroke Program Early CT Score regions using a medical and a smartphone reading system in a telestroke network.

INTRODUCTION: The aim of this study was to assess individual regions of the Alberta Stroke Program Early CT Score in noncontrast head computed tomography interpretations using a smartphone in a telestroke network, by comparison to a medical monitor. METHODS: The review board of our institution approved this retrospective study. A factorial design with 188 patients, four radiologists and two reading systems was used. Accuracy and reliability were evaluated. RESULTS: Very good interobserver agreements were observed on the total Alberta Stroke Program Early CT Score for both the medical and smartphone reading systems, with intraclass correlation coefficients of 0.91 and 0.84 respectively. Interobserver agreements were moderate to very good for the medical reading system (all intraclass correlation coefficients >0.74), whereas they were fair to very good for the smartphone (intraclass correlation coefficients ranged from 0.31-0.81). All intraobserver agreements were good (intraclass correlation coefficient >0.64), except for internal capsule (0.48) and M2 (0.55) regions. The areas under the receiver-operating curve ranged from 0.69-0.89 for the medical system, while for the smartphone ranged from 0.44-0.86. No statistical differences were observed between medical and smartphone reading systems for each region (all p > 0.05). DISCUSSION: If radiologists are better trained in the evaluation of the lesions in the insula, the internal capsule and the M2 regions, the total and the dichotomised Alberta Stroke Program Early CT Score will be more precise. Hence, ruling out contraindications to thrombolysis administration will be improved, allowing assessment of head computed tomography in a telestroke network using a smartphone to be a common practice.

Accuracy and Reliability of the Recommendation for IV Thrombolysis in Acute Ischemic Stroke Based on Interpretation of Head CT on a Smartphone or a Laptop.

OBJECTIVE. The objective of this study was to assess the accuracy and reliability of IV thrombolysis recommendations made after interpretation of head CT images of patients with symptoms of acute stroke displayed on smartphone or laptop reading systems compared with those made after interpretation of images displayed on a medical workstation monitor. MATERIALS AND METHODS. This retrospective study was institutional review board-approved, and the requirement for informed consent was waived. We used a factorial design including 2256 interpretations (188 patients, four neuroradiologists, and three reading systems). To evaluate the reliability, we calculated the intraobserver and interobserver agreements using the intraclass correlation coefficient (ICC) and the following interpretation variables: hemorrhagic lesions, intraaxial neoplasm, stroke dating (acute, subacute, and chronic), hyperdense arteries, and infarct size assessment. Accuracy equivalence tests were performed for the IV thrombolysis recommendation; for this variable, sensitivity, specificity, and ROC curves were evaluated. RESULTS. Good or very good interobserver and intraobserver agreements were obtained after interpretation of each variable. The IV thrombolysis recommendation showed very good interobserver agreements (ICC ≥ 0.85) and very good intraobserver agreements (ICC ≥ 0.81). For the IV thrombolysis recommendation, the AUC values (0.83-0.84) and sensitivities (0.94-0.95) were equivalent among all the reading systems at a 5% equivalent threshold. CONCLUSION. Our study found that mobile devices are reliable and accurate to help stroke teams to decide whether to administer IV thrombolysis in patients with acute stroke.

Evaluation of the Accuracy Equivalence of Head CT Interpretations in Acute Stroke Patients Using a Smartphone, a Laptop, or a Medical Workstation.

PURPOSE: To evaluate and compare the clinical performance of observers interpreting head CT images from patients with symptoms of acute stroke with a medical workstation or a smartphone or laptop reading system. MATERIALS AND METHODS: Our institutional review board approved this retrospective study and waived the requirement for informed consent. We employed a factorial design including 2,256 interpretations (188 patients × 4 neuroradiologists × 3 reading systems). Accuracy equivalence tests, at a threshold of 5% and 10%, were performed for the following variables: detection of imaging contraindications for intravenous recombinant tissue-type plasminogen activator administration (eg, hemorrhagic lesions), ischemic lesions, hyperdense arteries, and acute ischemic lesions. For each clinical variable, the sensitivity, specificity, and receiver operating characteristic (ROC) curves were evaluated. RESULTS: For each variable, the shapes of the ROC curves were very similar for all of the reading systems, indicating similar observer performance with different specificities and sensitivities. For all the clinical variables, the areas under the ROC curves were equivalent for all of the reading systems at a 10% threshold and were equivalent at a 5% threshold for hemorrhagic lesions, hyperdense middle cerebral artery, and acute ischemic lesion in the middle cerebral artery territory. There was no more than 30 seconds of difference between the reading time of the mobile devices compared with that for the medical workstation. CONCLUSION: The results of this pilot study showed equivalent diagnostic accuracy when using smartphone and laptops compared with medical monitors to interpret head CT images of patients with symptoms of acute stroke.

Mobile device for thrombolysis decisions for telestroke / Dispositivo móvil para decisiones de trombólisis mediante Telestroke

Abstract Aim: This study compares the reliability of brain CT interpretations performed using a diagnostic workstation and a mobile tablet computer in a telestroke context. Methods: A factorial design with 1,452 interpretations was used. Reliability was evaluated using the Fleiss' kappa coefficient on the agreements of the interpretation results on the lesion classification, presence of imaging contraindications to the intravenous recombinant tissue-type plasminogen activator (t-PA) administration, and on the Alberta Stroke Program Early CT Score (ASPECTS). Results: The intra-observer agreements were as follows: good agreement on the overall lesion classification (κ= 0.63, p<0.001), very good agreement on hemorrhagic lesions (κ= 0.89, p<0.001), and moderate agreements on both without acute lesion classification and acute ischemic lesion classification (κ= 0.59 and κ= 0.58 respectively, p<0.001). There was good intra-observer agreement on the dichotomized-ASPECTS (κ= 0.65, p<0.001). Conclusions: The results of our study allow us to conclude that the reliability of the mobile solution for interpreting brain CT images of patients with acute stroke was assured, which would allow efficient and low-cost telestroke services.

Resumen Objetivo: Este estudio compara la confiabilidad de interpretaciones de TAC de cráneo simple realizadas utilizando una estación de trabajo de diagnóstico y un computador tableta en un contexto de teleACV. Métodos: Se utilizó un diseño factorial con 1,452 interpretaciones. La confiabilidad se evaluó utilizando el coeficiente kappa de Fleiss en las concordancias sobre los resultados de la interpretación en la clasificación de la lesión, la presencia de contraindicaciones en la imagen para la administración intravenosa del activador del plasminógeno tisular (AP-t) y con el Alberta Stroke Programme Early CT Score (ASPECTS). Resultados: Se obtuvieron las siguientes concordancias intraobservadores: buena concordancia en la clasificación general de la lesión (κ= 0.63, p<0.001), muy buena concordancia en lesiones hemorrágicas (κ= 0.89, p< 0.001), y concordancia moderada en ambos sin clasificación de lesión aguda y clasificación de lesión isquémica aguda (κ= 0.59 y κ= 0.58 respectivamente, p<0.001). Hubo una buena concordancia intraobservadores en el ASPECTS dicotomizado (κ= 0.65, p<0.001). Conclusiones: Los resultados de nuestro estudio permiten concluir que la confiabilidad de la solución móvil para la interpretación de imágenes de TAC de cráneo simple de pacientes con accidente cerebrovascular agudo (ACV) estaba garantizada, lo que permitiría servicios de teleACV eficientes y de bajo costo.

Mobile device for thrombolysis decisions for telestroke.

AIM: This study compares the reliability of brain CT interpretations performed using a diagnostic workstation and a mobile tablet computer in a telestroke context. METHODS: A factorial design with 1,452 interpretations was used. Reliability was evaluated using the Fleiss' kappa coefficient on the agreements of the interpretation results on the lesion classification, presence of imaging contraindications to the intravenous recombinant tissue-type plasminogen activator (t-PA) administration, and on the Alberta Stroke Program Early CT Score (ASPECTS). RESULTS: The intra-observer agreements were as follows: good agreement on the overall lesion classification (κ= 0.63, p<0.001), very good agreement on hemorrhagic lesions (κ= 0.89, p<0.001), and moderate agreements on both without acute lesion classification and acute ischemic lesion classification (κ= 0.59 and κ= 0.58 respectively, p<0.001). There was good intra-observer agreement on the dichotomized-ASPECTS (κ= 0.65, p<0.001). CONCLUSIONS: The results of our study allow us to conclude that the reliability of the mobile solution for interpreting brain CT images of patients with acute stroke was assured, which would allow efficient and low-cost telestroke services.

OBJETIVO: Este estudio compara la confiabilidad de interpretaciones de TAC de cráneo simple realizadas utilizando una estación de trabajo de diagnóstico y un computador tableta en un contexto de teleACV. MÉTODOS: Se utilizó un diseño factorial con 1,452 interpretaciones. La confiabilidad se evaluó utilizando el coeficiente kappa de Fleiss en las concordancias sobre los resultados de la interpretación en la clasificación de la lesión, la presencia de contraindicaciones en la imagen para la administración intravenosa del activador del plasminógeno tisular (AP-t) y con el Alberta Stroke Programme Early CT Score (ASPECTS). RESULTADOS: Se obtuvieron las siguientes concordancias intraobservadores: buena concordancia en la clasificación general de la lesión (κ= 0.63, p<0.001), muy buena concordancia en lesiones hemorrágicas (κ= 0.89, p< 0.001), y concordancia moderada en ambos sin clasificación de lesión aguda y clasificación de lesión isquémica aguda (κ= 0.59 y κ= 0.58 respectivamente, p<0.001). Hubo una buena concordancia intraobservadores en el ASPECTS dicotomizado (κ= 0.65, p<0.001). CONCLUSIONES: Los resultados de nuestro estudio permiten concluir que la confiabilidad de la solución móvil para la interpretación de imágenes de TAC de cráneo simple de pacientes con accidente cerebrovascular agudo (ACV) estaba garantizada, lo que permitiría servicios de teleACV eficientes y de bajo costo.

Ruling Out Brain CT Contraindications prior to Intravenous Thrombolysis: Diagnostic Equivalence between a Primary Interpretation Workstation and a Mobile Tablet Computer.

OBJECTIVE: The aim of this study was to evaluate the equivalence of brain CT interpretations performed using a diagnostic workstation and a mobile tablet computer, in a telestroke service. MATERIALS AND METHODS: The ethics committee of our institution approved this retrospective study. A factorial design with 1452 interpretations was used. The assessed variables were the type of stroke classification, the presence of contraindications to the tPA administration, the presence of a hyperdense intracranial artery sign (HMCA), and the Alberta Stroke Program Early CT Score (ASPECTS) score. These variables were evaluated to determine the effect that the reading system had on their magnitudes. RESULTS: The achieved distribution of observed lesions using both the reading systems was not statistically different. The differences between the two reading systems to claim equivalence were 1.6% for hemorrhagic lesions, 4.5% for cases without lesion, and 5.2 for overall ischemic lesion. Equivalence was achieved at 2.1% for ASPECTS ≤ 6, 6.5% for the presence of imaging contraindication to the tPA administration, and 7.2% for the presence of HMCA. CONCLUSION: The diagnostic performance for detecting acute stroke is likely equivalent whether a tablet computer or a diagnostic workstation is used or not.

Reliability of the BI-RADS Final Assessment Categories and Management Recommendations in a Telemammography Context.

PURPOSE: The aim of this study was to evaluate the intradevice and interdevice reliability of four alternatives for telemammography-computed radiography, printed film, a film digitizer, and a digital camera-in terms of interpretation agreement when using the BI-RADS® lexicon. METHODS: The ethics committee of the authors' institution approved this retrospective study. A factorial design with repeated measures with 1,960 interpretations was used (70 patients, seven radiologists, and four devices). Reliability was evaluated using the κ coefficient for intradevice and interdevice agreement on malignancy classification and on BI-RADS final assessment category. RESULTS: Agreement on malignancy classification was higher than agreement for BI-RADS final assessment category. Interdevice agreement on malignancy classification between the film digitizer and computed radiography was ranked as almost perfect (P < .001), whereas interdevice agreement for the other alternatives was ranked as substantial (P < .001), with observed agreement ranging from 85% to 91% and κ values ranging from 0.70 to 0.81. Interdevice agreement on BI-RADS final assessment category was ranked as substantial or moderate (P < .001), with observed agreement ranging from 64% to 77% and κ values ranging from 0.52 to 0.69. Interdevice agreement was higher than intradevice agreement. CONCLUSIONS: The results of this study show very high interdevice agreement, especially for management recommendations derived from malignancy classification, which is one of the most important outcomes in screening programs. This study provides evidence to suggest the interchangeability of the devices evaluated, thereby enabling the provision of low-cost medical imaging services to underserved populations.

Noninferiority and Equivalence Evaluation of Clinical Performance among Computed Radiography, Film, and Digitized Film for Telemammography Services.

Objective. The aim of this study was to evaluate and compare the clinical performance of different alternatives to implement low-cost screening telemammography. We compared computed radiography, film printed images, and digitized films produced with a specialized film digitizer and a digital camera. Material and Methods. The ethics committee of our institution approved this study. We assessed the equivalence of the clinical performance of observers for cancer detection. The factorial design included 70 screening patients, four technological alternatives, and cases interpreted by seven radiologists, for a total of 1,960 observations. The variables evaluated were the positive predictive value (PPV), accuracy, sensitivity, specificity, and the area under the receiver operating characteristic curves (AUC). Result. The mean values for the observed variables were as follows: accuracy ranged from 0.77 to 0.82, the PPV ranged from 0.67 to 0.68, sensitivity ranged from 0.64 to 0.74, specificity ranged from 0.87 to 0.90, and the AUC ranged from 0.87 to 0.90. At a difference of 0.1 to claim equivalence, all alternatives were equivalent for all variables. Conclusion. Our findings suggest that telemammography screening programs may be provided to underserved populations at a low cost, using a film digitizer or a digital camera.

DICOM gray-scale standard display function: clinical diagnostic accuracy of chest radiography in medical-grade gray-scale and consumer-grade color displays.

OBJECTIVE: The purpose of this study was to compare the diagnostic accuracy achieved with and without the calibration method established by the DICOM standard in both medical-grade gray-scale displays and consumer-grade color displays. MATERIALS AND METHODS: This study involved 76 cases, six radiologists, three displays, and two display calibrations for a total of 2736 observations in a multireader-multicase factorial design. The evaluated conditions were interstitial opacities, pneumothorax, and nodules. CT was adopted as the reference standard. One medical-grade gray-scale display and two consumer-grade color displays were evaluated. Analyses of ROC curves, diagnostic accuracy (measured as AUC), accuracy of condition classification, and false-positive and false-negative rate comparisons were performed. The degree of agreement between readers was also evaluated. RESULTS: No significant differences in image quality perception by the readers in the presence or absence of calibration were observed. Similar forms of the ROC curves were observed. No significant differences were detected in the observed variables (diagnostic accuracy, accuracy of condition classification, false-positive rates, false-negative rates, and image-quality perception). Strong agreement between readers was also determined for each display with and without calibration. CONCLUSION: For the chest conditions and selected observers included in this study, no significant differences were observed between the three evaluated displays with respect to accuracy performance with and without calibration.

Evaluation of low-cost telemammography screening configurations: a comparison with film-screen readings in vulnerable areas.

The aim of this study was to evaluate the diagnostic accuracy for detecting breast cancer using different telemammography configurations, including combinations of both low-cost capture devices and consumer-grade color displays. At the same time, we compared each of these configurations to film-screen readings. This study used a treatment-by-reader-by-case factorial design. The sample included 70 mammograms with 34 malignant cases. The readers consisted of four radiologists who classified all of the cases according to the categories defined by the Breast Imaging Reporting and Data System (BI-RADS). The evaluated capture devices included a specialized film digitizer and a digital camera, and the evaluated displays included liquid crystal display (LCD) and light-emitting diode (LED) consumer-grade color displays. Receiver operating characteristic curves, diagnostic accuracy (measured as the area under these curves), accuracy of the composition classification, sensitivity, specificity, and the degree of agreement between readers in the detection of malignant cases were also evaluated. Comparisons of diagnostic accuracy between film-screen and the different combinations of digital configurations showed no significant differences for nodules, calcifications, and asymmetries. In addition, no differences were observed in terms of sensibility or specificity when the degree of malignancy using the film-screen method was compared to that provided with digital configurations. Similar results were observed for the classification of breast composition. Furthermore, all observed reader agreements of malignant detection between film-screen and digital configurations were substantial. These findings indicate that the evaluated digital devices showed comparable diagnostic accuracy to the reference treatment (film-screen).

Low-cost wearable data acquisition for stroke rehabilitation: a proof-of-concept study on accelerometry for functional task assessment.

BACKGROUND: An increasingly aging society and consequently rising number of patients with poststroke-related neurological dysfunctions are forcing the rehabilitation field to adapt to ever-growing demands. Although clinical reasoning within rehabilitation is dependent on patient movement performance analysis, current strategies for monitoring rehabilitation progress are based on subjective time-consuming assessment scales, not often applied. Therefore, a need exists for efficient nonsubjective monitoring methods. Wearable monitoring devices are rapidly becoming a recognized option in rehabilitation for quantitative measures. Developments in sensors, embedded technology, and smart textile are driving rehabilitation to adopt an objective, seamless, efficient, and cost-effective delivery system. This study aims to assist physiotherapists' clinical reasoning process through the incorporation of accelerometers as part of an electronic data acquisition system. METHODS: A simple, low-cost, wearable device for poststroke rehabilitation progress monitoring was developed based on commercially available inertial sensors. Accelerometry data acquisition was performed for 4 first-time poststroke patients during a reach-press-return task. RESULTS: Preliminary studies revealed acceleration profiles of stroke patients through which it is possible to quantitatively assess the functional movement, identify compensatory strategies, and help define proper movement. CONCLUSION: An inertial data acquisition system was designed and developed as a low-cost option for monitoring rehabilitation. The device seeks to ease the data-gathering process by physiotherapists to complement current practices with accelerometry profiles and aid the development of quantifiable methodologies and protocols.

Diagnostic accuracy of digitized chest X-rays using consumer-grade color displays for low-cost teleradiology services: a multireader-multicase comparison.

INTRODUCTION: In teleradiology services and in hospitals, the extensive use of visualization displays requires affordable devices. The purpose of this study was to compare three differently priced displays (a medical-grade grayscale display and two consumer-grade color displays) for image visualization of digitized chest X-rays. MATERIALS AND METHODS: The evaluated conditions were interstitial opacities, pneumothorax, and nodules using computed tomography as the gold standard. The comparison was accomplished in terms of receiver operating characteristic (ROC) curves, the diagnostic power measured as the area under ROC curves, accuracy in conditions classification, and main factors affecting accuracy, in a factorial study with 76 cases and six radiologists. RESULTS: The ROC curves for all of the displays and pathologies had similar shapes and no differences in diagnostic power. The proportion of cases correctly classified for each display was greater than 71.9%. The correctness proportions of the three displays were different (p<0.05) only for interstitial opacities. The evaluation of the main factors affecting these proportions revealed that the display factor was not significant for either nodule size or pneumothorax size (p>0.05). CONCLUSIONS: Although the image quality variables showed differences in the radiologists' perceptions of the image quality of the three displays, significant differences in the accuracy did not occur. The main effect on the variability of the proportions of correctly classified cases did not come from the display factor. This study confirms previous findings that medical-grade displays could be replaced by consumer-grade color displays with the same image quality.

Evaluation of three pneumothorax size quantification methods on digitized chest X-ray films using medical-grade grayscale and consumer-grade color displays.

This study focused on the effects of pneumothorax size quantification in digital radiology environments when a quantification method is selected according to the radiologist's criteria. The objective of this study was to assess the effects of factors, including the radiologist (with different experience), displays (medical-grade and consumer-grade displays), or display calibration, on the Rhea, Collins, and Light quantification methods. This study used a factorial design with 76 cases, including 16 pneumothorax cases observed by six radiologists on three displays with and without the DICOM standard calibration. The gold standard was established by two radiologists by using computed tomography. Analysis of variance (ANOVA) was performed on the pneumothorax sizes. For the three quantifications methods, none of the evaluated factors were significant. We conclude that radiologists, displays, and calibration do not significantly affect the quantification of pneumothorax size in different digital radiology environments.

Comparison between differently priced devices for digital capture of X-ray films using computed tomography as a gold standard: a multireader-multicase receiver operating characteristic curve study.

OBJECTIVE: Film digitizers are a specialized technology that is available for scanning X-ray radiographs; however, their cost makes them unaffordable for developing countries. Thus, less expensive alternatives are used. The purpose of this study was to compare three devices for digital capture of X-ray films: a film digitizer (US $15,000), a flatbed scanner (US $1800), and a 10-megapixel digital camera (US $450), in terms of diagnostic accuracy, defined as the area under the receiver operating characteristic curves and computed tomography as the gold standard. MATERIALS AND METHODS: The sample included 136 chest X-ray cases with computed tomography confirmation of the presence or absence of pneumothorax, interstitial opacities, or nodules. The readers were six radiologists who made observations of eight variables for each digital capture of the X-ray films: three main variables to determine the accuracy in the detection of the above-mentioned pathologies, four secondary variables to categorize other pathological classifications, and one variable regarding digital image quality. RESULTS: The receiver operating characteristic curves for each device and pathology were very similar. For the main variables, there was no significant statistical difference in diagnostic accuracy between the devices. For the secondary variables, >84% of cases were correctly classified, even those that were classified with the lowest image quality. High accuracy was determined for the three main variables (0.75 to 0.96), indicating good performance for all tested devices, despite their very different prices. CONCLUSIONS: Choosing a device for a teleradiology service should involve additional factors, such as capture time, maintenance concerns, and training requirements.