RESUMEN
BACKGROUND: Diagnostic work-up of leg ulcers is time- and cost-intensive. This study aimed at evaluating ulcer location as a diagnostic criterium and providing a diagnostic algorithm to facilitate differential diagnosis. PATIENTS AND METHODS: The study consisted of 277 patients with lower leg ulcers. The following five groups were defined: Venous leg ulcer, arterial ulcers, mixed ulcer, arteriolosclerosis, and vasculitis. Using computational surface rendering, predilection sites of different ulcer types were evaluated. The results were integrated in a multinomial logistic regression model to calculate the likelihood of a specific diagnosis depending on location, age, bilateral involvement, and ulcer count. Additionally, neural network image analysis was performed. RESULTS: The majority of venous ulcers extended to the medial malleolar region. Arterial ulcers were most frequently located on the dorsal aspect of the forefoot. Arteriolosclerotic ulcers were distinctly localized at the middle third of the lower leg. Vasculitic ulcers appeared to be randomly distributed and were markedly smaller, multilocular and bilateral. The multinomial logistic regression model showed an overall satisfactory performance with an estimated accuracy of 0.68 on unseen data. CONCLUSIONS: The presented algorithm based on ulcer location may serve as a basic tool to narrow down potential diagnoses and guide further diagnostic work-up.
Asunto(s)
Úlcera de la Pierna , Úlcera Varicosa , Humanos , Úlcera , Úlcera de la Pierna/diagnóstico , Úlcera de la Pierna/etiología , Úlcera Varicosa/diagnóstico , Pierna , AlgoritmosRESUMEN
We investigated whether human preferences hold the potential to improve diagnostic artificial intelligence (AI)-based decision support using skin cancer diagnosis as a use case. We utilized nonuniform rewards and penalties based on expert-generated tables, balancing the benefits and harms of various diagnostic errors, which were applied using reinforcement learning. Compared with supervised learning, the reinforcement learning model improved the sensitivity for melanoma from 61.4% to 79.5% (95% confidence interval (CI): 73.5-85.6%) and for basal cell carcinoma from 79.4% to 87.1% (95% CI: 80.3-93.9%). AI overconfidence was also reduced while simultaneously maintaining accuracy. Reinforcement learning increased the rate of correct diagnoses made by dermatologists by 12.0% (95% CI: 8.8-15.1%) and improved the rate of optimal management decisions from 57.4% to 65.3% (95% CI: 61.7-68.9%). We further demonstrated that the reward-adjusted reinforcement learning model and a threshold-based model outperformed naïve supervised learning in various clinical scenarios. Our findings suggest the potential for incorporating human preferences into image-based diagnostic algorithms.
Asunto(s)
Carcinoma Basocelular , Melanoma , Neoplasias Cutáneas , Humanos , Inteligencia Artificial , Algoritmos , Neoplasias Cutáneas/diagnóstico , Neoplasias Cutáneas/patología , Melanoma/diagnóstico , Melanoma/patología , Carcinoma Basocelular/diagnósticoRESUMEN
BACKGROUND: To deploy clinical decision support (CDS) systems in routine patient care they have to be certified as a medical device. The European Medical Device Regulation explicitly asks for the use of standards and interoperability in the approval process. OBJECTIVES: We extended an existing dermatological CDS system with emerging standards for CDS interoperability, to facilitate a future integration into existing healthcare infrastructure, and approval as a medical device. METHODS: The data collection part of a CDS system was extended with the endpoints required by the CDS Hooks specification. FHIR QuestionnaireResponse resources trigger a newly defined hook. RESULTS: One hundred and seventeen clinical observations and patient variables needed for the ranking of a disease were mapped to SNOMED CT or LOINC and modeled as FHIR Questionnaire which is rendered using LHC LForms in a SMART on FHIR app with the SMART Dev Sandbox. CONCLUSION: SMART on FHIR in combination with CDS Hooks facilitates the integration of existing CDS systems into EHR systems, potentially improving education and patient care.
Asunto(s)
Sistemas de Apoyo a Decisiones Clínicas , Aplicaciones Móviles , Humanos , Registros Electrónicos de Salud , Estándar HL7 , Encuestas y CuestionariosRESUMEN
BACKGROUND: There are many medical knowledge bases with potential for supporting medical professionals in their decision-making during routine care, yet usage of these sources remains low. Standardized linking of Clinical Decision Support (CDS) applications and existing medical knowledge bases is not a common practice. OBJECTIVES: Using existing eHealth standards to increase the utilization of knowledge bases and implement a prototype. METHODS: Linking an existing online knowledge base via a FHIR CodeSystem supplement to the Austrian national EHR (ELGA) terminology server and accessing these data using CDS Hooks and FHIR. RESULTS: We tested the approach by incorporating photosensitivity data of medications into a local copy of the Austrian terminology server. These data are directly used by a CDS Hooks compliant CDS service. CONCLUSION: The Austrian Terminology Server could be an important interface to access existing knowledge bases from within EHR systems. FHIR and CDS Hooks could lead the way for a simple and open integration of CDS services into EHR systems.
Asunto(s)
Sistemas de Apoyo a Decisiones Clínicas , Registros Electrónicos de Salud , Computadores , AustriaRESUMEN
Introduction: Classification of dermatoscopic images via neural networks shows comparable performance to clinicians in experimental conditions but can be affected by artefacts like skin markings or rulers. It is unknown whether specialized neural networks are more robust to artefacts. Objectives: Analyze robustness of 3 neural network architectures, namely ResNet-34, Faster R-CNN and Mask R-CNN. Methods: We identified common artefacts in the HAM10000, PH2 and the 7-point criteria evaluation datasets, and established a template-based method to superimpose artefacts on dermatoscopic images. The HAM10000-dataset with and without superimposed artefacts was used to train the networks, followed by analyzing their robustness against artefacts in test images. Performance was assessed via area under the precision recall curve and classification results. Results: ResNet-34 and Faster R-CNN models trained on regular images perform worse than Mask R-CNN on images with superimposed artefacts. Artefacts added to all tested images led to a decrease in area under the precision-recall curve values of 0.030 for ResNet-34 and 0.045 for Faster R-CNN in comparison to only 0.011 for Mask R-CNN. However, changes in model performance only became significant with 40% or more of the images having superimposed artefacts. A loss in performance occurred when the training was biased by selectively superimposing artefacts on images belonging to a certain class. Conclusions: As Mask R-CNN showed the least decrease in performance when confronted with artefacts, instance segmentation architectures may be helpful to counter the effects of artefacts, warranting further research on related architectures. Our artefact insertion mechanism could be useful for future research.
RESUMEN
BACKGROUND: Previous studies of artificial intelligence (AI) applied to dermatology have shown AI to have higher diagnostic classification accuracy than expert dermatologists; however, these studies did not adequately assess clinically realistic scenarios, such as how AI systems behave when presented with images of disease categories that are not included in the training dataset or images drawn from statistical distributions with significant shifts from training distributions. We aimed to simulate these real-world scenarios and evaluate the effects of image source institution, diagnoses outside of the training set, and other image artifacts on classification accuracy, with the goal of informing clinicians and regulatory agencies about safety and real-world accuracy. METHODS: We designed a large dermoscopic image classification challenge to quantify the performance of machine learning algorithms for the task of skin cancer classification from dermoscopic images, and how this performance is affected by shifts in statistical distributions of data, disease categories not represented in training datasets, and imaging or lesion artifacts. Factors that might be beneficial to performance, such as clinical metadata and external training data collected by challenge participants, were also evaluated. 25 331 training images collected from two datasets (in Vienna [HAM10000] and Barcelona [BCN20000]) between Jan 1, 2000, and Dec 31, 2018, across eight skin diseases, were provided to challenge participants to design appropriate algorithms. The trained algorithms were then tested for balanced accuracy against the HAM10000 and BCN20000 test datasets and data from countries not included in the training dataset (Turkey, New Zealand, Sweden, and Argentina). Test datasets contained images of all diagnostic categories available in training plus other diagnoses not included in training data (not trained category). We compared the performance of the algorithms against that of 18 dermatologists in a simulated setting that reflected intended clinical use. FINDINGS: 64 teams submitted 129 state-of-the-art algorithm predictions on a test set of 8238 images. The best performing algorithm achieved 58·8% balanced accuracy on the BCN20000 data, which was designed to better reflect realistic clinical scenarios, compared with 82·0% balanced accuracy on HAM10000, which was used in a previously published benchmark. Shifted statistical distributions and disease categories not included in training data contributed to decreases in accuracy. Image artifacts, including hair, pen markings, ulceration, and imaging source institution, decreased accuracy in a complex manner that varied based on the underlying diagnosis. When comparing algorithms to expert dermatologists (2460 ratings on 1269 images), algorithms performed better than experts in most categories, except for actinic keratoses (similar accuracy on average) and images from categories not included in training data (26% correct for experts vs 6% correct for algorithms, p<0·0001). For the top 25 submitted algorithms, 47·1% of the images from categories not included in training data were misclassified as malignant diagnoses, which would lead to a substantial number of unnecessary biopsies if current state-of-the-art AI technologies were clinically deployed. INTERPRETATION: We have identified specific deficiencies and safety issues in AI diagnostic systems for skin cancer that should be addressed in future diagnostic evaluation protocols to improve safety and reliability in clinical practice. FUNDING: Melanoma Research Alliance and La Marató de TV3.
Asunto(s)
Melanoma , Neoplasias Cutáneas , Inteligencia Artificial , Dermoscopía/métodos , Humanos , Melanoma/diagnóstico por imagen , Melanoma/patología , Reproducibilidad de los Resultados , Neoplasias Cutáneas/diagnóstico por imagen , Neoplasias Cutáneas/patologíaRESUMEN
BACKGROUND: The use of potentially inappropriate medication (PIM) in population of older adults may result in adverse drug events (ADE) already after short term exposure, especially when it is prescribed to patients with chronic kidney disease (CKD). In order to limit ADE in the treatment of older adults PIM lists have been constructed as a source of information for healthcare professionals. The aim of this study was to estimate the utilization of PIM and incidence of ADE in older adults (≥70 years) with CKD. METHODS: We conducted a retrospective population-wide cohort study including patients from Lower Austria who were 70 years or older and diagnosed with CKD in the period from 2008 to 2011. Utilization of PIM was estimated from prescriptions filled by target population. We estimated risks of hospitalization due to ADE within 30 days after incident PIM prescription and compared them to a PIM-free control group by using marginal structural models (MSM). RESULTS: We identified 11,547 patients (women: 50.6%, median age in 2008: 78 years) who fulfilled the inclusion criteria. In total 24.7 and 8.1% of all prescriptions from that period contained a medication with a substance listed in the EU (7)-PIM and AT-PIM list, respectively. Proton pump inhibitors and Ginkgo biloba were the most often prescribed PIMs in this population. 94.6 and 79.3% patients filled at least one EU(7)-PIM and AT-PIM prescription, respectively. Despite the relatively high utilization of PIM there was only a low incidence of clinically relevant ADE. No event type exceeded the threshold level of 1% in the analysis of risks of ADE after filling a prescription for PIM. Nevertheless, MSM analysis showed an increased risk for 11 drugs and reduced risk for 4 drugs. CONCLUSIONS: PIM prescription was common among older adults with CKD, however, only a small number of these drugs eventually led to hospitalization due to ADE within 30 days after incident PIM was filled. In the absence of a clinically important PIM-related increase in risk, an assessment of potential ADE severity to a PIM list by using a warning score system seems prudent.
Asunto(s)
Efectos Colaterales y Reacciones Adversas Relacionados con Medicamentos , Insuficiencia Renal Crónica , Minorías Sexuales y de Género , Anciano , Austria , Estudios de Cohortes , Efectos Colaterales y Reacciones Adversas Relacionados con Medicamentos/diagnóstico , Efectos Colaterales y Reacciones Adversas Relacionados con Medicamentos/epidemiología , Femenino , Homosexualidad Masculina , Humanos , Prescripción Inadecuada , Masculino , Lista de Medicamentos Potencialmente Inapropiados , Insuficiencia Renal Crónica/inducido químicamente , Insuficiencia Renal Crónica/diagnóstico , Insuficiencia Renal Crónica/tratamiento farmacológico , Estudios RetrospectivosRESUMEN
OBJECTIVE: Postscreening colorectal cancer (PSCRC) after screening colonoscopy is associated with endoscopists' performance and characteristics of resected lesions. Prior studies have shown that adenoma detection rate (ADR) is a decisive factor for PSCRC, but correlations with other parameters need further analysis and ADR may change over time. DESIGN: Cohort study including individuals undergoing screening colonoscopy between 1/2008 and 12/2019 performed by physicians participating in a quality assurance programme in Austria. Data were linked with hospitalisation data for the diagnosis of PSCRC (defined as CRC diagnosis >6 months after colonoscopy). ADR was defined dynamically in relation to the time point of subsequent colonoscopies; high-risk groups of patients were those with an adenoma ≥10 mm, or with high-grade dysplasia, or villous or tubulovillous histology, or a serrated lesion ≥10 mm or with dysplasia, or colonoscopies with ≥3 lesions. Main outcome was PSCRC for each risk group (negative colonoscopy, hyperplastic polyps, low-risk and high-risk group of patients) after colonoscopy by endoscopists with an ADR <20% compared with endoscopists with an ADR ≥20%. RESULTS: 352 685 individuals were included in the study (51.0% women, median age 60 years) of which 10.5% were classified as high-risk group. During a median follow-up of 55.4 months, 241 (0.06%) PSCRC were identified; of 387 participating physicians, 19.6% had at least one PSCRC (8.4% two or more). While higher endoscopist ADR decreased PSCRC incidence (HR per 1% increase 0.97, 95% CI 0.95 to 0.98), affiliation to the high-risk group of patients was also associated with higher PSCRC incidence (HR 3.27, 95% CI 2.36 to 4.00). Similar correlations were seen with regards to high-risk, and advanced adenomas. The risk for PSCRC was significantly higher after colonoscopy by an endoscopist with an ADR <20% as compared with an endoscopist with an ADR ≥20% in patients after negative colonoscopy (HR 2.01, 95% CI 1.35 to 3.0, p<0.001) and for the high-risk group of patients (HR 2.51, 95% CI 1.49 to 4.22, p<0.001). CONCLUSION: A dynamic calculation of the ADR takes into account changes over time but confirms the correlation of ADR and interval cancer. Both lesion characteristics and endoscopists ADR may play a similar role for the risk of PSCRC. This should be considered in deciding about appropriate surveillance intervals in the future.
Asunto(s)
Adenoma/diagnóstico por imagen , Adenoma/patología , Pólipos del Colon/diagnóstico por imagen , Colonoscopía/estadística & datos numéricos , Neoplasias Colorrectales/diagnóstico por imagen , Neoplasias Colorrectales/epidemiología , Anciano , Austria/epidemiología , Competencia Clínica , Pólipos del Colon/patología , Colonoscopía/normas , Neoplasias Colorrectales/patología , Bases de Datos Factuales , Detección Precoz del Cáncer/normas , Detección Precoz del Cáncer/estadística & datos numéricos , Femenino , Humanos , Incidencia , Masculino , Registro Médico Coordinado , Persona de Mediana Edad , Factores de Riesgo , Factores de Tiempo , Carga TumoralRESUMEN
The rapid increase in telemedicine coupled with recent advances in diagnostic artificial intelligence (AI) create the imperative to consider the opportunities and risks of inserting AI-based support into new paradigms of care. Here we build on recent achievements in the accuracy of image-based AI for skin cancer diagnosis to address the effects of varied representations of AI-based support across different levels of clinical expertise and multiple clinical workflows. We find that good quality AI-based support of clinical decision-making improves diagnostic accuracy over that of either AI or physicians alone, and that the least experienced clinicians gain the most from AI-based support. We further find that AI-based multiclass probabilities outperformed content-based image retrieval (CBIR) representations of AI in the mobile technology environment, and AI-based support had utility in simulations of second opinions and of telemedicine triage. In addition to demonstrating the potential benefits associated with good quality AI in the hands of non-expert clinicians, we find that faulty AI can mislead the entire spectrum of clinicians, including experts. Lastly, we show that insights derived from AI class-activation maps can inform improvements in human diagnosis. Together, our approach and findings offer a framework for future studies across the spectrum of image-based diagnostics to improve human-computer collaboration in clinical practice.
Asunto(s)
Inteligencia Artificial , Neoplasias Cutáneas/diagnóstico por imagen , Telemedicina , Interfaz Usuario-Computador , Toma de Decisiones Clínicas , Humanos , Redes Neurales de la Computación , Médicos , Neoplasias Cutáneas/patologíaRESUMEN
An amendment to this paper has been published and can be accessed via a link at the top of the paper.
RESUMEN
Equations predicting the risk of occurrence of cardiovascular disease (CVD) are used in primary care to identify high-risk individuals among the general population. To improve the predictive performance of such equations, we updated the Framingham general CVD 1991 and 2008 equations and the Pooled Cohort equations for atherosclerotic CVD within five years in a contemporary cohort of individuals who participated in the Austrian health-screening program from 2009-2014. The cohort comprised 1.7 M individuals aged 30-79 without documented CVD history. CVD was defined by hospitalization or death from cardiovascular cause. Using baseline and follow-up data, we recalibrated and re-estimated the equations. We evaluated the gain in discrimination and calibration and assessed explained variation. A five-year general CVD risk of 4.61% was observed. As expected, discrimination c-statistics increased only slightly and ranged from 0.73-0.79. The two original Framingham equations overestimated the CVD risk, whereas the original Pooled Cohort equations underestimated it. Re-estimation improved calibration of all equations adequately, especially for high-risk individuals. Half of the individuals were reclassified into another risk category using the re-estimated equations. Predictors in the re-estimated Framingham equations explained 7.37% of the variation, whereas the Pooled Cohort equations explained 5.81%. Age was the most important predictor.
Asunto(s)
Enfermedades Cardiovasculares/epidemiología , Adulto , Anciano , Austria/epidemiología , Enfermedades Cardiovasculares/mortalidad , Estudios de Cohortes , Femenino , Humanos , Estudios Longitudinales , Masculino , Persona de Mediana Edad , Sistema de Registros , Factores de RiesgoRESUMEN
BACKGROUND: The diagnosis of pigmented skin lesion is error prone and requires domain-specific expertise, which is not readily available in many parts of the world. Collective intelligence could potentially decrease the error rates of nonexperts. OBJECTIVE: The aim of this study was to evaluate the feasibility and impact of collective intelligence for the detection of skin cancer. METHODS: We created a gamified study platform on a stack of established Web technologies and presented 4216 dermatoscopic images of the most common benign and malignant pigmented skin lesions to 1245 human raters with different levels of experience. Raters were recruited via scientific meetings, mailing lists, and social media posts. Education was self-declared, and domain-specific experience was tested by screening tests. In the target test, the readers had to assign 30 dermatoscopic images to 1 of the 7 disease categories. The readers could repeat the test with different lesions at their own discretion. Collective human intelligence was achieved by sampling answers from multiple readers. The disease category with most votes was regarded as the collective vote per image. RESULTS: We collected 111,019 single ratings, with a mean of 25.2 (SD 18.5) ratings per image. As single raters, nonexperts achieved a lower mean accuracy (58.6%) than experts (68.4%; mean difference=-9.4%; 95% CI -10.74% to -8.1%; P<.001). Collectives of nonexperts achieved higher accuracies than single raters, and the improvement increased with the size of the collective. A collective of 4 nonexperts surpassed single nonexperts in accuracy by 6.3% (95% CI 6.1% to 6.6%; P<.001). The accuracy of a collective of 8 nonexperts was 9.7% higher (95% CI 9.5% to 10.29%; P<.001) than that of single nonexperts, an improvement similar to single experts (P=.73). The sensitivity for malignant images increased for nonexperts (66.3% to 77.6%) and experts (64.6% to 79.4%) for answers given faster than the intrarater mean. CONCLUSIONS: A high number of raters can be attracted by elements of gamification and Web-based marketing via mailing lists and social media. Nonexperts increase their accuracy to expert level when acting as a collective, and faster answers correspond to higher accuracy. This information could be useful in a teledermatology setting.
Asunto(s)
Inteligencia/genética , Neoplasias Cutáneas/diagnóstico , Telemedicina/métodos , Femenino , Humanos , Internet , Masculino , Neoplasias Cutáneas/patologíaRESUMEN
PURPOSE: Overactive bladder (OAB) syndrome has severe effects on quality of life. Certain drugs are known risk factors for OAB but have not been investigated in a population-wide cohort. The objective of this study was to investigate the role of prescription drugs in the etiology of the OAB. METHODS: Retrospective cohort study using a population-wide database of 4 185 098 OAB-naïve women followed Strengthening the Reporting of Observational Studies in Epidemiology guidelines. We investigated the subscription use of anticholinergic medication and 188 chemical substances, which are suspected triggers for OAB (trigger medications [TMs]). We hypothesized a relationship between the prescription for one or more TM and the prescription for anticholinergic medication against OAB (marker medication [MM]). RESULTS: The use of MM in Austria increased from 2009 to 2012 on average by 0.025 percentage points per year (95% confidence interval [CI]: 0.015-0.036). In December 2012, 1 in 123 women filled a prescription for any MM, equaling an average utilization of 0.84%. The relative risk of filling a prescription for a MM 6 months after filling a prescription for a TM was 2.70 (95% CI: 2.64-2.77). All investigated medication classes showed a higher risk for the prescription for MM. Medication from classes "genitourinary system and sex hormones" and "systemic anti-infectives" caused the highest increase in risk (109% and 89%, respectively). Prescriptions for class "cardiovascular system" caused the lowest increase in the risk (15%). CONCLUSION: Certain prescription medications are a significant risk factor for the need to take anticholinergic medication as a consequence.
Asunto(s)
Vigilancia de la Población , Medicamentos bajo Prescripción/efectos adversos , Vejiga Urinaria Hiperactiva/inducido químicamente , Vejiga Urinaria Hiperactiva/epidemiología , Adulto , Anciano , Austria/epidemiología , Estudios de Cohortes , Femenino , Humanos , Persona de Mediana Edad , Medicamentos bajo Prescripción/uso terapéutico , Estudios Retrospectivos , Vejiga Urinaria Hiperactiva/diagnósticoRESUMEN
The Main Association of Austrian Social Security Institutions collects pseudonymized claims data from Austrian social security institutions and information about hospital stays in a database for research purposes. For new studies the same data are repeatedly reprocessed and it is difficult to compare different study results even though the data is already preprocessed and prepared in a proprietary data model. Based on a study on adverse drug events in relation to inappropriate medication in geriatric patients the suitability of the Observational Medical Outcomes Partnership (OMOP) common data model (CDM) is analyzed and data is transformed into the OMOP CDM. 1,023 (99.7%) of drug codes and 3,812 (99.2%) of diagnoses codes coincide with the OMOP vocabularies. The biggest obstacles are missing mappings for the Local Vocabularies like the Austrian pharmaceutical registration numbers and the Socio-Economic Index to the OMOP vocabularies. OMOP CDM is a promising approach for the standardization of Austrian claims data. In the long run, the benefits of standardization and reproducibility of research should outweigh this initial drawback.
Asunto(s)
Bases de Datos Factuales/normas , Revisión de Utilización de Seguros/organización & administración , Anciano , Anciano de 80 o más Años , Austria/epidemiología , Efectos Colaterales y Reacciones Adversas Relacionados con Medicamentos/epidemiología , Estudios de Factibilidad , Geriatría , Humanos , Revisión de Utilización de Seguros/normas , Mal Uso de Medicamentos de Venta con Receta/estadística & datos numéricos , Reproducibilidad de los Resultados , Factores SocioeconómicosRESUMEN
BACKGROUND: Whether machine-learning algorithms can diagnose all pigmented skin lesions as accurately as human experts is unclear. The aim of this study was to compare the diagnostic accuracy of state-of-the-art machine-learning algorithms with human readers for all clinically relevant types of benign and malignant pigmented skin lesions. METHODS: For this open, web-based, international, diagnostic study, human readers were asked to diagnose dermatoscopic images selected randomly in 30-image batches from a test set of 1511 images. The diagnoses from human readers were compared with those of 139 algorithms created by 77 machine-learning labs, who participated in the International Skin Imaging Collaboration 2018 challenge and received a training set of 10â015 images in advance. The ground truth of each lesion fell into one of seven predefined disease categories: intraepithelial carcinoma including actinic keratoses and Bowen's disease; basal cell carcinoma; benign keratinocytic lesions including solar lentigo, seborrheic keratosis and lichen planus-like keratosis; dermatofibroma; melanoma; melanocytic nevus; and vascular lesions. The two main outcomes were the differences in the number of correct specific diagnoses per batch between all human readers and the top three algorithms, and between human experts and the top three algorithms. FINDINGS: Between Aug 4, 2018, and Sept 30, 2018, 511 human readers from 63 countries had at least one attempt in the reader study. 283 (55·4%) of 511 human readers were board-certified dermatologists, 118 (23·1%) were dermatology residents, and 83 (16·2%) were general practitioners. When comparing all human readers with all machine-learning algorithms, the algorithms achieved a mean of 2·01 (95% CI 1·97 to 2·04; p<0·0001) more correct diagnoses (17·91 [SD 3·42] vs 19·92 [4·27]). 27 human experts with more than 10 years of experience achieved a mean of 18·78 (SD 3·15) correct answers, compared with 25·43 (1·95) correct answers for the top three machine algorithms (mean difference 6·65, 95% CI 6·06-7·25; p<0·0001). The difference between human experts and the top three algorithms was significantly lower for images in the test set that were collected from sources not included in the training set (human underperformance of 11·4%, 95% CI 9·9-12·9 vs 3·6%, 0·8-6·3; p<0·0001). INTERPRETATION: State-of-the-art machine-learning classifiers outperformed human experts in the diagnosis of pigmented skin lesions and should have a more important role in clinical practice. However, a possible limitation of these algorithms is their decreased performance for out-of-distribution images, which should be addressed in future research. FUNDING: None.
Asunto(s)
Algoritmos , Dermoscopía , Internet , Aprendizaje Automático , Trastornos de la Pigmentación/patología , Neoplasias Cutáneas/patología , Adulto , Femenino , Humanos , Masculino , Reproducibilidad de los Resultados , Estudios RetrospectivosRESUMEN
BACKGROUND: Reuse of EHR data for selecting patients who are eligible for clinical research can substantially improve the recruitment process. ART-DECOR is an open-source tool that is commonly used to design and publish HL7 V3 templates of national (e.g. ELGA) and international EHR initiatives. OBJECTIVES: Extend ART-DECOR to allow the definition of criteria that may be used for patient selection. METHODS: Using the native ART-DECOR development framework we extended existing ART-DECOR template associations by allowing conditions to be formulated. RESULTS: An editor for the specification of conditions was implemented. The resulting criteria are internally translated to XPath expressions and can be immediately applied to CDA documents. As a prototypical application of our approach we implemented a "Trial Criteria Evaluator" tool that allows trial eligibility criteria to be composed of our ART-DECOR criteria and have them checked against a patient's CDA documents. CONCLUSION: Referring to HL7 templates, our criteria can be applied to documents of national EHR systems such as ELGA and hereby reach a broad patient cohort. Implementing our approach within ART-DECOR alleviates its reuse and enhancement by other researchers.
Asunto(s)
Registros Electrónicos de Salud , Selección de Paciente , Vocabulario Controlado , Atención a la Salud , HumanosRESUMEN
BACKGROUND: Health information exchange (HIE) among care providers who cooperate in the treatment of patients with diabetes mellitus (DM) has been rated as an important aspect of successful care. Patient-sharing relations among care providers permit inferences about corresponding information-sharing relations. OBJECTIVES: This study aimed to obtain information for an effective HIE platform design to be used in DM care by analyzing patient-sharing relations among various types of care providers (ToCPs), such as hospitals, pharmacies, and different outpatient specialists, within a nationwide claims dataset of Austrian DM patients. We focus on 2 parameters derived from patient-sharing networks: (1) the principal HIE partners of the different ToCPs involved in the treatment of DM and (2) the required participation rate of ToCPs in HIE platforms for the purpose of effective communication. METHODS: The claims data of 7.9 million Austrian patients from 2006 to 2007 served as our data source. DM patients were identified by their medication. We established metrics for the quantification of our 2 parameters of interest. The principal HIE partners were derived from the portions of a care provider's patient-sharing relations with different ToCPs. For the required participation rate of ToCPs in an HIE platform, we determine the concentration of patient-sharing relations among ToCPs. Our corresponding metrics are derived in analogy from existing work for the quantification of the continuity of care. RESULTS: We identified 324,703 DM patients treated by 12,226 care providers; the latter were members of 16 ToCPs. On the basis of their score for 2 of our parameters, we categorized the ToCPs into low, medium, and high. For the most important HIE partner parameter, pharmacies, general practitioners (GPs), and laboratories were the representatives of the top group, that is, our care providers shared the highest numbers of DM patients with these ToCPs. For the required participation rate of type of care provide (ToCP) in HIE platform parameter, the concentration of DM patient-sharing relations with a ToCP tended to be inversely related to the ToCPs member count. CONCLUSIONS: We conclude that GPs, pharmacies, and laboratories should be core members of any HIE platform that supports DM care, as they are the most important DM patient-sharing partners. We further conclude that, for implementing HIE with ToCPs who have many members (in Austria, particularly GPs and pharmacies), an HIE solution with high participation rates from these ToCPs (ideally a nationwide HIE platform with obligatory participation of the concerned ToCPs) seems essential. This will raise the probability of HIE being achieved with any care provider of these ToCPs. As chronic diseases are rising because of aging societies, we believe that our quantification of HIE requirements in the treatment of DM can provide valuable insights for many industrial countries.
RESUMEN
The suitability of the Observational Medical Outcomes Partnership (OMOP) common data model (CDM) for Austrian pseudonymized claims data from social security institutions and information about hospital stays is evaluated. 1,023 (99.7%) of ATC codes and 3,695 (98.6%) of ICD10 codes coincide with the OMOP vocabulary. Mappings for the local vocabularies like the Austrian pharmaceutical registration numbers, the Socio-Economic Index and professional groups, to the OMOP vocabulary do not exist. A standardization with the OMOP CDM is possible, however an initial, not negligible effort is required to adapt and incorporate the vocabulary.
Asunto(s)
Análisis de Datos , Revisión de Utilización de Seguros , Modelos Estadísticos , Austria , Bases de Datos Factuales , Estudios de FactibilidadRESUMEN
Importance: Convolutional neural networks (CNNs) achieve expert-level accuracy in the diagnosis of pigmented melanocytic lesions. However, the most common types of skin cancer are nonpigmented and nonmelanocytic, and are more difficult to diagnose. Objective: To compare the accuracy of a CNN-based classifier with that of physicians with different levels of experience. Design, Setting, and Participants: A CNN-based classification model was trained on 7895 dermoscopic and 5829 close-up images of lesions excised at a primary skin cancer clinic between January 1, 2008, and July 13, 2017, for a combined evaluation of both imaging methods. The combined CNN (cCNN) was tested on a set of 2072 unknown cases and compared with results from 95 human raters who were medical personnel, including 62 board-certified dermatologists, with different experience in dermoscopy. Main Outcomes and Measures: The proportions of correct specific diagnoses and the accuracy to differentiate between benign and malignant lesions measured as an area under the receiver operating characteristic curve served as main outcome measures. Results: Among 95 human raters (51.6% female; mean age, 43.4 years; 95% CI, 41.0-45.7 years), the participants were divided into 3 groups (according to years of experience with dermoscopy): beginner raters (<3 years), intermediate raters (3-10 years), or expert raters (>10 years). The area under the receiver operating characteristic curve of the trained cCNN was higher than human ratings (0.742; 95% CI, 0.729-0.755 vs 0.695; 95% CI, 0.676-0.713; P < .001). The specificity was fixed at the mean level of human raters (51.3%), and therefore the sensitivity of the cCNN (80.5%; 95% CI, 79.0%-82.1%) was higher than that of human raters (77.6%; 95% CI, 74.7%-80.5%). The cCNN achieved a higher percentage of correct specific diagnoses compared with human raters (37.6%; 95% CI, 36.6%-38.4% vs 33.5%; 95% CI, 31.5%-35.6%; P = .001) but not compared with experts (37.3%; 95% CI, 35.7%-38.8% vs 40.0%; 95% CI, 37.0%-43.0%; P = .18). Conclusions and Relevance: Neural networks are able to classify dermoscopic and close-up images of nonpigmented lesions as accurately as human experts in an experimental setting.