Your browser doesn't support javascript.
Evaluation of Diagnostic and Triage Accuracy and Usability of a Symptom Checker in an Emergency Department: Observational Study.
Fraser, Hamish S F; Cohan, Gregory; Koehler, Christopher; Anderson, Jared; Lawrence, Alexis; Pateña, John; Bacher, Ian; Ranney, Megan L.
  • Fraser HSF; Brown Center for Biomedical Informatics, Warren Alpert Medical School, Brown University, Providence, RI, United States.
  • Cohan G; School of Public Health, Brown University, Providence, RI, United States.
  • Koehler C; Warren Alpert Medical School, Brown University, Providence, RI, United States.
  • Anderson J; Department of Emergency Medicine, Brown University, Providence, RI, United States.
  • Lawrence A; Department of Emergency Medicine, Brown University, Providence, RI, United States.
  • Pateña J; Harvard Medical Faculty Physicians, Department of Emergency Medicine, St Luke's Hospital, New Bedford, MA, United States.
  • Bacher I; Brown-Lifespan Center for Digital Health, Providence, RI, United States.
  • Ranney ML; Brown Center for Biomedical Informatics, Warren Alpert Medical School, Brown University, Providence, RI, United States.
JMIR Mhealth Uhealth ; 10(9): e38364, 2022 09 19.
Article in English | MEDLINE | ID: covidwho-2054780
ABSTRACT

BACKGROUND:

Symptom checkers are clinical decision support apps for patients, used by tens of millions of people annually. They are designed to provide diagnostic and triage advice and assist users in seeking the appropriate level of care. Little evidence is available regarding their diagnostic and triage accuracy with direct use by patients for urgent conditions.

OBJECTIVE:

The aim of this study is to determine the diagnostic and triage accuracy and usability of a symptom checker in use by patients presenting to an emergency department (ED).

METHODS:

We recruited a convenience sample of English-speaking patients presenting for care in an urban ED. Each consenting patient used a leading symptom checker from Ada Health before the ED evaluation. Diagnostic accuracy was evaluated by comparing the symptom checker's diagnoses and those of 3 independent emergency physicians viewing the patient-entered symptom data, with the final diagnoses from the ED evaluation. The Ada diagnoses and triage were also critiqued by the independent physicians. The patients completed a usability survey based on the Technology Acceptance Model.

RESULTS:

A total of 40 (80%) of the 50 participants approached completed the symptom checker assessment and usability survey. Their mean age was 39.3 (SD 15.9; range 18-76) years, and they were 65% (26/40) female, 68% (27/40) White, 48% (19/40) Hispanic or Latino, and 13% (5/40) Black or African American. Some cases had missing data or a lack of a clear ED diagnosis; 75% (30/40) were included in the analysis of diagnosis, and 93% (37/40) for triage. The sensitivity for at least one of the final ED diagnoses by Ada (based on its top 5 diagnoses) was 70% (95% CI 54%-86%), close to the mean sensitivity for the 3 physicians (on their top 3 diagnoses) of 68.9%. The physicians rated the Ada triage decisions as 62% (23/37) fully agree and 24% (9/37) safe but too cautious. It was rated as unsafe and too risky in 22% (8/37) of cases by at least one physician, in 14% (5/37) of cases by at least two physicians, and in 5% (2/37) of cases by all 3 physicians. Usability was rated highly; participants agreed or strongly agreed with the 7 Technology Acceptance Model usability questions with a mean score of 84.6%, although "satisfaction" and "enjoyment" were rated low.

CONCLUSIONS:

This study provides preliminary evidence that a symptom checker can provide acceptable usability and diagnostic accuracy for patients with various urgent conditions. A total of 14% (5/37) of symptom checker triage recommendations were deemed unsafe and too risky by at least two physicians based on the symptoms recorded, similar to the results of studies on telephone and nurse triage. Larger studies are needed of diagnosis and triage performance with direct patient use in different clinical environments.
Subject(s)
Keywords

Full text: Available Collection: International databases Database: MEDLINE Main subject: Physicians / Decision Support Systems, Clinical / Emergency Service, Hospital Type of study: Experimental Studies / Observational study / Prognostic study / Randomized controlled trials Limits: Adolescent / Adult / Aged / Female / Humans / Middle aged / Young adult Language: English Journal: JMIR Mhealth Uhealth Year: 2022 Document Type: Article Affiliation country: 38364

Similar

MEDLINE

...
LILACS

LIS


Full text: Available Collection: International databases Database: MEDLINE Main subject: Physicians / Decision Support Systems, Clinical / Emergency Service, Hospital Type of study: Experimental Studies / Observational study / Prognostic study / Randomized controlled trials Limits: Adolescent / Adult / Aged / Female / Humans / Middle aged / Young adult Language: English Journal: JMIR Mhealth Uhealth Year: 2022 Document Type: Article Affiliation country: 38364