Should artificial intelligence have lower acceptable error rates than humans?

The first patient was misclassified in the diagnostic conclusion according to a local clinical expert opinion in a new clinical implementation of a knee osteoarthritis artificial intelligence (AI) algorithm at Bispebjerg-Frederiksberg University Hospital, Copenhagen, Denmark. In preparation for the evaluation of the AI algorithm, the implementation team collaborated with internal and external partners to plan workflows, and the algorithm was externally validated. After the misclassification, the team was left wondering: what is an acceptable error rate for a low-risk AI diagnostic algorithm? A survey among employees at the Department of Radiology showed significantly lower acceptable error rates for AI (6.8 %) than humans (11.3 %). A general mistrust of AI could cause the discrepancy in acceptable errors. AI may have the disadvantage of limited social capital and likeability compared to human co-workers, and therefore, less potential for forgiveness. Future AI development and implementation require further investigation of the fear of AI's unknown errors to enhance the trustworthiness of perceiving AI as a co-worker. Benchmark tools, transparency, and explainability are also needed to evaluate AI algorithms in clinical implementations to ensure acceptable performance.

Reliability and correlation of weight-bearing cone beam CT and Foot Posture Index (FPI) for measurements of foot posture: a test-retest study.

OBJECTIVE: To assess test-retest reliability and correlation of weight-bearing (WB) and non-weight-bearing (NWB) cone beam CT (CBCT) foot measurements and Foot Posture Index (FPI) MATERIALS AND METHODS: Twenty healthy participants (age 43.11±11.36, 15 males, 5 females) were CBCT-scanned in February 2019 on two separate days on one foot in both WB and NWB positions. Three radiology observers measured the navicular bone position. Plantar (ΔNAVplantar) and medial navicular displacements (ΔNAVmedial) were calculated as a measure of foot posture changes under loading. FPI was assessed by two rheumatologists on the same two days. FPI is a clinical measurement of foot posture with 3 rearfoot and 3 midfoot/forefoot scores. Test-retest reproducibility was determined for all measurements. CBCT was correlated to FPI total and subscores. RESULTS: Intra- and interobserver reliabilities for navicular position and FPI were excellent (intraclass correlation coefficient (ICC) .875-.997). In particular, intraobserver (ICC .0.967-1.000) and interobserver reliabilities (ICC .946-.997) were found for CBCT navicular height and medial position. Interobserver reliability of ΔNAVplantar was excellent (ICC .926 (.812; .971); MDC 2.22), whereas the ΔNAVmedial was fair-good (ICC .452 (.385; .783); MDC 2.42 mm). Using all observers' measurements, we could calculate mean ΔNAVplantar (4.25±2.08 mm) and ΔNAVmedial (1.55±0.83 mm). We demonstrated a small day-day difference in ΔNAVplantar (0.64 ±1.13mm; p<.05), but not for ΔNAVmedial (0.04 ±1.13mm; p=n.s.). Correlation of WBCT (WB navicular height - ΔNAVmedial) with total clinical FPI scores and FPI subscores, respectively, showed high correlation (ρ: -.706; ρ: -.721). CONCLUSION: CBCT and FPI are reliable measurements of foot posture, with a high correlation between the two measurements.

The Effect of Endoscopic Partial Plantar Fasciotomy on Morphologic and Functional Properties of the Foot.

BACKGROUND: The lifetime risk of plantar fasciitis is 10%, and operative treatment in the form of endoscopic partial plantar fascia release are often performed in cases refractory for nonsurgical treatment. The effect of the operation on the biomechanical properties of the foot has only been sparsely studied. METHODS: This is a prospective, observational study of 25 patients with plantar fasciitis, for a minimum of 3 months, verified by ultrasonographic scanning, who had endoscopic partial fasciotomy. A bony spur was resected if present. At the calcaneal insertion, the medial half of the central band of the plantar fascia was excised in full thickness. The biomechanical properties of the foot were evaluated before surgery and 12 months postoperatively. RESULTS: Foot length increased 0.17 cm (P = .03), the width of the central zone 0.35 cm (P = .019), the modified arch index 0.05 (P = .032), and the Foot Posture Index 1.0 (P = .0014). There were no significant changes in rearfoot eversion angle, ankle dorsiflexion and jump distance, or in magnetic resonance imaging-measured 3D navicular position from pre- to postoperation, with or without loading, and no changes in ultrasonographically measured heel pad thickness. A tantalum bead (0.7-mm-diameter) was inserted during operation into the most proximal part of the released medial plantar fascia. Radiographs obtained few days postoperatively and 1 year later revealed no changes in the tantalum-calcaneus distance in supine position, but an increase from 48.3 to 50.7 mm (P = .045) in one-leg standing, suggesting a higher flexibility of the remaining fascia. Patients with a body mass index above and below 27.0 demonstrated no significant differences in any of the assessments at 12 months. CONCLUSION: There were minimal changes in the measured foot morphologic and functional properties at 1-year follow-up, after endoscopic partial plantar fascia release. LEVEL OF EVIDENCE: Level II, prospective cohort study.

Weight-bearing cone-beam CT: the need for standardised acquisition protocols and measurements to fulfill high expectations-a review of the literature.

Weight bearing CT (WBCT) of the lower extremity is gaining momentum in evaluation of the foot/ankle and knee. A growing number of international studies use WBCT, which is promising for improving our understanding of anatomy and biomechanics during natural loading of the lower extremity. However, we believe there is risk of excessive enthusiasm for WBCT leading to premature application of the technique, before sufficiently robust protocols are in place e.g. standardised limb positioning and imaging planes, choice of anatomical landmarks and image slices used for individual measurements. Lack of standardisation could limit benefits from introducing WBCT in research and clinical practice because useful imaging information could become obscured. Measurements of bones and joints on WBCT are influenced by joint positioning and magnitude of loading, factors that need to be considered within a 3-D coordinate system. A proportion of WBCT studies examine inter- and intraobserver reproducibility for different radiological measurements in the knee or foot with reproducibility generally reported to be high. However, investigations of test-retest reproducibility are still lacking. Thus, the current ability to evaluate, e.g. the effects of surgery or structural disease progression, is questionable. This paper presents an overview of the relevant literature on WBCT in the lower extremity with an emphasis on factors that may affect measurement reproducibility in the foot/ankle and knee. We discuss the caveats of performing WBCT without consensus on imaging procedures and measurements.

External validation of an artificial intelligence tool for radiographic knee osteoarthritis severity classification.

PURPOSE: To externally validate an artificial intelligence (AI) tool for radiographic knee osteoarthritis severity classification on a clinical dataset. METHOD: This retrospective, consecutive patient sample, external validation study used weight-bearing, non-fixed-flexion posterior-anterior knee radiographs from a clinical production PACS. The index test was ordinal Kellgren-Lawrence grading by an AI tool, two musculoskeletal radiology consultants, two reporting technologists, and two resident radiologists. Grading was repeated by all readers after at least four weeks. Reference test was the consensus of the two consultants. The primary outcome was quadratic weighted kappa. Secondary outcomes were ordinal weighted accuracy, multiclass accuracy and F1-score. RESULTS: 50 consecutive patients between September 24, 2019 and October 22, 2019 were retrospectively included (3 excluded) totaling 99 knees (1 excluded). Quadratic weighted kappa for the AI tool and the consultant consensus was 0.88 CI95% (0.82-0.92). Agreement between the consultants was 0.89 CI95% (0.85-0.93). Intra-rater agreements for the consultants were 0.96 CI95% (0.94-0.98) and 0.94 CI95% (0.91-0.96) respectively. For the AI tool it was 1 CI95% (1-1). For the AI tool, ordinal weighted accuracy was 97.8% CI95% (96.9-98.6 %). Average multiclass accuracy and F1-score were 84% (83/99) CI95% (77-91%) and 0.67 CI95% (0.51-0.81). CONCLUSIONS: The AI tool achieved the same good-to-excellent agreement with the radiology consultant consensus for radiographic knee osteoarthritis severity classification as the consultants did with each other.