Evaluation of an architecture for intelligent query and exploration of time-oriented clinical data.

OBJECTIVE: Evaluate KNAVE-II, a knowledge-based framework for visualization, interpretation, and exploration of longitudinal clinical data, clinical concepts and patterns. KNAVE-II mediates queries to a distributed temporal-abstraction architecture (IDAN), which uses a knowledge-based problem-solving method specializing in on-the-fly computation of clinical queries. METHODS: A two-phase, balanced cross-over study to compare efficiency and satisfaction of a group of clinicians when answering queries of variable complexity about time-oriented clinical data, typical for oncology protocols, using KNAVE-II, versus standard methods: both paper charts and a popular electronic spreadsheet (ESS) in Phase I; an ESS in Phase II. The measurements included the time required to answer and the correctness of answer for each query and each complexity category, and for all queries, assessed versus a predetermined gold standard set by a domain expert. User satisfaction was assessed by the Standard Usability Score (SUS) tool-specific questionnaire and by a "Usability of Tool Comparison" comparative questionnaire developed for this study. RESULTS: In both evaluations, subjects answered higher-complexity queries significantly faster using KNAVE-II than when using paper charts or an ESS up to a mean of 255 s difference per query versus the ESS for hard queries (p=0.0003) in the second evaluation. Average correctness scores when using KNAVE-II versus paper charts, in the first phase, and the ESS, in the second phase, were significantly higher over all queries. In the second evaluation, 91.6% (110/120) of all of the questions asked within queries of all levels produced correct answers using KNAVE-II, opposed to only 57.5% (69/120) using the ESS (p<0.0001). User satisfaction with KNAVE-II was significantly superior compared to using either a paper chart or the ESS (p=0.006). Clinicians ranked KNAVE-II superior to both paper and the ESS. CONCLUSIONS: An evaluation of the functionality and usability of KNAVE-II and its supporting knowledge-based temporal-mediation architecture has produced highly encouraging results regarding saving of physician time, enhancement of accuracy of clinical assessment, and user satisfaction.

Evaluation of KNAVE-II: a tool for intelligent query and exploration of patient data.

We present the results of a preliminary evaluation of KNAVE-II, a distributed knowledge-based computational framework for visualization, interpretation, and exploration of longitudinal clinical data and of multiple levels of concepts derivable from these data. KNAVE-II uses a distributed architecture to access at run-time clinical time-oriented data, a domain-specific knowledge base containing properties of the clinical data, and a knowledge-based problem-solving method for computing on-the-fly interpretations of these data. The purpose of the evaluation was to compare efficiency and user satisfaction when answering clinical queries of variable complexity about clinical time-oriented data using KNAVE-II, versus using methods available in standard clinical settings: paper chart or electronic spreadsheet (ESS). Subjects answered high-complexity queries significantly faster using KNAVE-II than when using paper or ESS. User satisfaction with KNAVE-II was significantly superior compared to satisfaction using paper or ESS, based on a standard usability scale. Users also explicitly ranked KNAVE-II as superior to paper and the ESS.

Interactive visualization and exploration of time-oriented clinical data using a distributed temporal-abstraction architecture.

KNAVE-II is a system for visualization and exploration of large amounts of time-oriented clinical data and of multiple levels of clinically meaningful abstractions derivable from these data. KNAVE-II uses a distributed temporal-abstraction architecture that integrates a set of knowledge services, each interacting with a domain-specific knowledge source, a set of data-access services, each interacting with a clinical data source, and a computational service for deriving knowledge-based abstractions of the data.

A Web-Based system for interactive visualization and exploration of time-oriented clinical data and their abstractions.

In this theater-style demonstration, the speakers will demonstrate KNAVE-II, a Web-based distributed system for interactive visualization and exploration of large amounts of time-oriented clinical data from multiple sources, and of clinically meaningful concepts (abstractions) derivable from these data. The KNAVE-II system and its complete underlying architecture provide a solution to the data overload problem.