Development of a novel hybrid cognitive model validation framework for implementation under COVID-19 restrictions.

ABSTRACT

The purpose of this study was to develop a method for validation of cognitive models consistent with the remote working situation arising from COVID-19 restrictions in place in Spring 2020. We propose a framework for structuring validation tasks and applying a scoring system to determine initial model validity. We infer an objective validity level for cognitive models requiring no in-person observations, and minimal reliance on remote usability and observational studies. This approach has been derived from the necessity of the COVID-19 response, however, we believe this approach can lower costs and reduce timelines to initial validation in post-Covid-19 studies, enabling faster progress in the development of cognitive engineering systems. A three-stage hybrid validation framework was developed based on existing validation methods and was adapted to enable compliance with the specific limitations derived from COVID-19 response restrictions. This validation method includes elements of argument-based validation combined with a cognitive walkthrough analysis, and reflexivity assessments. We conducted a case study of the proposed framework on a developmental cognitive model of cardiovascular surgery to demonstrate application of a real-world validation task. This framework can be easily and quickly implemented by a small research team and provides a structured validation method to increase confidence in assumptions as well as to provide evidence to support validity claims in the early stages of model development.