ABSTRACT
Background: The increasing development and use of digital health interventions requires good quality costing information to inform development and commissioning choices about resource allocation decisions. The Narrative Experiences Online (NEON) Intervention is a web-application that delivers recorded mental health recovery narratives to its users. Two randomized controlled trials are testing the NEON Intervention in people with experience of psychosis (NEON) and people experiencing non-psychosis mental health problems (NEON-O). Aim: This study describes and estimates the cost components and total cost of developing and delivering the NEON Intervention. Materials and methods: Total costs for the NEON Trial (739 participants) and NEON-O Trial (1,024 participants) were estimated by: identifying resource use categories involved in intervention development and delivery; accurate measurement or estimation of resource use; and a valuation of resource use to generate overall costs, using relevant unit costs. Resource use categories were identified through consultation with literature, costing reporting standards and iterative consultation with health researchers involved in NEON Intervention development and delivery. Sensitivity analysis was used to test assumptions made. Results: The total cost of developing the NEON Intervention was £182,851. The largest cost components were software development (27%); Lived Experience Advisory Panel workshops (23%); coding the narratives (9%); and researchers' time to source narratives (9%). The total cost of NEON Intervention delivery during the NEON Trial was £118,663 (£349 per NEON Intervention user). In the NEON-O Trial, the total delivery cost of the NEON Intervention was £123,444 (£241 per NEON Intervention user). The largest cost components include updating the narrative collection (50%); advertising (19%); administration (14%); and software maintenance (11%). Uncertainty in the cost of administration had the largest effect on delivery cost estimates. Conclusion: Our work shows that developing and delivering a digital health intervention requires expertise and time commitment from a range of personnel. Teams developing digital narrative interventions need to allocate substantial resources to curating narrative collections. Implications for practice: This study identifies the development and delivery resource use categories of a digital health intervention to promote the consistent reporting of costs and informs future decision-making about the costs of delivering the NEON Intervention at scale. Trial registration: NEON Trial: ISRCTN11152837, registered 13 August 2018, http://www.isrctn.com/ISRCTN11152837. NEON-O Trial: ISRCTN63197153, registered 9 January 2020, http://www.isrctn.com/ISRCTN63197153.
ABSTRACT
BACKGROUND: The internet enables sharing of narratives about health concerns on a substantial scale, and some digital health narratives have been integrated into digital health interventions. Narratives describing recovery from health problems are a focus of research, including those presented in recorded (eg, invariant) form. No clinical trial has been conducted on a web-based intervention providing access to a collection of Recorded Recovery Narratives (RRNs). OBJECTIVE: This study presents knowledge produced through the development of the Narrative Experiences Online (NEON) Intervention, a web-based intervention incorporating the algorithmic recommendation of RRNs. METHODS: Knowledge was gathered through knowledge integration (KI) activities. KI1 synthesized previous studies to produce the NEON Impact Model describing how accessing RRNs produces health-related outcomes. KI2 developed curation principles for the NEON Collection of RRNs through consultation with the NEON Lived Experience Advisory Panel and the curation of a preliminary collection. KI3 identified harm minimization strategies for the NEON Intervention through consultation with the NEON International Advisory Board and Lived Experience Advisory Panel. The NEON Intervention was finalized through 2 research studies (RS). In RS1, mental health service users (N=40) rated the immediate impact of randomly presented narratives to validate narrative feedback questions used to inform the recommendation algorithm. In RS2, mental health service users (n=25) were interviewed about their immediate response to a prototype of the NEON Intervention and trial procedures and then were interviewed again after 1 month of use. The usability and acceptability of the prototype and trial procedures were evaluated and refinements were made. RESULTS: KI1 produced the NEON Impact Model, which identifies moderators (recipient and context), mechanisms of connection (reflection, comparison, learning, and empathy), processes (identification of change from narrative structure or content and internalization of observed change), and outcomes (helpful and unhelpful). KI2 identified 22 curation principles, including a mission to build a large, heterogeneous collection to maximize opportunities for connection. KI3 identified seven harm minimization strategies, including content warnings, proactive and reactive blocking of narratives, and providing resources for the self-management of emotional distress. RS1 found variation in the impact of narratives on different participants, indicating that participant-level feedback on individual narratives is needed to inform a recommender system. The order of presentation did not predict narrative feedback. RS2 identified amendments to web-based trial procedures and the NEON Intervention. Participants accessed some narratives multiple times, use reduced over the 4-week period, and narrative feedback was provided for 31.8% (105/330) of narrative accesses. CONCLUSIONS: RRNs can be integrated into web-based interventions. Evaluating the NEON Intervention in a clinical trial is feasible. The mixed methods design for developing the NEON Intervention can guide its extension to other clinical populations, the design of other web-based mental health interventions, and the development of narrative-based interventions in mental health.
