RESUMEN
Background: The Trial Innovation Network (TIN) is a collaborative initiative within the National Center for Advancing Translational Science (NCATS) Clinical and Translational Science Awards (CTSA) Program. To improve and innovate the conduct of clinical trials, it is exploring the uses of gamification to better engage the trial workforce and improve the efficiencies of trial activities. The gamification structures described in this article are part of a TIN website gamification toolkit, available online to the clinical trial scientific community. Methods: The game designers used existing electronic trial platforms to gamify the tasks required to meet trial start-up timelines to create friendly competitions. Key indicators and familiar metrics were mapped to scoreboards. Webinars were organized to share and applaud trial and game performance. Results: Game scores were significantly associated with an increase in achieving start-up milestones in activation, institutional review board (IRB) submission, and IRB approval times, indicating the probability of completing site activation faster by using games. Overall game enjoyment and feelings that the game did not apply too much pressure appeared to be an important moderator of performance in one trial but had little effect on performance in a second. Conclusion: This retrospective examination of available data from gaming experiences may be a first-of-kind use in clinical trials. There are signals that gaming may accelerate performance and increase enjoyment during the start-up phase of a trial. Isolating the effect of gamification on trial outcomes will depend on a larger sampling from future trials, using well-defined, hypothesis-driven statistical analysis plans.
RESUMEN
BACKGROUND AND PURPOSE: Intracerebral hemorrhage (ICH) and intraventricular hemorrhage (IVH) clinical trials rely on manual linear and semi-quantitative (LSQ) estimators like the ABC/2, modified Graeb and IVH scores for timely volumetric estimation from CT. Deep learning (DL) volumetrics of ICH have recently approached the accuracy of gold-standard planimetry. However, DL and LSQ strategies have been limited by unquantified uncertainty, in particular when ICH and IVH estimates intersect. Bayesian deep learning methods can be used to approximate uncertainty, presenting an opportunity to improve quality assurance in clinical trials. METHODS: A DL model was trained to simultaneously segment ICH and IVH using diagnostic CT data from the Minimally Invasive Surgery Plus Alteplase for ICH Evacuation (MISTIE) III and Clot Lysis: Evaluating Accelerated Resolution of IVH (CLEAR) III clinical trials. Bayesian uncertainty approximation was performed using Monte-Carlo dropout. We compared the performance of our model with estimators used in the CLEAR IVH and MISTIE II trials. The reliability of planimetry, DL, and LSQ volumetrics in the setting of high ICH and IVH intersection is quantified using consensus estimates. RESULTS: Our DL model produced volume correlations and median Dice scores of .994 and .946 for ICH in MISTIE II, and .980 and .863 for IVH in CLEAR IVH, respectively, outperforming LSQ estimates from the clinical trials. We found significant linear relationships between ICH uncertainty, Dice scores (r = -.849), and relative volume difference (r = .735). CONCLUSION: In our validation clinical trial dataset, DL models with Bayesian uncertainty approximation provided superior volumetric estimates to LSQ methods with real-time estimates of model uncertainty.
