Postsurgical Pain Risk Stratification to Enhance Pain Management Workflow in Adult Patients: Design, Implementation, and Pilot Evaluation.

BACKGROUND: Exposure to opioids after surgery is the initial contact for some people who develop chronic opioid use disorder. Hence, effective postoperative pain management, with less reliance on opioids, is critical. The Perioperative Opioid Quality Improvement (POQI) program developed (1) a digital health platform leveraging patient-survey-reported risk factors and (2) a postsurgical pain risk stratification algorithm to personalize perioperative care by integrating several commercially available digital health solutions into a combined platform. Development was reduced in scope by the COVID-19 pandemic. OBJECTIVE: This pilot study aims to assess the screening performance of the risk algorithm, quantify the use of the POQI platform, and evaluate clinicians' and patients' perceptions of its utility and benefit. METHODS: A POQI platform prototype was implemented in a quality improvement initiative at a Canadian tertiary care center and evaluated from January to September 2022. After surgical booking, a preliminary risk stratification algorithm was applied to health history questionnaire responses. The estimated risk guided the patient assignment to a care pathway based on low or high risk for persistent pain and opioid use. Demographic, procedural, and medication administration data were extracted retrospectively from the electronic medical record. Postoperative inpatient opioid use of >90 morphine milligram equivalents per day was the outcome used to assess algorithm performance. Data were summarized and compared between the low- and high-risk groups. POQI use was assessed by completed surveys on postoperative days 7, 14, 30, 60, 90, and 120. Semistructured patient and clinician interviews provided qualitative feedback on the platform. RESULTS: Overall, 276 eligible patients were admitted for colorectal procedures. The risk algorithm stratified 203 (73.6%) as the low-risk group and 73 (26.4%) as the high-risk group. Among the 214 (77.5%) patients with available data, high-risk patients were younger than low-risk patients (age: median 53, IQR 40-65 years, vs median 59, IQR 49-69 years, median difference five years, 95% CI 1-9; P=.02) and were more often female patients (45/73, 62% vs 80/203, 39.4%; odds ratio 2.5, 95% CI 1.4-4.5; P=.002). The risk stratification was reasonably specific (true negative rate=144/200, 72%) but not sensitive (true positive rate=10/31, 32%). Only 39.7% (85/214) patients completed any postoperative quality of recovery questionnaires (only 14, 6.5% patients beyond 60 days after surgery), and 22.9% (49/214) completed a postdischarge medication survey. Interviewed participants welcomed the initiative but noted usability issues and poor platform education. CONCLUSIONS: An initial POQI platform prototype was deployed operationally; the risk algorithm had reasonable specificity but poor sensitivity. There was a significant loss to follow-up in postdischarge survey completion. Clinicians and patients appreciated the potential impact of preemptively addressing opioid exposure but expressed shortcomings in the platform's design and implementation. Iterative platform redesign with additional features and reevaluation are required before broader implementation.

Dashboard of Short-Term Postoperative Patient Outcomes for Anesthesiologists: Development and Preliminary Evaluation.

BACKGROUND: Anesthesiologists require an understanding of their patients' outcomes to evaluate their performance and improve their practice. Traditionally, anesthesiologists had limited information about their surgical outpatients' outcomes due to minimal contact post discharge. Leveraging digital health innovations for analyzing personal and population outcomes may improve perioperative care. BC Children's Hospital's postoperative follow-up registry for outpatient surgeries collects short-term outcomes such as pain, nausea, and vomiting. Yet, these data were previously not available to anesthesiologists. OBJECTIVE: This quality improvement study aimed to visualize postoperative outcome data to allow anesthesiologists to reflect on their care and compare their performance with their peers. METHODS: The postoperative follow-up registry contains nurse-reported postoperative outcomes, including opioid and antiemetic administration in the postanesthetic care unit (PACU), and family-reported outcomes, including pain, nausea, and vomiting, within 24 hours post discharge. Dashboards were iteratively co-designed with 5 anesthesiologists, and a department-wide usability survey gathered anesthesiologists' feedback on the dashboards, allowing further design improvements. A final dashboard version has been deployed, with data updated weekly. RESULTS: The dashboard contains three sections: (1) 24-hour outcomes, (2) PACU outcomes, and (3) a practice profile containing individual anesthesiologist's case mix, grouped by age groups, sex, and surgical service. At the time of evaluation, the dashboard included 24-hour data from 7877 cases collected from September 2020 to February 2023 and PACU data from 8716 cases collected from April 2021 to February 2023. The co-design process and usability evaluation indicated that anesthesiologists preferred simpler designs for data summaries but also required the ability to explore details of specific outcomes and cases if needed. Anesthesiologists considered security and confidentiality to be key features of the design and most deemed the dashboard information useful and potentially beneficial for their practice. CONCLUSIONS: We designed and deployed a dynamic, personalized dashboard for anesthesiologists to review their outpatients' short-term postoperative outcomes. This dashboard facilitates personal reflection on individual practice in the context of peer and departmental performance and, hence, the opportunity to evaluate iterative practice changes. Further work is required to establish their effect on improving individual and department performance and patient outcomes.

Identifying Risk Factors, Patient-Reported Experience and Outcome Measures, and Data Capture Tools for an Individualized Pain Prediction Tool in Pediatrics: Focus Group Study.

BACKGROUND: The perioperative period is a data-rich environment with potential for innovation through digital health tools and predictive analytics to optimize patients' health with targeted prehabilitation. Although some risk factors for postoperative pain following pediatric surgery are already known, the systematic use of preoperative information to guide personalized interventions is not yet widespread in clinical practice. OBJECTIVE: Our long-term goal is to reduce the incidence of persistent postsurgical pain (PPSP) and long-term opioid use in children by developing personalized pain risk prediction models that can guide clinicians and families to identify targeted prehabilitation strategies. To develop such a system, our first objective was to identify risk factors, outcomes, and relevant experience measures, as well as data collection tools, for a future data collection and risk modeling study. METHODS: This study used a patient-oriented research methodology, leveraging parental/caregiver and clinician expertise. We conducted virtual focus groups with participants recruited at a tertiary pediatric hospital; each session lasted approximately 1 hour and was composed of clinicians or family members (people with lived surgical experience and parents of children who had recently undergone a procedure requiring general anesthesia) or both. Data were analyzed thematically to identify potential risk factors for pain, as well as relevant patient-reported experience and outcome measures (PREMs and PROMs, respectively) that can be used to evaluate the progress of postoperative recovery at home. This guidance was combined with a targeted literature review to select tools to collect risk factor and outcome information for implementation in a future study. RESULTS: In total, 22 participants (n=12, 55%, clinicians and n=10, 45%, family members) attended 10 focus group sessions; participants included 12 (55%) of 22 persons identifying as female, and 12 (55%) were under 50 years of age. Thematic analysis identified 5 key domains: (1) demographic risk factors, including both child and family characteristics; (2) psychosocial risk factors, including anxiety, depression, and medical phobias; (3) clinical risk factors, including length of hospital stay, procedure type, medications, and pre-existing conditions; (4) PREMs, including patient and family satisfaction with care; and (5) PROMs, including nausea and vomiting, functional recovery, and return to normal activities of daily living. Participants further suggested desirable functional requirements, including use of standardized and validated tools, and longitudinal data collection, as well as delivery modes, including electronic, parent proxy, and self-reporting, that can be used to capture these metrics, both in the hospital and following discharge. Established PREM/PROM questionnaires, pain-catastrophizing scales (PCSs), and substance use questionnaires for adolescents were subsequently selected for our proposed data collection platform. CONCLUSIONS: This study established 5 key data domains for identifying pain risk factors and evaluating postoperative recovery at home, as well as the functional requirements and delivery modes of selected tools with which to capture these metrics both in the hospital and after discharge. These tools have been implemented to generate data for the development of personalized pain risk prediction models.

Identification of Requirements for a Postoperative Pediatric Pain Risk Communication Tool: Focus Group Study With Clinicians and Family Members.

BACKGROUND: Pediatric surgery is associated with a risk of postoperative pain that can impact the family's quality of life. Although some risk factors for postoperative pain are known, these are often not consistently communicated to families. In addition, although tools for risk communication exist in other domains, none are tailored to pediatric surgery. OBJECTIVE: As part of a larger project to develop pain risk prediction tools, we aimed to design an easy-to-use tool to effectively communicate a child's risk of postoperative pain to both clinicians and family members. METHODS: With research ethics board approval, we conducted virtual focus groups (~1 hour each) comprising clinicians and family members (people with lived surgical experience and parents of children who had recently undergone surgery/medical procedures) at a tertiary pediatric hospital to understand and evaluate potential design approaches and strategies for effectively communicating and visualizing postoperative pain risk. Data were analyzed thematically to generate design requirements and to inform iterative prototype development. RESULTS: In total, 19 participants (clinicians: n=10, 53%; family members: n=9, 47%) attended 6 focus group sessions. Participants indicated that risk was typically communicated verbally by clinicians to patients and their families, with severity indicated using a descriptive or a numerical representation or both, which would only occasionally be contextualized. Participants indicated that risk communication tools were seldom used but that families would benefit from risk information, time to reflect on the information, and follow-up with questions. In addition, 9 key design requirements and feature considerations for effective risk communication were identified: (1) present risk information clearly and with contextualization, (2) quantify the risk and contextualize it, (3) include checklists for preoperative family preparation, (4) provide risk information digitally to facilitate recall and sharing, (5) query the family's understanding to ensure comprehension of risk, (6) present the risk score using multimodal formats, (7) use color coding that is nonthreatening and avoids limitations with color blindness, (8) present the most significant factors contributing to the risk prediction, and (9) provide risk mitigation strategies to potentially decrease the patient's level of risk. CONCLUSIONS: Key design requirements for a pediatric postoperative pain risk visualization tool were established and guided the development of an initial prototype. Implementing a risk communication tool into clinical practice has the potential to bridge existing gaps in the accessibility, utilization, and comprehension of personalized risk information between health care professionals and family members. Future iterative codesign and clinical evaluation of this risk communication tool are needed to confirm its utility in practice.

Prediction Model Performance With Different Imputation Strategies: A Simulation Study Using a North American ICU Registry.

OBJECTIVES: To evaluate the performance of pragmatic imputation approaches when estimating model coefficients using datasets with varying degrees of data missingness. DESIGN: Performance in predicting observed mortality in a registry dataset was evaluated using simulations of two simple logistic regression models with age-specific criteria for abnormal vital signs (mentation, systolic blood pressure, respiratory rate, WBC count, heart rate, and temperature). Starting with a dataset with complete information, increasing degrees of biased missingness of WBC and mentation were introduced, depending on the values of temperature and systolic blood pressure, respectively. Missing data approaches evaluated included analysis of complete cases only, assuming missing data are normal, and multiple imputation by chained equations. Percent bias and root mean square error, in relation to parameter estimates obtained from the original data, were evaluated as performance indicators. SETTING: Data were obtained from the Virtual Pediatric Systems, LLC, database (Los Angeles, CA), which provides clinical markers and outcomes in prospectively collected records from 117 PICUs in the United States and Canada. PATIENTS: Children admitted to a participating PICU in 2017, for whom all required data were available. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Simulations demonstrated that multiple imputation by chained equations is an effective strategy and that even a naive implementation of multiple imputation by chained equations significantly outperforms traditional approaches: the root mean square error for model coefficients was lower using multiple imputation by chained equations in 90 of 99 of all simulations (91%) compared with discarding cases with missing data and lower in 97 of 99 (98%) compared with models assuming missing values are in the normal range. Assuming missing data to be abnormal was inferior to all other approaches. CONCLUSIONS: Analyses of large observational studies are likely to encounter the issue of missing data, which are likely not missing at random. Researchers should always consider multiple imputation by chained equations (or similar imputation approaches) when encountering even only small proportions of missing data in their work.

An Artificial Neural Network-Based Pediatric Mortality Risk Score: Development and Performance Evaluation Using Data From a Large North American Registry.

BACKGROUND: In the pediatric intensive care unit (PICU), quantifying illness severity can be guided by risk models to enable timely identification and appropriate intervention. Logistic regression models, including the pediatric index of mortality 2 (PIM-2) and pediatric risk of mortality III (PRISM-III), produce a mortality risk score using data that are routinely available at PICU admission. Artificial neural networks (ANNs) outperform regression models in some medical fields. OBJECTIVE: In light of this potential, we aim to examine ANN performance, compared to that of logistic regression, for mortality risk estimation in the PICU. METHODS: The analyzed data set included patients from North American PICUs whose discharge diagnostic codes indicated evidence of infection and included the data used for the PIM-2 and PRISM-III calculations and their corresponding scores. We stratified the data set into training and test sets, with approximately equal mortality rates, in an effort to replicate real-world data. Data preprocessing included imputing missing data through simple substitution and normalizing data into binary variables using PRISM-III thresholds. A 2-layer ANN model was built to predict pediatric mortality, along with a simple logistic regression model for comparison. Both models used the same features required by PIM-2 and PRISM-III. Alternative ANN models using single-layer or unnormalized data were also evaluated. Model performance was compared using the area under the receiver operating characteristic curve (AUROC) and the area under the precision recall curve (AUPRC) and their empirical 95% CIs. RESULTS: Data from 102,945 patients (including 4068 deaths) were included in the analysis. The highest performing ANN (AUROC 0.871, 95% CI 0.862-0.880; AUPRC 0.372, 95% CI 0.345-0.396) that used normalized data performed better than PIM-2 (AUROC 0.805, 95% CI 0.801-0.816; AUPRC 0.234, 95% CI 0.213-0.255) and PRISM-III (AUROC 0.844, 95% CI 0.841-0.855; AUPRC 0.348, 95% CI 0.322-0.367). The performance of this ANN was also significantly better than that of the logistic regression model (AUROC 0.862, 95% CI 0.852-0.872; AUPRC 0.329, 95% CI 0.304-0.351). The performance of the ANN that used unnormalized data (AUROC 0.865, 95% CI 0.856-0.874) was slightly inferior to our highest performing ANN; the single-layer ANN architecture performed poorly and was not investigated further. CONCLUSIONS: A simple ANN model performed slightly better than the benchmark PIM-2 and PRISM-III scores and a traditional logistic regression model trained on the same data set. The small performance gains achieved by this two-layer ANN model may not offer clinically significant improvement; however, further research with other or more sophisticated model designs and better imputation of missing data may be warranted.