Trading-off privacy and utility: the Washington State experience assessing the performance of a public health digital exposure notification system for coronavirus disease 2019.

OBJECTIVE: Digital exposure notifications (DEN) systems were an emergency response to the coronavirus disease 2019 (COVID-19) pandemic, harnessing smartphone-based technology to enhance conventional pandemic response strategies such as contact tracing. We identify and describe performance measurement constructs relevant to the implementation of DEN tools: (1) reach (number of users enrolled in the intervention); (2) engagement (utilization of the intervention); and (3) effectiveness in preventing transmissions of COVID-19 (impact of the intervention). We also describe WA State's experience utilizing these constructs to design data-driven evaluation approaches. METHODS: We conducted an environmental scan of DEN documentation and relevant publications. Participation in multidisciplinary collaborative environments facilitated shared learning. Compilation of available data sources and their relevance to implementation and operation workflows were synthesized to develop implementation evaluation constructs. RESULTS: We identified 8 useful performance indicators within reach, engagement, and effectiveness constructs. DISCUSSION: We use implementation science to frame the evaluation of DEN tools by linking the theoretical constructs with the metrics available in the underlying disparate, deidentified, and aggregate data infrastructure. Our challenges in developing meaningful metrics include limited data science competencies in public health, validation of analytic methodologies in the complex and evolving pandemic environment, and the lack of integration with the public health infrastructure. CONCLUSION: Continued collaboration and multidisciplinary consensus activities can improve the utility of DEN tools for future public health emergencies.

A mixed method approach to understanding the impact of COVID-19 on patients with or at risk for aortic dissection.

Aortic dissection (AD) is a life-threatening rare disease that occurs as a spontaneous tear in the wall of the aorta. Survivors of AD go on to have a chronic disease process that requires lifelong follow-up and management. Although the COVID-19 pandemic has strained health systems and impacted practice in the United States, the effects of these impacts on people living with or at risk for AD is not well understood. This mixed methods project examined the experiences of people in the AD community during the COVID-19 pandemic between March and October 2020. Results reveal that the AD community lacked clear guidance on the role aortic health status plays in COVID-19 risk and experienced significant disruptions in aortic healthcare. At the same time, the new expansion in access to medical care with telehealth conferred unforeseen benefits in the form of reduced barriers for access to specialized aortic health care.

Lived experiences of people with or at risk for aortic dissection: A qualitative assessment.

Aortic dissection (AD) is a life-changing event that is often accompanied by a loss of normal quality of life. Survivors of AD go on to have a chronic disease that requires lifestyle modification, medical management, and surgical repair of the aorta. Clinical care includes multiple disciplines, health care settings, and often different geographic locations. This qualitative assessment examined the experiences of people with and at risk for AD. The following four themes emerged: "unnecessary drama" at diagnosis, unmet needs for information and support, the burden of self-advocacy and care coordination, and living with unaddressed mental health impacts. Our findings inform recommendations to advance patient-centered care delivery for individuals with and at risk for AD, improving communication of timely and relevant information, and an approach to care that acknowledges the whole person in clinical decision making.

Augmenting systems-level implementation of patient-reported outcomes for depression care through the use of structured analysis and design technique.

Background: Health systems increasingly need to implement complex practice changes such as the routine capture of patient-reported outcome (PRO) measures. Yet, health systems have met challenges when trying to bring practice change to scale across systems at large. While implementation science can guide the evaluation of implementation determinants, teams first need tools to systematically understand and compare workflow activities across practice sites. Structured analysis and design technique (SADT), a system engineering method of workflow modeling, may offer an opportunity to enhance the scalability of implementation evaluation for complex practice change like PROs. Method: We utilized SADT to identify the core workflow activities needed to implement PROs across diverse settings and goals for use, establishing a generalizable PRO workflow diagram. We then used the PRO workflow diagram to guide implementation monitoring and evaluation for a 1-year pilot implementation of the electronic Patient Health Questionnaire-9 (ePHQ). The pilot occurred across multiple clinical settings and for two clinical use cases: depression screening and depression management. Results: SADT identified five activities central to the use of PROs in clinical care: deploying PRO measures, collecting PRO data, tracking PRO completion, reviewing PRO results, and documenting PRO data for future use. During the 1-year pilot, 8,596 patients received the ePHQ for depression screening via the patient portal, of which 1,719 (21%) submitted the ePHQ; 367 patients received the ePHQ for depression management, of which 174 (47%) submitted the ePHQ. We present three case examples of how the SADT PRO workflow diagram augmented implementation monitoring, tailoring, and evaluation activities. Conclusions: Use of a generalizable PRO workflow diagram aided the ability to systematically assess barriers and facilitators to fidelity and identify needed adaptations. The use of SADT offers an opportunity to align systems science and implementation science approaches, augmenting the capacity for health systems to advance system-level implementation. Plain Language Summary: Health systems increasingly need to implement complex practice changes such as the routine capture of patient-reported outcome (PRO) measures. Yet these system-level changes can be challenging to manage given the variability in practice sites and implementation context across the system at large. We utilized a systems engineering method-structured analysis and design technique-to develop a generalizable diagram of PRO workflow that captures five common workflow activities: deploying PRO measures, collecting PRO data, tracking PRO completion, reviewing PRO results, and documenting PRO data for future use. Next, we used the PRO workflow diagram to guide our implementation of PROs for depression care in multiple clinics. Our experience showed that use of a standard workflow diagram supported our implementation evaluation activities in a systematic way. The use of structured analysis and design technique may enhance future implementation efforts in complex health settings.

A learning health systems approach to integrating electronic patient-reported outcomes across the health care organization.

INTRODUCTION: Foundational to a learning health system (LHS) is the presence of a data infrastructure that can support continuous learning and improve patient outcomes. To advance their capacity to drive patient-centered care, health systems are increasingly looking to expand the electronic capture of patient data, such as electronic patient-reported outcome (ePRO) measures. Yet ePROs bring unique considerations around workflow, measurement, and technology that health systems may not be poised to navigate. We report on our effort to develop generalizable learnings that can support the integration of ePROs into clinical practice within an LHS framework. METHODS: Guided by action research methodology, we engaged in iterative cycles of planning, acting, observing, and reflecting around ePRO use with two primary goals: (1) mobilize an ePRO community of practice to facilitate knowledge sharing, and (2) establish guidelines for ePRO use in the context of LHS practice. Multiple, emergent data collection activities generated generalizable guidelines that document the tangible best practices for ePRO use in clinical care. We organized guidelines around thematic areas that reflect LHS structures and stakeholders. RESULTS: Three core thematic areas (and 24 guidelines) emerged. The theme of governance reflects the importance of leadership, knowledge management, and facilitating organizational learning around best practice models for ePRO use. The theme of integration considers the intersection of workflow, technology, and human factors for ePROs across areas of care delivery. Lastly, the theme of reporting reflects critical considerations for curating data and information, designing system functions and interactions, and presentation of ePRO data to support the translation of knowledge to action. CONCLUSIONS: The guidelines produced from this work highlight the complex, multidisciplinary nature of implementing change within LHS contexts, and the value of action research approaches to enable rapid, iterative learning that leverages the knowledge and experience of communities of practice.

"What Will Happen If I Say Yes?" Perspectives on a Standardized Firearm Access Question Among Adults With Depressive Symptoms.

OBJECTIVE: Addressing firearm access is recommended when patients are identified as being at risk of suicide. However, the practice of assessing firearm access is controversial, and no national guidelines exist to inform practice. This study qualitatively explored patient perspectives on a routine question about firearm access to optimize the patient centeredness of this practice in the context of suicide risk. METHODS: Electronic health record data were used to identify primary care patients reporting depressive symptoms, including suicidal thoughts, within 2 weeks of sampling. Participants completed a semistructured telephone interview (recorded and transcribed), which focused broadly on the experience of being screened for suicidality and included specific questions to elicit beliefs and opinions about being asked a standard firearm access question. Directive (deductive) and conventional (inductive) content analysis was used to analyze responses to the portion of the interview focused on firearm assessment and disclosure. RESULTS: Thirty-seven patients in Washington State ages 20-95 completed the qualitative interview by phone. Organizing themes included apprehensions about disclosing access to firearms related to privacy, autonomy, and firearm ownership rights; perceptions regarding relevance of the firearm question, informed by experiences with suicidality and common beliefs and misconceptions about the inevitability of suicide; and suggestions for connecting questions about firearms and other lethal means to suicide risk. CONCLUSIONS: Clarifying the purpose and use of routine firearm access assessment, contextualizing firearm questions within injury prevention broadly, and addressing misconceptions about suicide prevention may help encourage disclosure of firearm access and increase the patient centeredness of this practice.

Design of digital walking programs that engage prostate cancer survivors: Needs and preferences from focus groups.

The majority of prostate cancer survivors do not meet physical activity (PA) recommendations. Although technology has shown to promote PA, engagement has been a challenge. This mixed method study characterizes survivors' needs and preferences for digital walking programs Through focus groups and surveys, we engaged prostate cancer support groups to describe PA motivators and barriers, interest in improving PA, and preferences for design features of a future digital walking program. Identified motivators (peers, positive thinking) and barriers (health issues) reflect PA needs that impact engagement. The most preferred features include: (1) well-curated, specific content, (2) individualized feedback from trusted sources, (3) moderated peer discussion, and (4) support from small teams and peer mentors. These findings inform digital PA programs that survivors will find engaging and can promote PA.

Capturing the patient voice: implementing patient-reported outcomes across the health system.

PURPOSE: Supporting the capture and use of patient-reported outcomes (PROs) at the point-of-care enriches information about important clinical and quality of life outcomes. Yet the ability to scale PROs across healthcare systems has been limited by knowledge gaps around how to manage the diversity of PRO uses and leverage health information technology. In this study, we report learnings and practice insights from UW Medicine's practice transformation efforts to incorporate patient voice into multiple areas of care. METHODS: Using a participatory, action research approach, we engaged with UW Medicine clinical and administrative stakeholders experienced with PRO implementation to inventory PRO implementations across the health system, characterize common clinical uses for PROs, and develop recommendations for system-wide governance and implementation of PROs. RESULTS: We identified a wide breadth of PRO implementations (n = 14) in practice and found that nearly half (47%) of employed PRO measures captured shared clinical domains (e.g., depression). We developed three vignettes (use cases) that illustrate how users interact with PROs, characterize common ways PRO implementations support clinical care across the health system (1) Preventive care, (2) Chronic/Specialty care, and (3) Surgical/Interventional care), and elucidate opportunities to enhance efficient PRO implementations through system-level standards and governance. CONCLUSIONS: Practice transformation efforts increasingly require integration of the patient voice into clinical care, often through the use of PROs. Learnings from our work highlight the importance of proactively considering how PROs will be used across the layers of healthcare organizations to optimize the design and governance of PROs.

Opportunities and challenges to advance the use of electronic patient-reported outcomes in clinical care: a report from AMIA workshop proceedings.

Despite the demonstrated value of patient-centered care, health systems have been slow to integrate the patient's voice into care delivery through patient-reported outcomes (PROs) with electronic tools. This is due in part to the complex interplay of technology, workflow, and human factors that shape the success of electronic PROs (ePROs) use. The 2018 American Medical Informatics Association Annual Symposium served as the setting for a half-day interactive workshop with diverse stakeholders to discuss proposed best practices for the planning, design, deployment, and evaluation of ePROs. We provide this collective commentary that synthesizes participant feedback regarding critical challenges that prohibit the scale and spread of ePROs across healthcare delivery systems, including governance and leadership, workflow and human factors, informatics, and data science. In order to realize the promise of ePROs at scale, adaptable approaches are critical to balance the needs of individual users with health systems at large.

Framework To Guide The Collection And Use Of Patient-Reported Outcome Measures In The Learning Healthcare System.

BACKGROUND: Web-based collection of patient-reported outcome measures (PROMs) in clinical practice is expanding rapidly as electronic health records include web portals for patients to report standardized assessments of their symptoms. As the value of PROMs in patient care expands, a framework to guide the implementation planning, collection, and use of PROs to serve multiple goals and stakeholders is needed. METHODS: We identified diverse clinical, quality, and research settings where PROMs have been successfully integrated into care and routinely collected and analyzed drivers of successful implementation. Findings are based on key informant interviews with 46 individuals representing 38 organizations, of whom 40 participated in a webinars series, and 25 attended an in-person workshop designed to enable broad stakeholder input, review and refinement of the proposed PROMs implementation model. Stakeholders identified differing uses of PROMs to support: 1) individual patient care decisions, 2) quality improvement initiatives, 3) payer mandates, and 4) population health and research. RESULTS: The implementation framework and steps that are consistently identified by stakeholders as best practices to guide PROM capture and use are described. Of note, participants indicate that web-based informatics tools are necessary but not sufficient for PROM use, suggesting that successful PROM implementation requires integration into clinic operations and careful planning for user's analytic needs. Each of the four identified uses may require implementation modifications at each step to assure optimal use. CONCLUSIONS: The proposed framework will guide future PROM implementation efforts across learning health care systems to assure that complete PROMs are captured at the correct time, and with associated risk factors, to generate meaningful information to serve diverse stakeholders.

Incorporating Patient-Reported Outcomes Into Health Care To Engage Patients And Enhance Care.

The provision of patient-centered care requires a health care environment that fosters engagement between patients and their health care team. One way to encourage patient-centered care is to incorporate patient-reported outcomes into clinical settings. Collecting these outcomes in routine care ensures that important information only the patient can provide is captured. This provides insights into patients' experiences of symptoms, quality of life, and functioning; values and preferences; and goals for health care. Previously embraced in the research realm, patient-reported outcomes have started to play a role in successful shared decision making, which can enhance the safe and effective delivery of health care. We examine the opportunities for using patient-reported outcomes to enhance care delivery and outcomes as health care information needs and technology platforms change. We highlight emerging practices in which patient-reported outcomes provide value to patients and clinicians and improve care delivery. Finally, we examine present and future challenges to maximizing the use of patient-reported outcomes in the clinic.

Infrastructure to support learning health systems: are we there yet? Innovative solutions and lessons learned from American Recovery and Reinvestment Act CER investments.

The 11 big health data networks participating in the AcademyHealth Electronic Data Methods Forum represent cutting-edge efforts to harness the power of big health data for research and quality improvement. This paper is a comparative case study based on site visits conducted with a subset of these large infrastructure grants funded through the Recovery Act, in which four key issues emerge that can inform the evolution of learning health systems, including the importance of acknowledging the challenges of scaling specialized expertise needed to manage and run CER networks; the delicate balance between privacy protections and the utility of distributed networks; emerging community engagement strategies; and the complexities of developing a robust business model for multi-use networks.

American Recovery and Reinvestment Act-comparative effectiveness research infrastructure investments: emerging data resources, tools and publications.

The Recovery Act provided a substantial, one-time investment in data infrastructure for comparative effectiveness research (CER). A review of the publications, data, and tools developed as a result of this support has informed understanding of the level of effort undertaken by these projects. Structured search queries, as well as outreach efforts, were conducted to identify and review resources from American Recovery and Reinvestment Act of 2009 CER projects building electronic clinical data infrastructure. The findings from this study provide a spectrum of productivity across a range of topics and settings. A total of 451 manuscripts published in 192 journals, and 141 data resources and tools were identified and address gaps in evidence on priority populations, conditions, and the infrastructure needed to support CER.

Opportunities and challenges for comparative effectiveness research (CER) with Electronic Clinical Data: a perspective from the EDM forum.

INTRODUCTION: The Electronic Data Methods (EDM) Forum brings together perspectives from the Prospective Outcome Systems using Patient-specific Electronic data to Compare Tests and therapies (PROSPECT) studies, the Scalable Distributed Research Networks, and the Enhanced Registries projects. This paper discusses challenges faced by the research teams as part of their efforts to develop electronic clinical data (ECD) infrastructure to support comparative effectiveness research (CER). The findings reflect a set of opportunities for transdisciplinary learning, and will ideally enhance the transparency and generalizability of CER using ECD. METHODS: Findings are based on 6 exploratory site visits conducted under naturalistic inquiry in the spring of 2011. Themes, challenges, and innovations were identified in the visit summaries through coding, keyword searches, and review for complex concepts. RESULTS: : The identified overarching challenges and emerging opportunities include: the substantial level of effort to establish and sustain data sharing partnerships; the importance of understanding the strengths and limitations of clinical informatics tools, platforms, and models that have emerged to enable research with ECD; the need for rigorous methods to assess data validity, quality, and context for multisite studies; and, emerging opportunities to achieve meaningful patient and consumer engagement and work collaboratively with multidisciplinary teams. DISCUSSION: The new infrastructure must evolve to serve a diverse set of potential users and must scale to address a range of CER or patient-centered outcomes research (PCOR) questions. To achieve this aim-to improve the quality, transparency, and reproducibility of CER and PCOR-a high level of collaboration and support is necessary to foster partnership and best practices as part of the EDM Forum.

Building the informatics infrastructure for comparative effectiveness research (CER): a review of the literature.

BACKGROUND: Technological advances in clinical informatics have made large amounts of data accessible and potentially useful for research. As a result, a burgeoning literature addresses efforts to bridge the fields of health services research and biomedical informatics. The Electronic Data Methods Forum review examines peer-reviewed literature at the intersection of comparative effectiveness research and clinical informatics. The authors are specifically interested in characterizing this literature and identifying cross-cutting themes and gaps in the literature. METHODS: A 3-step systematic literature search was conducted, including a structured search of PubMed, manual reviews of articles from selected publication lists, and manual reviews of research activities based on prospective electronic clinical data. Two thousand four hundred thirty-five citations were identified as potentially relevant. Ultimately, a full-text review was performed for 147 peer-reviewed papers. RESULTS: One hundred thirty-two articles were selected for inclusion in the review. Of these, 88 articles are the focus of the discussion in this paper. Three types of articles were identified, including papers that: (1) provide historical context or frameworks for using clinical informatics for research, (2) describe platforms and projects, and (3) discuss issues, challenges, and applications of natural language processing. In addition, 2 cross-cutting themes emerged: the challenges of conducting research in the absence of standardized ontologies and data collection; and unique data governance concerns related to the transfer, storage, deidentification, and access to electronic clinical data. Finally, the authors identified several current gaps on important topics such as the use of clinical informatics for cohort identification, cloud computing, and single point access to research data.

The Electronic Data Methods (EDM) forum for comparative effectiveness research (CER).

BACKGROUND: AcademyHealth convened the Electronic Data Methods (EDM) Forum to collect, synthesize, and share lessons from eleven projects that are building infrastructure and using electronic clinical data for comparative effectiveness research (CER) and patient-centered outcomes research (PCOR). This paper provides a brief review of participating projects and provides a framework of common challenges. METHODS: EDM Forum staff conducted a text review of relevant grant programs' funding opportunity announcements; projects' research plans; and available information on projects' websites. Additional information was obtained from presentations provided by each project; phone calls with project principal investigators, affiliated partners, and staff from the Agency for Healthcare Research and Quality (AHRQ); and six site visits. RESULTS: Projects participating in the EDM Forum are building infrastructure and developing innovative strategies to address a set of methodological, and data and informatics challenges, here identified in a common framework. The eleven networks represent more than 20 states and include a range of partnership models. Projects vary substantially in size, from 11,000 to more than 7.5 million individuals. Nearly all of the AHRQ priority populations and conditions are addressed. DISCUSSION: In partnership with the projects, the EDM Forum is focused on identifying and sharing lessons learned to advance the national dialogue on the use of electronic clinical data to conduct CER and PCOR. These efforts have the shared goal of addressing challenges in traditional research studies and data sources, and aim to build infrastructure and generate evidence to support a learning health care system that can improve patient outcomes.

A tall order on a tight timeframe: stakeholder perspectives on comparative effectiveness research using electronic clinical data.

UNLABELLED: BACKGROUND & SIGNIFICANCE: The AcademyHealth Electronic Data Methods Forum aims to advance the national dialogue on the use of electronic clinical data (ECD) for comparative effectiveness research (CER), patient-centered outcomes research, and quality improvement by facilitating exchange and collaboration among eleven research projects and external stakeholders. AcademyHealth conducted a mixed-method needs assessment with the Electronic Data Methods Forum's key stakeholders to assess: stakeholder views on developing new infrastructure for CER using ECD; current gaps in knowledge with respect to CER; and expectations for a learning health system. METHODS: AcademyHealth conducted 50 stakeholder interviews between August 2011 and November 2011 with participants from the following seven stakeholder groups: government, business/payer, industry, healthcare delivery, patient/consumer, nonprofit/policy and research. With input from key collaborators, AcademyHealth designed a semi-structured interview guide and a short survey. Reviewers used the qualitative data analysis software NVivo to code the transcripts and to identify and manage complex concepts. Quantitative data from the questionnaire has been integrated with the final analysis as relevant. RESULTS: The analysis of recurring concepts in the interviews focus on five central themes: stakeholders have substantial expectations for CER using ECD, both with respect to addressing the limitations of traditional research studies, and generating meaningful evidence for decision-making and improving patient outcomes; stakeholders are aware of many challenges related to implementing CER with ECD, including the need to develop appropriate governance, assess and manage data quality, and develop methods to address confounding in observational data; stakeholders continue to struggle to define 'patient-centeredness' in CER using ECD, adding complexity to attaining this goal; stakeholders express that improving translation and dissemination of CER, and how research can be 'useful' at the point of care, can help mitigate negative perceptions of the CER 'brand'; and stakeholders perceive a need for a substantial 'culture shift' to facilitate collaborative science and new ways of conducting biomedical and outcomes research. Many stakeholders proposed approaches or solutions they felt might address the challenges identified.