Collaborative learning framework for online stakeholder engagement.

BACKGROUND: Public and stakeholder engagement can improve the quality of both research and policy decision making. However, such engagement poses significant methodological challenges in terms of collecting and analysing input from large, diverse groups. OBJECTIVE: To explain how online approaches can facilitate iterative stakeholder engagement, to describe how input from large and diverse stakeholder groups can be analysed and to propose a collaborative learning framework (CLF) to interpret stakeholder engagement results. METHODS: We use 'A National Conversation on Reducing the Burden of Suicide in the United States' as a case study of online stakeholder engagement and employ a Bayesian data modelling approach to develop a CLF. RESULTS: Our data modelling results identified six distinct stakeholder clusters that varied in the degree of individual articulation and group agreement and exhibited one of the three learning styles: learning towards consensus, learning by contrast and groupthink. Learning by contrast was the most common, or dominant, learning style in this study. CONCLUSION: Study results were used to develop a CLF, which helps explore multitude of stakeholder perspectives; identifies clusters of participants with similar shifts in beliefs; offers an empirically derived indicator of engagement quality; and helps determine the dominant learning style. The ability to detect learning by contrast helps illustrate differences in stakeholder perspectives, which may help policymakers, including Patient-Centered Outcomes Research Institute, make better decisions by soliciting and incorporating input from patients, caregivers, health-care providers and researchers. Study results have important implications for soliciting and incorporating input from stakeholders with different interests and perspectives.

Bayesian Semi- and Non-parametric Models for Longitudinal Data with Multiple Membership Effects in R.

We introduce growcurves for R that performs analysis of repeated measures multiple membership (MM) data. This data structure arises in studies under which an intervention is delivered to each subject through the subject's participation in a set of multiple elements that characterize the intervention. In our motivating study design under which subjects receive a group cognitive behavioral therapy (CBT) treatment, an element is a group CBT session and each subject attends multiple sessions that, together, comprise the treatment. The sets of elements, or group CBT sessions, attended by subjects will partly overlap with some of those from other subjects to induce a dependence in their responses. The growcurves package offers two alternative sets of hierarchical models: 1. Separate terms are specified for multivariate subject and MM element random effects, where the subject effects are modeled under a Dirichlet process prior to produce a semi-parametric construction; 2. A single term is employed to model joint subject-by-MM effects. A fully non-parametric dependent Dirichlet process formulation allows exploration of differences in subject responses across different MM elements. This model allows for borrowing information among subjects who express similar longitudinal trajectories for flexible estimation. growcurves deploys "estimation" functions to perform posterior sampling under a suite of prior options. An accompanying set of "plot" functions allow the user to readily extract by-subject growth curves. The design approach intends to anticipate inferential goals with tools that fully extract information from repeated measures data. Computational efficiency is achieved by performing the sampling for estimation functions using compiled C++.

Bayesian hierarchical multivariate formulation with factor analysis for nested ordinal data.

This article devises a Bayesian multivariate formulation for analysis of ordinal data that records teacher classroom performance along multiple dimensions to assess aspects characterizing good instruction. Study designs for scoring teachers seek to measure instructional performance over multiple classroom measurement event sessions at varied occasions using disjoint intervals within each session and employment of multiple ratings on intervals scored by different raters; a design which instantiates a nesting structure with each level contributing a source of variation in recorded scores. We generally possess little a priori knowledge of the existence or form of a sparse generating structure for the multivariate dimensions at any level in the nesting that would permit collapsing over dimensions as is done under univariate modeling. Our approach composes a Bayesian data augmentation scheme that introduces a latent continuous multivariate response linked to the observed ordinal scores with the latent response mean constructed as an additive multivariate decomposition of nested level means that permits the extraction of de-noised continuous teacher-level scores and the associated correlation matrix. A semi-parametric extension facilitates inference for teacher-level dependence among the dimensions of classroom performance under multi-modality induced by sub-groupings of rater perspectives. We next replace an inverse Wishart prior specified for the teacher covariance matrix over dimensions of instruction with a factor analytic structure to allow the simultaneous assessment of an underlying sparse generating structure. Our formulation for Bayesian factor analysis employs parameter expansion with an accompanying post-processing sign re-labeling step of factor loadings that together reduce posterior correlations among sampled parameters to improve parameter mixing in our Markov chain Monte Carlo (MCMC) scheme. We evaluate the performance of our formulation on simulated data and make an application for the assessment of the teacher covariance structure with a dataset derived from a study of middle and high school algebra teachers.

Bayesian Hierarchical Semiparametric Modelling of Longitudinal Post-treatment Outcomes from Open Enrolment Therapy Groups.

There are several challenges to testing the effectiveness of group therapy-based interventions in alcohol and other drug use (AOD) treatment settings. Enrollment into AOD therapy groups typically occurs on an open (rolling) basis. Changes in therapy group membership induce a complex correlation structure among client outcomes, with relatively small numbers of clients attending each therapy group session. Primary outcomes are measured post-treatment, so each datum reflects the effect of all sessions attended by a client. The number of post-treatment outcomes assessments is typically very limited. The first feature of our modeling approach relaxes the assumption of independent random effects in the standard multiple membership model by employing conditional autoregression (CAR) to model correlation in random therapy group session effects associated with clients' attendance of common group therapy sessions. A second feature specifies a longitudinal growth model under which the posterior distribution of client-specific random effects, or growth parameters, is modeled non-parametrically. The Dirichlet process prior helps to overcome limitations of standard parametric growth models given limited numbers of longitudinal assessments. We motivate and illustrate our approach with a data set from a study of group cognitive behavioral therapy to reduce depressive symptoms among residential AOD treatment clients.

Bayesian Non-Parametric Hierarchical Modeling for Multiple Membership Data in Grouped Attendance Interventions.

We develop a dependent Dirichlet process (DDP) model for repeated measures multiple membership (MM) data. This data structure arises in studies under which an intervention is delivered to each client through a sequence of elements which overlap with those of other clients on different occasions. Our interest concentrates on study designs for which the overlaps of sequences occur for clients who receive an intervention in a shared or grouped fashion whose memberships may change over multiple treatment events. Our motivating application focuses on evaluation of the effectiveness of a group therapy intervention with treatment delivered through a sequence of cognitive behavioral therapy session blocks, called modules. An open-enrollment protocol permits entry of clients at the beginning of any new module in a manner that may produce unique MM sequences across clients. We begin with a model that composes an addition of client and multiple membership module random effect terms, which are assumed independent. Our MM DDP model relaxes the assumption of conditionally independent client and module random effects by specifying a collection of random distributions for the client effect parameters that are indexed by the unique set of module attendances. We demonstrate how this construction facilitates examining heterogeneity in the relative effectiveness of group therapy modules over repeated measurement occasions.

The frequency and cost of treatment perceived to be futile in critical care.

IMPORTANCE: Physicians often perceive as futile intensive care interventions that prolong life without achieving an effect that the patient can appreciate as a benefit. The prevalence and cost of critical care perceived to be futile have not been prospectively quantified. OBJECTIVE: To quantify the prevalence and cost of treatment perceived to be futile in adult critical care. DESIGN, SETTING, AND PARTICIPANTS: To develop a common definition of futile care, we convened a focus group of clinicians who care for critically ill patients. On a daily basis for 3 months, we surveyed critical care specialists in 5 intensive care units (ICUs) at an academic health care system to identify patients whom the physicians believed were receiving futile treatment. Using a multivariate model, we identified patient and clinician characteristics associated with patients perceived to be receiving futile treatment. We estimated the total cost of futile treatment by summing the charges of each day of receiving perceived futile treatment and converting to costs. MAIN OUTCOME AND MEASURE: Prevalence of patients perceived to be receiving futile treatment. RESULTS: During a 3-month period, there were 6916 assessments by 36 critical care specialists of 1136 patients. Of these patients, 904 (80%) were never perceived to be receiving futile treatment, 98 (8.6%) were perceived as receiving probably futile treatment, 123 (11%) were perceived as receiving futile treatment, and 11 (1%) were perceived as receiving futile treatment only on the day they transitioned to palliative care. The patients with futile treatment assessments received 464 days of treatment perceived to be futile in critical care (range, 1-58 days), accounting for 6.7% of all assessed patient days in the 5 ICUs studied. Eighty-four of the 123 patients perceived as receiving futile treatment died before hospital discharge and 20 within 6 months of ICU care (6-month mortality rate of 85%), with survivors remaining in severely compromised health states. The cost of futile treatment in critical care was estimated at $2.6 million. CONCLUSIONS AND RELEVANCE: In 1 health system, treatment in critical care that is perceived to be futile is common and the cost is substantial.