A Semiparametric Inverse Reinforcement Learning Approach to Characterize Decision Making for Mental Disorders.

Major depressive disorder (MDD) is one of the leading causes of disability-adjusted life years. Emerging evidence indicates the presence of reward processing abnormalities in MDD. An important scientific question is whether the abnormalities are due to reduced sensitivity to received rewards or reduced learning ability. Motivated by the probabilistic reward task (PRT) experiment in the EMBARC study, we propose a semiparametric inverse reinforcement learning (RL) approach to characterize the reward-based decision-making of MDD patients. The model assumes that a subject's decision-making process is updated based on a reward prediction error weighted by the subject-specific learning rate. To account for the fact that one favors a decision leading to a potentially high reward, but this decision process is not necessarily linear, we model reward sensitivity with a non-decreasing and nonlinear function. For inference, we estimate the latter via approximation by I-splines and then maximize the joint conditional log-likelihood. We show that the resulting estimators are consistent and asymptotically normal. Through extensive simulation studies, we demonstrate that under different reward-generating distributions, the semiparametric inverse RL outperforms the parametric inverse RL. We apply the proposed method to EMBARC and find that MDD and control groups have similar learning rates but different reward sensitivity functions. There is strong statistical evidence that reward sensitivity functions have nonlinear forms. Using additional brain imaging data in the same study, we find that both reward sensitivity and learning rate are associated with brain activities in the negative affect circuitry under an emotional conflict task.

Learning optimal biomarker-guided treatment policy for chronic disorders.

Electroencephalogram (EEG) provides noninvasive measures of brain activity and is found to be valuable for the diagnosis of some chronic disorders. Specifically, pre-treatment EEG signals in the alpha and theta frequency bands have demonstrated some association with antidepressant response, which is well-known to have a low response rate. We aim to design an integrated pipeline that improves the response rate of patients with major depressive disorder by developing a treatment policy guided by the resting state pre-treatment EEG recordings and other treatment effects modifiers. First, we design an innovative automatic site-specific EEG preprocessing pipeline to extract features with stronger signals than raw data. We then estimate the conditional average treatment effect (CATE) using causal forests and use a doubly robust technique to improve efficiency in the estimation of the average treatment effect. We present evidence of heterogeneity in the treatment effect and the modifying power of the EEG features, as well as a significant average treatment effect, a result that cannot be obtained with conventional methods. Finally, we employ an efficient policy learning algorithm to learn an optimal depth-2 treatment assignment decision tree and compare its performance with Q-Learning and outcome-weighted learning via simulation studies and an application to a large multi-site, double-blind, randomized controlled clinical trial, EMBARC.

Identifying temporal pathways using biomarkers in the presence of latent non-Gaussian components.

Time-series data collected from a network of random variables are useful for identifying temporal pathways among the network nodes. Observed measurements may contain multiple sources of signals and noises, including Gaussian signals of interest and non-Gaussian noises, including artifacts, structured noise, and other unobserved factors (eg, genetic risk factors, disease susceptibility). Existing methods, including vector autoregression (VAR) and dynamic causal modeling do not account for unobserved non-Gaussian components. Furthermore, existing methods cannot effectively distinguish contemporaneous relationships from temporal relations. In this work, we propose a novel method to identify latent temporal pathways using time-series biomarker data collected from multiple subjects. The model adjusts for the non-Gaussian components and separates the temporal network from the contemporaneous network. Specifically, an independent component analysis (ICA) is used to extract the unobserved non-Gaussian components, and residuals are used to estimate the contemporaneous and temporal networks among the node variables based on method of moments. The algorithm is fast and can easily scale up. We derive the identifiability and the asymptotic properties of the temporal and contemporaneous networks. We demonstrate superior performance of our method by extensive simulations and an application to a study of attention-deficit/hyperactivity disorder (ADHD), where we analyze the temporal relationships between brain regional biomarkers. We find that temporal network edges were across different brain regions, while most contemporaneous network edges were bilateral between the same regions and belong to a subset of the functional connectivity network.

Clinical Trials Studying Suicide Risk Reduction: Who is Excluded From Participation.

OBJECTIVE: The use of exclusion criteria in clinical trials can cause research participants to differ markedly from clinical populations, which negatively impacts generalizability of results. This study identifies and quantifies common and recurring exclusion criteria in clinical trials studying suicide risk reduction, and estimates their impact on eligibility among a clinical sample of adults in an emergency department with high suicide risk. METHOD: Recent trials were identified by searching PubMed (terms suicide, efficacy, effectiveness, limited to clinical trials in prior 5 years). Common exclusion criteria were identified using Qualitative Content Analysis. A retrospective chart review examined a one-month sample of all adults receiving psychiatric evaluation in a large urban academic emergency department. RESULTS: The search yielded 27 unique clinical trials studying suicide risk reduction as a primary or secondary outcome. After research fundamentals (e.g. informed consent, language fluency), the most common exclusion criteria involved psychosis (77.8%), cognitive problems (66.7%), and substance use (63.0%). In the clinical sample of adults with high suicide risk (N = 232), psychosis exclusions would exclude 53.0% of patients and substance use exclusions would exclude 67.2% of patients. Overall, 5.6% of emergency psychiatry patients would be eligible for clinical trials that use common exclusion criteria. CONCLUSIONS: Recent clinical trials studying suicide risk reduction have low generalizability to emergency psychiatry patients with high suicide risk. Trials enrolling persons with psychosis and substance use in particular are needed to improve generalizability to this clinical population.

Exclusion criteria limit who can enroll in trials studying suicide risk reduction.Trials most frequently exclude psychosis, cognitive problems, and substance use.Trials have poor generalizability to emergency psychiatry patients.

Hydrogel Alginate Considerations for Improved 3D Matrix Stability and Cell Graft Viability and Function in Studying Type 1 Diabetes In Vitro.

Biomedical devices such as islet-encapsulating systems are used for treatment of type 1 diabetes (T1D). Despite recent strides in preventing biomaterial fibrosis, challenges remain for biomaterial scaffolds due to limitations on cells contained within. The study demonstrates that proliferation and function of insulinoma (INS-1) cells as well as pancreatic rat islets may be improved in alginate hydrogels with optimized gel%, crosslinking, and stiffness. Quantitative polymerase chain reaction (qPCR)-based graft phenotyping of encapsulated INS-1 cells and pancreatic islets identified a hydrogel stiffness range between 600 and 1000 Pa that improved insulin Ins and Pdx1 gene expression as well as glucose-sensitive insulin-secretion. Barium chloride (BaCl2 ) crosslinking time is also optimized due to toxicity of extended exposure. Despite possible benefits to cell viability, calcium chloride (CaCl2 )-crosslinked hydrogels exhibited a sharp storage modulus loss in vitro. Despite improved stability, BaCl2 -crosslinked hydrogels also exhibited stiffness losses over the same timeframe. It is believed that this is due to ion exchange with other species in culture media, as hydrogels incubated in dIH2 O exhibited significantly improved stability. To maintain cell viability and function while increasing 3D matrix stability, a range of useful media:dIH2 O dilution ratios for use are identified. Such findings have importance to carry out characterization and optimization of cell microphysiological systems with high fidelity in vitro.

Change in food choice during acute treatment and the effect on longer-term outcome in patients with anorexia nervosa.

BACKGROUND: Restriction of food intake is a central pathological feature of anorexia nervosa (AN). Maladaptive eating behavior and, specifically, limited intake of calorie-dense foods are resistant to change and contribute to poor long-term outcomes. This study is a preliminary examination of whether change in food choices during inpatient treatment is related to longer-term clinical course. METHODS: Individuals with AN completed a computerized Food Choice Task at the beginning and end of inpatient treatment to determine changes in high-fat and self-controlled food choices. Linear regression and longitudinal analyses tested whether change in task behavior predicted short-term outcome (body mass index [BMI] at discharge) and longer-term outcome (BMI and eating disorder psychopathology). RESULTS: Among 88 patients with AN, BMI improved significantly with hospital treatment (p < 0.001), but Food Choice Task outcomes did not change significantly. Change in high-fat and self-controlled choices was not associated with BMI at discharge (r = 0.13, p = 0.22 and r = 0.10, p = 0.39, respectively). An increase in the proportion of high-fat foods selected (ß = 0.91, p = 0.02) and a decrease in the use of self-control (ß = -1.50, p = 0.001) predicted less decline in BMI over 3 years after discharge. CONCLUSIONS: Short-term treatment is associated with improvement in BMI but with no significant change, on average, in choices made in a task known to predict actual eating. However, the degree to which individuals increased high-fat choices during treatment and decreased the use of self-control over food choice were associated with reduced weight loss over the following 3 years, underscoring the need to focus on changing eating behavior in treatment of AN.

Eligibility of emergency psychiatry patients for clinical trials studying depression.

BACKGROUND: Differences often exist between people with depression who are eligible for clinical trials and those seen in clinical practice. The impact of exclusion criteria on eligibility has been previously reported for inpatients and outpatients, but has not been assessed for emergency psychiatry patients; a group that overlaps with inpatients and outpatients but also has important distinctives. Understanding the frequencies of commonly used exclusion criteria in this population could inform interpretation of existing data (generalizability) and highlight opportunities/needs for future trials. METHODS: We reviewed 67 clinical trials studying depression using Qualitative Content Analysis to identify common and recurring exclusion criteria. We examined the frequency of these exclusion criteria among a clinical sample of emergency psychiatry patients. RESULTS: Most clinical trials had exclusions for basic research requirements, age, symptom severity, psychosis, and substance use. Applying 9 commonly used exclusion criteria to the clinical population resulted in a 3.3 % eligibility rate (95 % CI 1.2 %-7.0 %). Exclusions for psychosis (85.1 % of trials), substance use (83.6 % of trials), and suicide risk (65.7 % of trials) would likely exclude 93 % of emergency psychiatry patients. The prevalence of psychosis, substance use, and suicide risk was much higher among emergency psychiatry patients than among previously studied populations. LIMITATIONS: Some eligibility criteria could not be measured. The Qualitative Content Analysis consolidated similar exclusion criteria, losing potentially important nuances in wordings. CONCLUSIONS: Exclusion criteria commonly used in contemporary clinical trials of depression limit generalizability to emergency psychiatry patients, due in large part to exclusions for psychosis, substance use, and suicide risk.

Temporal Generative Models for Learning Heterogeneous Group Dynamics of Ecological Momentary Data.

One of the goals of precision psychiatry is to characterize mental disorders in an individualized manner, taking into account the underlying dynamic processes. Recent advances in mobile technologies have enabled the collection of Ecological Momentary Assessments (EMAs) that capture multiple responses in real-time at high frequency. However, EMA data is often multi-dimensional, correlated, and hierarchical. Mixed-effects models are commonly used but may require restrictive assumptions about the fixed and random effects and the correlation structure. The Recurrent Temporal Restricted Boltzmann Machine (RTRBM) is a generative neural network that can be used to model temporal data, but most existing RTRBM approaches do not account for the potential heterogeneity of group dynamics within a population based on available covariates. In this paper, we propose a new temporal generative model, the Heterogeneous-Dynamics Restricted Boltzmann Machine (HDRBM), to learn the heterogeneous group dynamics and demonstrate the effectiveness of this approach on simulated and real-world EMA data sets. We show that by incorporating covariates, HDRBM can improve accuracy and interpretability, explore the underlying drivers of the group dynamics of participants, and serve as a generative model for EMA studies.

A general framework of nonparametric feature selection in high-dimensional data.

Nonparametric feature selection for high-dimensional data is an important and challenging problem in the fields of statistics and machine learning. Most of the existing methods for feature selection focus on parametric or additive models which may suffer from model misspecification. In this paper, we propose a new framework to perform nonparametric feature selection for both regression and classification problems. Under this framework, we learn prediction functions through empirical risk minimization over a reproducing kernel Hilbert space. The space is generated by a novel tensor product kernel, which depends on a set of parameters that determines the importance of the features. Computationally, we minimize the empirical risk with a penalty to estimate the prediction and kernel parameters simultaneously. The solution can be obtained by iteratively solving convex optimization problems. We study the theoretical property of the kernel feature space and prove the oracle selection property and Fisher consistency of our proposed method. Finally, we demonstrate the superior performance of our approach compared to existing methods via extensive simulation studies and applications to two real studies.

A latent state space model for estimating brain dynamics from electroencephalogram (EEG) data.

Modern neuroimaging technologies have substantially advanced the measurement of brain activity. Electroencephalogram (EEG) as a noninvasive neuroimaging technique measures changes in electrical voltage on the scalp induced by brain cortical activity. With its high temporal resolution, EEG has emerged as an increasingly useful tool to study brain connectivity. Challenges with modeling EEG signals of complex brain activity include interactions among unknown sources, low signal-to-noise ratio, and substantial between-subject heterogeneity. In this work, we propose a state space model that jointly analyzes multichannel EEG signals and learns dynamics of different sources corresponding to brain cortical activity. Our model borrows strength from spatially correlated measurements and uses low-dimensional latent states to explain all observed channels. The model can account for patient heterogeneity and quantify the effect of a subject's covariates on the latent space. The EM algorithm, Kalman filtering, and bootstrap resampling are used to fit the state space model and provide comparisons between patient diagnostic groups. We apply the developed approach to a case-control study of alcoholism and reveal significant attenuation of brain activity in response to visual stimuli in alcoholic subjects compared to healthy controls.

Estimating individualized treatment rules for multicategory type 2 diabetes treatments using electronic health records.

In this article, we propose a general framework to learn optimal treatment rules for type 2 diabetes (T2D) patients using electronic health records (EHRs). We first propose a joint modeling approach to characterize patient's pretreatment conditions using longitudinal markers from EHRs. The estimation accounts for informative measurement times using inverse-intensity weighting methods. The predicted latent processes in the joint model are used to divide patients into a finite of subgroups and, within each group, patients share similar health profiles in EHRs. Within each patient group, we estimate optimal individualized treatment rules by extending a matched learning method to handle multicategory treatments using a one-versus-one approach. Each matched learning for two treatments is implemented by a weighted support vector machine with matched pairs of patients. We apply our method to estimate optimal treatment rules for T2D patients in a large sample of EHRs from the Ohio State University Wexner Medical Center. We demonstrate the utility of our method to select the optimal treatments from four classes of drugs and achieve a better control of glycated hemoglobin than any one-size-fits-all rules.

Mixed-Response State-Space Model for Analyzing Multi-Dimensional Digital Phenotypes.

Digital technologies (e.g., mobile phones) can be used to obtain objective, frequent, and real-world digital phenotypes from individuals. However, modeling these data poses substantial challenges since observational data are subject to confounding and various sources of variabilities. For example, signals on patients' underlying health status and treatment effects are mixed with variation due to the living environment and measurement noises. The digital phenotype data thus shows extensive variabilities between- and within-patient as well as across different health domains (e.g., motor, cognitive, and speaking). Motivated by a mobile health study of Parkinson's disease (PD), we develop a mixed-response state-space (MRSS) model to jointly capture multi-dimensional, multi-modal digital phenotypes and their measurement processes by a finite number of latent state time series. These latent states reflect the dynamic health status and personalized time-varying treatment effects and can be used to adjust for informative measurements. For computation, we use the Kalman filter for Gaussian phenotypes and importance sampling with Laplace approximation for non-Gaussian phenotypes. We conduct comprehensive simulation studies and demonstrate the advantage of MRSS in modeling a mobile health study that remotely collects real-time digital phenotypes from PD patients.

Multilayer Exponential Family Factor models for integrative analysis and learning disease progression.

Current diagnosis of neurological disorders often relies on late-stage clinical symptoms, which poses barriers to developing effective interventions at the premanifest stage. Recent research suggests that biomarkers and subtle changes in clinical markers may occur in a time-ordered fashion and can be used as indicators of early disease. In this article, we tackle the challenges to leverage multidomain markers to learn early disease progression of neurological disorders. We propose to integrate heterogeneous types of measures from multiple domains (e.g., discrete clinical symptoms, ordinal cognitive markers, continuous neuroimaging, and blood biomarkers) using a hierarchical Multilayer Exponential Family Factor (MEFF) model, where the observations follow exponential family distributions with lower-dimensional latent factors. The latent factors are decomposed into shared factors across multiple domains and domain-specific factors, where the shared factors provide robust information to perform extensive phenotyping and partition patients into clinically meaningful and biologically homogeneous subgroups. Domain-specific factors capture remaining unique variations for each domain. The MEFF model also captures nonlinear trajectory of disease progression and orders critical events of neurodegeneration measured by each marker. To overcome computational challenges, we fit our model by approximate inference techniques for large-scale data. We apply the developed method to Parkinson's Progression Markers Initiative data to integrate biological, clinical, and cognitive markers arising from heterogeneous distributions. The model learns lower-dimensional representations of Parkinson's disease (PD) and the temporal ordering of the neurodegeneration of PD.

Evaluating effectiveness of public health intervention strategies for mitigating COVID-19 pandemic.

Coronavirus disease 2019 (COVID-19) pandemic is an unprecedented global public health challenge. In the United States (US), state governments have implemented various non-pharmaceutical interventions (NPIs), such as physical distance closure (lockdown), stay-at-home order, mandatory facial mask in public in response to the rapid spread of COVID-19. To evaluate the effectiveness of these NPIs, we propose a nested case-control design with propensity score weighting under the quasi-experiment framework to estimate the average intervention effect on disease transmission across states. We further develop a method to test for factors that moderate intervention effect to assist precision public health intervention. Our method takes account of the underlying dynamics of disease transmission and balance state-level pre-intervention characteristics. We prove that our estimator provides causal intervention effect under assumptions. We apply this method to analyze US COVID-19 incidence cases to estimate the effects of six interventions. We show that lockdown has the largest effect on reducing transmission and reopening bars significantly increase transmission. States with a higher percentage of non-White population are at greater risk of increased R t $$ {R}_t $$ associated with reopening bars.

Self-matched learning to construct treatment decision rules from electronic health records.

Electronic health records (EHRs) collected from large-scale health systems provide rich subject-specific information on a broad patient population at a lower cost compared to randomized controlled trials. Thus, EHRs may serve as a complementary resource to provide real-world data to construct individualized treatment rules (ITRs) and achieve precision medicine. However, in the absence of randomization, inferring treatment rules from EHR data may suffer from unmeasured confounding. In this article, we propose a self-matched learning method inspired by the self-controlled case series (SCCS) design to mitigate this challenge. We alleviate unmeasured time-invariant confounding between patients by matching different periods of treatments within the same patient (self-controlled matching) to infer the optimal ITRs. The proposed method constructs a within-subject matched value function for optimizing ITRs and bears similarity to the SCCS design. We examine assumptions that ensure Fisher consistency, and show that our method requires weaker assumptions on unmeasured confounding than alternative methods. Through extensive simulation studies, we demonstrate that self-matched learning has comparable performance to other existing methods when there are no unmeasured confounders, but performs markedly better when unobserved time-invariant confounders are present, which is often the case for EHRs. Sensitivity analyses show that the proposed method is robust under different scenarios. Finally, we apply self-matched learning to estimate the optimal ITRs from type 2 diabetes patient EHRs, which shows our estimated decision rules lead to greater advantages in reducing patients' diabetes-related complications.

Optimizing relapse prevention and changing habits (REACH+) in anorexia nervosa.

INTRODUCTION: Relapse rates in anorexia nervosa (AN) are high, even after full weight restoration. This study aims to develop a relapse prevention treatment that specifically addresses persistent maladaptive behaviors (habits). Relapse Prevention and Changing Habits (REACH+) aims to support patients in developing routines that promote weight maintenance, encourage health, and challenge habits that perpetuate illness. The clinical trial design uses the Multiphase Optimization STrategy (MOST) framework to efficiently identify which components of treatment contribute to positive outcomes. METHODS: Participants will be 60 adults with AN who have achieved weight restoration in an inpatient setting. Treatment will consist of 6 months of outpatient telehealth sessions. REACH+ consists of behavior, cognitive, and motivation components, as well as food monitoring and a skill consolidation phase. A specialized online platform extends therapy between sessions. Participants will be randomly assigned to different versions of each component in a fractional factorial design. Outcomes will focus on maintenance of remission, measured by rate of weight loss and end-of-trial status. Interventions that contribute to remission will be included in an optimized treatment package, suitable for a large-scale clinical trial of relapse prevention in AN.

Conditional Gaussian graphical model for estimating personalized disease symptom networks.

The co-occurrence of symptoms may result from the direct interactions between these symptoms and the symptoms can be treated as a system. In addition, subject-specific risk factors (eg, genetic variants, age) can also exert external influence on the system. In this work, we develop a covariate-dependent conditional Gaussian graphical model to obtain personalized symptom networks. The strengths of network connections are modeled as a function of covariates to capture the heterogeneity among individuals and subgroups of individuals. We assess the performance of our proposed method by simulation studies and an application to a large natural history study of Huntington's disease to investigate the networks of symptoms in multiple clinical domains (motor, cognitive, psychiatric) and identify important brain imaging biomarkers that are associated with the connections. We show that the symptoms in the same clinical domain interact more often with each other than cross domains and the psychiatric subnetwork is the densest network. We validate the findings using the subjects' symptom measurements at follow-up visits.

Semiparametric regression analysis of bivariate censored events in a family study of Alzheimer's disease.

Assessing disease comorbidity patterns in families represents the first step in gene mapping for diseases and is central to the practice of precision medicine. One way to evaluate the relative contributions of genetic risk factor and environmental determinants of a complex trait (e.g., Alzheimer's disease [AD]) and its comorbidities (e.g., cardiovascular diseases [CVD]) is through familial studies, where an initial cohort of subjects are recruited, genotyped for specific loci, and interviewed to provide extensive disease history in family members. Because of the retrospective nature of obtaining disease phenotypes in family members, the exact time of disease onset may not be available such that current status data or interval-censored data are observed. All existing methods for analyzing these family study data assume single event subject to right-censoring so are not applicable. In this article, we propose a semiparametric regression model for the family history data that assumes a family-specific random effect and individual random effects to account for the dependence due to shared environmental exposures and unobserved genetic relatedness, respectively. To incorporate multiple events, we jointly model the onset of the primary disease of interest and a secondary disease outcome that is subject to interval-censoring. We propose nonparametric maximum likelihood estimation and develop a stable Expectation-Maximization (EM) algorithm for computation. We establish the asymptotic properties of the resulting estimators and examine the performance of the proposed methods through simulation studies. Our application to a real world study reveals that the main contribution of comorbidity between AD and CVD is due to genetic factors instead of environmental factors.

Learning Individualized Treatment Rules for Multiple-Domain Latent Outcomes.

For many mental disorders, latent mental status from multiple-domain psychological or clinical symptoms may perform as a better characterization of the underlying disorder status than a simple summary score of the symptoms, and they may also serve as more reliable and representative features to differentiate treatment responses. Therefore, in order to address the complexity and heterogeneity of treatment responses for mental disorders, we provide a new paradigm for learning optimal individualized treatment rules (ITRs) by modeling patients' latent mental status. We first learn the multi-domain latent states at baseline from the observed symptoms under a restricted Boltzmann machine (RBM) model, through which patients' heterogeneous symptoms are represented using an economical number of latent variables and yet remains flexible. We then optimize a value function defined by the latent states after treatment by exploiting a transformation of the observed symptoms based on the RBM without modeling the relationship between the latent mental states before and after treatment. The optimal treatment rules are derived using a weighted large margin classifier. We derive the convergence rate of the proposed estimator under the latent models. Simulation studies are conducted to test the performance of the proposed method. Finally, we apply the developed method to real world studies and we demonstrate the utility and advantage of our method in tailoring treatments for patients with major depression, and identify patient subgroups informative for treatment recommendations.