Sustainability of Adolescent Screening and Brief Intervention Services in Primary Care After Removal of Implementation Supports.

OBJECTIVE: Although many health care organizations have sought to increase the integration of substance use services into clinical practice, such practice changes can prove difficult to sustain. METHOD: Seven primary care clinics participated in an implementation study of screening and brief intervention (BI) services for adolescent patients (ages 12-17). All sites delivered screening and brief advice (BA) for low-risk use using a uniform protocol. Clinics were randomized to deliver BI using generalist (provider-delivered) or specialist (behavioral health clinician-delivered) models. Implementation was facilitated by multiple supporting activities (e.g., trainings, local "champion," electronic health record [EHR] integration of screening and documentation, individualized feedback, project-specific materials, etc.). Data on the penetration of screening, BA, and BI delivery (N = 14,486 adolescent patient visits) were abstracted from the EHR for the 20-month implementation phase and a 12-month sustainability phase (during which implementation supports were removed). RESULTS: Penetration of screening continued to slowly increase across the implementation-to-sustainability phases (62% vs. 70%; p = .04). Although uptake during implementation was low for BA (29%) and BI (22%), there was no significant decrease in service provision during the sustainability phase. Although overall delivery of BI was significantly higher in generalist compared with specialist sites (p < .001), sustainability did not differ by generalist versus specialist conditions. There were considerable differences in penetration across clinic sites. CONCLUSIONS: Clinics sustained a high level of substance use screening. Uptake of intervention services was low but did not decrease further following the cessation of implementation supports. This study illustrates the challenges of successfully implementing and sustaining substance use services in adolescent primary care.

Costs and Implementation Effectiveness of Generalist Versus Specialist Models for Adolescent Screening and Brief Intervention in Primary Care.

OBJECTIVE: This study analyzed the marginal service and program costs, and conducted a cost-effectiveness analysis (CEA) of two models of implementation of adolescent substance screening, brief intervention, and referral to treatment (SBIRT). METHOD: SBIRT was implemented at seven clinics in a multisite, cluster-randomized trial, through a Specialist model (behavioral health counselor-delivered brief intervention), and a Generalist model (primary care provider-delivered brief intervention). The CEA calculated marginal costs using an activity-based costing methodology for direct SBIRT services, and effectiveness was measured by the proportion of brief interventions delivered among patients who screened positive for alcohol, tobacco, or other drugs. Site-level program costs comprised start-up and maintenance (training and technical assistance). Costs were estimated in 2017 U.S. dollars. RESULTS: The marginal cost of SBIRT per patient with a positive screen for brief intervention was $6.72 in the Specialist model and $6.05 in the Generalist model. Implementation effectiveness was 7.2% (SE = 2.9%) in the Specialist model and 37.7% (SE = 5.6%) in the Generalist model. The program costs to provide SBIRT for 1 year per site were $13,548 for the Specialist site and $12,081 for the Generalist. CONCLUSIONS: The Generalist model was more effective in implementing brief intervention and less expensive than the Specialist model. Results were robust to sensitivity analysis. Brief intervention delivered by primary care providers rather than by handoff to a behavioral health counselor may ensure greater penetration and a lower cost of these services in primary care settings.

Adolescent SBIRT implementation: Generalist vs. Specialist models of service delivery in primary care.

BACKGROUND: Drug, alcohol, and tobacco use among adolescents pose significant short- and long-term health consequences and are associated with more severe use as adults. Screening, brief intervention, and referral to treatment in primary care settings has the potential to deliver preventive interventions to a diverse range of adolescents, but optimal implementation of these services needs to be determined. The purpose of this study was to compare implementation of two different SBIRT service delivery models in primary care settings. METHODS: This cluster-randomized trial assigned 7 primary care clinics of a federally qualified health center to implement brief interventions (BI) using a Generalist model (4 sites), in which BIs were delivered by the primary care provider (PCP), or a Specialist model (3 sites), in which BIs were delivered by a behavioral health counselor (BHC) for adolescent patients ages 12-17 years. Implementation was tracked through the clinic's electronic health record, spanning 9639 clinic visits over 20 months. Multilevel logistic regression modeling was used to compare Generalist and Specialist strategies on penetration of BI for patients scoring ≥2 on the CRAFFT substance use screen, delivered by the PCP in the Generalist sites, and via warm hand-off to a BHC in the Specialist sites. RESULTS: Approximately 62% of adolescent patient visits were screened with the CRAFFT (with <4% screening positive with a CRAFFT score ≥ 2). The Generalist Condition had significantly higher self-reported penetration of BI delivery than the Specialist Condition (38% vs. 8%; Adjusted Odds Ratio = 6.53; p = .005). DISCUSSION: Despite having co-located behavioral health services at all sites, a Specialist approach to providing BI was less effectively implemented than a Generalist approach in this FQHC. BI delivered by PCPs rather than by hand-off to a BHC may ensure greater penetration of these services in primary care settings. Both implementation models provided a framework for identifying and intervening with adolescent primary care patients whose substance use might have otherwise gone undetected.

Application of System Dynamics to Inform a Model of Adolescent SBIRT Implementation in Primary Care Settings.

System dynamics (SD) modeling is used to compare and contrast strategies for effective implementation of an evidence-based adolescent behavioral health treatment in primary care settings. With qualitative and quantitative data from an on-going cluster-randomized trial in 7 federally qualified health center sites, two implementation conditions were compared: generalist vs. specialist. In the generalist approach, the primary care provider (PCP) delivered brief intervention (BI) for substance misuse (n = 4 clinics). In the specialist approach, BIs were delivered by behavioral health counselors (BHCs) (n = 3 clinics). The resultant SD model compared 'basecase' dynamics to strategic approaches to deploying continuous technical assistance (TA) and performance feedback reporting (PFR). The basecase effectively represented the SBIRT intervention, which reflected actual monthly volume of adolescent primary care visits (N = 9639), screenings (N = 5937), positive screenings (N = 246), and brief interventions (BIs; N = 50) over the 20-month implementation period. Insights gained suggest that implementation outcomes are sensitive to frequency of PFR, with bimonthly events generating the most rapid and sustained screening results. Simulated trends indicated that availability of the BHC directly impacts success of the specialist model. Similarly, understanding PCPs' perception of severity of need for intervention is key to outcomes in either condition.

Start-Up Costs of SBIRT Implementation for Adolescents in Urban U.S. Federally Qualified Health Centers.

OBJECTIVE: Understanding the costs to implement Screening, Brief Intervention, and Referral to Treatment (SBIRT) for adolescent substance use in primary care settings is important for providers in planning for services and for decision makers considering dissemination and widespread implementation of SBIRT. We estimated the start-up costs of two models of SBIRT for adolescents in a multisite U.S. Federally Qualified Health Center (FQHC). In both models, screening was performed by a medical assistant, but models differed on delivery of brief intervention, with brief intervention delivered by a primary care provider in the generalist model and a behavioral health specialist in the specialist model. METHOD: SBIRT was implemented at seven clinics in a multisite, cluster randomized trial. SBIRT implementation costs were calculated using an activity-based costing methodology. Start-up activities were defined as (a) planning activities (e.g., changing existing electronic medical record system and tailoring service delivery protocols); and (b) initial staff training. Data collection instruments were developed to collect staff time spent in start-up activities and quantity of nonlabor resources used. RESULTS: The estimated average costs to implement SBIRT were $5,182 for the specialist model and $3,920 for the generalist model. Planning activities had the greatest impact on costs for both models. Overall, more resources were devoted to planning and training activities in specialist sites, making the specialist model costlier to implement. CONCLUSIONS: The initial investment required to implement SBIRT should not be neglected. The level of resources necessary for initial implementation depends on the delivery model and its integration into current practice.

The concurrent validity of the Problem Oriented Screening Instrument for Teenagers (POSIT) substance use/abuse subscale in adolescent patients in an urban federally qualified health center.

BACKGROUND: The Problem Oriented Screening Instrument for Teenagers (POSIT) substance use/abuse subscale has been validated with high school students, adolescents with criminal justice involvement, and adolescent substance use treatment samples using the Diagnostic and Statistical Manual of Mental Disorders (DSM)-III-R and DSM-IV. This study examines the concurrent validity of the POSIT's standard 17-item substance use/abuse subscale and a revised, shorter 11-item version using DSM-5 substance use disorder diagnoses. METHODS: Adolescents (N = 525; 93% African American, 55% female) 12-17 years of age awaiting primary care appointments at a Federally Qualified Health Center in Baltimore, Maryland completed the 17-item POSIT substance use/abuse subscale and items from a modified World Mental Health Composite International Diagnostic Interview corresponding to DSM-5 alcohol use disorder (AUD) and cannabis use disorder (CUD). Receiver operating characteristic curves, sensitivities, and specificities were examined with DSM-5 AUD, CUD, and a diagnosis of either or both disorders for the standard and revised subscales using risk cutoffs of either 1 or 2 POSIT "yes" responses. RESULTS: For the 17-item subscale, sensitivities were generally high using either cutoff (range: 0.79-1.00), although a cutoff of 1 was superior (sensitivities were 1.00 for AUD, CUD, and for either disorder). Specificities were also high using either cutoff (range: 0.81-0.95) but were higher using a cutoff of 2. For the 11-item subscale, a cutoff of 1 yielded higher sensitivities than a cutoff of 2 (ranges for 1 and 2: 0.96-1.00 and 0.79-0.86, respectively). Specificities for this subscale were higher using a cutoff of 2 (ranges for 1 and 2: 0.82-0.89 and 0.89-0.96, respectively). CONCLUSIONS: Findings suggest that the POSIT's substance use/abuse subscale is a potentially useful tool for screening adolescents in primary care for AUD or CUD using a cutoff of 1 or 2. The briefer, revised subscale may be preferable to the standard subscale in busy pediatric practices.

SBIRT Implementation for Adolescents in Urban Federally Qualified Health Centers.

BACKGROUND: Alcohol, tobacco, and other drug use remains highly prevalent among US adolescents and is a threat to their well-being and to the public health. Evidence from clinical trials and meta-analyses supports the effectiveness of Screening, Brief Intervention and Referral to Treatment (SBIRT) for adolescents with substance misuse but primary care providers have been slow to adopt this evidence-based approach. The purpose of this paper is to describe the theoretically informed methodology of an on-going implementation study. METHODS: This study protocol is a multi-site, cluster randomized trial (N=7) guided by Proctor's conceptual model of implementation research and comparing two principal approaches to SBIRT delivery within adolescent medicine: Generalist vs. Specialist. In the Generalist Approach, the primary care provider delivers brief intervention (BI) for substance misuse. In the Specialist Approach, BIs are delivered by behavioral health counselors. The study will also examine the effectiveness of integrating HIV risk screening within an SBIRT model. Implementation Strategies employed include: integrated team development of the service delivery model, modifications to the electronic medical record, regular performance feedback and supervision. Implementation outcomes, include: Acceptability, Appropriateness, Adoption, Feasibility, Fidelity, Costs/Cost-Effectiveness, Penetration, and Sustainability. DISCUSSION: The study will fill a major gap in scientific knowledge regarding the best SBIRT implementation strategy at a time when SBIRT is poised to be brought to scale under health care reform. It will also provide novel data to inform the expansion of the SBIRT model to address HIV risk behaviors among adolescents. Finally, the study will generate important cost data that offer guidance to policymakers and clinic directors about the adoption of SBIRT in adolescent health care.

Validation and performance of the Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) among adolescent primary care patients.

BACKGROUND AND AIMS: The World Health Organization's Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) has strong support as a clinical screening tool and research instrument, but has only been validated with adults. This study evaluated the ASSIST and ASSIST-Lite in an adolescent population. DESIGN: Internal consistency, concurrent validity, discriminant validity and diagnostic accuracy were examined for tobacco, alcohol and cannabis ASSIST scores. An abbreviated version (the ASSIST-Lite) was evaluated for cannabis. SETTING: Three community health centers in Baltimore, MD, USA. PARTICIPANTS: A total of 525 primary care patients, ages 12-17 years. MEASUREMENTS: Measures included the ASSIST, the CRAFFT screening tool and items from the Composite International Diagnostic Interview (CIDI) corresponding to substance use disorder criteria in the Diagnostic and Statistical Manual of Mental Disorders, 5th edition. FINDINGS: The ASSIST had good internal consistency (α = 0.68-0.88), good concurrent validity with the CRAFFT (r = 0.41-0.76; P < 0.001), and was able to discriminate between gradations of cannabis problem severity. In receiver operating characteristics analysis of optimal clinical cut-points, the ASSIST accurately identified tobacco, alcohol and cannabis use disorders (sensitivities = 95-100%; specificities = 79-93%; area under the curve [AUC] = 0.90-0.94), but did so at minimally low cut-points (indicative of any use in the past 3 months). The ASSIST-Lite performed similarly to the ASSIST in identifying cannabis use disorders (sensitivity = 96%; specificity = 88%; AUC = 0.92), also at a minimally low cut-point. However, confirmatory factor analysis of the ASSIST indicated poor model fit. CONCLUSIONS: The Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) is promising as a research and screening/brief assessment tool with adolescents, but revisions to clinical risk thresholds are warranted. The ASSIST-Lite is sufficiently informative for rapid clinical screening of adolescents for cannabis use disorders.

The CRAFFT cut-points and DSM-5 criteria for alcohol and other drugs: a reevaluation and reexamination.

BACKGROUND: The CRAFFT, previously validated against DSM-IV (Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition) diagnostic criteria, is the most widely used screening instrument for alcohol and other substance misuse in adolescents. The present secondary analysis study sought to compare the CRAFFT with the new DSM-5 diagnostic criteria in order to assess the CRAFFT's psychometric properties and determine the optimal cut-point for identifying adolescents in need of further assessment. METHODS: Participants were primary care patients aged 12-17 (N = 525) who were recruited while waiting for a medical appointment in an urban federally qualified health center in Baltimore, Maryland, USA. Participants were administered the CRAFFT and the Composite International Diagnostic Interview, second edition, modified to include the new DSM-5 craving item. The authors examined the performance of the CRAFFT in identifying any problem use (defined as 1 or more DSM-5 criteria) and any DSM-5 substance use disorder (2 or more DSM-5 criteria) for alcohol or drugs other than tobacco. The authors examined sensitivity, specificity, and receiver operating characteristic areas under the curve (AUC) to determine the optimal CRAFFT cut-point(s) for predicting any problem use and any DSM-5 substance use disorder (SUD). RESULTS: Examining the CRAFFT as a continuous measure, AUC values were 0.93 for problem use or higher and 0.97 for DSM-5 SUD. Consistent with previously recommended cut-points for the CRAFFT, the cut-point of 2 performed optimally for identifying adolescents both exhibiting problem use of alcohol or drugs and meeting DSM-5 SUD criteria for alcohol or other drugs. CONCLUSIONS: Despite changes in the DSM substance use diagnostic criteria, the CRAFFT continues to demonstrate excellent sensitivity and specificity at its established cut-point of 2. Additional studies examining the CRAFFT in light of the new DSM-5 diagnostic criteria with more diverse populations are warranted.

Concordance between DSM-5 and DSM-IV nicotine, alcohol, and cannabis use disorder diagnoses among pediatric patients.

BACKGROUND: The recently published Diagnostic and Statistical Manual of Mental Disorders-5 (DSM-5) includes several major revisions to substance use diagnoses. Studies have evaluated the impact of these changes among adult samples but research with adolescent samples is lacking. METHODS: 525 adolescents (93% African American) awaiting primary care appointments in Baltimore, Maryland were recruited for a study evaluating a substance use screening instrument. Participants were assessed for DSM-5 nicotine, alcohol, and cannabis use disorder, DSM-IV alcohol and cannabis abuse, and DSM-IV dependence for all three substances during the past year using the modified Composite International Diagnostic Interview-2, Substance Abuse Module. Contingency tables examining DSM-5 vs. DSM-IV joint frequency distributions were examined for each substance. RESULTS: Diagnoses were more prevalent using DSM-5 criteria compared with DSM-IV for nicotine (4.0% vs. 2.7%), alcohol (4.6% vs. 3.8%), and cannabis (10.7% vs. 8.2%). Cohen's κ, Somers' d, and Cramer's V ranged from 0.70 to 0.99 for all three substances. Of the adolescents categorized as "diagnostic orphans" under DSM-IV, 7/16 (43.8%), 9/29 (31.0%), and 13/36 (36.1%) met criteria for DSM-5 disorder for nicotine, alcohol, and cannabis, respectively. Additionally, 5/17 (29.4%) and 1/21 (4.8%) adolescents who met criteria for DSM-IV abuse did not meet criteria for a DSM-5 diagnosis for alcohol and cannabis, respectively. CONCLUSIONS: Categorizing adolescents using DSM-5 criteria may result in diagnostic net widening-particularly for cannabis use disorders-by capturing adolescents who were considered diagnostic orphans using DSM-IV criteria. Future research examining the validity of DSM-5 substance use disorders with larger and more diverse adolescent samples is needed.

Validity of brief screening instrument for adolescent tobacco, alcohol, and drug use.

BACKGROUND AND OBJECTIVE: The National Institute on Alcohol Abuse and Alcoholism developed an alcohol screening instrument for youth based on epidemiologic data. This study examines the concurrent validity of this instrument, expanded to include tobacco and drugs, among pediatric patients, as well as the acceptability of its self-administration on an iPad. METHODS: Five hundred and twenty-five patients (54.5% female; 92.8% African American) aged 12 to 17 completed the Brief Screener for Tobacco, Alcohol, and other Drugs (BSTAD) via interviewer-administration or self-administration using an iPad. Diagnostic and Statistical Manual, Fifth Edition substance use disorders (SUDs) were identified using a modified Composite International Diagnostic Interview-2 Substance Abuse Module. Receiver operating characteristic curves, sensitivities, and specificities were obtained to determine optimal cut points on the BSTAD in relation to SUDs. RESULTS: One hundred fifty-nine (30.3%) adolescents reported past-year use of ≥1 substances on the BSTAD: 113 (21.5%) used alcohol, 84 (16.0%) used marijuana, and 50 (9.5%) used tobacco. Optimal cut points for past-year frequency of use items on the BSTAD to identify SUDs were ≥6 days of tobacco use (sensitivity = 0.95; specificity = 0.97); ≥2 days of alcohol use (sensitivity = 0.96; specificity = 0.85); and ≥2 days of marijuana use (sensitivity = 0.80; specificity = 0.93). iPad self-administration was preferred over interviewer administration (z = 5.8; P < .001). CONCLUSIONS: The BSTAD is a promising screening tool for identifying problematic tobacco, alcohol, and marijuana use in pediatric settings. Even low frequency of substance use among adolescents may indicate need for intervention.