Assessing Laboratory Method Validations for Informing Inference Across Survey Cycles in the National Health and Nutrition Examination Survey.

Background and objectives Laboratory tests conducted on survey respondents' biological specimens are a major component of the National Health and Nutrition Examination Survey. The National Center for Health Statistics' Division of Health and Nutrition Examination Surveys performs internal analytic method validation studies whenever laboratories undergo instrumental or methodological changes, or when contract laboratories change. These studies assess agreement between methods to evaluate how methodological changes could affect data inference or compromise consistency of measurements across survey cycles. When systematic differences between methods are observed, adjustment equations are released with the data documentation for analysts planning to combine survey cycles or conduct a trend analysis. Adjustment equations help ensure that observed differences from methodological changes are not misinterpreted as population changes. This report assesses the reliability of statistical methods used by the Division of Health and Nutrition Examination Surveys when conducting method validation studies to address concerns that adjustment equations are being overproduced (recommended too frequently). Methods Public-use 2017-2018 National Health and Nutrition Examination Survey laboratory data were used to simulate "new" measurements for 120 analytic method validation studies. Blinded studies were analyzed to determine the final adjustment recommendation for each study using difference plots, descriptive statistics, t-tests, and Deming regressions. Final recommendations were compared with simulated difference types to assess how often spurious results were observed. Concordance estimates (concordance, misclassification, sensitivity, specificity, and positive and negative predictive values) informed assessments. Results Adjustment equations were appropriately recommended for 75.0% of the studies, over-recommended for 5.8%, under-recommended for 15.8%, and recommended with an inappropriate technique for 3.3%. Across simulated difference types, sensitivity ranged from 65.9% to 84.4% and specificity from 74.7% to 97.5%. Conclusions Findings from this report suggest that the current methodology used by the Division of Health and Nutrition Examination Surveys performs moderately well. Based on these data and analyses, underadjustment was more prevalent than overadjustment, suggesting that the current methodology is conservative.

Calibration Weighting Methods for the National Center for Health Statistics Research and Development Survey.

Objectives The Research and Development Survey (RANDS) is a series of web-based, commercial panel surveys that have been conducted by the National Center for Health Statistics (NCHS) since 2015. RANDS was designed for methodological research purposes,including supplementing NCHS' evaluation of surveys and questionnaires to detect measurement error, and exploring methods to integrate data from commercial survey panels with high-quality data collections to improve survey estimation. The latter goal of improving survey estimation is in response to limitations of web surveys, including coverage and nonresponse bias. To address the potential bias in estimates from RANDS,NCHS has investigated various calibration weighting methods to adjust the RANDS panel weights using one of NCHS' national household surveys, the National Health Interview Survey. This report describes calibration weighting methods and the approaches used to calibrate weights in web-based panel surveys at NCHS.

Multiple imputation of missing data with skip-pattern covariates: a comparison of alternative strategies.

Multiple imputation (MI) is a widely used approach to address missing data issues in surveys. Variables included in MI can have various distributional forms with different degrees of missingness. However, when variables with missing data contain skip patterns (i.e. questions not applicable to some survey participants are thus skipped), implementation of MI may not be straightforward. In this research, we compare two approaches for MI when skip-pattern covariates with missing values exist. One approach imputes missing values in the skip-pattern variables only among applicable subjects (denoted as imputation among applicable cases (IAAC)). The second approach imputes skip-pattern covariates among all subjects while using different recoding methods on the skip-pattern variables (denoted as imputation with recoded non-applicable cases (IWRNC)). A simulation study is conducted to compare these methods. Both approaches are applied to the 2015 and 2016 Research and Development Survey data from the National Center for Health Statistics.

Propensity-score Adjusted Estimates for Selected Health Outcomes From the Research and Development Survey.

To evaluate the quality of web surveys, the National Center for Health Statistics' Division of Research and Methodology has been conducting a series of studies with survey data from commercially recruited panels,referred to as the Research and Development Survey (RANDS). This report describes the propensity-score adjusted estimates from the second round of RANDS (RANDS 2) using the 2016 National Health Interview Survey (NHIS).

Identifying relative strength of methadone versus health and social services in comprehensive substance use disorder treatment using a variance decomposition approach.

BACKGROUND: The purpose of this study is to identify the relative strengths of association of medication and health and social services in comprehensive substance use disorder (SUD) treatment. OBJECTIVES: The study uses a novel variance decomposition method to assess the relative strength of association of six active ingredients of comprehensive SUD treatment: methadone medication, access services, SUD counseling, matched service ratio, client-provider relationship, and treatment duration. METHODS: The study uses data from the National Treatment Improvement Evaluation Study (1992-1997), a dataset with an unusual number of services and service strategies measured. The data include 3012 clients from 45 SUD treatment programs. Linear mixed models are used to assess the relation of service variables to the outcome of posttreatment substance use. Variance decomposition methods are used to assess the relative importance of the ingredients in the treatment model. RESULTS: Along with a random intercept and background variables, receipt of methadone accounted for the greatest relative strength of association at 35.4%, compared with 23.8% for treatment duration, 15.4% for client-provider relationship, and 11.2% for matched service ratio. Access and SUD counseling accounted for modest strengths of association at 1% and 3% each. CONCLUSION: Findings indicate somewhat greater strength of association of methadone compared with other services and service strategies and overall, reinforce the importance of both medication and services and service strategies in the design and development of effective models of service delivery. SIGNIFICANCE: This study, among the first to evaluate the relative importance of specific services and service strategies of comprehensive SUD treatment, provides insights relevant to the development of effective models of service delivery.

The National Center for Health Statistics' 2015 and 2016 Research and Development Surveys.

Objective: This report provides a general description of the background and operation of the first two rounds of the Research and Development Survey (RANDS), a series of cross-sectional surveys from probability-sampled commercial survey panels. The Division of Research and Methodology of the National Center for Health Statistics (NCHS) conducted the first two rounds of RANDS in 2015 and 2016. RANDS 1 and 2 are being used primarily for question design evaluation and for investigating statistical methodologies for estimation. Methods: NCHS contracted with Gallup, Inc. to conduct RANDS 1 in Fall 2015 and RANDS 2 in Spring 2016. RANDS 1 and 2 were conducted using a web survey mode and included survey questions from the National Health Interview Survey (NHIS) that were specifically chosen to provide comparison and evaluation of the survey methodology properties of web surveys and traditional household surveys. In this report, some demographic and health estimates are provided from both sources to describe the RANDS data. Results: In RANDS 1, 2,304 out of the original 9,809 invited panel members completed the survey, for a completion rate of 23.5%. In RANDS 2, 2,480 of the initial 8,231 invited respondents completed the survey, for a completion rate of 30.1%. RANDS 1 and 2 participants were similar to the quarterly NHIS participants with respect to sex, census region, and whether they had worked for pay in the previous week. Other characteristics varied, including age, race and ethnicity, and income. Most health estimates differed between RANDS and NHIS. Public-use versions of the RANDS data can be found at: https://www.cdc.gov/nchs/rands. Conclusion: RANDS is an ongoing platform for research to understand the properties of probability-sampled recruited panels of primarily web users, investigating and developing statistical methods for using such data in conjunction with large nationally representative health surveys, and for extending question-design evaluations.

Overview and Initial Results of the National Center for Health Statistics' Research and Development Survey.

The National Center for Health Statistics is assessing the usefulness of recruited web panels in multiple research areas. One research area examines the use of close-ended probe questions and split-panel experiments for evaluating question-response patterns. Another research area is the development of statistical methodology to leverage the strength of national survey data to evaluate, and possibly improve, health estimates from recruited panels. Recruited web panels, with their lower cost and faster production cycle, in combination with established population health surveys, may be useful for some purposes for statistical agencies. Our initial results indicate that web survey data from a recruited panel can be used for question evaluation studies without affecting other survey content. However, the success of these data to provide estimates that align with those from large national surveys will depend on many factors, including further understanding of design features of the recruited panel (e.g. coverage and mode effects), the statistical methods and covariates used to obtain the original and adjusted weights, and the health outcomes of interest.

Comparison of Quarterly and Yearly Calibration Data for Propensity Score Adjusted Web Survey Estimates.

While web surveys have become increasingly popular as a method of data collection, there is concern that estimates obtained from web surveys may not reflect the target population of interest. Web survey estimates can be calibrated to existing national surveys using a propensity score adjustment, although requirements for the size and collection timeline of the reference data set have not been investigated. We evaluate health outcomes estimates from the National Center for Health Statistics' Research and Development web survey. In our study, the 2016 National Health Interview Survey as well as its quarterly subsets are considered as reference datasets for the web data. It is demonstrated that the calibrated health estimates overall vary little when using the quarterly or yearly data, suggesting that there is flexibility in selecting the reference dataset. This finding has many practical implications for constructing reference data, including the reduced cost and burden of a smaller sample size and a more flexible timeline.

On a scale as a sum of manifest variables.

The most common approach for a scale construction is to create a scale as a sum of manifest variables (a "sum scale"). When we use the sum scale for analysis, we implicitly assume that there is a one-dimensional latent structure representing the manifest data on a multidimensional space. In this commentary, we review basics of identifying a latent structure using measured variables with a minimum linear algebra. We demonstrate the technique using Fisher's iris data as an illustration. We examine the relationships between resulting latent variables and the sum scale to evaluate goodness of the sum scale. As a practical solution, in general, we could create a sum scale using a set of positively and highly correlated measured variables. More care is needed when the data are not unidimensional.

National Health and Nutrition Examination Survey: Estimation Procedures, 2011-2014.

This report describes the methods used to create NHANES 2011-2014 sample weights and variance units for the public-use data files, including sample weights for selected subsamples, such as the fasting subsample. The impacts of sample design changes on estimation for NHANES 2011-2014 and the addition of the NHANES National Youth Fitness Survey (NNYFS) 2012 are described. Approaches that data users can employ to modify sample weights when combining survey cycles or when combining subsamples are also included.

Gender disparities in utilization and outcome of comprehensive substance abuse treatment among racial/ethnic groups.

This study examined gender differences within Black, Latino, and White subgroups in the utilization of comprehensive services and their relation to posttreatment substance use. Survey data were collected during the National Treatment Improvement Evaluation Study (NTIES), a prospective, longitudinal, multisite study of substance abuse treatment programs and their clients in the United States. The analytic sample consisted of 1,812 blacks (734 women and 1,078 men), 486 Latinos (147 women and 339 men), and 844 whites (147 women and 339 men) from 59 service delivery organizations. Results related to service utilization indicated that compared to men, women in all racial and ethnic groups needed and received more services targeted to their needs and reported more positive relations with service providers. Gender was a significant moderator of the relationship between service receipt and treatment outcomes for all racial and ethnic groups, but especially for the Latino subsample. Findings point to the need to consider race-specific gender differences in the development of culturally competent, comprehensive substance abuse treatment.

Client and program characteristics associated with wait time to substance abuse treatment entry.

BACKGROUND: Wait time is among the most commonly cited barriers to access among individuals seeking entry to substance abuse treatment, yet relatively little is known about what contributes to it. OBJECTIVES: To address this gap, this study draws from a national sample of substance abuse treatment clients and programs to estimate the proportion of clients entering treatment who waited more than 1 month to receive it (outpatient, residential, or methadone) and to identify client and program characteristics associated with wait time. METHODS: This study used data from the National Treatment Improvement Evaluation Study (1992-1997). The data include 2920 clients from 57 substance abuse treatment programs. Generalized linear modeling was used to identify client and program characteristics associated with wait time to treatment entry. RESULTS: Results of modeling indicate that being African-American (OR: 1.40; CI: 1.04, 1.88), being referred by criminal justice (OR: 1.70; CI: 1.18, 2.43), and receiving methadone (OR: 3.90; CI: 1.00, 15.16) were associated with increased odds of waiting more than 1 month. Conversely, having a diagnosis of HIV/AIDS (OR: 0.38; CI: 0.19, 0.77) was associated with decreased odds of waiting for more than 1 month. CONCLUSION: A significant proportion of clients waited more than 1 month on enter treatment. Greater odds of such wait times were associated with being African-American, criminal justice-referred, and receiving methadone. SIGNIFICANCE: This study is the first to use a national sample to examine the prevalence of wait time to substance abuse treatment entry and to identify client and program characteristics associated with it.

Client-Provider relationship in comprehensive substance abuse treatment: differences in residential and nonresidential settings.

As the substance abuse service system shifts from primarily residential to primarily nonresidential settings, it becomes important to understand how substance abuse treatment processes and outcomes may vary across service setting. Research increasingly indicates that, along with specific treatment and service strategies, client-provider relationship is an important ingredient in effective substance abuse treatment. This study uses a moderator-mediator analysis of a comprehensive service model to examine how the relation between client-provider relationship and substance abuse treatment outcomes may differ in residential and nonresidential settings. The study used data collected for the National Treatment Improvement Evaluation Study, a prospective, cohort-based study of U.S. substance abuse treatment programs and their clients, with an analytic sample of 59 publicly funded service delivery units and 3,027 clients. Structural equation modeling is used to assess the structural relations and causal connections between treatment process and treatment outcome variables. Results indicate that for nonresidential settings, a better client-provider relationship is directly related to improved outcomes of treatment duration and reduced posttreatment substance use and is indirectly related to both outcomes through provision of services matched to client needs. In residential settings, the quality of the client-provider relationship is unrelated to process or outcome variables. The findings point to the importance of the client-provider relationship in all settings but particularly in outpatient settings where there are limited physical constraints on the treatment process.

Improving health and social outcomes with targeted services in comprehensive substance abuse treatment.

BACKGROUND: Improved life functioning along with substance use reduction is increasingly recognized as the definition of effective addiction treatment. OBJECTIVES: To assess whether targeted health and social services contribute to improved physical/mental health and employment. METHODS: This study used data from the National Treatment Improvement Evaluation Study (N = 3027) and modeled the improvement in physical/mental health and employment at discharge or 12 months after discharge compared with intake measures as a function of receipts of matched services. RESULTS: Receiving matched medical service improves physical health only at treatment discharge; receiving matched mental health services improves mental health at discharge and 12 months after discharge; receiving matched vocational services improves employment only 12 months after discharge. CONCLUSIONS: Need-service matching contributes to improved health and social outcomes when longitudinal assessments of treatment outcomes are used to evaluate treatment effectiveness. SCIENTIFIC SIGNIFICANCE: Study findings document the value of targeted services for achieving success in health and social functioning in comprehensive substance abuse treatment.

The impact of comprehensive services in substance abuse treatment for women with a history of intimate partner violence.

This study examines the impact of comprehensive services on posttreatment substance use among women with a history of intimate partner violence. The sample includes 1,123 women from 50 treatment facilities derived from the National Treatment Improvement Evaluation Study (NTIES). Generalized linear mixed modeling was used to determine whether a history of intimate partner violence moderates the association between service receipt and posttreatment substance use. Significant interactions were found between history of intimate partner violence and concrete ( p = .016) and family services (p = .023) in predicting substance use.

Gender differences in client-provider relationship as active ingredient in substance abuse treatment.

The client-provider relationship is increasingly evaluated as an active ingredient in the delivery of substance abuse treatment services. This study examines gender differences in client-provider relationship as an important treatment ingredient affecting retention in treatment and reduced post-treatment substance use. The study uses data collected for the National Treatment Improvement Evaluation Study (NTIES), a prospective, cohort study of U.S. substance abuse treatment programs and their clients. Data on individual characteristics were collected at the pre-treatment interview; on client-provider relationship and services received at treatment exit; and on post-treatment drug use at 12 months post-treatment. The analytic sample consists of 3027 clients from 59 service delivery units (1922 men and 1105 women). Structural equation modeling (SEM) was used to assess the structural relations and causal connections between relationship and service variables and treatment outcome variables. Results indicate that a positive client-provider relationship is related directly to longer duration and reduced post-treatment drug use for the total sample and for men analyzed separately. For women, a positive client-provider relationship was related directly to treatment duration and only indirectly to reduced post-treatment drug use. The findings point to the significance of including client-provider relationship in service delivery models--both as a therapeutic element as well as an element facilitative of matching services to specific client needs.

Need-service matching in substance abuse treatment: racial/ethnic differences.

This study examines the impact of ancillary health and social services matched to client needs in substance abuse treatment for African Americans, Latinos and Whites. The study uses data collected from 1992 to 1997 for the National Treatment Improvement Evaluation Study, a prospective cohort study of substance abuse treatment programs and their clients. The analytic sample consists of 3142 clients (1812 African Americans, 486 Latinos, 844 Whites) from 59 treatment facilities. Results show that racial/ethnic minorities are underserved compared to Whites in the substance abuse service system. Different racial/ethnic groups come into treatment with distinct needs and receive distinct services. Although groups respond differentially to service types, substance abuse counseling and matching services to needs is an effective strategy both for retaining clients in treatment and for reducing post-treatment substance use for African Americans and Whites. Receipt of access services was related to reduced post-treatment substance use for Latinos. Study findings are relevant to planning special services for African Americans and Latinos.

Closing the Need-Service Gap: Gender Differences in Matching Services to Client Needs in Comprehensive Substance Abuse Treatment.

Despite the broad recognition in social work that services are more effective when they are tailored to individual client needs, we have only limited evidence of the impact that services matched to client needs have on treatment outcomes. This study examines gender differences in the impact of matched services, access services, and outcome-targeted services on substance abuse treatment outcomes by using data collected from 1992 through 1997 for the National Treatment Improvement Evaluation Study, a prospective, cohort study of substance abuse treatment programs and clients. The analytic sample consists of 3,027 clients (1,105 women and 1,922 men) who reported needed services from 59 treatment facilities. Findings from the study indicate that overall programs have only limited success in targeting services to client needs, but when they do, receipt of substance abuse counseling and matched services predicts both remaining in treatment and reduced posttreatment substance use for both women and men, but especially for women.

Gender and racial/ethnic disparities in the impact of HIV prevention programming in substance abuse treatment.

OBJECTIVE: The objective of the study was to evaluate the capacity of HIV prevention programs offered in substance abuse treatment to reduce HIV-related risk behavior for women and men and for Black, Latino, and White groups. METHODS: Prospective data was collected at intake, discharage, and 12 months post-treatment from 1992 to 1997 for the National Treatment Improvement Evaluation Study with a sample consisting of 3,142 clients from 59 service delivery units: 972 females, 1,870 males, 1,812 Blacks, 486 Latinos, and 844 Whites. RESULTS: Study findings show that receipt of HIV prevention programming as part of substance abuse treatment services resulted in reductions in HIV-related risk behavior for the sample overall and for women as well as men. However, although Blacks received more prevention services than Latinos and Whites, the significant positive effect of HIV services on reduced HIVrisk behavior held only for Whites. CONCLUSIONS: Racial/ethnic disparities exist in the capacity for HIV prevention programming offered as part of substance abuse treatment to reduce HIV-risk behavior. The findings highlight the need for the development of culturally competent service delivery strategies to enhance the impact of these services for all groups.