Using enrollment records to evaluate self-reports of monthly coverage in the redesigned current population survey health insurance module.

OBJECTIVE: To evaluate the veracity of self-reports of month-level health insurance coverage in the Current Population Survey Annual Social and Economic Supplement (CPS). DATA SOURCES AND STUDY SETTING: The CHIME (Comparing Health Insurance Measurement Error) study used health insurance enrollment records from a large regional Midwest insurer as sample for primary data collection in spring 2015. STUDY DESIGN: A sample of individuals enrolled in a range of public and private coverage types (including Medicaid and marketplace) was administered the CPS health insurance module, which included questions about month-level coverage, by type, over a 17-18-month time span. Survey data was then matched to enrollment records covering that same time frame, and concordance between the records and self-reports was assessed. DATA COLLECTION/EXTRACTION METHODS: Sample was drawn by the insurer's informatics specialists and Census Bureau interviewers conducted the survey. Following data collection, updated enrollment records were matched to the survey data to produce a person-level file of coverage by type at the month-level. PRINCIPAL FINDINGS: For 91% of the overall sample, coverage status and type were reported accurately for at least 75% of observed months. Results varied somewhat by stability of coverage. Among those who were continuously covered throughout the 17-18 month observation period (which comprised 64% of the overall sample), that level of reporting accuracy was observed for 94% of the sample; for those who had censored spells (34% of the overall sample), the figure was 87%; and among those with gaps and/or changes according to the records (2% of the overall sample), for 82% of the group at least 75% of months were reported accurately. CONCLUSIONS: Findings suggest that reporting accuracy of month-level coverage in the CPS is high and that the survey could become a valuable new data source for studying the dynamics of coverage, including the Medicaid unwinding.

Factors associated with accurate reporting of public and private health insurance type.

OBJECTIVE: To examine factors associated with accurate reporting of private and public health insurance coverage. DATA SOURCES: Minnesota health plan enrollment records provided the sample for the Comparing Health Insurance Measurement Error (CHIME) study, a survey conducted in 2015 that randomly assigned enrollees to treatments that included health insurance questions from the American Community Survey (ACS) or the redesigned Current Population Survey Annual Social and Economic Supplement (CPS). STUDY DESIGN: Reverse record check study that compared CHIME study survey responses to enrollment records of coverage type (direct purchase on and off the Marketplace, Medicaid, or MinnesotaCare), service use, subsidy receipt, and duration of coverage from a major insurer. DATA COLLECTION METHODS: Using matched enrollment and CHIME survey data and logistic regression, we examined correlates of accurate insurance type reporting, including characteristics of the insurance coverage, the covered individual, respondent, and family. PRINCIPAL FINDINGS: Reporting accuracy across treatment and coverage type is high (77%-84%). As with past research, accurate reporting of public insurance is higher for people with characteristics consistent with eligibility for public insurance for both survey treatments. For the ACS treatment, reports of direct purchase insurance are more accurate for enrollees who receive a premium subsidy. CONCLUSIONS: Given the complexity of health insurance measurement and frequently changing policy environment, differences in reporting accuracy across treatments or coverage types are not surprising. Several results have important implications for data editing and modeling routines. First, adding premium and subsidy questions in federal surveys should prove useful given the finding that subsidy receipt is associated with reporting accuracy. Second, across both survey treatments, people whose opportunity structures (race, ethnicity, and income) match public program eligibility are accurate reporters of this coverage. This evidence supports using these commonly collected demographic variables in simulation, imputation, and editing routines.

Medicaid expansion and the Medicaid undercount in the American Community Survey.

OBJECTIVE: To measure discordance between aggregate estimates of means-tested coverage from the American Community Survey (ACS) and administrative counts and examine the association of discordance with ACA Medicaid expansion. DATA SOURCES: 2010-2016 ACS and counts of Medicaid and Children's Health Insurance Program enrollment from the Centers for Medicare & Medicaid Services. STUDY DESIGN: State-by-year counts of means-tested coverage from the ACS were compared to administrative counts using percentage differences. Discordance was compared for states that did and did not adopt expansion using difference-in-differences. We then contrasted the effect of expansion on means-tested coverage estimated from the ACS with results from administrative data. DATA COLLECTION/EXTRACTION: Survey and administrative data. PRINCIPAL FINDINGS: One year before expansion there was a 0.8 and 4 percent overcount in expansion and nonexpansion states, respectively. By 2016, there was a 10.64 percent undercount in expansion states vs a 0.02 percent undercount in nonexpansion states. The ACS suggests that expansion increased means-tested coverage in the full population by three percentage points, relative to five percentage points suggested by administrative records. CONCLUSIONS: Discordance between the ACS and administrative records has increased over time. The ACS underestimates the impact of Medicaid expansion, relative to administrative counts.

Assessing the accuracy of survey reports of health insurance coverage using enrollment data.

OBJECTIVE: To measure the accuracy of survey-reported data on features and type of health insurance coverage. DATA SOURCE: Enrollment records from a private insurer were used as sample for primary survey data collection in spring of 2015 using the Current Population Survey health insurance module. STUDY DESIGN: A reverse record check study where households with individuals enrolled in a range of public and private health insurance plans (including the marketplace) were administered a telephone survey that included questions about general source of coverage (eg, employer), program name (eg, Medicaid), portal, premium, and subsidies. DATA COLLECTION/EXTRACTION METHODS: Survey data were matched back to enrollment records, which indicated coverage status at the time of the survey. Concordance between the records and survey data was assessed. PRINCIPAL FINDINGS: Correct reporting of general source of coverage ranged from 77.8 percent to 98.3 percent across coverage type, premium ranged from 91.6 percent to 96.4 percent, and subsidy ranged from 83.0 percent to 91.0 percent. Using a conceptual algorithm to categorize coverage type resulted in sensitivity of 98.3 percent for employer-sponsored enrollees, and 70.6 percent-77.6 percent for the other coverage types, while specificity ranged from 93.9 percent to 99.4 percent across coverage types. CONCLUSIONS: Survey reports of features of coverage suggest they are viable items to include in an algorithm to categorize coverage type. Findings have implications beyond the CPS, particularly for marketplace enrollees.

Understanding the New Current Population Survey Health Insurance Questions.

OBJECTIVE: To compare estimates of health coverage from the pre- and post- redesign of the Current Population Survey (CPS) Annual Social and Economic Supplement. DATA SOURCES/STUDY SETTING: The CPS 2013 Content Test. STUDY DESIGN: A test of the old and new CPS in which the control panel was a subset of the CPS production cases interviewed by phone and the test panel was conducted in parallel (also by phone) with a sample that had already completed the final rotation of the CPS. Outcome variables tested include uninsured and coverage type by subgroup and calendar year versus point-in-time estimates. DATA COLLECTION/EXTRACTION METHODS: Census Bureau telephone interviewers. PRINCIPAL FINDINGS: The odds of having coverage in the past calendar year were higher under the new than the old CPS. Within the new CPS, calendar year estimates of coverage were higher than and distinct from point-in-time estimates. There were few statistically significant differences in coverage across demographic subgroups. CONCLUSIONS: The new method reduced presumed underreporting of past year coverage, and the integrated point-in-time/calendar-year series effectively generated distinct measures of each within the same questionnaire.

Preparing to measure health coverage in federal surveys post-reform: lessons from Massachusetts.

In preparation for health reform in 2014, qualitative research was conducted with Massachusetts residents to explore how to adapt surveys to accommodate reporting information about health exchanges. Questions about exchange participation were effective when state-specific exchange program names were offered, but generic terms such as "marketplace" and "exchange" did not resonate with respondents. However, respondents were able to understand new questions about premiums and subsidies and to answer with a high degree of accuracy. These questions, taken in tandem with answers on plan type, were sufficient to distinguish among Medicaid, subsidized exchange coverage, and unsubsidized coverage, even without the benefit of state-specific exchange program names.

Measurement error in health insurance reporting.

In the United States, surveys serve as the only source of data for the number of uninsured people; they also provide rich data for exploring the relationships between health insurance coverage and individuals' life circumstances, such as employment, income, and health status, enabling researchers to assess the effectiveness of various aspects of the health care system. The Current Population Survey (CPS) is one of the most influential surveys measuring health insurance, but it is not without critics. To address outstanding questions about the data quality of the CPS health insurance questions, qualitative testing was conducted to assess various aspects of the questionnaire from the respondent's perspective. A testing protocol was developed largely based on previous health survey methods literature, and test subjects were probed about their comprehension of the questions, particular terms and phrases, and their strategies for formulating an answer. Several design features were identified as problematic, including the overall questionnaire structure, the calendar year reference period, the household-level design, and the wording of questions on public coverage.