Creating more effective health plan quality reports for consumers: lessons from a synthesis of qualitative testing.

OBJECTIVE: Social marketing techniques such as consumer testing have only recently been applied to develop effective consumer health insurance information. This article discusses lessons learned from consumer testing to create consumer plan choice materials. DATA SOURCES/STUDY SETTING: Data were collected from 268 publicly and privately insured consumers in three studies between 1994 and 1999. STUDY DESIGN: Iterative testing and revisions were conducted to design seven booklets to help Medicaid, Medicare, and employed consumers choose a health plan. DATA COLLECTION METHODS: Standardized protocols were used in 11 focus groups and 182 interviews to examine the content, comprehension, navigation, and utility of the booklets. PRINCIPAL FINDINGS: A method is suggested to help consumers narrow their plan choices by breaking down the process into smaller decisions using a set of guided worksheets. CONCLUSION: Implementing these lessons is challenging and not often done well. This article gives examples of evidence-based approaches to address cognitive barriers that designers of consumer health insurance information can adapt to their needs.

Providing information to help Medicare beneficiaries choose a health plan.

Many Medicare beneficiaries have limited knowledge of the Medicare program and related health insurance options. This is due in part to the complexity of the Medicare program and supplemental health insurance market. A recent congressional mandate through the Balanced Budget Act of 1997 called for broad dissemination of information to educate beneficiaries about their health plan options and to encourage informed health plan decision-making. In response, the Health Care Financing Administration (HCFA) launched the National Medicare Education Program (NMEP) to support the educational objectives of the BBA. This paper provides an overview of the components of the NMEP information campaign. We also review lessons learned from our experience in designing and testing a prototype consumer handbook that explains the different health plan options to Medicare beneficiaries. Through our discussion of the handbook, we highlight several ways to communicate information effectively about a complex publicly funded program to an older adult population.

Beneficiaries' perceptions of new Medicare health plan choice print materials.

This article presents findings from a study involving seven focus groups with aged and disabled Medicare beneficiaries in the Kansas City area regarding their impressions of a pilot version of the Medicare & You 1999 handbook and the Medicare Consumer Assessment of Health Plans Study (CAHPS) survey report. Beneficiaries generally had positive reactions to both booklets and viewed the handbook as an important reference tool. Based on the findings, we present policy recommendations for the development and dissemination of Medicare health plan information to beneficiaries.

What do consumers want and need to know in making health care choices?

This article assesses the presumption that consumer choice in health care is based on a rational weighing of alternatives--that information consumers about plan or provider performance, when coupled with information on cost plus service scope and limitation, will lead consumers to select high-quality, low-priced plans or providers. The authors review research on what health care consumers know, what they want to know, and what others think they should know. They also consider how people use information in making decisions and what this implies for what consumers really need to know to make effective decisions. The article concludes that assuming a rational consumer does not account for choice among options in the increasingly complex health care context facing consumers today. Based on this review, the article identifies gaps in the knowledge and sketches out a prospective research agenda in the area of consumer health care decision making.

The use of cognitive testing to develop and evaluate CAHPS 1.0 core survey items. Consumer Assessment of Health Plans Study.

OBJECTIVES: The main goal of the Consumer Assessments of Health Plans Study (CAHPS) is to develop an integrated set of tested, standardized surveys to obtain meaningful information from health plan enrollees about their experiences. The CAHPS project benefits from the complementary strengths of psychometric and cognitive testing. METHODS: The CAHPS team conducted 150 cognitive interviews across three organizations in different geographic locations using multiple interview methods with different consumer populations. This article explains how cognitive testing was used in the CAHPS survey development process and shares the main findings from the cognitive interviews. RESULTS: A modified report format is more appropriate when asking about specific aspects of plan enrollees' experiences, whereas a rating format is useful for asking about overall assessments. Specifying a longer reference period is preferable to asking about the most recent visit when capturing experiences with care, because some respondents get frustrated when they cannot include experiences other than the most recent visit. Explicit screeners and tailored inapplicable response categories are beneficial in mail questionnaires, so people know that they should not answer questions about which they have no relevant experience. CONCLUSION: Cognitive testing was integral in the development and refinement of the CAHPS instrument. The cognitive testing findings contributed to an improved instrument that should capture consumers' health care and plan experiences with less response error than one not subjected to such testing. The cognitive testing process and findings can be useful to other researchers with similar survey development goals.

Psychometric properties of the CAHPS 1.0 survey measures. Consumer Assessment of Health Plans Study.

OBJECTIVES: Consumer surveys are being used increasingly to assess the quality of care provided by health plans, physician groups, and clinicians. The purpose of the Consumer Assessment of Health Plans Study (CAHPS) is to develop an integrated and standardized set of surveys designed to collect reliable and valid information about health plan performance from consumers. This article reports psychometric results for the CAHPS 1.0 survey items in samples of individuals with Medicaid or private health insurance coverage. METHODS: Reliability estimates for CAHPS 1.0 measures were estimated in a sample of 5,878 persons on Medicaid and 11,393 persons with private health insurance. Correlations of the CAHPS global rating of the health plan with willingness to recommend the plan and intention to re-enroll were estimated in a sample of 313 persons on Medicaid. The association of the rating of the health plan with ratings using a 5-point Excellent-to-Poor response scale also was investigated in the latter sample and in a sample of 539 persons with private health insurance. RESULTS: The CAHPS measures appeared to have good reliability, particularly at the health-plan level. Responses from 300 consumers per health plan tend to yield estimates that are reliable enough for health plan comparisons, especially among the privately insured. The global health plan rating was significantly correlated with consumers' willingness to recommend the plan to family and friends and to their intention to re-enroll in the plan if given a choice. CONCLUSIONS: The CAHPS 1.0 survey instrument appears to have excellent psychometric properties.

Special issues addressed in the CAHPS survey of Medicare managed care beneficiaries. Consumer Assessment of Health Plans Study.

OBJECTIVES: This article describes the process through which the MMC survey was developed and examines issues in using this survey with Medicare beneficiaries that have implications for all CAHPS surveys. These include the ability of Medicare beneficiaries to use MMC navigational features, whether access measures are meaningful for this population, and whether beneficiaries' familiarity with managed care influences their health plan assessments. BACKGROUND: The Health Care Financing Administration (HCFA) is mandated to provide comparative plan information, based partly on consumer surveys, to Medicare beneficiaries. The Consumer Assessments of Health Plans Study (CAHPS) is an integrated set of tested, standardized surveys of health plan enrollees. To meet its goal, HCFA has invested in the development of a CAHPS survey of beneficiaries enrolled in Medicare Managed Care plans (MMC). METHODS: Cognitive interviews were completed with 31 Medicare beneficiaries. A field test also was conducted with beneficiaries to examine patterns of survey response. A sample of 956 eligible individuals was selected from six health plans. Using a combination of mail and telephone data collection, 663 (69%) questionnaires were completed. This article provides selective results from these tests. RESULTS: The use of screening questions, skip instructions, and tailored "not applicable" response options appeared to facilitate the response task. Some CAHPS access questions were not meaningful to Medicare beneficiaries. The data do not support the need to adjust for length of plan enrollment. CONCLUSION: Analyses suggested changes to improve the MMC survey and to make other CAHPS surveys consistent with these changes.