Case-mix adjustment of the National CAHPS benchmarking data 1.0: a violation of model assumptions?

OBJECTIVE: To compare models for the case-mix adjustment of consumer reports and ratings of health care. DATA SOURCES: The study used the Consumer Assessment of Health Plans (CAHPS) survey 1.0 National CAHPS Benchmarking Database data from 54 commercial and 31 Medicaid health plans from across the United States: 19,541 adults (age > or = 18 years) in commercial plans and 8,813 adults in Medicaid plans responded regarding their own health care, and 9,871 Medicaid adults responded regarding the health care of their minor children. STUDY DESIGN: Four case-mix models (no adjustment; self-rated health and age; health, age, and education; and health, age, education, and plan interactions) were compared on 21 ratings and reports regarding health care for three populations (adults in commercial plans, adults in Medicaid plans, and children in Medicaid plans). The magnitude of case-mix adjustments, the effects of adjustments on plan rankings, and the homogeneity of these effects across plans were examined. DATA EXTRACTION: All ratings and reports were linearly transformed to a possible range of 0 to 100 for comparability. PRINCIPAL FINDINGS: Case-mix adjusters, especially self-rated health, have substantial effects, but these effects vary substantially from plan to plan, a violation of standard case-mix assumptions. CONCLUSION: Case-mix adjustment of CAHPS data needs to be re-examined, perhaps by using demographically stratified reporting or by developing better measures of response bias.

Racial and ethnic differences in parents' assessments of pediatric care in Medicaid managed care.

OBJECTIVE: This study examines whether parents' reports and ratings of pediatric health care vary by race/ethnicity and language in Medicaid managed care. DATA SOURCES: The data analyzed are from the National Consumer Assessment of Health Plans (CAHPS) Benchmarking Database 1.0 and consist of 9,540 children enrolled in Medicaid managed care plans in Arkansas, Kansas, Minnesota, Oklahoma, Vermont, and Washington state from 1997 to 1998. DATA COLLECTION: The data were collected by telephone and mail, and surveys were administered in Spanish and English. The mean response rate for all plans was 42.1 percent. STUDY DESIGN: Data were analyzed using multiple regression models. The dependent variables are CAHPS 1.0 ratings (personal doctor, specialist, health care, health plan) and reports of care (getting needed care, timeliness of care, provider communication, staff helpfulness, plan service). The independent variables are race/ethnicity (white, African American, American Indian, Asian, and Hispanic), Hispanic language (English or Spanish), and Asian language (English or other), controlling for gender, age, education, and health status. PRINCIPAL FINDINGS: Racial/ethnic minorities had worse reports of care than whites. Among Hispanics and Asians language barriers had a larger negative effect on reports of care than race/ethnicity. For example, while Asian non-English-speakers had lower scores than whites for staff helpfulness (beta = -20.10), timeliness of care (beta = -18.65), provider communication (beta = -17.19), plan service (beta = -10.95), and getting needed care (beta = -8.11), Asian English speakers did not differ significantly from whites on any of the reports of care. However, lower reports of care for racial/ethnic groups did not translate necessarily into lower ratings of care. CONCLUSIONS: Health plans need to pay increased attention to racial/ethnic differences in assessments of care. This study's finding that language barriers are largely responsible for racial/ethnic disparities in care suggests that linguistically appropriate health care services are needed to address these gaps.

Differences in CAHPS adult survey reports and ratings by race and ethnicity: an analysis of the National CAHPS benchmarking data 1.0.

OBJECTIVE: To examine racial/ethnic group differences in adults' reports and ratings of care using data from the National Consumer Assessment of Health Plans (CAHPS) survey Benchmarking Database (NCBD) 1.0. DATA SOURCE: Adult data from the NCBD 1.0 is comprised of CAHPS 1.0 survey data from 54 commercial and 31 Medicaid health plans from across the United States. A total of 28,354 adult respondents (age > or = 18 years) were included in this study. Respondents were categorized as belonging to one of the following racial/ethnic groups: Hispanic (n = 1,657), white (n = 20,414), black or African American (n = 2,942), Asian and Pacific Islander (n = 976), and American Indian or Alaskan native (n = 588). STUDY DESIGN: Four single-item global ratings (personal doctor, specialty care, overall rating of health plan, and overall rating of health care) and five multiple-item report composites (access to needed care, provider communication, office staff helpfulness, promptness of care, and health plan customer service) from CAHPS 1.0 were examined. Statistical Analyses. Multiple regression models were estimated to assess differences in global ratings and report composites between whites and members of other racial/ethnic groups, controlling for age, gender, perceived health status, educational attainment, and insurance type. PRINCIPAL FINDINGS: Members of racial/ethnic minority groups, with the exception of Asians/Pacific Islanders, reported experiences with health care similar to those of whites. However, global ratings of care by Asians/Pacific Islanders are similar to those of whites. CONCLUSIONS: Improvements in quality of care for Asians/Pacific Islanders are needed. Comparisons of care in racially and ethnically diverse populations based on global ratings of care should be interpreted cautiously.

Confirmatory factor analysis of the Consumer Assessment of Health Plans Study (CAHPS) 1.0 Core Survey.

The National Consumer Assessment of Health Plans Study (CAHPS) Benchmarking Database was used to assess the factor structure and invariance of the CAHPS 1.0 Core Survey. Separate analyses were conducted with Latino and non-Latino Caucasian consumers drawn from commercial and Medicaid sectors (N = 15,092). Results demonstrated that the 23 CAHPS 1.0 report items measure consumer reports of experiences with 5 aspects of health plan performance: Access to Care, Timeliness of Care, Provider Communication, Health Plan Consumer Service, and Office Staff Helpfulness. Four items assessed global ratings of care. Analyses revealed an identical pattern of fixed and free factor loadings across all samples. Magnitude of factor loadings and correlations among factors was essentially equivalent within a common health service sector. A higher order factor analysis revealed that rating and reports of care showed marked convergence.

Development of the 25-item National Eye Institute Visual Function Questionnaire.

OBJECTIVE: To develop and test the psychometric properties of a 25-item version of the National Eye Institute Visual Function Questionnaire (NEI VFQ-25). DESIGN: Prospective observational cohort study of persons with 1 of 5 chronic eye diseases or low vision who were scheduled for nonurgent visits in ophthalmology practices and a reference sample of persons without eye disease. SETTING: Eleven university-based ophthalmology practices and the NEI Clinical Center. PATIENTS: Eligible participants had to have 1 of the following eye conditions: age-related cataracts, age-related macular degeneration, diabetic retinopathy, primary open-angle glaucoma, cytomegalovirus retinitis, or low vision from any cause. Seven of the 12 sites also enrolled persons in a reference sample. Reference sample participants had no evidence of underlying eye disease but were scheduled for either screening eye examinations or correction of refractive error. All eligible persons had to be 21 years or older, English speaking, and cognitively able to give informed consent and participate in a health status interview. MEASUREMENTS AND MAIN RESULTS: To provide the data needed to create the NEI VFQ-25, all subjects completed an interview that included the 51-item NEI VFQ. Estimates of internal consistency indicate that the subscales of the NEI VFQ-25 are reliable. The validity of the NEI VFQ-25 is supported by high correlations between the short- and long-form versions of the measure, observed between-group differences in scores for persons with different eye diseases of varying severity, and the moderate-to-high correlations between the NEI VFQ-25 subscales that have the most to do with central vision and measured visual acuity. CONCLUSIONS: The reliability and validity of the NEI VFQ-25 are comparable to those of the 51-item NEI VFQ field test version of the survey. This shorter version will be more feasible in settings such as clinical trials where interview length is a critical consideration. In addition, preliminary analyses indicate that the psychometric properties of the NEI VFQ-25 are robust for the eye conditions studied; this suggests that the measure will provide reproducible and valid data when used across multiple conditions of varying severity.

Using data to enhance the expert panel process. Rating indications of alcohol-related problems in older adults.

OBJECTIVE: To enhance the validity of a well-known expert panel process, we used data from patient surveys to identify and correct rating errors. METHODS: We used the two-round RAND/UCLA panel method to rate indications of harmful (presence of problems), hazardous (at risk for problems), and nonhazardous (no known risks) drinking in older adults. Results from the panel provided guidelines for classifying older individuals as harmful, hazardous, or nonhazardous drinkers, using a survey. The classifications yielded unexpectedly high numbers of harmful and hazardous drinkers. We hypothesized possible misclassifications of drinking risks and used the survey data to identify indications that may have led to invalid ratings. We modified problematic indications and asked three clinician panelists to evaluate the clinical usefulness of the modifications in a third panel round. We revised the indications based on panelist response and reexamined drinking classifications. RESULTS: Using the original indications, 48% of drinkers in the sample were classified as harmful, 31% as hazardous, and 21% as nonhazardous. A review of the indications revealed framing bias in the original rating task and vague definitions of certain symptoms and conditions. The modified indications resulted in classifications of 22% harmful, 47% hazardous, and 31% nonhazardous drinkers. CONCLUSIONS: Analysis of survey data led to identification and correction of specific errors occurring during the panel-rating process. The validity of the RAND/UCLA method can be enhanced using data-driven modifications.

Health behaviors, health status, and access to and use of health care: a population-based study of lesbian, bisexual, and heterosexual women.

BACKGROUND: There is a dearth of validated information about lesbian and bisexual women's health. To better understand some of these issues, we used population-based data to assess variations in health behaviors, health status, and access to and use of health care based on sexual orientation. METHODS: Our study population was drawn from a population-based sample of women, the 1997 Los Angeles County Health Survey. Participants reported their sexual orientation and these analyses included 4697 women: 4610 heterosexual women, 51 lesbians, and 36 bisexual women. We calculated adjusted relative risks to assess the effect of sexual orientation on important health issues. RESULTS: Lesbians and bisexual women were more likely than heterosexual women to use tobacco products and to report any alcohol consumption, but only lesbians were significantly more likely than heterosexual women to drink heavily. Lesbians and bisexual women were less likely than heterosexual women to have health insurance, more likely to have been uninsured for health care during the preceding year, and more likely to have had difficulty obtaining needed medical care. During the preceding 2 years, lesbians, but not bisexual women, were less likely than heterosexual women to have had a Papanicolaou test and a clinical breast examination. CONCLUSIONS: In this first population-based study of lesbian and bisexual women's health, we found that lesbians and bisexual women were more likely than heterosexual women to have poor health behaviors and worse access to health care. These findings support our hypothesis that sexual orientation has an independent effect on health behaviors and receipt of care, and indicate the need for the increased systematic study of the relationship between sexual orientation and various aspects of health and health care. Arch Fam Med. 2000;9:1043-1051

Development of subscales from the symptoms/problems and effects of kidney disease scales of the kidney disease quality of life instrument.

BACKGROUND: The Kidney Disease Quality of Life Instrument (KDQOL) was developed to provide clinicians with a comprehensive assessment of the important domains of health-related quality of life (HRQOL) for patients with end-stage renal disease who are undergoing hemodialysis. OBJECTIVE: The purpose of this study was to develop subscales from the 55 items comprising the Symptoms/Problems and Effects of Kidney Disease scales of the KDQOL and to measure the internal consistency reliability of these subscales. METHODS: The 55 items from the Symptoms/Problems and Effects of Kidney Disease scales were arranged into substantively meaningful clusters using an affinity mapping procedure. The resulting subscales were assessed for internal consistency reliability using data from a sample of 165 individuals with kidney disease who had completed the KDQOL. RESULTS: Eleven multi-item subscales were identified: pain, psychological dependency, cognitive functioning, social functioning, dialysis-related symptoms, cardiopulmonary symptoms, sleep, energy, cramps, diet, and appetite. Four items (clotting or other problems with access site, high blood pressure, numbness in hands or feet, and blurred vision) were not included in any of these subscales. Internal consistency reliability estimates for the 11 subscales ranged from 0.66 to 0.92. These subscales correlated with the scales from the 36-Item Short-Form Health Survey as hypothesized (ie, corresponding pain, energy, and social functioning scales had the highest correlations). In addition, several subscales were significantly associated, as hypothesized, with other variables such as the number of disability days. CONCLUSIONS: The results of this study further support the reliability and validity of the KDQOL. The 11 subscales identified yield more detailed information on the HRQOL of patients with kidney disease and provide a basis for specific improvements in the quality of care delivered to these patients.

A comparison of self-reported utilization of ophthalmic care for diabetes in managed care versus fee-for-service.

PURPOSE: To assess the association between structural factors in the health care delivery system and self-reported utilization of ophthalmic services by patients with diabetes in the Medical Outcomes Study (MOS). METHODS: Self-reported utilization of ophthalmic services within the 6 months preceding enrollment into the MOS among 522 of 567 individuals with diabetes in the MOS longitudinal panel was measured. Use of eye care services was regressed (logistic model) on patient demographics, geographic location, physician specialty, type of practice, and finance plan (prepaid or fee-for-service). RESULTS: None of the variables was significantly associated with a higher or lower likelihood of having used ophthalmic services in the preceding 6 months. Thus, no difference between prepaid or fee-for-service plans or among solo practice, large multispecialty groups, or HMOs were identified. Having seen an internist, family practitioner, or diabetes specialist for diabetes care was not related to use of ophthalmic services. CONCLUSIONS: Despite a presumed greater interest in preventive health, prepaid health plans were no more or less likely than the fee-for-service sector to have patients with diabetes reporting an eye examination within the prior 6 months. Thus, steps to improve the rate of eye examinations of diabetics may need to focus beyond the structural elements of the health care delivery system.

Member ratings of health care provided by 48 physician groups.

BACKGROUND: Satisfaction with health plan performance has been assessed frequently, but assessment of physician group performance is rare. OBJECTIVE: To present ratings of the care provided by physician groups to enrollees in a variety of capitated health maintenance organization plans. METHODS: A random sample was drawn of adult enrollees receiving managed health care from 48 physician groups in a group practice association. Each individual in the sample was mailed a 12-page questionnaire and 7093 were returned (59% response rate). The mean age of those returning the questionnaire was 51 years; 65% were women. RESULTS: Reliability estimates for 6 multi-item satisfaction scales were excellent, and noteworthy differences in ratings among groups were observed. In particular, ratings of overall quality ranged from a low of 28 to a high of 68 (mean, 50; SD, 10). Average scores for physician groups were strongly correlated across all scales, but no single group scored consistently highest or lowest on the different scales. Negative ratings of care were significantly related to the following: intention to switch to another physician group, difficulty in getting appointments, lengthy waiting periods in the reception area and examination room, the inability to get consistent care from one physician for routine visits, and not being informed by the office staff when there was a delay in seeing the primary care provider. CONCLUSIONS: Monitoring of health care quality at the physician group level is possible, and could be used for benchmarking, internal quality improvement, and for providing information to the public about how these physician groups will meet its needs.

Identifying the content area for the 51-item National Eye Institute Visual Function Questionnaire: results from focus groups with visually impaired persons.

OBJECTIVE: To identify the content area for a questionnaire designed to measure vision-targeted health-related quality of life and to determine whether problems with vision-related functioning are qualitatively similar across different common eye diseases. DESIGN: Twenty-six condition-specific focus groups were conducted with 246 patients from 5 geographic regions to identify the content area for a questionnaire for use among persons with diabetic retinopathy, glaucoma, macular degeneration, cytomegalovirus retinitis, and cataract. A standard protocol was used to structure each focus group discussion. Sessions were audiotaped, transcribed, and coded in preparation for a content analysis. SETTINGS: Five university-based ophthalmology practices and 1 nonprofit eye care foundation. PARTICIPANTS: Eligible participants had to have 1 of the following eye conditions: age-related cataracts, age-related macular degeneration, diabetic retinopathy, primary open angle glaucoma, cytomegalovirus retinitis, or low vision from any cause. All eligible persons were older than 21 years, spoke English, and had sufficient cognitive function to provide informed consent. RESULTS: Among the 246 participants, 2623 problems with vision-related functioning were mentioned. The mean number of problems per person ranged from 13.5 for those with diabetic retinopathy to 7.9 for persons with glaucoma. For the sample overall, reading problems were mentioned most frequently, followed by driving, general problems with seeing clearly, and mental health complaints caused by vision. Although the proportion of persons who reported each problem varied by condition, at least some persons with each eye disease reported each problem. The 3 most common descriptors associated with each problem were difficulty or ease of performance (13%), psychological distress associated with performance of the activity (11%), and complete inability to participate in a visual activity (11%). CONCLUSION: An item-generation strategy for a new questionnaire using a standardized focus group method identified content areas and aspects of visual disability that are not included in currently available vision-specific instruments that assess the impact of common eye diseases on visual functioning in every-day life. Although participants mentioned problems that were unique to their disease, across conditions the problems mentioned were similar. These findings provide empirical evidence of content validity for a vision-targeted, health-related quality-of-life survey designed for use across conditions.

Consumer satisfaction with CMHC services.

To examine the relationship between consumer satisfaction with community mental health clinic (CMHC) services and patterns of outpatient service use, we conducted a survey of 210 schizophrenics in Mississippi, the majority of whom were African American. Subjects with lowest CMHC satisfaction were those who did not identify the CMHC as their primary source of outpatient mental health care. They were more likely to be white, single, and to either receive no outpatient mental health care (46%) or to seek care from sources other than the CMHC (54%), many of which might provide substandard care, such as family doctors, ministers, folk healers, or hospital emergency rooms. Among those who identified the CMHC as their primary source of mental health care, we found little evidence that satisfaction was associated with type, variety, or frequency of services. Even though clinics offered similar services, there were differences in consumer satisfaction ratings by clinics, suggesting that qualities of the clinic itself may influence consumer satisfaction.

The impact of blurred vision on functioning and well-being.

OBJECTIVE: To determine the impact of blurred vision on functional status and well-being, the authors used a cross-sectional analysis of Medical Outcomes Study (MOS) 2-year follow-up data. METHODS: One thousand six hundred forty-two respondents were asked to report how often they experienced blurred vision not correctable by glasses or contact lenses, as well as other various symptoms, at the 2-year follow-up. Participants completed the SF-36, a general functioning and well-being instrument. Demographic, medical, and other data also were collected. The association of these symptoms, including blurred vision, and tracer medical conditions with SF-36 scores were evaluated. RESULTS: The unique impact of blurred vision on role limitations due to physical health problems was significantly greater than the impact of hypertension, history of myocardial infarction, type II diabetes mellitus, indigestion, trouble urinating, and headache. Blurred vision also had a significantly greater negative impact on energy than Type I diabetes mellitus, on social function than indigestion, and on physical functioning than trouble urinating. CONCLUSIONS: Having blurred vision more than once or twice a month has a detectable and significant impact on functional status and well-being, especially in role limitations due to physical health problems. This demonstrates the impact of a common visual symptom on health status and well-being, as measured by the SF-36. In addition, comparison of the impact of various symptoms and conditions provides important and potentially clinically relevant information.

The criminalization of persons with serious mental illness living in rural areas.

In the context of restrictive admissions policies at public inpatient facilities, rates of arrest and incarceration of persons with serious mental illness (SMI) have been increasing, leading to the perception that SMI persons are being unduly "criminalized." This paper describes the characteristics of persons with SMI (N = 177) in Mississippi who have been (1) jailed without charges while awaiting a hospital bed and (2) jailed for a criminal offense. Seventy-five percent (N = 132) of the 177 subjects had been held in local jails awaiting state hospital admission at least once in their lives, most for more than five days. Fifteen percent (N = 26) had contact with the police for a criminal offense in the past year. Rural residence markedly increased the risk for waiting in jail (OR = 4.24) but was not related to committing a criminal offense. Protective factors for any type of criminal justice contact were female gender, caucasian ethnicity, better compliance with medication regimes, and nonrural residence. The phenomenon of rural criminalization, i.e., waiting in jail without criminal charges, differs qualitatively from the criminalization that occurs in urban areas and may require a different solution. To avoid criminalization of SMI persons in rural areas, public mental health systems must develop effective crisis interventions in the community or work with local law enforcement officials and medical and mental health facilities to create more immediate access to acute inpatient care.

Characteristics and service use of seriously mentally ill persons living in rural areas.

OBJECTIVE: Data from a cross-sectional survey of 210 current or former inpatients at Mississippi State Hospital and 197 informants, usually patients' family members, were used to compare the demographic, clinical, and service use characteristics of patients living in rural areas with those of patients living in nonrural areas. METHODS: All subjects completed structured face-to-face interviews. Information on recent use of community mental health center (CMHC) services was obtained from CMHC records. Characteristics of rural patients (N = 107) and nonrural patients (N = 103) were compared using chi square and t tests. RESULTS: Rural patients were less likely to have completed high school and more likely to be living with family members. Although rural and nonrural subjects were clinically similar, rural subjects were less likely to have received certain community mental health services such as case management and day treatment and were more likely to have used physical health services and to have been involved with the criminal justice system. CONCLUSIONS: Differences between rural and nonrural patients in patterns of service use are most likely a reflection of less availability or accessibility of mental health services in rural areas, resulting in spillover into physical health and criminal justice sectors. Rural community mental health centers need to formally incorporate families, local medical providers, and law enforcement officials into their network of care.

Modified capitation and treatment incentives for end stage renal disease.

This study developed a modified capitation payment method for the Medicare end stage renal disease (ESRD) program designed to support appropriate treatment choices and protect health plans from undue financial risk. The payment method consists of risk-adjusted monthly capitated payments for individuals on dialysis or with functioning kidney grafts, lump sum event payments for expected incremental costs of kidney transplantations or graft failures, and outlier payments for expensive patients. The methodology explained 25 percent of variation in annual payments per patient. Risk adjustment captured substantial variations across patient groups. Outlier payments reduced health plan risk by up to 15 percent.

The relationship between visual acuity and functioning and well-being among diabetics.

Given the enormous recent interest in functional capabilities related to vision, the goal of this study was to examine the relationship of standard clinical measures of vision (e.g. Snellen acuity) to functioning and well-being. The association between Snellen visual acuity, Amsler grid distortion and presence of diabetic retinopathy with self-reported functioning and well-being (SF-36) were examined in a sample of 327 diabetics from the Medical Outcomes Study (MOS). There was little or no correlation between Snellen visual acuity, Amsler grid distortion or diabetic retinopathy and functioning and well-being (i.e. SF-36 scales). Maximum product-moment correlation was 0.15 with worst eye visual acuity, 0.13 with best eye visual acuity, 0.08 with presence of retinopathy, and 0.10 with Amsler grid distortion. Analysis of variance revealed that visual acuity (both best and worst eye) was statistically related only to the physical function scale; no other exam measure was related to any other SF-36 scale score. Snellen visual acuity, Amsler distortion and diabetic retinopathy correlate weakly with patient self-reported functioning and well-being. Thus, the information provided by functioning and well-being measures is complementary to that of standard clinical measures of visual ability.

Agreement between self reports and proxy reports of quality of life in epilepsy patients.

Agreement between self reports and proxy reports of quality of life was examined in a sample of 292 patients with epilepsy and their designated proxies. Patients and proxies completed an 89-item Quality of Life Inventory (QOLIE-89), with the items rephrased for the proxy. Results reveal moderate correlations (product-moment r ranging from 0.29 to 0.56 for 17 multi-item scales) between self reports and proxy reports. Agreement was good for measures of function that are directly observable and relatively poor for more subjective measures. Mean scale scores were significantly different between patients and proxies for only five of 17 multi-item scales. Proxy respondents systematically reported better functioning than did patients in three scales assessing cognitive functioning (all p < 0.001). By contrast, patients reported more positive health perceptions and less seizure distress than proxies. Patient educational attainment correlated inversely with degree of disagreement between patient and proxy reports for six of the 17 QOL scales and for the overall score. In addition, proxy educational attainment correlated positively with agreement for four scales. This study indicates that for group level comparisons, proxy respondents can be substituted for adults with epilepsy having low to moderate seizure frequency. However, for individual level assessments proxies should be used with caution.

Outcome assessment for epilepsy surgery: the impact of measuring health-related quality of life.

Epilepsy surgery is an increasingly common treatment for intractable epilepsy; yet there is no clear consensus among experts on how to report epilepsy surgery outcome. Most published outcome reporting systems focus on seizure frequency and type but differ in how they define clinically distinct outcome categories. We used a reliable and valid measure of self-reported health-related quality of life (HRQOL), the Epilepsy Surgery Inventory (ESI)-55, as an external standard by which to evaluate seven previously published, seizure-based outcome classification systems. The ESI-55 was administered to 133 adults who had previously undergone surgery for intractable epilepsy, and results were linked to data on their seizure occurrence before and after surgery (over the year prior to their HRQOL reports). These 133 patients were classified according to each seizure-based outcome system, and variation in HRQOL across outcome groups was evaluated using analysis of variance. Results reveal noteworthy variation in the extent to which different systems reflect patients' HRQOL at follow-up. We modified existing systems to derive a seizure-based surgery outcome system that most closely reflects HRQOL when applied over the latest 1-year postoperative interval.