Case-mix adjustment of the National CAHPS benchmarking data 1.0: a violation of model assumptions?

OBJECTIVE: To compare models for the case-mix adjustment of consumer reports and ratings of health care. DATA SOURCES: The study used the Consumer Assessment of Health Plans (CAHPS) survey 1.0 National CAHPS Benchmarking Database data from 54 commercial and 31 Medicaid health plans from across the United States: 19,541 adults (age > or = 18 years) in commercial plans and 8,813 adults in Medicaid plans responded regarding their own health care, and 9,871 Medicaid adults responded regarding the health care of their minor children. STUDY DESIGN: Four case-mix models (no adjustment; self-rated health and age; health, age, and education; and health, age, education, and plan interactions) were compared on 21 ratings and reports regarding health care for three populations (adults in commercial plans, adults in Medicaid plans, and children in Medicaid plans). The magnitude of case-mix adjustments, the effects of adjustments on plan rankings, and the homogeneity of these effects across plans were examined. DATA EXTRACTION: All ratings and reports were linearly transformed to a possible range of 0 to 100 for comparability. PRINCIPAL FINDINGS: Case-mix adjusters, especially self-rated health, have substantial effects, but these effects vary substantially from plan to plan, a violation of standard case-mix assumptions. CONCLUSION: Case-mix adjustment of CAHPS data needs to be re-examined, perhaps by using demographically stratified reporting or by developing better measures of response bias.

Differences in CAHPS adult survey reports and ratings by race and ethnicity: an analysis of the National CAHPS benchmarking data 1.0.

OBJECTIVE: To examine racial/ethnic group differences in adults' reports and ratings of care using data from the National Consumer Assessment of Health Plans (CAHPS) survey Benchmarking Database (NCBD) 1.0. DATA SOURCE: Adult data from the NCBD 1.0 is comprised of CAHPS 1.0 survey data from 54 commercial and 31 Medicaid health plans from across the United States. A total of 28,354 adult respondents (age > or = 18 years) were included in this study. Respondents were categorized as belonging to one of the following racial/ethnic groups: Hispanic (n = 1,657), white (n = 20,414), black or African American (n = 2,942), Asian and Pacific Islander (n = 976), and American Indian or Alaskan native (n = 588). STUDY DESIGN: Four single-item global ratings (personal doctor, specialty care, overall rating of health plan, and overall rating of health care) and five multiple-item report composites (access to needed care, provider communication, office staff helpfulness, promptness of care, and health plan customer service) from CAHPS 1.0 were examined. Statistical Analyses. Multiple regression models were estimated to assess differences in global ratings and report composites between whites and members of other racial/ethnic groups, controlling for age, gender, perceived health status, educational attainment, and insurance type. PRINCIPAL FINDINGS: Members of racial/ethnic minority groups, with the exception of Asians/Pacific Islanders, reported experiences with health care similar to those of whites. However, global ratings of care by Asians/Pacific Islanders are similar to those of whites. CONCLUSIONS: Improvements in quality of care for Asians/Pacific Islanders are needed. Comparisons of care in racially and ethnically diverse populations based on global ratings of care should be interpreted cautiously.

Using data to enhance the expert panel process. Rating indications of alcohol-related problems in older adults.

OBJECTIVE: To enhance the validity of a well-known expert panel process, we used data from patient surveys to identify and correct rating errors. METHODS: We used the two-round RAND/UCLA panel method to rate indications of harmful (presence of problems), hazardous (at risk for problems), and nonhazardous (no known risks) drinking in older adults. Results from the panel provided guidelines for classifying older individuals as harmful, hazardous, or nonhazardous drinkers, using a survey. The classifications yielded unexpectedly high numbers of harmful and hazardous drinkers. We hypothesized possible misclassifications of drinking risks and used the survey data to identify indications that may have led to invalid ratings. We modified problematic indications and asked three clinician panelists to evaluate the clinical usefulness of the modifications in a third panel round. We revised the indications based on panelist response and reexamined drinking classifications. RESULTS: Using the original indications, 48% of drinkers in the sample were classified as harmful, 31% as hazardous, and 21% as nonhazardous. A review of the indications revealed framing bias in the original rating task and vague definitions of certain symptoms and conditions. The modified indications resulted in classifications of 22% harmful, 47% hazardous, and 31% nonhazardous drinkers. CONCLUSIONS: Analysis of survey data led to identification and correction of specific errors occurring during the panel-rating process. The validity of the RAND/UCLA method can be enhanced using data-driven modifications.

Development of subscales from the symptoms/problems and effects of kidney disease scales of the kidney disease quality of life instrument.

BACKGROUND: The Kidney Disease Quality of Life Instrument (KDQOL) was developed to provide clinicians with a comprehensive assessment of the important domains of health-related quality of life (HRQOL) for patients with end-stage renal disease who are undergoing hemodialysis. OBJECTIVE: The purpose of this study was to develop subscales from the 55 items comprising the Symptoms/Problems and Effects of Kidney Disease scales of the KDQOL and to measure the internal consistency reliability of these subscales. METHODS: The 55 items from the Symptoms/Problems and Effects of Kidney Disease scales were arranged into substantively meaningful clusters using an affinity mapping procedure. The resulting subscales were assessed for internal consistency reliability using data from a sample of 165 individuals with kidney disease who had completed the KDQOL. RESULTS: Eleven multi-item subscales were identified: pain, psychological dependency, cognitive functioning, social functioning, dialysis-related symptoms, cardiopulmonary symptoms, sleep, energy, cramps, diet, and appetite. Four items (clotting or other problems with access site, high blood pressure, numbness in hands or feet, and blurred vision) were not included in any of these subscales. Internal consistency reliability estimates for the 11 subscales ranged from 0.66 to 0.92. These subscales correlated with the scales from the 36-Item Short-Form Health Survey as hypothesized (ie, corresponding pain, energy, and social functioning scales had the highest correlations). In addition, several subscales were significantly associated, as hypothesized, with other variables such as the number of disability days. CONCLUSIONS: The results of this study further support the reliability and validity of the KDQOL. The 11 subscales identified yield more detailed information on the HRQOL of patients with kidney disease and provide a basis for specific improvements in the quality of care delivered to these patients.

Member ratings of health care provided by 48 physician groups.

BACKGROUND: Satisfaction with health plan performance has been assessed frequently, but assessment of physician group performance is rare. OBJECTIVE: To present ratings of the care provided by physician groups to enrollees in a variety of capitated health maintenance organization plans. METHODS: A random sample was drawn of adult enrollees receiving managed health care from 48 physician groups in a group practice association. Each individual in the sample was mailed a 12-page questionnaire and 7093 were returned (59% response rate). The mean age of those returning the questionnaire was 51 years; 65% were women. RESULTS: Reliability estimates for 6 multi-item satisfaction scales were excellent, and noteworthy differences in ratings among groups were observed. In particular, ratings of overall quality ranged from a low of 28 to a high of 68 (mean, 50; SD, 10). Average scores for physician groups were strongly correlated across all scales, but no single group scored consistently highest or lowest on the different scales. Negative ratings of care were significantly related to the following: intention to switch to another physician group, difficulty in getting appointments, lengthy waiting periods in the reception area and examination room, the inability to get consistent care from one physician for routine visits, and not being informed by the office staff when there was a delay in seeing the primary care provider. CONCLUSIONS: Monitoring of health care quality at the physician group level is possible, and could be used for benchmarking, internal quality improvement, and for providing information to the public about how these physician groups will meet its needs.

Consumer satisfaction with CMHC services.

To examine the relationship between consumer satisfaction with community mental health clinic (CMHC) services and patterns of outpatient service use, we conducted a survey of 210 schizophrenics in Mississippi, the majority of whom were African American. Subjects with lowest CMHC satisfaction were those who did not identify the CMHC as their primary source of outpatient mental health care. They were more likely to be white, single, and to either receive no outpatient mental health care (46%) or to seek care from sources other than the CMHC (54%), many of which might provide substandard care, such as family doctors, ministers, folk healers, or hospital emergency rooms. Among those who identified the CMHC as their primary source of mental health care, we found little evidence that satisfaction was associated with type, variety, or frequency of services. Even though clinics offered similar services, there were differences in consumer satisfaction ratings by clinics, suggesting that qualities of the clinic itself may influence consumer satisfaction.

Characteristics and service use of seriously mentally ill persons living in rural areas.

OBJECTIVE: Data from a cross-sectional survey of 210 current or former inpatients at Mississippi State Hospital and 197 informants, usually patients' family members, were used to compare the demographic, clinical, and service use characteristics of patients living in rural areas with those of patients living in nonrural areas. METHODS: All subjects completed structured face-to-face interviews. Information on recent use of community mental health center (CMHC) services was obtained from CMHC records. Characteristics of rural patients (N = 107) and nonrural patients (N = 103) were compared using chi square and t tests. RESULTS: Rural patients were less likely to have completed high school and more likely to be living with family members. Although rural and nonrural subjects were clinically similar, rural subjects were less likely to have received certain community mental health services such as case management and day treatment and were more likely to have used physical health services and to have been involved with the criminal justice system. CONCLUSIONS: Differences between rural and nonrural patients in patterns of service use are most likely a reflection of less availability or accessibility of mental health services in rural areas, resulting in spillover into physical health and criminal justice sectors. Rural community mental health centers need to formally incorporate families, local medical providers, and law enforcement officials into their network of care.

Modified capitation and treatment incentives for end stage renal disease.

This study developed a modified capitation payment method for the Medicare end stage renal disease (ESRD) program designed to support appropriate treatment choices and protect health plans from undue financial risk. The payment method consists of risk-adjusted monthly capitated payments for individuals on dialysis or with functioning kidney grafts, lump sum event payments for expected incremental costs of kidney transplantations or graft failures, and outlier payments for expensive patients. The methodology explained 25 percent of variation in annual payments per patient. Risk adjustment captured substantial variations across patient groups. Outlier payments reduced health plan risk by up to 15 percent.

Long-term functioning and well-being outcomes associated with physical activity and exercise in patients with chronic conditions in the Medical Outcomes Study.

This study was carried out to determine whether levels of physical activity of patients with various chronic diseases are associated with subsequent functioning and well-being. It was an observational 2-year longitudinal design. The setting was offices of medical and mental health practices within health maintenance organizations, large multispecialty groups, and solo practices or small single-specialty group practices in three U.S. cities. Included in the study were 1758 adult patients with one or more of the following: diabetes, hypertension, congestive heart failure, recent myocardial infarction, depressive symptoms, or current depressive disorder. Outcome measures included physical, role, and functioning; energy/fatigue; pain intensity; sleep problems; depressed affect, anxiety, positive affect, and overall psychological distress/well-being; health distress; and current health perceptions. Cross-sectional (base-line), 2-year endpoint, and change score relationships were evaluated between baseline levels of physical activity and each outcome, controlling for chronic conditions, comorbidity, smoking, alcohol use, overweight, self-reported adherence, and other patient and study characteristics. Higher baseline levels of exercise were uniquely associated with better functioning and well-being at baseline and 2 years later for some measures. The magnitude of the differences varied by disease group, but tended to be between 0.17 and 0.39 of the baseline SD. Greater levels of exercise are associated with feeling and functioning better for patients with chronic conditions over a 2-year period, suggesting that this is a fruitful area for further study using controlled interventions.