The reliability and validity of lung cancer and melanoma clinical quality survival measures.

OBJECTIVE: To develop a risk adjustment approach and test reliability and validity for oncology survival measures. DATA SOURCES AND STUDY SETTING: We used the National Cancer Institute's Surveillance, Epidemiology, and End Results (SEER)-Medicare data from 2010 to 2013, with mortality data through 2015. STUDY DESIGN: We developed 2-year risk-standardized survival rates (RSSR) for melanoma, non-small cell lung cancer (NSCLC), and small cell lung cancer (SCLC). Patients were attributed to group practices based on the plurality of visits. We identified the risk-adjustment variables via bootstrap and calculated the RSSRs. Reliability was tested via three approaches: (1) signal-to-noise ratio (SNR) reliability, (2) split-half, and (3) test-retest using bootstrap. We tested known group validity by stage at diagnosis using Cohen's d. DATA COLLECTION/EXTRACTION METHODS: We selected all patients enrolled in Medicare and linked to SEER during the measurement period with an incident first primary diagnosis of stage I-IV melanoma, NSCLC, or SCLC. We excluded patients with missing data on month and/or stage of diagnosis. PRINCIPAL FINDINGS: Results are based on patients with melanoma (n = 4344); NSCLC (n = 16,080); and SCLC (n = 2807) diagnosed between 2012 and 2013. The median (interquartile range) for the RSSRs at the group practice-level were 0.89 (0.83-0.87) for melanoma, 0.37 (0.30-0.43) for NSCLC, and 0.19 (0.11-0.25) for SCLC. C-statistics for the models ranged from 0.725 to 0.825. The reliability varied by approach with median SNR 0.20, 0.25, and 0.13; median test-retest 0.59, 0.57, and 0.56; median split-half reliability 0.21, 0.29, and 0.29 for melanoma, NSCLC, and SCLC, respectively. Cohen's d for stage I-IIIa and IIIb+ was 1.27, 0.86, 0.60 for melanoma, NSCLC, and SCLC, respectively. CONCLUSIONS: Our results suggest that these cancer survival measures demonstrated adequate test-retest reliability and expected findings for the known-group validity analysis. If data limitations and feasibility challenges can be addressed, implementation of these quality measures may provide a survival metric used for oncology quality improvement efforts.

Development and Evaluation of a Risk-Adjusted Measure of Intraoperative Hypotension in Patients Having Nonemergent, Noncardiac Surgery.

BACKGROUND: Intraoperative hypotension is common and associated with organ injury and death, although randomized data showing a causal relationship remain sparse. A risk-adjusted measure of intraoperative hypotension may therefore contribute to quality improvement efforts. METHODS: The measure we developed defines hypotension as a mean arterial pressure <65 mm Hg sustained for at least 15 cumulative minutes. Comparisons are based on whether clinicians have more or fewer cases of hypotension than expected over 12 months, given their patient mix. The measure was developed and evaluated with data from 225,389 surgeries in 5 hospitals. We assessed discrimination and calibration of the risk adjustment model, then calculated the distribution of clinician-level measure scores, and finally estimated the signal-to-noise reliability and predictive validity of the measure. RESULTS: The risk adjustment model showed acceptable calibration and discrimination (area under the curve was 0.72 and 0.73 in different validation samples). Clinician-level, risk-adjusted scores varied widely, and 36% of clinicians had significantly more cases of intraoperative hypotension than predicted. Clinician-level score distributions differed across hospitals, indicating substantial hospital-level variation. The mean signal-to-noise reliability estimate was 0.87 among all clinicians and 0.94 among clinicians with >30 cases during the 12-month measurement period. Kidney injury and in-hospital mortality were most common in patients whose anesthesia providers had worse scores. However, a sensitivity analysis in 1 hospital showed that score distributions differed markedly between anesthesiology fellows and attending anesthesiologists or certified registered nurse anesthetists; score distributions also varied as a function of the fraction of cases that were inpatients. CONCLUSIONS: Intraoperative hypotension was common and was associated with acute kidney injury and in-hospital mortality. There were substantial variations in clinician-level scores, and the measure score distribution suggests that there may be opportunity to reduce hypotension which may improve patient safety and outcomes. However, sensitivity analyses suggest that some portion of the variation results from limitations of risk adjustment. Future versions of the measure should risk adjust for important patient and procedural factors including comorbidities and surgical complexity, although this will require more consistent structured data capture in anesthesia information management systems. Including structured data on additional risk factors may improve hypotension risk prediction which is integral to the measure's validity.

Reliability of the Deployment Resiliency Assessment.

This article describes the reliability of the instruments embedded in a mental health screening instrument designed to detect risky drinking, depression, and post-traumatic stress disorder among members of the Armed Forces. The instruments were generally reliable, however, the risky drinking screen (Alcohol Use Disorders Identification Test-Consumption) had unacceptable reliability (α = 0.58). This was the first attempt to assess psychometric properties of a screening and assessment instrument widely used for members of the Armed Forces.

Mental disorders among non-elderly nursing home residents.

Many Medicaid beneficiaries aged 22 to 64 with serious mental illness may be admitted to nursing facilities rather than psychiatric facilities as a result of Medicaid policies prohibiting coverage of inpatient psychiatric care in institutions of mental disease while requiring states to cover nursing facility care. Using nationwide Medicaid Analytic Extract claims from 2002, we found that nearly 16% of nursing home residents aged 22 to 64 had a diagnosed mental disorder, while 45.5% received antipsychotic medication, but these rates varied widely across states. Further research is necessary to determine whether, among the nation's youngest nursing home residents, care in nursing homes is potentially substituting for care in institutions for mental disease or community-based settings.

How many nursing home residents live with a mental illness?

OBJECTIVE: A number of data sets can be used to estimate the size of the nursing home population that has mental illness; however, estimates vary because of differences in methods of data collection. This study sought to compare estimates from three nationally representative data sets of the number of nursing home residents who have a mental illness, determine which data set provides the best national-level estimate, and identify the types of policy and monitoring questions that can best be answered with each. METHODS: The study compared estimates of the number of nursing home residents who had either a primary or any diagnosed mental illness from the National Nursing Home Survey (NNHS), the Minimum Data Set (MDS), and the Medicaid Analytic eXtract (MAX) files. RESULTS: The NNHS produced the most valid national-level estimates of residents with a mental illness--nearly 102,000 with a primary diagnosis in 2004 (6.8% of residents), of which about 23,000 were under age 65 and 79,000 were aged 65 and older. However, data from the NNHS cannot be broken down to the state level; therefore, state- and facility-level estimates would have to be generated with the MDS or MAX data sets. CONCLUSIONS: Policy makers and program managers need to be aware of the strengths and limitations of the data they use in order to make informed decisions. Users of the NNHS, MDS, and MAX data sets should be aware of the differences in recorded diagnoses among the three, especially the relatively limited diagnoses in the MAX and imprecise diagnoses in the MDS.

Reliability of a structured assessment for nonclinicians to detect delirium among new admissions to postacute care.

OBJECTIVE: To evaluate the interrater reliability of a structured delirium assessment method for nonclinician interviewers in elderly patients newly admitted for postacute care. DESIGN: Prospective assessment using dyads of nonclinician raters. SETTING: Postacute (Medicare) units at 6 skilled nursing facilities. PARTICIPANTS: Forty elderly patients newly admitted for postacute care from medical or surgical units at acute care hospitals. MEASUREMENTS: Subjects underwent dual delirium assessments within 5 days of admission. The standardized delirium assessment included the Mini-Mental Status Exam and Digit Span to assess overall cognitive function, the Delirium Symptom Interview to elicit specific delirium symptoms, the Memorial Delirium Assessment Scale to measure the severity of delirium, and the Confusion Assessment Method (CAM) to make the diagnosis of delirium. A coding protocol that linked observations to specific coding was used to improve reliability. RESULTS: The structured delirium assessment process produced very high interobserver agreement for all instruments. Kappa for agreement on delirium diagnosis was 0.95. CONCLUSIONS: Nonclinician interviewers using a structured delirium assessment achieved reliability that rivaled or exceeded that of trained clinical assessors in other studies. Nonclinicians may offer an effective alternative for the assessment of delirium among postacute patients in skilled nursing facilities.

The effect of urinary incontinence on quality of life in older nursing home residents.

OBJECTIVES: To determine whether nursing home residents with urinary incontinence (UI) have worse quality of life (QoL) than continent residents, whether the relationship between UI and QoL differs across strata of cognitive and functional impairment, and whether change in continence status is associated with change in QoL. DESIGN: Retrospective cohort study using a Minimum Data Set (MDS) database to determine cross-sectional and longitudinal (6 month) associations between UI and QoL. SETTING: All Medicare- or Medicaid-licensed nursing homes in Kansas, Maine, Mississippi, New York, and South Dakota during 1994 to 1996. PARTICIPANTS: All residents aged 65 and older, excluding persons unable to void or with potentially unstable continence or QoL status (recent nursing home admission, coexistent delirium, large change in functional status, comatose, near death). MEASUREMENTS: UI was defined as consistent leakage at least twice weekly over 3 months and continence as consistent dryness over 3 months. QoL was measured using the validated MDS-derived Social Engagement Scale. RESULTS: Of 133,111 eligible residents, 90,538 had consistent continence status, 58,850 (65%) of whom were incontinent. UI was significantly associated with worse QoL in residents with moderate cognitive and functional impairment. New or worsening UI over 6 months was associated with worse QoL (odds ratio = 1.46, 95% confidence interval = 1.36-1.57) and was second only to cognitive decline and functional decline in predicting worse QoL. CONCLUSION: This is the first study to quantitatively demonstrate that prevalent and new or worsening UI decreases QoL even in frail, functionally and cognitively impaired nursing home residents. These results provide a crucial incentive to improve continence care and quality in nursing homes and a rationale for targeting interventions to those residents most likely to benefit.

Outcomes of older people admitted to postacute facilities with delirium.

OBJECTIVES: To compare outcomes of patients admitted to postacute skilled nursing facilities with delirium, subsyndromal delirium, and no delirium. DESIGN: Observational cohort study. SETTING: Seven skilled nursing facilities that specialize in postacute care within a single metropolitan region. PARTICIPANTS: Five hundred four subjects chosen from 1,248 consenting subjects aged 65 and older who underwent mental status testing within 5 days of admission to the participating facilities. Subjects who met full Confusion Assessment Method (CAM) criteria were classified as delirious, those with one or more CAM criteria were classified as having subsyndromal delirium, and those with no CAM features were classified as having no delirium. All subjects with delirium and with available medical records were included. A random subset of subjects with no delirium and subsyndromal delirium with available medical records was included. MEASUREMENTS: The medical records of all subjects underwent a structured review by trained research nurses who were masked to the subjects' initial delirium status. Records were reviewed for the development of new complications within the postacute setting and to determine whether the subjects were discharged within 30 days and, if so, the discharge destination. The National Death Index was used to assess 6-month mortality. RESULTS: Subjects with delirium were more likely to experience one or more complications than subjects with no delirium (73% vs 41%, P < .01). Within 30 days of postacute admission, subjects with delirium were more than twice as likely to be rehospitalized (30% vs 13%), and less than half as likely to be discharged to the community (30% vs 73%) than subjects without delirium (differences P < .01). Subjects with subsyndromal delirium had outcomes intermediate between those with and without delirium. Finally, subjects admitted to the postacute setting with delirium experienced a 6-month mortality rate of 25.0%, compared with 5.7% in subjects admitted without delirium. Subjects with subsyndromal delirium had a 6-month mortality rate of 18.3%. CONCLUSION: Patients admitted to postacute skilled nursing facilities with delirium are more likely to experience complications, rehospitalization, and death than patients without delirium. These findings support the need for improved case finding and management of delirium in postacute care.

Delirium symptoms in post-acute care: prevalent, persistent, and associated with poor functional recovery.

OBJECTIVES: To determine the prevalence of delirium symptoms at the time of admission to post-acute facilities, the persistence of delirium symptoms in this setting, and the association of delirium symptoms with functional recovery. DESIGN: Prospective cohort study. SETTING: Eighty-five post-acute care facilities: 55 rehabilitation hospitals and 30 skilled nursing facilities in 29 states. PARTICIPANTS: Five hundred fifty-one consenting patients aged 65 and older newly admitted to participating facilities from acute care hospitals. MEASUREMENTS: Data were collected as part of a field study effort related to the Minimum Data Set (MDS). Basic demographic data, medical comorbidity, delirium symptoms, and functional status--activities of daily living (ADLs) and instrumental activities of daily living (IADLs)--were obtained from MDS assessments performed within 4 days of admission and again 1 week later by the patient's primary nurse. Six delirium symptoms (easily distracted, periods of altered perception, disorganized speech, periods of restlessness, periods of lethargy, and mental function varies over the course of a day) were assessed after appropriate training. RESULTS: Of the 551 patients (mean age +/- standard deviation 78 +/- 7, 64% women), 126 had delirium symptoms on post-acute admission, for an overall prevalence of 23%. In patients with delirium symptoms on the admission assessment, 1 week later, 14% had completely resolved, 22% had fewer delirium symptoms, 52% had the same number of symptoms, and 12% had more symptoms. Of those with no delirium symptoms on admission, 4% had new symptoms 1 week later. Patients who had the same number of or more delirium symptoms at the second assessment had significantly worse ADL and IADL recovery than those with fewer or resolved delirium symptoms or those with no delirium symptoms at either assessment. Persistent delirium symptoms remained significantly associated with worse ADL and IADL recovery after adjusting for age, comorbidity, dementia, and baseline functional status. CONCLUSIONS: The data from this study provide strong preliminary evidence that, in patients newly admitted to post-acute care facilities from acute care hospitals, delirium symptoms are prevalent, persistent, and associated with poor functional recovery. Educational efforts are warranted to help post-acute facility staff recognize and manage this common and morbid condition.

A screening system for Michigan's home- and community-based long-term care programs.

PURPOSE: To develop a screening system for Michigan's MI Choice publicly funded home- and community-based services programs, to aid in identifying both individuals eligible for services and their most appropriate level of care (LOC). DESIGN AND METHODS: Identify assessment items from the Minimum Data Set for Home Care (MDS-HC) assessment instrument that are predictive of five LOCs determined by expert care managers: nursing home, home care, intermittent personal care, homemaker, and information and referral (without services). RESULTS: The algorithm based on approximately 30 client characteristics agrees with expert opinions substantially better (kappa =.62) than systems based on activities of daily living and instrumental activities of daily living only (kappa <.40). IMPLICATIONS: The screening algorithm can be used both over the telephone to identify clients who will not be fully assessed (as they are unlikely to receive services) and in person to recommend the appropriate LOC.