Inter-Rater and Intra-Rater Reliability of the INSPECT (Interactive Nutrition Specific Physical Exam Competency Tool) Measured in Multi-Site Acute Care Settings.

Even as new medical modalities, diagnostics, and technologies are rapidly changing healthcare, providing patients with safe, high-quality care remains the central focus. To provide safe patient care, healthcare providers are obligated to demonstrate and maintain the necessary competence. As more healthcare disciplines move toward a competency-based education model, it is essential to extend the competence verification from the academic educational level to the patient's bedside. The nutrition-focused physical exam (NFPE) is a competency recently adopted by registered dietitian nutritionists (RDNs) for assessing patients' nutritional status. Being a newly acquired skill, validated tools are required to measure NFPE competence during routine clinical practice. The Interactive Nutrition Specific Physical Exam Competency Tool (INSPECT) is a new tool developed specifically to observe and measure RDNs' NFPE competence in clinical settings. The INSPECT was designed and validated for content using expert RDNs' input in the first and second phases of the study. This current study aimed to assess the reliability of the INSPECT through multi-site observations by clinical supervisors evaluating RDNs' NFPE competency during patient assessment. The INSPECT exhibited good inter-rater reliability (ICC = 0.78 for the first assessment and ICC = 0.68 for the second assessment), moderate to strong intra-rater reliability for 37 of 41 items (Spearman rho = 0.54 to 1.0), and excellent internal consistency (Cronbach's α = 0.86 for the first assessment and α = 0.92 for the second assessment). In total, 10 out of the 11 INSPECT subsets showed good to excellent internal consistency (α ranging from 0.70 to 0.98). The results demonstrate that the INSPECT is a reliable tool, is stable over time, and has good agreement and excellent consistency between raters. The INSPECT can be a valuable tool to measure, promote and maintain RDNs' NFPE competence in authentic acute care settings.

Item-Level Analysis of a Newly Developed Interactive Nutrition Specific Physical Exam Competency Tool (INSPECT) Using the Rasch Measurement Model.

The Interactive Nutrition Specific Physical Exam Competency Tool (INSPECT) is a tool designed specifically to observe and measure registered dietitian nutritionists' (RDNs) nutrition-focused physical exam (NFPE) competence in authentic acute care settings. The initial INSPECT items were generated and tested for content and face validity using expert RDNs' input. The INSPECT was further examined for inter-rater, intra-rater, and internal consistency using clinical supervisor observations of RDNs performing NFPE on patients in real-life acute care settings. These previous studies showed the INSPECT to have excellent content validity, acceptable face validity, good inter-rater reliability, moderate to strong intra-rater reliability, and excellent internal consistency. In the current study, the Rasch measurement model was applied to examine the item-level properties of the INSPECT. Results confirm that the INSPECT measured a single construct. All items fit the established criteria for clinical observations of >0.5 and <1.7, had positive point measure correlations, met the Wright Unidimensionality Index criteria of ≥0.9, exhibited one latent construct with >40% variance explained by the Rasch dimension as well as a sub-dimension based on item difficulty from the principal component analysis of the first contrast Rasch residuals. Rasch rating scale analysis revealed that the rating scale and majority of the items (39/41) fit the Rasch model. Rasch item hierarchy analysis matched the a priori hypothesized hierarchy for the top-most and bottom-most items. Ceiling effects were seen for three items (hand hygiene, personal protective equipment, and patient position) and one item (handgrip using hand dynamometer) reached the floor effect. Rasch reliability assessment demonstrated high person reliability (0.86), high item reliability (0.96), and person separation of 3.56 ability levels. The principal component analysis of residuals revealed two factors based on item difficulty, one for micronutrient exam and another for macronutrient exam, initial steps, and bedside manner. The resulting two factors may likely be due to a sub-dimension of the latent NFPE trait. Overall, the INSPECT items were found to have good item-level psychometrics. Continued testing of the INSPECT with RDNs at different ability levels will help to determine cut-off scores ranging from novice to expert. Establishing cut-off scores for the INSPECT will further enhance the utility of the tool.

Content and Face Validation of a Novel, Interactive Nutrition Specific Physical Exam Competency Tool (INSPECT) to Evaluate Registered Dietitians' Competence: A Delphi Consensus from the United States.

The nutrition-focused physical examination (NFPE) is an integral component of nutrition assessment performed by registered dietitian nutritionists (RDNs) to determine signs of malnutrition and other nutrition-related complications. Increased use of this essential skill among RDNs and the transformation of dietetics education to a competency-based model in the near future calls for appropriately validated tools to measure RDNs' NFPE competence. To fill the need for a validated competency tool, this study developed an Interactive Nutrition-Specific Physical Exam Competency Tool (INSPECT) utilizing the initial 70 items identified in the first phase of the study. The second phase of this study aimed to test the preliminary version of the INSPECT for content and face validity. An expert panel of 17 members provided consensus recommendations through the Delphi process. Internal consistency of the consensus was measured with Cronbach's alpha (α) and α of ≥0.70 was defined as acceptable a priori. Inter-rater agreement among the expert panel was determined using the intraclass correlation coefficient (ICC) and an a priori ICC of 0.75 to 0.9 was established as good and >0.9 as excellent agreement. The results showed acceptable face validity (α = 0.71) and excellent content validity for the INSPECT, with an internal consistency of α = 0.97 in the first round and α = 0.96 in the second round. The inter-rater agreement was also excellent with ICC = 0.95 for each of the Delphi rounds. A total of 52 items were retained from the preliminary version of the INSPECT. Open feedback from the experts allowed for the consolidation of 11 similar items for better scoring and evaluation and thus, a total of 41 items were included in the final version of the INSPECT. The final version of the INSPECT is currently being studied in real-life, multi-site clinical settings among practicing RDNs to examine construct validity, reliability, and item-level psychometric properties. Ultimately, the validated INSPECT will be available for the competency evaluation of RDNs practicing in clinical settings.

Item Generation in the Development of an Interactive Nutrition Specific Physical Exam Competency Tool (INSPECT): A Qualitative Study Utilizing Technology-Based Focus Groups in the United States.

An alarming 30% to 50% prevalence rate of disease-related malnutrition among hospitalized patients compels the need for early diagnosis and treatment of malnutrition. Registered Dietitian Nutritionists (RDNs) can utilize the nutrition-focused physical examination (NFPE) as one of the nutrition assessment criteria to accurately diagnose malnutrition. Although RDNs are striving to employ NFPE in practice, a lack of experience and adequate training impedes full utilization of this technique. This results in wide skill variations requiring continuous evaluation of RDNs' NFPE competency. However, a standardized, validated competency tool is not widely available and hence this study aims to develop a standardized, interactive nutrition-specific physical exam competency tool (INSPECT). As a first step in the development of INSPECT, a qualitative, technology-based focus group approach with 7 content and practice experts was utilized to generate appropriate tool items. A total of 70 NFPE items under 9 areas including 12 items for muscle loss, 4 items for subcutaneous fat loss, 31 items for micronutrient deficiencies, 1 item for fluid status, 2 items for handgrip strength, 5 items for initial preparation, 4 items for bedside manner, 8 items for swallowing, and 3 items for abdominal evaluation were generated. This study successfully utilized technology-based focus groups to generate appropriate NFPE items for the competency tool development. Using the items, an initial version of INSPECT has been developed, which is presently being investigated for content and face validity. The final version will undergo field tests and will be examined for reliability, validity, and item-level psychometric properties.

Psychometric evaluation of the disabilities of the arm, shoulder and hand (DASH) in patients with orthopedic shoulder impairments seeking outpatient rehabilitation.

STUDY DESIGN: This is a cross-sectional, observational study. INTRODUCTION: The disability of the arm, shoulder and hand (DASH) questionnaire is an upper-extremity specific outcome measure commonly used in routine clinical care and clinical trials. PURPOSE OF THE STUDY: Our purposes were to: (1) examine the psychometric properties of the DASH questionnaire using factor analysis, one- and two-parameter item response theory models, and (2) develop a functional staging map illustrating the relationships between the item difficulty hierarchy of the DASH items and the patient's DASH total score. METHODS: Data from 2724 patients with orthopedic shoulder impairments seeking outpatient physical therapy in 79 clinics in the US were analyzed. RESULTS: Factor analysis supported a general factor, explaining 62.2% of the total variance. The coverage of DASH items was suitable for patients with shoulder impairments with no ceiling or floor effect. Endorsed items representing the most difficult items were related to feeling less capable, executing recreational activities with force or impact, and performing recreational activities in which you move your arm freely. Items with higher discriminating abilities were those related to do heavy household chores, garden/yard work, and change a light bulb overhead. With a separation index equaled to 4.94, the DASH can differentiate persons into at least 6 statistically distinct person strata. None of the DASH items exhibited differential item functioning by gender or symptom acuity, except two items by age group. DISCUSSION: Besides a total (summed) final score obtained from a specific measure, the keyform and functional staging plots/maps can be used to assist clinicians in clinical interpretation of the scores. CONCLUSIONS: Results supported the clinical usage of the DASH questionnaire in patients with orthopedic shoulder impairments seeking outpatient rehabilitation.

Global rating of change: perspectives of patients with lumbar impairments and of their physical therapists.

Background: Global rating of change (GROC) scores provide a means of measuring patients' self-perceived change in health status over time. Objectives: The purposes of the study were to examine agreement of GROC ratings between the patient (GROCp) and the treating therapist (GROCt), the impact on the minimal clinically important difference (MCID) estimates by reporting resources, and the relationships between the subjective GROC scores and objective outcome measures. Methods: Data from 52,767 patients with orthopedic lumbar spine impairments treated in 623 outpatient rehabilitation clinics in 34 states (United States) were analyzed. Patient's functional status (FS) before and after the treatment was measured using the Lumbar Computerized Adaptive Testing (LCAT) survey (0-100 scale). The GROC ratings were obtained from both the patients and their caring therapists at discharge. Results: On average, GROCp (mean [(SD] = 3.1 [4.1]) was 1.9 points lower than GROCt (mean [SD] = 5.0 [1.9]). There were negligible differences between MCID estimates derived from the patients' GROC ratings (MCID cutoff = 9.2) and from the therapists' GROC ratings (MCID cutoff = 9.5). While only analyzing patient data whose GROCp ≥ 0, the intraclass correlation coefficient between GROCp and GROCt was 0.61, the GROCp correlated with FS at intake at r = 0.12, with FS at discharge at r = 0.56, and with FS change scores at r = 0.47. Conclusions: Therapists tended to report higher GROC ratings than their patients did but there were negligible differences between MCID estimates derived from GROCp and that derived from GROCt.

Rasch Analysis of the Activities-Specific Balance Confidence Scale in Older Adults Seeking Outpatient Rehabilitation Services.

Background The Activities-specific Balance Confidence (ABC) scale measures confidence in performing various ambulatory activities without falling or experiencing a sense of unsteadiness. Objectives This study (1) examined the ABC scale (0-100) using Rasch analysis, (2) assessed statistically reliable change, and (3) developed a functional staging to guide clinical interpretation of a patient's improvement. Methods The authors examined rating-scale structure, item difficulty hierarchy, item fit, person-item match, separation index, differential item functioning, test precision, and unidimensionality. Additionally, this cross-sectional study of 5012 older patients seeking outpatient rehabilitation therapy in 123 clinics estimated the minimal detectable change and developed a functional staging. Results The item "walk outside on icy sidewalks" was the most difficult item, while the item "reach for a small can off a shelf at eye level" was the easiest item. Overall, average patient ability estimates of 56.2 ± 20.3 were slightly higher than the average item difficulty estimates of 45.9 ± 7.8. With a separation index equal to 3.65, the ABC scale items can differentiate individuals into 5.2 statistically distinct strata. Most ABC scale items were free of differential item functioning. For example, "walk outside on icy sidewalks" was easier for patients who were underweight. Results supported unidimensionality of the ABC scale, with the first factor explaining 77% of the total variance. The estimated minimal detectable change was 15 points. The authors provided an example of functional staging application. Conclusion Results supported sound psychometric properties and clinical usage of the ABC scale for older adults seeking outpatient rehabilitation therapy. J Orthop Sports Phys Ther 2018;48(7):574-583. Epub 30 Mar 2018. doi:10.2519/jospt.2018.8023.

Quality of Life Perspectives of People With Amyotrophic Lateral Sclerosis and Their Caregivers.

This study explored differences in perspectives on quality of life (QOL) between people affected by amyotrophic lateral sclerosis (ALS) and their caregivers. QOL is often thought of as related to physical limitations, without consideration of other factors (e.g., cognitive, emotional) that may be stronger predictors of QOL in people with long-term degenerative diseases. Because QOL is complex and influenced by multiple factors, people with ALS and their caregivers may have different perspectives on what constitutes QOL. This study investigated potential discrepancies in QOL perspectives between people with ALS and their caregivers. Thirty dyads from the Augusta University Health ALS Clinic completed a measure of QOL, and we compared the results and identified patterns. The most prominent finding was that members of the dyads misunderstood the mental experiences of one another.

Influence of fear-avoidance beliefs on functional status outcomes for people with musculoskeletal conditions of the shoulder.

BACKGROUND: The influence of elevated fear-avoidance beliefs on change in functional status is unclear. OBJECTIVE: The purpose of this study was to determine the influence of fear-avoidance on recovery of functional status during rehabilitation for people with shoulder impairments. DESIGN: A retrospective longitudinal cohort study was conducted. METHODS: Data were collected from 3,362 people with musculoskeletal conditions of the shoulder receiving rehabilitation. At intake and discharge, upper-extremity function was measured using the shoulder Computerized Adaptive Test. Pain intensity was measured using an 11-point numerical rating scale. Completion rate at discharge was 57% for function and 47% for pain intensity. A single-item screen was used to classify patients into groups with low versus elevated fear-avoidance beliefs at intake. A general linear model (GLM) was used to describe how change in function is affected by fear avoidance in 8 disease categories. This study also accounted for within-clinic correlation and controlled for other important predictors of functional change in functional status, including various demographic and health-related variables. The parameters of the GLM and their standard errors were estimated with the weighted generalized estimating equations method. RESULTS: Functional change was predicted by the interaction between fear and disease categories. On further examination of 8 disease categories using GLM adjusted for other confounders, improvement in function was greater for the low fear group than for the elevated fear group among people with muscle, tendon, and soft tissue disorders (Δ=1.37, P<.01) and those with osteopathies, chondropathies, and acquired musculoskeletal deformities (Δ=5.52, P<.02). These differences were below the minimal detectable change. Limitations Information was not available on whether therapists used information on level of fear to implement treatment plans. CONCLUSIONS: The influence of fear-avoidance beliefs on change in functional status varies among specific shoulder impairments.

Examination of the factor structure of the Disabilities of the Arm, Shoulder, and Hand questionnaire.

OBJECTIVE: The Disabilities of the Arm, Shoulder, and Hand (DASH) questionnaire is a global scale evaluating the impact of upper-extremity disorders. We sought to validate or challenge the unidimensional factor structure of the DASH. METHOD: Secondary analysis was performed on data collected from outpatient clinics. Factor analysis was performed in two steps, exploratory factor analysis (EFA) followed by confirmatory factor analysis (CFA). To provide further support for unidimensionality, fit statistics were calculated using the Andrich Rasch rating-scale model. RESULTS: EFA revealed three potential factors (eigenvalues = 18.40, 1.56, and 1.54). CFA was performed fitting a three-factor model. Tucker-Lewis Index (.99) and standardized root mean square residual (.05) values indicated good fit. Comparative fit index (.89) and root mean square error of approximation (.13) did not. When divided into three constructs, only one item misfit. CONCLUSION: More research is needed to determine situations in which division of the DASH may enhance interpretability.

Development of an item bank for a computerised adaptive test of upper-extremity function.

PURPOSE: The purpose of this study was to determine the psychometric characteristics of an upper-extremity item bank as a precursor to developing a computer adaptive patient reported outcome instrument. The Activity dimension of the World Health Organization's International Classification of Functioning, Disability and Health (ICF) provided the conceptual framework for the items. METHOD: Factor and Rasch analyses were used to evaluate the psychometric properties of the item bank, including: monotonicity, local independence, dimensionality, item difficulty hierarchy and match between sample ability and item difficulty. RESULTS: Monotonicity of the rating scale was supported. Nine item pairs were locally dependent, and thus one item from each pair was removed from subsequent analyses. There was evidence for two unidimensional constructs; gross upper-extremity and fine hand. Both constructs showed good internal consistency and person separation. In general, the order of item difficulty within each construct replicated the hypothesised item difficulty order. The fine hand construct had a ceiling effect. CONCLUSIONS: The above study of our newly developed upper-extremity item bank empirically verified the intended item difficulty order, identified separate constructs (i.e. gross upper-extremity and fine hand) and provided insights into eliminating the ceiling effect of one of the constructs. These findings are critical precursors to the development of upper-extremity components of the ICF Activity Measure, an ICF-based, CAT located on the web at: www.icfmeasure.phhp.ufl.edu.

Creating a clinically useful data collection form for the DASH questionnaire.

PURPOSE: We generated a clinically useful data collection form for the Disabilities of the Arm, Shoulder, and Hand (DASH) questionnaire. This data collection form is designed to aid in measuring change and goal setting. METHOD: Rasch analysis was used to generate three data collection forms for constructs on the DASH (gross motor, fine motor, and symptoms; N = 960). A form was completed to represent the findings from 1 study participant. Admission and discharge data were illustrated for one of the three sections (gross motor activity). Possible goals were indicated on the admission form. The discharge form illustrates whether these goals have been achieved. RESULTS: Figures illustrate the utility of the forms in observing functional change from admission to discharge and how the forms aid in goal setting. CONCLUSION: Use of the data collection form has many positive implications. This type of form could aid in goal setting and treatment planning.

Ability to detect change in patient function: responsiveness designs and methods of calculation.

UNLABELLED: In hand clinics, the goal of enabling patient improvement is fostered by the use of assessments with the ability to detect change (responsiveness). Thus, for commonly used assessments, investigations are needed to determine a standardized change index, the amount of change exceeding error estimates (minimal detectable change or MDC), and the amount of change shown to make a clinically relevant difference (clinically important difference or CID). The purpose of study was to serve as an introduction for hand therapists to responsiveness designs and indices and to highlight their application within the clinical setting. The study design used was a narrative review. Method papers and research studies addressing responsiveness were selected and summarized. Currently, several good studies of responsiveness have been conducted. However, there is a need to move beyond the calculation of standardized change indices to include the calculation of clinically meaningful values. For many of the assessments used in hand clinics, there is still a call for investigation of the amount of change, which exceeds error estimates (MDC) and the amount of change shown to make a clinically relevant difference (CID). LEVEL OF EVIDENCE: N/A.

A comparison of the ability of two upper extremity assessments to measure change in function.

STUDY DESIGN: Retrospective Measurement Comparison. INTRODUCTION: Upper extremity musculoskeletal disorders affect millions, thus, discerning optimal assessments for measuring change in upper extremity function is critical. PURPOSE OF THE STUDY: To compare responsiveness (ability to measure change) of the Disabilities of Arm, Shoulder, and Hand (DASH) and Upper Extremity Functional Index (UEFI). METHODS: Statistical analyses included Rasch analysis to place the instruments on the same scale, analysis of variance to compare change scores, correlations to compare change scores with global ratings, and the use of receiver operating characteristic (ROC) curves to determine meaningful change scores and overall error. RESULTS: Change scores on the DASH and UEFI and correlations between change scores and global ratings were similar. Areas under the ROC curves for the DASH and UEFI were 67% and 65%, respectively. CONCLUSIONS: Neither assessment has a clear advantage over the other when measuring clinical change. LEVEL OF EVIDENCE: Not applicable.

Utilizing Rasch measurement models to develop a computer adaptive self-report of walking, climbing, and running.

PURPOSE: The purpose of this paper is to show how the Rasch model can be used to develop a computer adaptive self-report of walking, climbing, and running. METHOD: Our instrument development work on the walking/climbing/running construct of the ICF Activity Measure was used to show how to develop a computer adaptive test (CAT). Fit of the items to the Rasch model and validation of the item difficulty hierarchy was accomplished using Winsteps software. Standard error was used as a stopping rule for the CAT. Finally, person abilities were connected to items difficulties using Rasch analysis 'maps'. RESULTS: All but the walking one mile item fit the Rasch measurement model. A CAT was developed which selectively presented items based on the last calibrated person ability measure and was designed to stop when standard error decreased to a pre-set criterion. Finally, person ability measures were connected to the ability to perform specific walking/climbing/ running activities using Rasch maps. CONCLUSIONS: Rasch measurement models can be useful in developing CAT measures for rehabilitation and disability. In addition to CATs reducing respondent burden, the connection of person measures to item difficulties may be important for the clinical interpretation of measures.

Assessing arm and hand function after stroke: a validity test of the hierarchical scoring system used in the motor assessment scale for stroke.

OBJECTIVE: To evaluate the validity of the scoring hierarchy for the 3 upper-limb items on the Motor Assessment Scale (MAS). DESIGN: Application of Rasch analysis to 3 independent measurement scales, each representing the upper-arm function, hand movements, and advanced hand activities items of the MAS. SETTING: Inpatient and outpatient occupational therapy (OT) programs in a department of rehabilitation of an urban hospital center. PARTICIPANTS: One hundred patients (67 men, 33 women; average age, 54.3+/-14.4 y; average time since stroke onset, 104 d) attending OT for stroke rehabilitation. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURE: The MAS for stroke (upper-arm function, hand movements, and advanced hand activities sections). RESULTS: Rasch analysis provided support for the validity of hierarchical scoring criteria for the upper-arm scale. This analysis, however, identified inconsistencies in the hierarchical scoring criteria for the hand function and advanced hand activities scales and, when considering measurement error, only small differences in difficulty level between several behavioral criteria. CONCLUSIONS: The findings lead to suggestions for changes in the behavioral criteria hierarchy for upper-limb items on the MAS and highlight the importance of using statistical analyses to test the validity of proposed hierarchies of behavioral criteria in functional assessments.