Assessment of the Psychometric Properties of the Danish VISA-P.

Purpose: The objective of the current study was to conduct a rigorous assessment of the psychometric properties of the Victorian Institute of Sports Assessment-patellar tendinopathy (VISA-P). Methods: Rasch analysis, confirmatory factor analysis (CFA), and multivariable linear regression were used to assess the psychometric properties of the VISA-P questionnaire in 184 Danish patients with patellar tendinopathy who had symptoms ranging from under 3 months to over 1 year. A group of 100 healthy Danish persons was included as a reference for known-group validation. Results: The analyses revealed that the 8-item VISA-P did not fit a unidimensional model, yielded at best a 3-factor model, and exhibited differential item functioning (DIF) across healthy subjects versus people with patellar tendinopathy. Conclusion: VISA-P in its present form does not satisfy a measurement model and is not a robust scale for measuring patellar tendinopathy. A new PROM for patellar tendinopathy should be developed and appropriately validated, and meanwhile, simple pain scoring (e.g., numeric rating scales) and functional tests are suggested as more appropriate outcome measures for studies of patellar tendinopathy.

A catalogue of PROMs in sports science: Quality assessment of PROM development and validation.

Choosing the most adequate PROM for a study is a non-trivial process. The aim of this study was to provide a catalogue with analyses of content and construct validity of PROMs relevant to research in sports science, including all published local translations. The most commonly used PROMs in sports research were selected from a PubMed search "patient reported outcome measures sports", identifying 439 articles and 194 different PROMs. Articles describing development of the 61 selected PROMs were assessed for content validity, and all articles regarding construct validity of each PROM and all published translations (in total 622 articles) were analyzed. A catalogue with assessments of the 61 PROMs was produced. The majority were of inferior validity, with few exceptions. The most common reason for this was that the PROM had not been developed by methods that ensure high content validity. Another major reason for inferior validity was that construct validity had not been secured by adequate statistical methods. In conclusion, this catalogue provides a tool for researchers to facilitate choosing the most valid PROM for studies in sports research. Furthermore, it shows for popular PROMs where further validation is needed, and for fields in musculoskeletal medicine where valid PROMs are lacking. It is suggested that a targeted effort is made to develop valid PROMs for major conditions in musculoskeletal research. The current method is easier to practice compared with assessment after COSMIN guidelines.

Psychometric validation of PROM instruments.

The aim was to provide an overview of the different statistical methods for validation of patient-reported outcome measures, ranging from simple statistical methods available in all software packages to advanced statistical models that require specialized software. A non-technical summary of classical test theory (CTT) and modern test theory (MTT) is provided. Specifically, confirmatory factor analysis, item response theory, and Rasch analysis is outlined. One CTT and three MTT methods were used to validate the two subscales (Symptoms and Quality of Life) from the Knee Injury and Osteoarthritis Outcome Score (KOOS). For each methodology, two analyses were considered: (i) a unidimensional analysis ignoring the pre-specified dimensionality, and (ii) a two-dimensional analysis using the pre-specified dimensionality. While CTT did not adequately address central issues regarding the validity of the KOOS subscales, the three MTT methods yielded very similar results. In conclusion, MTT methods offer analysis of all relevant properties related to the validity of patient-reported outcome measures, while this is not the case for CTT. Claims about sufficient validity based on CTT methods are inadequate and should not be trusted.

Choosing the most appropriate PROM for clinical studies in sports medicine.

Choosing the most appropriate patient-reported outcome measure (PROM) for a clinical study is essential in order to achieve trustworthy results. This choice will depend on (a) the objective of the study and hence the research question; (b) the choice of a theoretical framework, such as the World Health Organization's International Classification of Functioning, Disability, and Health (ICF); (c) whether there currently is a PROM that possesses high content validity and high construct validity for the specific patient group and objective, and if not; (d) the decision on whether to use a suboptimal PROM or develop and validate a new PROM. This paper presents the steps that should be followed in order to assess the relevance of PROMs and suggests ways to enhance the choice depending on the goal of the study.

What is a PROM and why do we need it?

The purpose of this article was to introduce the reader to the nature of patient-reported outcome measures (PROMs) and pitfalls in their use. PROMs collect subjective information directly from the patient regarding specific or general conditions and add to clinical and functional outcomes, and turn unmeasurable subjective qualities into quantitative measures. PROMs are questionnaires consisting of items: questions or statements with predefined response options. The items in an adequate PROM have been developed by involvement of patients with the condition in focus, and the PROM has been validated for these patients using suitable statistical methods. An adequate well-targeted PROM is more responsive than an inadequate PROM. Unfortunately, many studies use inadequate PROMs as outcomes. The methods used to generate PROMs should be described as thoroughly as those used to develop any other types of measurement instruments, and the choice of PROM should always be explained and thereby justified. If the PROM used is not adequate, the consequences for the interpretation of the results should be discussed. In many cases, an adequate PROM does not exist. If the best available PROM is chosen, there are methods to validate the adequacy of the chosen PROM, which make an interpretation of the study results possible.

Are adequate PROMs used as outcomes in randomized controlled trials? an analysis of 54 trials.

Results by patient-reported outcome measures (PROMs) from randomized controlled trials (RCTs) in musculoskeletal research often influence healthcare strategies. We aimed to evaluate to which extent these RCTs use adequate PROMs, and how this influences the results and conclusions. We identified RCTs of sports research relevance with PROMs as primary outcomes published in 13 preselected journals between January 1, 2008, and November 1, 2019; all journals regularly publish results from musculoskeletal research. Five journals have a high impact factor (>15), and eight with lower impact factors are widely read journals. It was assessed whether the RCTs had used PROMs with high content validity and whether the most adequate PROMs were used (ie, the most well developed and well validated for the patients enrolled in the study). We registered journal impact factor, year of publication, existence of a registered protocol, and whether the study showed significant difference between interventions. A total of 54 RCTs with 56 primary outcomes comprising 26 different PROMs were identified. For 13 RCTs (24%), a protocol was not published. In only 24 of RCTs (44%), the most appropriate PROM had been used as primary outcome, independent of a registered protocol, ranking of the journal, and year of publication. In seven cases, PROMs were used to evaluate a condition that they had not been developed for. RCTs that used the most adequate PROM showed significantly more often (46%) difference in outcomes in contrast to RCTs that used inadequate PROMs (22%) (P = 0.0483). In conclusion, in the majority of RCTs, the most adequate PROM had not been used. Studies, in which the most adequate PROM had been used as outcome, were significantly more likely to show significant difference between interventions. The extent to which protocols were not available was surprisingly high. Journals should request that adequate PROMs are used in RCTs, and if this is not the case that it is discussed how it might influence the results and conclusions. Likewise, it should be requested that a protocol is published or registered.

Potential problems in the use of patient reported outcome measures (PROMs) and reporting of PROM data in sports science.

To use an inadequate patient reported outcome measure (PROM) or use a PROM in an inappropriate way potentially influences the quality of measurement. The objectives of this study were to define potential inadequate uses of PROMs in sports research studies and estimate how often they occur. A consensus group consisting of medical researchers, statisticians, and psychometricians identified and defined potentially irregular applications of PROMs. Occurrence of these in 349 consecutive articles in sports medicine in which PROMs were used as primary outcomes was reviewed. In all, 14 different potential problems were defined, and one or several occurred in 172 of the articles (49%). These were as follows: using a PROM that was developed for a different patient group (100 cases), using two or more PROMs with identical questions (94), aggregation of domain sum scores (82), combinations of subjective and objective measures (27), using a PROM to diagnose or evaluate the individual patient (7), using a PROM for a single limb (3), recall bias (3), exclusion of domains or items (3), construction of a PROM for a specific occasion (2), categorization of the scale (2), and mixing different versions of a PROM (1). Adaption of scale scores (e. g., to percentage) when results are reported (144) carries a risk of miscalculation and distorted impression of results. Data related to uncertainty about completing the PROM and the handling of missing data were not provided in the manuscripts. In conclusion, potential problems in the use and reporting of PROMs are common in sports research, and this can influence the validity of reported results.

How to develop a condition-specific PROM.

Developing new patient-reported outcome measures (PROMs) for application in clinical studies can be necessary if an adequate PROM does not exist. For adequate measurement, it is essential that the PROM has face validity (ie, is perceived to be relevant by clinicians and researchers) and has high content validity (ie, content relevance and content coverage for the targeted patient group). The steps needed to create PROMs that possess face and content validity for a specific condition are described in this paper. Face validity is achieved by item identification and generation through literature review. Content validity is confirmed through repetitive cognitive interviews of patients from the targeted patient group in order to generate a consensus-based pilot-version of the new PROM. This qualitative process ensures that items are appropriately worded, understandable, and minimizes doubts about how items should be answered. A practical example of this process is presented, which shows the development of the Knee Numeric-Entity Evaluation Score (KNEES-ACL), a condition-specific PROM for patients with deficiency of the anterior cruciate ligament (ACL).

How to translate and locally adapt a PROM. Assessment of cross-cultural differential item functioning.

Translating patient-reported outcome measures (PROMs) can alter the meaning of items and undermine the PROM's psychometric properties (quantified as cross-cultural differential item functioning [DIF]). The aim of this paper was to present the theoretical background for PROM translation, adaptation, and cross-cultural validation, and assess how PROMs used in sports medicine research have been translated and adapted. We also assessed DIF for the Knee Injury and Osteoarthritis Outcome Score (KOOS) across Danish, Norwegian, and Swedish versions. We conducted a search in PubMed and Scopus to identify the method of translation, adaptation, and validation of PROMs relevant to musculoskeletal research. Additionally, 150 preoperative KOOS questionnaires were obtained from the Scandinavian knee ligament reconstruction registries, and cross-cultural DIF was evaluated using confirmatory factor analysis and Rasch analysis. There were 392 studies identified, describing the translation of 61 PROMs. Ninety-four percent were performed with forward-backward technique. Forty-nine percent used cognitive interviews to ensure appropriate wording, understandability, and adaptation to the target culture. Only two percent were validated according to modern test theory. No study assessed cross-cultural DIF. One KOOS subscale showed no cross-cultural DIF, two had DIF with respect to some (but not all) items, and thus conversion tables could be constructed, and two KOOS subscales could not be pooled. Most PROM translations are of undocumented quality, despite the common conclusion that they are valid and reliable. Scores from three of five KOOS subscales can be pooled across the Danish, Norwegian, and Swedish versions, but two of these must be adjusted for DIF.

Are PROMs used adequately in sports research? An analysis of 54 randomized controlled trials with PROMs as endpoint.

Deviations from adequate use and reporting of PROMs may be problematic and misleading. The aim of this study was to investigate the extent of such problems in randomized clinical trials (RCTs). RCTs involving sports medicine research that used PROMs as primary outcomes were identified in 13 preselected journals. The articles were reviewed for nine potential problems related to how the PROM was used and how the data had been reported. The potential problems were as follows: aggregating subscale scores; combining patient-reported scores with physical, clinical, or para-clinical measures; using a PROM to diagnose or evaluate the individual patient; using a PROM for one leg or arm; selectively excluding domains or items; constructing a PROM for the specific occasion; mixing PROM formats (ie, digital, paper, telephone, e-mail, in person); ambiguous instructions for how the PROM should be completed; and recall bias. As covariates, we registered journal impact factor, year of publication, and existence of a registered protocol. In 29 (53.7%) of 54 identified RCTs, at least one potential problem was identified, the most common being aggregation of domain scores. This was not different with a published protocol or dependent on journal rankings, except for exclusion of domains, which was most common in high-ranking journals. Aggregation of domain scores was significantly less common in recently published articles compared with older articles (P = .03). Potential problematic use of PROMs and reporting of PROM data are common in RCTs, also in high-ranking journals, but less so in more recent articles.

Adaptation of the Activity Measure Post-Acute Care (AM-PAC) from English to Mandarin using the dual-panel translation approach.

PURPOSE: The aims of this study were to translate and adapt the Activity Measure Post-Acute Care (AM-PAC) from US English to Mandarin using the dual-panel method, and to assess its psychometric properties in an outpatient rehabilitation setting. METHODS: The AM-PAC outpatient short forms were translated using the dual-panel method. The translated AM-PAC was tested in 550 Chinese-speaking rehabilitation outpatients. Floor and ceiling effects were evaluated and internal consistency was assessed using Cronbach's alpha. Spearman correlation was used to assess the concurrent validity of the AM-PAC with the Barthel Index and the Mini-Mental State Examination. Test-retest reliability was determined by administering the AM-PAC twice to 57 participants within a 2-7 day interval. RESULTS: Some ceiling effects (>20%) were observed in the Applied Cognition subscale. All subscales exhibited good internal consistency (α > 0.70). Supportive evidence for concurrent validity was found in strong correlations between Basic Mobility subscale and Barthel Index (r = 0.68), and Daily Activity subscale and Barthel Index (r = 0.70); and moderate correlations between Applied Cognition subscale and Mini-Mental State Examination (r = 0.50). Test-retest reliability for all subscales was high (intraclass correlation coefficient =0.89-0.98). CONCLUSIONS: The dual-panel approach was successfully used to translate the AM-PAC from English to Mandarin. Adequate reliability and validity in rehabilitation outpatients in Taiwan were established. Implications for Rehabilitation The dual-panel method is a modern translation technique, which was successfully used to adapt the Activity Measure Post-Acute Care from English to Mandarin. The Mandarin version of Activity Measure Post-Acute Care demonstrates adequate internal consistency, concurrent validity, and test-retest reliability in rehabilitation outpatients. The Activity Measure Post-Acute Care is superior to existing functional measures used to monitor activity performance for rehabilitation patients in Taiwan.

Ensuring face validity in patient-related outcome scores--a matter of content.

BACKGROUND: Patient reported outcome (PRO) questionnaires are increasingly used to measure treatment effect in patients with knee pathology. PROs commonly used to assess outcome in patients with knee conditions can be generic, knee-specific, or condition-specific. Most PROs have been created based on clinician-based consensus and are not patient-centered. Items (questions plus their response options) in PROs can be generated by clinicians or through patient interviews. Items created by clinicians possess face validity. The objective of this study was to find all existing PRO items with potentially relevant content for patients with knee pathology. METHODS: An exhaustive literature search was conducted for PRO questionnaires in English, German, and Scandinavian languages used to assess outcome in patients with knee pathology. The items from the collected PROs were assessed for content redundancy and item reduction was carried out to isolate items of unique content. These items were grouped into one of the components of the ICF classification system. RESULTS: Thirty-one PROs used for assessment of patients with knee problems were identified, yielding 539 items. Approximately 70% of these items consisted of redundant content matter. The item pool was reduced to a pool of 157 items. CONCLUSION: The search yielded 157 items of unique content. CLINICAL RELEVANCE: The identified items can be used to build condition-specific PRO questionnaires for patients with knee pathology.