Mapping Short Warwick and Edinburgh Mental Wellbeing Scale (SWEMWBS) to Recovering Quality of Life (ReQoL) to estimate health utilities.

BACKGROUND: The Short Warwick and Edinburgh Mental Wellbeing Scale (SWEMWBS) is a widely used non-preference-based measure of mental health in the UK. The primary aim of this paper is to construct an algorithm to translate the SWEMWBS scores to utilities using the Recovering Quality of Life Utility Index (ReQoL-UI) measure. METHODS: Service users experiencing mental health difficulties were recruited in two separate cross-sectional studies in the UK. The following direct mapping functions were used: Ordinary Least Square, Tobit, Generalised Linear Models. Indirect (response) mapping was performed using seemingly unrelated ordered probit to predict responses to each of the ReQoL-UI items and subsequently to predict using UK tariffs of the ReQoL-UI from SWEMWBS. The performance of all models was assessed by the mean absolute errors, root mean square errors between the predicted and observed utilities and graphical representations across the SWEMWBS score range. RESULTS: Analyses were based on 2573 respondents who had complete data on the ReQoL-UI items, SWEMWBS items, age and sex. The direct mapping methods predicted ReQoL-UI scores across the range of SWEMWBS scores reasonably well. Very little differences were found among the three regression specifications in terms of model fit and visual inspection when comparing modelled and actual utility values across the score range of the SWEMWBS. However, when running simulations to consider uncertainty, it is clear that response mapping is superior. CONCLUSIONS: This study presents mapping algorithms from SWEMWBS to ReQoL as an alternative way to generate utilities from SWEMWBS. The algorithm from the indirect mapping is recommended to predict utilities from the SWEMWBS.

Psychometric evaluation of the German version of the Recovering Quality of Life (ReQoL) measures in patients with affective disorders.

BACKGROUND: The generic self-reported Recovering Quality of Life (ReQoL) measures were developed for measuring recovery-focused health-related quality of life (HrQoL) in persons with mental health conditions. The aim of this study was to assess the psychometric properties of the German version of the ReQoL measures in patients with affective disorders in Germany. METHODS: Data from a patient sub-sample in a randomized controlled trial have been used (N = 393). The internal consistency and the test-retest reliability of the ReQoL measures were assessed using Cronbach's Alpha and the intra-class correlation coefficient (ICC). The concurrent validity and the known-group validity of the ReQoL measures were assessed using Pearson's Correlation coefficient and Cohen's d. The responsiveness was assessed using Glass' Δ and the standardized response mean (SRM). RESULTS: The reliability among the items of the ReQoL-20 was overall excellent. The ICC of the ReQoL-20 was r = 0.70, indicating moderate test-retest reliability. The concurrent validity of the ReQoL-20 with the clinical measure PHQ-9 was strong with a correlation coefficient of r = - 0.76. The known-group validity of the ReQoL-20 using PHQ-9 cut-off points was large with an effect size of d = 1.63. The ReQoL measures were sensitive to treatment response and remission of symptoms measured by the PHQ-9 with large effect sizes/SRM. DISCUSSION: The psychometric properties of the ReQoL measures for the assessment of patients with affective disorders were overall good. With the ReQoL, valid and reliable measures for the assessment of recovery-focused HrQoL for persons with affective disorders are available in German language.

Developing an International Standard Set of Patient-Reported Outcome Measures for Psychotic Disorders.

OBJECTIVE: The objective of this project was to develop a set of patient-reported outcome measures for adolescents and adults who meet criteria for a psychotic disorder. METHODS: A research team and an international consensus working group, including service users, clinicians, and researchers, worked together in an iterative process by using a modified Delphi consensus technique that included videoconferencing calls, online surveys, and focus groups. The research team conducted systematic literature searches to identify outcomes, outcome measures, and risk adjustment factors. After identifying outcomes important to service users, the consensus working group selected outcome measures, risk adjustment factors, and the final set of outcome measures. International stakeholder groups consisting of >100 professionals and service users reviewed and commented on the final set. RESULTS: The consensus working group identified four outcome domains: symptoms, recovery, functioning, and treatment. The domains encompassed 14 outcomes of importance to service users. The research team identified 131 measures from the literature. The consensus working group selected nine measures in an outcome set that takes approximately 35 minutes to complete. CONCLUSIONS: A set of patient-reported outcome measures for use in routine clinical practice was identified. The set is free to service users, is available in at least two languages, and reflects outcomes important to users. Clinicians can use the set to improve clinical decision making, and administrators and researchers can use it to learn from comparing program outcomes.

Measuring health-related quality of life and well-being: a head-to-head psychometric comparison of the EQ-5D-5L, ReQoL-UI and ICECAP-A.

OBJECTIVE: This study aimed to assess the psychometric properties of three generic preference-based measures and compare their performance in a sample of Hong Kong general population. METHODS: Data used for this analysis were obtained from a cross-sectional telephone-based survey in July 2020. Participants were asked to complete several measures, including The EuroQol five-dimensional five levels (EQ-5D-5L), Recovering Quality of Life-Utility Index (ReQoL-UI) and ICEpop CAPability measure for adults (ICECAP-A). Acceptability, reliability, convergent and discriminant validity of three measures were assessed as well as the agreement between these instruments. RESULTS: Based on data from 500 participants to the survey, a lower mean score of the ICECAP-A (mean = 0.85) was observed compared to the other two measures (meanReQoL-UI = 0.92; meanEQ-5D-5L = 0.92). All three measures showed an acceptable internal consistency reliability (Cronbach's alpha = 0.74, 0.82 and 0.77, respectively) as well as good test-retest reliability (intra-class correlation coefficient = 0.74, 0.82 and 0.77, respectively). Correlation analyses confirmed satisfactory convergent validity and the ability of the measures to differentiate between participants with different health or from socioeconomic status groups. The Bland-Altman plot revealed poor agreement between the three measures. CONCLUSIONS: This study confirmed that EQ-5D-5L, ReQoL-UI and ICECAP-A were psychometrically robust to measure HRQoL in the general HK population. The EQ-5D-5L was more suitable for assessing physical HRQoL, whereas the ICECAP-A and ReQoL-UI were more appropriate for measuring interventions aimed at improving people's well-being and mental health.

Psychometric Evaluation of the Chinese Recovering Quality of Life (ReQoL) Outcome Measure and Assessment of Health-Related Quality of Life During the COVID-19 Pandemic.

OBJECTIVE: The primary objective was to translate the Recovering Quality of Life (ReQoL) measures from English to traditional Chinese and assess their psychometric properties in Hong Kong (HK) Chinese population. The secondary objective was to investigate the mental health-related quality of life (HRQoL) of this sample during the coronavirus disease 2019 (COVID-19) pandemic. METHOD: Recovering Quality of Life was translated to Traditional Chinese adhering to standard guideline recommended by the official distributors. Five hundred members of the general population were successfully recruited to participate in a telephone-based survey. The following psychometric properties of the ReQoL were evaluated: construct, convergent, and known-group validity and internal consistency and test-retest reliability. The item measurement invariance was assessed on the basis of differential item functioning (DIF). Multiple regression analysis was used to assess the relationship between respondents' characteristics and mental HRQoL. RESULTS: Results of confirmatory factor analysis (CFA) supported a two-factor structure of the ReQoL. The ReQoL showed significant correlations with the other mental health, quality of life, and well-being measures, which indicated a satisfactory convergent validity. Known-group validity confirmed that ReQoL is able to differentiate between people with different mental health status. The (Cronbach's alpha = 0.91 and 0.76 for positive [PF] and negative [NF] factor), and McDonald's omega of 0.89 (PF = 0.94, NF = 0.82) indicated the ReQoL has good reliability as well as test-retest reliability with an intraclass correlation coefficient of 0.75. Four items showed negligible DIF with respect to age. Respondents who were highly educated and without psychological problems reported a high ReQoL score. CONCLUSION: Traditional Chinese ReQoL was shown to be a valid and reliable instrument to assess the recovery-focused quality of life in HK general population. Future studies are needed to appraise its psychometric properties in local people experiencing mental disorders.

Estimating a Preference-Based Index for Mental Health From the Recovering Quality of Life Measure: Valuation of Recovering Quality of Life Utility Index.

BACKGROUND: There are increasing concerns about the appropriateness of generic preference-based measures to capture health benefits in the area of mental health. OBJECTIVES: The aim of this study is to estimate preference weights for a new measure, Recovering Quality of Life (ReQoL-10), to better capture the benefits of mental healthcare. METHODS: Psychometric analyses of a larger sample of mental health service users (n = 4266) using confirmatory factor analyses and item response theory were used to derive a health state classification system and inform the selection of health states for utility assessment. A valuation survey with members of the UK public representative in terms of age, sex, and region was conducted using face-to-face interviewer administered time-trade-off with props. A series of regression models were fitted to the data and the best performing model selected for the scoring algorithm. RESULTS: The ReQoL-Utility Index (UI) classification system comprises 6 mental health items and 1 physical health item. Sixty-four health states were valued by 305 participants. The preferred model was a random effects model, with significant and consistent coefficients and best model fit. Estimated utilities modeled for all health states ranged from -0.195 (state worse than dead) to 1 (best possible state). CONCLUSIONS: The development of the ReQoL-UI is based on a novel application of item response theory methods for generating the classification system and selecting health states for valuation. Conventional time-trade-off was used to elicit utility values that are modeled to enable the generation of QALYs for use in cost-utility analysis of mental health interventions.

An item response theory analysis of an item pool for the recovering quality of life (ReQoL) measure.

PURPOSE: ReQoL-10 and ReQoL-20 have been developed for use as outcome measures with individuals aged 16 and over, experiencing mental health difficulties. This paper reports modelling results from the item response theory (IRT) analyses that were used for item reduction. METHODS: From several stages of preparatory work including focus groups and a previous psychometric survey, a pool of items was developed. After confirming that the ReQoL item pool was sufficiently unidimensional for scoring, IRT model parameters were estimated using Samejima's Graded Response Model (GRM). All 39 mental health items were evaluated with respect to item fit and differential item function regarding age, gender, ethnicity, and diagnosis. Scales were evaluated regarding overall measurement precision and known-groups validity (by care setting type and self-rating of overall mental health). RESULTS: The study recruited 4266 participants with a wide range of mental health diagnoses from multiple settings. The IRT parameters demonstrated excellent coverage of the latent construct with the centres of item information functions ranging from - 0.98 to 0.21 and with discrimination slope parameters from 1.4 to 3.6. We identified only two poorly fitting items and no evidence of differential item functioning of concern. Scales showed excellent measurement precision and known-groups validity. CONCLUSION: The results from the IRT analyses confirm the robust structure properties and internal construct validity of the ReQoL instruments. The strong psychometric evidence generated guided item selection for the final versions of the ReQoL measures.

Public involvement in health outcomes research: lessons learnt from the development of the recovering quality of life (ReQoL) measures.

BACKGROUND: To provide a model for Public involvement (PI) in instrument development and other research based on lessons learnt in the co-production of a recently developed mental health patient reported outcome measure called Recovering Quality of Life (ReQoL). While service users contributed to the project as research participants, this paper focuses on the role of expert service users as research partners, hence referred to as expert service users or PI. METHODS: At every stage of the development, service users influenced the design, content and face validity of the measure, collaborating with other researchers, clinicians and stakeholders who were central to this research. Expert service users were integral to the Scientific Group which was the main decision-making body, and also provided advice through the Expert Service User Group. RESULTS: During the theme and item generation phase (stage 1) expert service users affirmed the appropriateness of the seven domains of the Patient Reported Outcome Measure (activity, hope, belonging and relationships, self-perception, wellbeing, autonomy, and physical health). Expert service users added an extra 58 items to the pool of 180 items and commented on the results from the face and content validity testing (stage 2) of a refined pool of 88. In the item reduction and scale generation phase (stage 3), expert service users contributed to discussions concerning the ordering and clustering of the themes and items and finalised the measures. Expert service users were also involved in the implementation and dissemination of ReQoL (stage 4). Expert service users contributed to the interpretation of findings, provided inputs at every stage of the project and were key decision-makers. The challenges include additional work to make the technical materials accessible, extra time to the project timescales, including time to achieve consensus from different opinions, sometimes strongly held, and extra costs. CONCLUSION: This study demonstrates a successful example of how PI can be embedded in research, namely in instrument development. The rewards of doing so cannot be emphasised enough but there are challenges, albeit surmountable ones. Researchers should anticipate and address those challenges during the planning stage of the project.

Exploring the item sets of the Recovering Quality of Life (ReQoL) measures using factor analysis.

PURPOSE: This paper presents two studies exploring the latent structure of item sets used in the development of the Recovering Quality of Life mental health outcome measures: ReQoL-10 and ReQoL-20. METHOD: In study 1, 2262 participants completed an initial set of 61 items. In study 2, 4266 participants completed a reduced set of 40 items. Study 2 evaluated two formats of the questionnaires: one version where the items were intermingled and one where the positively worded and negatively worded items were presented as two separate blocks. Exploratory and confirmatory factor analyses were conducted on both datasets where models were specified using ordinal treatment of the item responses. Dimensionality based on the conceptual framework and methods effects reflecting the mixture of positively worded and negatively worded items were explored. Factor invariance was tested across the intermingled and block formats. RESULTS: In both studies, a bi-factor model (study 1: RMSEA = 0.061; CFI = 0.954; study 2: RMSEA = 0.066; CFI = 0.971) with one general factor and two local factors (positively worded questions and negatively worded questions) was preferred. The loadings on the general factor were higher than on the two local factors suggesting that the ReQoL scale scores can be understood in terms of a general factor. Insignificant differences were found between the intermingled and block formats. CONCLUSIONS: The analyses confirmed that the ReQoL item sets are sufficiently unidimensional to proceed to item response theory analysis. The model was robust across different ordering of positive and negative items.

Integrating Qualitative and Quantitative Data in the Development of Outcome Measures: The Case of the Recovering Quality of Life (ReQoL) Measures in Mental Health Populations.

While it is important to treat symptoms, there is growing recognition that in order to help people with mental health problems lead meaningful and fulfilling lives, it is crucial to capture the impact of their conditions on wider aspects of their social lives. We constructed two versions of the Recovering Quality of Life (ReQoL) measure—ReQoL-10 and ReQoL-20—for use in routine settings and clinical trials from a larger pool of items by combining qualitative and quantitative evidence covering six domains. Qualitative evidence was gathered through interviews and focus groups with over 76 service users, clinicians, and a translatability assessment. Psychometric evidence generated from data from over 6200 service users was obtained from confirmatory factor models and item response theory analyses. In this paper we present an approach based on a traffic light pictorial format that was developed to present qualitative and quantitative evidence to a group of service users, clinicians, and researchers to help to make the final selection. This work provides a pragmatic yet rigorous approach to combining qualitative and quantitative evidence to ensure that ReQoL is psychometrically robust and has high relevance to service users and clinicians. This approach can be extended to the development of patient reported outcome measures in general.

The importance of content and face validity in instrument development: lessons learnt from service users when developing the Recovering Quality of Life measure (ReQoL).

PURPOSE: Service user involvement in instrument development is increasingly recognised as important, but is often not done and seldom reported. This has adverse implications for the content validity of a measure. The aim of this paper is to identify the types of items that service users felt were important to be included or excluded from a new Recovering Quality of Life measure for people with mental health difficulties. METHODS: Potential items were presented to service users in face-to-face structured individual interviews and focus groups. The items were primarily taken or adapted from current measures and covered themes identified from earlier qualitative work as being important to quality of life. Content and thematic analysis was undertaken to identify the types of items which were either important or unacceptable to service users. RESULTS: We identified five key themes of the types of items that service users found acceptable or unacceptable; the items should be relevant and meaningful, unambiguous, easy to answer particularly when distressed, do not cause further upset, and be non-judgemental. Importantly, this was from the perspective of the service user. CONCLUSIONS: This research has underlined the importance of service users' views on the acceptability and validity of items for use in developing a new measure. Whether or not service users favoured an item was associated with their ability or intention to respond accurately and honestly to the item which will impact on the validity and sensitivity of the measure.

Recovering Quality of Life (ReQoL): a new generic self-reported outcome measure for use with people experiencing mental health difficulties.

BACKGROUND: Outcome measures for mental health services need to adopt a service-user recovery focus. Aims To develop and validate a 10- and 20-item self-report recovery-focused quality of life outcome measure named Recovering Quality of Life (ReQoL). METHOD: Qualitative methods for item development and initial testing, and quantitative methods for item reduction and scale construction were used. Data from >6500 service users were factor analysed and item response theory models employed to inform item selection. The measures were tested for reliability, validity and responsiveness. RESULTS: ReQoL-10 and ReQoL-20 contain positively and negatively worded items covering seven themes: activity, hope, belonging and relationships, self-perception, well-being, autonomy, and physical health. Both versions achieved acceptable internal consistency, test-retest reliability (>0.85), known-group differences, convergence with related measures, and were responsive over time (standardised response mean (SRM) > 0.4). They performed marginally better than the Short Warwick-Edinburgh Mental Well-being Scale and markedly better than the EQ-5D. CONCLUSIONS: Both versions are appropriate for measuring service-user recovery-focused quality of life outcomes. Declaration of interest M.B. and J.Co. were members of the research group that developed the Clinical Outcomes in Routine Evaluation (CORE) outcome measures.