Why it is hard to use PROMs and PREMs in routine health and care.

Patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) show the results of healthcare activities as rated by patients and others. Patients or their proxies record feedback using questionnaires. These can enhance quality for all and tailored care for individuals. This paper describes obstacles that inhibit widespread use of PROMs and PREMs and some potential solutions.Implementation is a prerequisite for any innovation to succeed. Health and care services are complex and people need to be engaged at every level. Most people are cautious about proven innovations such as PROMs and PREMs but champions and leaders can help them engage. The NASSS framework (reasons for Non-adoption, Abandonment and failure to Scale up, Spread or Sustain digital health innovations) helps indicate that implementation is complex why it may be resisted.The Plan-Do-Study-Act (PDSA) approach aids implementation and helps ensure that everyone knows who should do what, when, where, how and why. Noise is an under-appreciated problem, especially when tracking patients over time such as before and after treatment. Interoperability of PROMs and PREMs with electronic health records should use Fast Health Interoperability Resources and internationally accepted coding schemes such as SNOMED CT and LOINC.Most projects need multiple measures to meet the needs of everyone involved. Measure selection should focus on their relevance, ease of use, and response rates.If these problems are avoided or mitigated, PROMs and PREMs can help deliver better patient outcomes, patient experience, staff satisfaction and health equity.

Routine measurement of patient experience.

Patient experience is a key pillar of healthcare quality. We describe a framework of three short generic measures covering Patient Experience, Result Satisfaction and Service Integration. The Result Satisfaction measure is described for the first time.These measures capture twelve aspects of patient experience covering the relationship between patients and clinicians (Patient Experience), the immediate results of the consultation or treatment as perceived by patients (Result Satisfaction) and collaboration between different healthcare services and silos (Service Integration). Each measure has four items.These measures are compared with three national measures: the Friends and Family Test and the General Practice Patient Survey used in England, and HCAHPS used in US hospitals. The expected benefits of national measures are not being achieved and we need to think again about how best to tailor health services to meet patients' expectations.The three measures described (Patient Experience, Result Satisfaction and Service Integration) are generic, short and have low reading ages. They share common forms and scoring schemes, which mean that they can be used individually or in combination at all levels of a healthcare provider.

The realities and expectations of community involvement in COVID-19 research: a Consumer Reference Group perspective.

BACKGROUND: Older adults have been disproportionately impacted by the COVID-19 pandemic. COVID-19 restrictions such as stay at home orders and physical distancing measures have been implemented to reduce older adults' risk of infection, however, such measures can have negative effects on older adults' mental health and social wellbeing. In 2020, the research team received funding as part of an Australian COVID-19 research grants program to investigate how services can better meet the mental health and social support needs of older adults during COVID-19. A Consumer Reference Group (CRG) was established to provide a community perspective on all research activities. MAIN BODY: The CRG comprised of eight older adults aged 65 years and older living in Western Australia. Two members of the CRG were involved in the initial grant proposal, and one member worked for a not-for-profit organisation that provides support and advocacy for older adults. The CRGs role was to provide consumer and community perspectives on the research design, advise on study materials, facilitate links between consumers, the community, and researchers, and advocate on behalf of consumers and the community. The CRG was encouraged to reflect on the research project, their contributions, and the outcomes obtained. In this commentary, we document the CRGs contributions to the project, and record their reflections, including what went well, what were some challenges, the realities of conducting research during COVID-19, and lessons learnt. CONCLUSION: The CRG were active participants in the research process. They shared their perspectives and made important contributions to the project. Through collaboration with the CRG, we were able to reach four key messages, underpinned by consumers lived experiences, that were used to co-develop knowledge translation products. These were disseminated to service providers and older adults.

Since the start of the COVID-19 pandemic, health and social measures have been introduced to reduce the spread of the virus, including lockdowns, physical distancing, and mask mandates. Older adults (aged 60 years and older) are considered particularly vulnerable to COVID-19 and have therefore faced some of the greatest restrictions to reduce their risk of infection. These restrictions can have a negative effect on older adults social and emotional wellbeing. In 2020 the research team received funding to investigate how services could better meet the mental health and social support needs of older Australians during the pandemic. To enable a community perspective on all research activities, a Consumer Reference Group (CRG) of eight older adults living in Western Australia was established. Two of the eight CRG members were involved in the initial grant proposal. The CRG's role was to share their thoughts on the research design, study materials, and to provide links to and advocate for consumers and the community. This commentary reports reflections from the CRC on what went well, what some of the challenges were, the realities of conducting this research during COVID-19, and what lessons were learnt. Through collaboration with the CRG key messages for the research project were reached and used to inform infographics, which were then disseminated to inform service delivery providers and older adults of the research outcomes.

Readiness for five digital technologies in general practice: perceptions of staff in one part of southern England.

BACKGROUND: Our aim was to understand how digital readiness within general practice varies between different technologies and to identify how demographic, workplace and external factors affect this. The technologies considered include electronic patient records, telehealth (text messaging and video consultations), patient online access, patient clinical apps and wearables, and social media. METHOD: A digital readiness survey tool was developed and used in one area of southern England during Spring 2020. Semistructured qualitative interviews were also carried out with some practice staff and digital technology company representatives. RESULTS: GPs, nurses and non-clinical staff submitted 287 responses from 27 general practices (out of 33 invited).Staff digital readiness differs significantly between technologies. The mean perceived digital competency scores on 0-100 scale (high is good) were electronic patient records (75.7), telehealth (64.2), patient online access (65.8), patient clinical apps and wearables (50.8), and social media (51.2).Younger general practice staff, those in post for 5 or less years are more digitally competent and confident than older staff. This applies to both clinical and non-clinical staff. Older patient population, rurality and smaller practice size are associated with lower digital readiness. Readiness to use digital technology may have improved since the start of the COVID-19 pandemic but barriers remain in poor IT and mobile infrastructure, software usability and interoperability, and concerns about information governance. CONCLUSIONS: Improving digital readiness in general practice is complex and multifactorial. Issues may be alleviated by using dedicated digital implementation teams and closer collaboration between stakeholders (GPs and their staff, patients, funders, technology companies and government).

Development of social contact and loneliness measures with validation in social prescribing.

AIMS: This paper describes two patient-reported measures of social contact and loneliness, which are closely related concepts. The first measure (R-Outcomes Social Contact measure) was developed from scratch, based on customer needs and literature review. It covers emotional and social aspects using positive terms. The second measure (R-Outcomes Loneliness measure) is adapted from the GSS Loneliness Harmonised Standard. Both measures are patient-reported outcome measures, based on patients' own perception of how they feel. METHOD: This development started in 2016 in response to customers' requests to measure social contact/loneliness for patients in social prescribing projects.Both measures are compared with three other loneliness measures (the GSS Loneliness Harmonised Standard, De Jong Gierveld and Campaign to End Loneliness). Both measures are short (36 and 21 words, respectively). Mean improvement is reported as a positive number on a 0-100 scale (where high is good).We tested the psychometric performance and construct validity of the R-Outcomes Social Contact measure using secondary analysis of anonymised data collected before and after social prescribing interventions in one part of Southern England. RESULTS: In the validation study, 728 responses, collected during 2019-2020, were analysed. 90% were over 70 years old and 62% women. Cronbach's α=0.76, which suggests that it is appropriate to use a single summary score. Mean Social Contact scores before and after social prescribing intervention were 59.9 (before) and 66.7 (after, p<0.001).Exploratory factor analysis shows that measures for social contact, health status, health confidence, patient experience, personal well-being, medication adherence and social determinants of health are correlated but distinct factors. Construct validation shows that the results are consistent with nine hypotheses, based on the loneliness literature. CONCLUSION: The R-Outcomes Social Contact measure has good psychometric and construct validation results in a population referred to social prescribing. It is complementary to other R-Outcomes measures.

Measure what we want: a taxonomy of short generic person-reported outcome and experience measures (PROMs and PREMs).

INTRODUCTION: Health and care systems are complex and multifaceted, but most person-reported outcome and experience measures (PROMs and PREMs) address just one aspect. Multiple aspects need measuring to understand how what we do impacts patients, staff and services, and how these are affected by external factors. This needs survey tools that measure what people want, are valid, sensitive, quick and easy to use, and suitable for people with multiple conditions. METHODS: We have developed a coherent family of short generic PROMs and PREMs that can be used in combination in a pick-and-mix way. Each measure has evolved iteratively over several years, based on literature review, user inputs and field testing. Each has has a common format with four items with four response options and is designed for digital data collection with standardised analytics and data visualisation tools. We focused on brevity and low reading age. RESULTS: The results are presented in tabular format and as a taxonomy. The taxonomy is categorised by respondent type (patient or staff) and measure type. PROMs have subdomains: quality of life, individual care and community; PREMs have subdomains: service provided, provider culture and innovation. We show 22 patient-reported measures and 17 staff-reported measures. Previously published measures have been validated. Others are described for the first time. DISCUSSION AND CONCLUSIONS: This family of measures is broad in scope but is not claimed to be comprehensive. Measures share a common look and feel, which enables common methods of data collection, reporting and data visualisation. They are used in service evaluation, quality improvement and as key performance indicators. The taxonomy helps to organise the whole, explain what each measure does and identify gaps and overlaps.

Comparison of staff and resident health status ratings in care homes.

BACKGROUND: Many care home residents cannot self-report their own health status. Previous studies have shown differences between staff and resident ratings. In 2012, we collected 10 168 pairs of health status ratings using the howRu health status measure. This paper examines differences between staff and resident ratings. METHOD: HowRu is a short generic person-reported outcome measure with four items: pain or discomfort (discomfort), feeling low or worried (distress), limited in what you can do (disability) and require help from others (dependence). A summary score (howRu score) is also calculated. Mean scores are shown on a 0-100 scale. High scores are better than low scores. Differences between resident and staff reports (bias) were analysed at the item and summary level by comparing distributions, analysing correlations and a modification of the Bland-Altman method. RESULTS AND CONCLUSIONS: Distributions are similar superficially but differ statistically. Spearman correlations are between 0.55 and 0.67. For items, more than 92.9% of paired responses are within one class; for the howRu summary score, 66% are within one class. Mean differences (resident score minus staff score) on 0-100 scale are pain and discomfort (-1.11), distress (0.67), discomfort (1.56), dependence (3.92) and howRu summary score (1.26). The variation is not the same for different severities. At higher levels of pain and discomfort, staff rated their discomfort and distress as better than residents. On the other hand, staff rated disability and dependence as worse than did residents. This probably reflects differences in perspectives. Red amber green (RAG) thresholds of 10 and 5 points are suggested for monitoring changes in care home mean scores.

A Taxonomy for Survey Measures Used to Evaluate Complex Digital Health Innovations.

Evaluators need to measure whether innovations help patients and staff, but have lacked the tools needed to do this as part of routine care. We provide a taxonomy for the classification of survey measures, which can be used together on a pick and mix basis. These are described in the context of the evaluation of digital health innovations.

Health status of care home residents: practicality and construct validity of data collection by staff at scale.

BACKGROUND: Medical diagnoses and assessed need for care are the prerequisites for planning and delivery of care to residents of care homes. Assessing the effectiveness of care is difficult. The aim of this study was to test the practicality and construct validity of the howRu health status measure using secondary analysis of a large data set. METHOD: The data came from a Bupa Care Homes Census in 2012, which covered 24 506 residents in 395 homes internationally (UK, Australia and New Zealand). Staff completed optical mark readable forms about each resident using a short generic health status measure, howRu. Response rates were used to assess practicality and expected relationships between health status and independent variables were used to assess the construct validity. RESULTS AND DISCUSSION: 19,438 forms were returned (79.3%) in 360 care homes (91.1%); complete health status data were recorded for 18 617 residents (95.8% of those returned). Missing values for any health status items mostly came from a small number of homes. The relationships between howRu and independent variables support construct validity. Factor analysis suggests three latent variables (discomfort, distress and disability/dependence). CONCLUSIONS: HowRu proved easy to use and practical at scale. The howRu health status measure shows good construct validity.

Development and initial testing of a Health Confidence Score (HCS).

Introduction: Patients need to feel confident about looking after their own health. This is needed to improve patient outcomes and clinical support. With few suitable tools available to measure self-care health confidence, we developed and validated a short, generic survey instrument for use in evaluation and quality improvement. Methods: The Health Confidence Score (HCS) was developed through literature review, patient and expert focus groups and discussions. This paper reports an initial survey (n = 1031, study 1) which identified some issues and a further face-to-face survey (n = 378, study 2) to test the construct and concurrent validity of the final version. Scores were correlated against the My Health Confidence (MHC) rating scale, howRu (health status measure) and relevant demographics. Results: The HCS is short (50 words) with good readability (reading age 8). It has four items covering health knowledge, capability to self-manage, access to help and shared decision-making; each has four response options (strongly agree, agree, neutral disagree). Items are reported independently and as a summary score.The mean summary score was 76.7 (SD 20.4) on 0-100 scale. Cronbach's alpha = 0.82. Exploratory factor analysis suggested that the four items relate to a single dimension. Correlation of the HCS summary score with MHC was high (Spearman r = 0.76). It was also associated with health status (Spearman r = 0.49), negatively with number of medications taken (r=-0.29) and age (r=-0.22) and not with ethnicity, having children or education level. Conclusions: The HCS is short, easy to use, with good psychometric properties and construct validity. Each item is meaningful independently and the summary score gives an overall picture of health confidence.

Monitoring work well-being, job confidence and care provided by care home staff using a self-report survey.

Background and method: In care homes, staff well-being, job confidence and opinion of the care provided to residents are central to morale and care quality. In this study, care home staff in the East Midlands region of England completed self-reported outcome and experience surveys in two rounds. Mean scores for each home are shown using a scale from 0 (all chose lowest option) to 100 (all chose highest option). High scores are good. Results: In round 1, 332 staff in 15 homes submitted responses; in round 2, 207 staff in 9 homes. Mean scores in round 1 and round 2 were similar, although those of some homes scores differed significantly, cancelling each other out. Overall, Work Wellbeing mean score was 83 (care home range 48-97), with worthwhileness (92) the highest ranked item and anxiety at work (78) the lowest. Job Confidence mean score was 84 (range 59-94), with able to manage the work (86) highest and involvement in decisions that affect staff (79) lowest. Care Provided mean score was 86 (range 59-97), with treat people kindly (91) highest and well organised (80) lowest. Homes rated as outstanding by the Care Quality Commission had higher scores on average than those rated good, which were higher than those rated as needing improvement. Conclusions: This study has demonstrated the practicality of measuring staff views of their Work Wellbeing, Job Confidence and Care Provided in care homes. Rather than wait for adverse quality outcomes to be detected, this approach offers a way to track staff morale and declared capability over time.

Personal Wellbeing Score (PWS)-a short version of ONS4: development and validation in social prescribing.

Aims: Our aim was to develop a short generic measure of subjective well-being for routine use in patient-centred care and healthcare quality improvement alongside other patient-reported outcome and experience measures. Methods: The Personal Wellbeing Score (PWS) is based on the Office of National Statistics (ONS) four subjective well-being questions (ONS4) and thresholds. PWS is short, easy to use and has the same look and feel as other measures in the same family of measures. Word length and reading age were compared with eight other measures.Anonymous data sets from five social prescribing projects were analysed. Internal structure was examined using distributions, intra-item correlations, Cronbach's α and exploratory factor analysis. Construct validity was assessed based on hypothesised associations with health status, health confidence, patient experience, age, gender and number of medications taken. Scores on referral and after referral were used to assess responsiveness. Results: Differences between PWS and ONS4 include brevity (42 vs 114 words), reading age (9 vs 12 years), response options (4 vs 11), positive wording throughout and a summary score. 1299 responses (60% female, average age 81 years) from people referred to social prescribing services were analysed; missing values were less than 2%. PWS showed good internal reliability (Cronbach's α=0.90). Exploratory factor analysis suggested that all PWS items relate to a single dimension. PWS summary scores correlate positively with health confidence (r=0.60), health status (r=0.58), patient experience (r=0.30) and age group (r=0.24). PWS is responsive to social prescribing intervention. Conclusions: The PWS is a short variant of ONS4. It is easy to use with good psychometric properties, suitable for routine use in quality improvement and health services research.

Digital innovation evaluation: user perceptions of innovation readiness, digital confidence, innovation adoption, user experience and behaviour change.

BACKGROUND: Innovation spread is a key policy objective for health systems world-wide, but adoption success varies enormously. We have developed a set of short generic user-reported measures to help understand how and why healthcare innovations spread. This work builds on the literature and on practical experience in developing and using patient-reported outcome measures. MEASURES: The Innovation Readiness Score measures user perceptions of how much they are open to and up-to-date with new ideas, and whether their organisations are receptive to and capable of innovation. It is based on Rogers' classification of innovativeness (innovator, early adopter, early majority, etc).The Digital Confidence Score rates users' digital literacy and confidence to use digital products, with dimensions of familiarity, social pressure, support and digital self-efficacy.The Innovation Adoption Score rates the adoption process in terms of coherence and reflective thought before, during and after implementation. It is based on Normalisation Process Theory.The User Satisfaction measure assesses a digital product in terms of usefulness, ease of use, support and satisfaction.The Behaviour Change measure covers user perceptions of their capability, opportunity and motivation to change behaviour, based on the COM-B model.These measures have been mapped onto Greenhalgh's NASSS Framework (non-adoption, abandonment and challenges to scale-up, spread and sustainability of health and care technologies). CONCLUSION: These tools measure different aspects of digital health innovations and may help predict the success of innovation dissemination, diffusion and spread programmes.

The Load Model: an alternative to QALY.

BACKGROUND: QALYs are widely used in health economic evaluation, but remain controversial, largely because they do not reflect how many people behave in practice. This paper presents a new conceptual model (Load Model) and illustrates it in comparison with the QALY model. METHODS: Load is the average annual weight attributed to morbidity and mortality over a defined period, using weightings based on preference judgements. Morbidity Load is attributed to states of illness, according to their perceived severity. When people are in full health, Load is zero (no morbidity). Death is treated as an event with negative consequences, incurred in the year following death. Deaths may be weighted equally, with a fixed negative weight such as -100, or differ according to the context of death. After death, Load is zero. In a worked example, we use the standard gamble method to obtain a weighting for an illness state, for both Load and QALY models. A judge is indifferent between certainty of 1.5 years' illness followed by death, or a 50/50 chance of 1.5 years' full health or 1-year illness, each followed by death. The weightings calculated are applied to a hypothetical life, 72 years in full health followed by 3 years with illness then death, using both models. Three other hypothetical outcomes are also compared. RESULTS: For an example life, the relative size of the morbidity component compared with the mortality component is much higher in the Load model than in the QALY model. When comparing alternative outcomes, there are also substantial differences between the two models. CONCLUSIONS: In the Load model the weight of morbidity, relative to mortality, is very different from that in the QALY model. Given the role of the QALYs in economic evaluation, the implications of an alternative, which generates very different results, warrant further exploration.

Open Source Paradigm: A Synopsis of The Cathedral and the Bazaar for Health and Social Care.

BACKGROUND: Open source software (OSS) is becoming more fashionable in health and social care, although the ideas are not new. However progress has been slower than many had expected. OBJECTIVE: The purpose is to summarise the Free/Libre Open Source Software (FLOSS) paradigm in terms of what it is, how it impacts users and software engineers and how it can work as a business model in health and social care sectors. METHOD: Much of this paper is a synopsis of Eric Raymond's seminal book The Cathedral and the Bazaar, which was the first comprehensive description of the open source ecosystem, set out in three long essays. Direct quotes from the book are used liberally, without reference to specific passages. The first part contrasts open and closed source approaches to software development and support. The second part describes the culture and practices of the open source movement. The third part considers business models. CONCLUSION: A key benefit of open source is that users can access and collaborate on improving the software if they wish. Closed source code may be regarded as a strategic business risk that that may be unacceptable if there is an open source alternative. The sharing culture of the open source movement fits well with that of health and social care.

Performance of EQ-5D, howRu and Oxford hip & knee scores in assessing the outcome of hip and knee replacements.

BACKGROUND: We aimed to compare the performance of EQ-5D-3 L and howRu, which are short generic patient-reported outcome measures (PROMs), in assessing the outcome of hip and knee replacements, using the Oxford Hip Score (OHS) and the Oxford Knee Scores (OKS) for comparison. METHODS: Outcome was assessed as the difference between pre-surgery and 6-month post-surgery scores. We used a large sample from the NHS PROMs database, which used EQ-5D-3 L, and a small cohort of patients having the same operations collected by MyClinicalOutcomes (MCO), which used howRu. Both cohorts completed the OHS (hips) or the OKS (knees). RESULTS: The change (outcome) between pre-op and post-op scores as measured by howRu was greater than that measured by EQ-5D, relative to that measured by OHS or OKS. For hip replacements, the correlation for change measured by howRu and OHS was r = 0.77 (0.66-0.85). The corresponding correlation for change measured by EQ-5D Index and OHS was r = 0.64 (0.63-0.64). For knee replacements the correlation between change in howRu and OKS was r = 0.86 (0.75-0.92); between EQ-5D Index and OKS r = 0.59 (0.58-0.60). CONCLUSIONS: For hip and knee replacement, the outcome measured by howRu was more highly correlated with that measured by the condition-specific Oxford Hip and Knee Scores than were EQ-5D Index or EQ-VAS. The magnitude of change before and after surgery was also greater.

Metadata requirements for portals.

Consensus around the requirements for metadata in patient and clinical portals would provide a sound basis for the adoption of standards. We propose a set of requirements for metadata in a way that is generic and platform independent. These requirements cover both Clinical Documents and Clinical Statements, addressing the what, who, when and where of each item.