Reliability testing of the Health of the Nation Outcome Scales 2018.

WHAT IS KNOWN ON THE SUBJECT: The Health of the Nation Outcome Scales (HoNOS) is a widely used clinical measure designed to rate and monitor the outcomes of service users accessing specialist mental healthcare. Since its development (in 1996), numerous research studies have confirmed the HoNOS captures the aspects of care that it purports to (validity), and that clinicians' ratings are consistent both over time, and between different raters (reliability). WHAT THE PAPER ADDS TO EXISTING KNOWLEDGE: In 2018, the HoNOS was reviewed with updates made to some terminology and other revisions intended to remove ambiguity in the guidance for raters. However, although the new version (HoNOS 2018) was accompanied by a recommendation that its validity and reliability be re-tested this was not undertaken. To our knowledge, this is the first study to re-assess the updated tool's reliability by measuring the level of agreement between different raters. Our findings confirm that there is an acceptable level of consistency between student mental health nurses that have been trained to use the (new) HoNOS 2018. WHAT ARE THE IMPLICATIONS FOR PRACTICE: The HoNOS is nationally mandated for use by all specialist mental healthcare providers in the UK. Our findings provide some assurance that, with appropriate update training and monitoring of organisational-level data sets, the original HoNOS glossary can safely be replaced with the HoNOS 2018 to ensure more contemporary routine outcome measurement can occur. ABSTRACT: INTRODUCTION: The Health of the Nation Outcome Scales (HoNOS) is a well-established clinician rated outcome measure for use in mental health services. Following an international review, an updated version (HoNOS 2018) was published with a recommendation that its psychometric properties be re-tested prior to widespread implementation. To date, only one such study has been published. AIMS: To test the inter-rater agreement levels for HoNOS 2018. METHOD: Third-year student mental health nurses received training to complete the HoNOS 2018. Following this timetabled session, they were each invited to independently rate two, randomly selected, videos of (simulated) patient interviews. The resulting data were then analysed to calculate the tool's internal consistency and inter-rater agreement levels. RESULTS: The 55 participants provided 106 ratings from four vignettes. Cronbach's alphas and McDonalds omegas confirmed the revised tool's internal consistency was acceptable. Average measure intraclass correlation coefficients for the four patient vignettes indicated excellent reliability. IMPLICATIONS FOR PRACTICE: This study provides initial assurance that the HoNOS 2018 is a reliable clinician rated outcome measure suitable for use in routine clinical practice by relatively inexperienced mental health practitioners with limited training.

Assessing the content validity of the revised Health of the Nation Outcome Scales 65+: the HoNOS Older Adults.

AIMS AND METHOD: Recently, the Health of the Nation Outcome Scales 65+ (HoNOS65+) were revised. Twenty-five experts from Australia and New Zealand completed an anonymous web-based survey about the content validity of the revised measure, the HoNOS Older Adults (HoNOS OA). RESULTS: All 12 HoNOS OA scales were rated by most (≥75%) experts as 'important' or 'very important' for determining overall clinical severity among older adults. Ratings of sensitivity to change, comprehensibility and comprehensiveness were more variable, but mostly positive. Experts' comments provided possible explanations. For example, some experts suggested modifying or expanding the glossary examples for some scales (e.g. those measuring problems with relationships and problems with activities of daily living) to be more older adult-specific. CLINICAL IMPLICATIONS: Experts agreed that the HoNOS OA measures important constructs. Training may need to orient experienced raters to the rationale for some revisions. Further psychometric testing of the HoNOS OA is recommended.

Assessing the Content Validity of the Revised Health of the Nation Outcome Scales (HoNOS 2018).

The Health of the Nation Outcome Scales (HoNOS) comprises 12 scales that cover the kinds of problems that may be experienced by working-age adults in contact with specialised mental health services. Drawing on 20 years' experience in clinical practice, a collaborative, international review of the HoNOS was undertaken and a revised measure (known as the HoNOS 2018) was published. In this study, 32 experts from Australia, England and New Zealand completed an anonymous web-based survey to assess the relevance, comprehensiveness and comprehensibility (aspects of content validity) of the HoNOS 2018. The experts rated 11 of the 12 HoNOS 2018 scales as 'important' or 'very important' for determining the overall clinical severity (item-level content validity index or I-CVI ≥ 0.75). Evaluations of the scales' ability to capture change, comprehensiveness and comprehensibility were more variable, but generally positive. Experts' comments provided further insights into this variability; for example, they noted that some scales combine multiple phenomena, which can result in ambiguity in item wording and assessment challenges. Results from this study suggest that the revisions have not altered the importance of the scales. Given the measure's breadth of content, training remains important for ensuring rating fidelity. Inter-rater reliability and utility testing are indicated.

Review and update of the Health of the Nation Outcome Scales for Elderly People (HoNOS65+).

Aims and methodThe Health of the Nation Outcome Scales for Elderly People (HoNOS65+) has been used widely for 20 years, but has not been updated to reflect contemporary clinical practice. The Royal College of Psychiatrists convened an advisory board, with expertise from the UK, Australia and New Zealand, to propose amendments. The aim was to improve rater experience when using the HoNOS65+ glossary by removing ambiguity and inconsistency, rather than a more radical revision. RESULTS: Views and experience from the countries involved were used to produce a series of amendments intended to improve intra- and interrater reliability and improve validity. This update will be called HoNOS Older Adults to reflect the changing nature of the population and services provided to meet their needs. These improvements are reported verbatim, together with the original HoNOS65+ to aid comparison.Clinical implicationsFormal examination of the psychometric properties of the revised measure is needed. However, clinician training will remain crucial.Declaration of interestNone.

A review and update of the Health of the Nation Outcome Scales (HoNOS).

Aims and method The Health of the Nation Outcome Scales (HoNOS) and its older adults' version (HoNOS 65+) have been used widely for 20 years, but their glossaries have not been revised to reflect clinicians' experiences or changes in service delivery. The Royal College of Psychiatrists convened an international advisory board, with UK, Australian and New Zealand expertise, to identify desirable amendments. The aim was to improve rater experience by removing ambiguity and inconsistency in the glossary rather than more radical revision. RESULTS: Changes proposed to the HoNOS are reported. HoNOS 65+ changes will be reported separately. Based on the views and experience of the countries involved, a series of amendments were identified. Clinical implications While effective clinician training remains critically important, these revisions aim to improve intra- and interrater reliability and improve validity. Next steps will depend on feedback from HoNOS users. Reliability and validity testing will depend on funding. Declaration of interest None.

New models of care: a liaison psychiatry service for medically unexplained symptoms and frequent attenders in primary care.

Aims and method This paper describes the process of setting up and the early results from a new liaison psychiatry service in primary care for people identified as frequent general practice attenders with long-term conditions or medically unexplained symptoms. Using a rapid evidence synthesis, we identified existing service models, mechanisms to identify and refer patients, and outcomes for the service. Considering this evidence, with local contingencies we defined options and resources. We agreed a model to set up a service in three diverse general practices. An evaluation explored the feasibility of the service and of collecting data for clinical, service and economic outcomes. Results High levels of patient and staff satisfaction, and reductions in the utilisation of primary and secondary healthcare, with associated cost savings are reported. Clinical implications A multidisciplinary liaison psychiatry service integrated in primary care is feasible and may be evaluated using routinely collected data.