Patient evaluation of hospital outcomes: an analysis of open-ended comments from extreme clusters in a national survey.

OBJECTIVES: A recent study identified patients in six distinct response groups based on their evaluations of outcomes related to overall satisfaction, malpractice and benefit of treatment. This study validates the response clusters by analysing and comparing open-ended comments from the extreme positive and extreme negative response groups. DESIGN: Qualitative content analysis. SETTING: Data from open-ended comment fields provided by patients who completed a national patient-experience survey carried out in Norway in 2011. 10 514 patients responded to the questionnaire and 3233 provided comments. A random sample of 50 open-ended comments from respondents representing cluster 1 ('excellent services'), cluster 5 ('services have clear improvement needs') and outliers ('very poor services') was reviewed. RESULTS: 3 distinct patient profiles were identified. More than half of the comments in cluster 1 included descriptions of positive healthcare experiences, one addressed patient safety issues. Only 1 of the comments in cluster 5 was positive, and 12 were related to safety. All comments from the outliers were negative, and more than three-quarters reported experiences related to malpractice or adverse events. Recurring themes did not differ significantly between the three respondent groups, but significant differences were found for the descriptions and severity of the experiences. CONCLUSIONS: Patients in negative response groups had distinct and much poorer healthcare descriptions than those in the extreme positive group, supporting the interpretation of quality differences between these groups. Further research should assess ways of combining statistical cluster information and qualitative comments, which could be used for local quality improvement and public reporting.

The Cancer Patient Experiences Questionnaire (CPEQ): reliability and construct validity following a national survey to assess hospital cancer care from the patient perspective.

OBJECTIVES: Patient experience questionnaires have been criticised owing to the lack of supporting psychometric evidence. The objective of this study was to describe the development and psychometric evaluation of the Cancer Patient Experiences Questionnaire (CPEQ) in Norway. DESIGN: Questionnaire development was based on a literature review of existing questionnaires, patient interviews, expert-group consultations, pretesting of questionnaire items and a national survey. Psychometric evaluation included exploratory and confirmatory factor analysis, and tests of internal consistency reliability and test-retest reliability. SETTING: Data were collected using a postal survey of cancer patients attending 54 hospitals in all 4 health regions. The subjects were 14 227 adult cancer patients who had attended an outpatient clinic or who had been discharged from an inpatient ward. Patients with all types of cancer were included. Data quality, internal consistency reliability and construct validity were assessed. RESULTS: Of the 13 846 patients who received the CPEQ, 7212 (52%) responded. Exploratory factor analysis identified six scales of outpatient experiences relating to nurse contact, doctor contact, information, organisation, patient safety and contact with next of kin, and seven scales of inpatient experiences, with the addition of hospital standard to the aforementioned scales. All but two of the scales met the criterion of 0.70 for Cronbach's α testing, and test-retest correlations ranged from 0.57 to 0.85. Confirmatory factor analysis supported the interpretation of six and seven scales for outpatients and inpatients, respectively. Statistically significant associations based on explicit hypotheses provided evidence for the construct validity of the scales. One additional scale measuring the hospital level was identified (α=0.85). CONCLUSIONS: The CPEQ is a self-report instrument that includes the most important aspects of patient experiences with cancer care at hospitals. The instrument was tested following a national survey in Norway; good evidence is provided herein for the internal consistency reliability, test-retest reliability and construct validity.

The Nordic Patient Experiences Questionnaire (NORPEQ): cross-national comparison of data quality, internal consistency and validity in four Nordic countries.

OBJECTIVES: To evaluate the Nordic Patient Experiences Questionnaire (NORPEQ) for data quality, reliability and validity following surveys of patients in Finland, Norway, Sweden and the Faroe Islands. DESIGN, METHODS AND PARTICIPANTS: The NORPEQ was mailed to 500 patients randomly selected after receiving inpatient treatment in Finland, Norway and Sweden. The NORPEQ was also included in a national survey in Norway and in the Faroe Islands. Dimensionality was assessed using principal component analysis and internal consistency by item-total correlation and Cronbach's α. Construct validity was assessed by correlating NORPEQ scores with variables known to be related to patient experiences. SETTING: Somatic hospitals in Finland, Faroe Islands, Norway and Sweden. PRIMARY AND SECONDARY OUTCOME MEASURES: Item missing, internal consistency reliability and construct validity. RESULTS: Response rates ranged from 45.8% in Norway to 84% for Sweden. Levels of missing data were low for all items across the surveys. Principal component analysis identified one component with six experiences items. Mean NORPEQ scores ranged from 74 to 79 on the 0-100 scale, where 100 represents the best possible experiences. Cronbach's α ranged from 0.84 in Finland to 0.88 in Sweden. CONCLUSIONS: The NORPEQ is a brief measure of patient experiences that covers important aspects of the healthcare encounter. It shows good evidence of reliability and validity. PRACTICE IMPLICATIONS: The NORPEQ instrument is recommended for cross-national comparisons of healthcare experiences for the four Nordic countries.

User-experience surveys with maternity services: a randomized comparison of two data collection models.

OBJECTIVE: To compare two ways of combining postal and electronic data collection for a maternity services user-experience survey. DESIGN: Cross-sectional survey. SETTING: Maternity services in Norway. PARTICIPANTS: All women who gave birth at a university hospital in Norway between 1 June and 27 July 2010. INTERVENTION: Patients were randomized into the following groups (n= 752): Group A, who were posted questionnaires with both electronic and paper response options for both the initial and reminder postal requests; and Group B, who were posted questionnaires with an electronic response option for the initial request, and both electronic and paper response options for the reminder postal request. MAIN OUTCOME MEASURES: Response rate, the amount of difference in background variables between respondents and non-respondents, main study results and estimated cost-effectiveness. RESULTS: The final response rate was significantly higher in Group A (51.9%) than Group B (41.1%). None of the background variables differed significantly between the respondents and non-respondents in Group A, while two variables differed significantly between the respondents and non-respondents in Group B. None of the 11 user-experience scales differed significantly between Groups A and B. The estimated costs per response for the forthcoming national survey was €11.7 for data collection Model A and €9.0 for Model B. CONCLUSIONS: The model with electronic-only response option in the first request had lowest response rate. However, this model performed equal to the other model on non-response bias and better on estimated cost-effectiveness, and is the better of the two models in large-scale user experiences surveys with maternity services.

Usefulness of a national parent experience survey in quality improvement: views of paediatric department employees.

OBJECTIVES: This study presents results from an electronic survey among paediatric department employees, addressing employees' attitudes and use of results from a national parent experience survey carried out in 2005. METHODS: Electronic questionnaire survey of employees from each of the 20 paediatric departments included in the national survey, with a response rate of 87%. RESULTS: The employees had favourable opinions of user experience surveys, and the results from the national survey were well known among both managers and other personnel. User experience surveys were considered important, and 56% reported that they had implemented improvement actions addressing problems identified in the national survey. Managers reported more often than staff without managerial responsibility that the results had been informally discussed, and that the survey was useful for their own department. Department leaders were more positive to the usefulness of the survey than non-leaders. Significant differences in attitudes were found between physicians and other health personnel. CONCLUSION: Employees in the paediatric departments were positive to user experience surveys, and the surveys have a potential to be actively used in quality improvement actions. Effects of the quality improvement initiatives should be assessed in future parent experience surveys.

Family physicians' experiences with community mental health centers: a multilevel analysis.

OBJECTIVE: The measurement of professionals' views of quality is a key component in the evaluation of health care delivery; however, few studies have assessed the feasibility of using such evaluations as quality indicators. The effects of individual and organizational factors on family physicians' experiences with community mental health centers were assessed by responses to a self-completed questionnaire and publicly available data. METHODS: Questionnaires were mailed in spring 2006 to family physicians in Norway for their evaluation of 71 community mental health centers; 2,120 (61%) practitioners responded. They evaluated centers on five scales: competence, discharge letter, contact with and help from the center in emergency situations, guidance from the center to practitioners, and workforce situation. Multilevel regression analyses were used to divide the variance between the two levels and to analyze associations with the five scales for individual- and center-level variables. RESULTS: Center characteristics accounted for a substantial part of the variation; the intraclass correlation coefficient ranged from 10% for discharge letter to 23% for workforce situation. General impression of psychiatric services, knowledge of the center, negative episodes with the center, gender, and the center's location were significantly related to the scales. Explained variance at the individual level ranged from 24% for emergency situations to 38% for competence. Explained variance at the center level ranged from 40% for emergency situations to 70% for competence. CONCLUSIONS: A substantial part of the variation at both levels was explained for all scales. Family physicians' evaluation of community mental health centers had substantial variation at the center level, supporting the use of such evaluations as quality indicators.

Patient experiences with treatment in private practice compared with public mental health services.

The study compared patient experiences with psychiatric treatment provided by private practitioners and public outpatient clinics. Questionnaires were completed by 642 outpatients in private practice and 6,677 outpatients in public clinics. The questionnaire included a measure of patient experiences comprising six items: treatment outcome, enough time for contact and dialogue with clinician, clinicians' understanding of patient's situation, suitability of therapy and treatment, clinician follow-up of planned actions, and influence on treatment. Patients in private practice had generally better experiences than patients in public outpatient treatment. The difference between private and public patients was largest for patients with poor self-evaluated mental health or those who had just one consultation in the previous three months. Private practitioners appear to have an important role in mental health services delivery, and patients have relatively good experiences with services. Further studies that assess the patient - clinician interaction in different mental health services may give further insights into potential service improvements.

[Parent assessment of outpatient child and adolescent mental health services]. / Foresattes vurdering av barne- og ungdomspsykiatriske poliklinikker.

BACKGROUND: Parent assessment of outpatient child and adolescent mental health services (CAMHS) is now part of the Norwegian quality indicator system. This national survey included a validated questionnaire that assessed three aspects of care from a parental perspective. MATERIAL AND METHODS: Questionnaires were mailed to 17,080 parents of children and adolescents receiving care from CAMHS on an outpatient basis between 1 September and 31 December 2006. Two postal reminders were sent and telephone interviews were conducted with a sample of non-respondents. 7 906 (46%) parents returned a completed questionnaire. Three scales about treatment providers, treatment outcome and information/influence were identified (scales scored 0 to 100 where 100 represent the best experiences). RESULTS: The parents were most satisfied with the treatment providers (national mean 75). The national mean for the treatment outcome was 72 and for the scale concerning information and patients' possibilities to influence decisions the mean score was 59. The clinic scores varied considerably; the score for information and influence varied most (48-68). Few scores were however significantly different from the national average. INTERPRETATION: Most parents have good experience with the CAMHS. The scale concerning information and influence has the greatest potential for improvement. Several CAMHS had higher scores for all aspects of parent experience and may have useful information to convey to other clinics.

Parent experiences of paediatric care (PEPC) questionnaire: reliability and validity following a national survey.

AIM: To describe the development and evaluation of a parent completed questionnaire to measure parent experiences of inpatient paediatric care, the parent experiences of paediatric care (PEPC). METHODS: Literature review, parent interviews, pre-testing and a national survey of 6144 parents of children who were inpatients at one of the 20 paediatric departments within Norway in 2005. RESULTS: Three thousand three hundred and eight (53.8%) parents responded to the questionnaire. Low levels of missing data suggest that the PEPC is acceptable. The questionnaire includes six scales as supported by the results of factor analysis: nursing services (seven items), doctor services (five items), organisation (four items), information--examinations and tests (two items), information--discharge (three items) and hospital facilities (four items). Cronbach's alpha and test-retest correlations ranged from 0.7 to 0.9. Comparisons of scale scores with several variables including overall satisfaction with care and pain control, supported validity. CONCLUSION: The PEPC questionnaire includes important aspects of hospital care from the perspective of the parent. It has good evidence for internal consistency, test-retest reliability and validity and is recommended in surveys of parent experiences of paediatric inpatient care.

Patient experiences in relation to respondent and health service delivery characteristics: a survey of 26,938 patients attending 62 hospitals throughout Norway.

AIMS: To assess the association between patient experiences, aspects of healthcare delivery, and patient characteristics for adult somatic inpatients attending hospitals throughout Norway. METHODS: The Patient Experiences Questionnaire (PEQ) was mailed to 26,938 patients attending inpatient clinics at 62 Norwegian hospitals during 2002 and 2003 within a six-week period. Reminders were mailed at four weeks. Scores for the PEQ were regressed on whether the patient felt that he/she had received incorrect treatment, had spent the night in a corridor bed, had been an emergency or routine admission, his/her number of previous admissions, hospital teaching status, hospital size and location, health status, and sociodemographic variables. RESULTS: A total of 13,700 (50.9%) patients responded. Patients who felt that they had received incorrect treatment had significantly poorer scores for all 10 PEQ dimensions. Those spending the night in a corridor bed had significantly poorer scores for six dimensions. Emergency admissions and the previous number of inpatient stays were significantly associated with poorer experiences for 10 and 7 dimensions respectively. Hospital size and university status had negative relationships with scores for six and four dimensions respectively. CONCLUSIONS: Whether the patient felt that he/she had received the incorrect treatment had by far the strongest association with patient experiences. Future studies of patient experiences and satisfaction should include this variable. As found in previous research, patient experiences were significantly associated with age and health status.

[Assessment of community mental health centres by general practitioners]. / Fastlegenes vurdering av kvaliteten ved distriktspsykiatriske sentre.

BACKGROUND: General practitioners' evaluation of Community Mental Health Centres (CMHC) has become a part of the Norwegian system of quality indicators. On the basis of a national survey, we present the way in which GPs have assessed the quality of 77 CMHCs. MATERIAL AND METHODS: A questionnaire was posted to every GP in Norway from November 2005 to March 2006. This was followed up with 2-3 reminders and telephone interviews. 2,415 out of 3,704 GPs (65%) responded. RESULTS: GPs gave highest score for professional competence, the average being 55 on a scale of 0-100, where 100 is best. They gave lowest score for counselling (30) and staff situation (45). The average score given to the various CMHCs varied a lot for all dimensions, the largest difference being for counselling, with the highest average 58 and the lowest 8. There were many significant differences between various CMHCs' scores and the national average. When it came to counselling, the average score for 20 CMHCs was significantly different from that for the other CMHCs. INTERPRETATION: As GPs are important collaborators and users of the CMHCs, their views are valuable. The present findings need to be discussed. Many of the community mental health centres received all in all positive feedback from the GPs, suggesting that they may have useful experiences to share.

[PIPEQ--a method for measurement of user satisfaction in mental health services]. / PasOpp--en metode for å måle brukererfaringer i psykisk helsevern.

BACKGROUND: This study describes the development and testing of the Psychiatric In-Patient Experiences Questionnaire (PIPEQ). MATERIAL AND METHODS: Questionnaire development was based on a literature review, qualitative interviews with patients, and expert opinion. The questionnaire was tested in a postal survey among inpatients at three community health centres affiliated with the psychiatric clinic of Stavanger University hospital during spring 2005. 244 patients received the questionnaire; either they got the questionnaire by mail after discharge or at the institution before discharge. RESULTS: 68 (28%) patients responded to the questionnaire. Items had low levels of missing data. Factor analysis showed that 11 widely applicable items contribute to a measure of overall experiences. Item-total correlations ranged from 0.55 to 0.82. Cronbach's alpha exceeded the criterion of 0.9. Construct validity was supported by comparisons with variables related to patient experience, including overall satisfaction, extent of patient say in the medication decision, necessity of coercion, and mental health status. INTERPRETATION: The PIPEQ is a brief instrument that covers important aspects of health care experience for psychiatric inpatients and has good evidence for internal reliability and construct validity.

[Data collection methods and results in user surveys in mental health care]. / Innsamlingsmåte og resultater i brukerundersøkelser i psykisk helsevern.

BACKGROUND: Questionnaires on patient experience are increasingly used as quality indicators in the health services. There is limited evidence relating to alternative approaches to surveying patients within this field. We wanted to assess the effect of different methods of data collection on response rates and scores produced by a self-administered questionnaire on patient experience for adult inpatients receiving mental health care. MATERIAL AND METHODS: Data were collected from adult inpatients treated at three community mental health centres affiliated with the psychiatric clinic at Stavanger University hospital in spring 2005. The inclusion period was nine weeks, with three designs at consecutive time periods: A, a postal survey following discharge; B, a clinical survey before discharge; and C, patients given the choice of A or B. RESULTS: The response rate was highest with the postal design (38% vs. 24% and 23% respectively), but the differences were related to one additional reminder in the postal design. Out of the 11 questions, 4 had significantly poorer scores for the postal design. Questionnaire scores were significantly poorer with the postal design; 50 (on a scale from 0 to 100), vs. 59 and 63 in design B and C respectively. INTERPRETATION: The choice of data collection methods influences the results in user surveys in mental health care. This is an important consideration in the planning of studies and in the interpretation of the results, and in the comparison of results between studies using different data collection methods.

The Psychiatric Out-Patient Experiences Questionnaire (POPEQ): data quality, reliability and validity in patients attending 90 Norwegian clinics.

The aim of this study was to develop and evaluate the Psychiatric Out-Patient Experiences Questionnaire (POPEQ). The instrument was developed following a literature review, patient interviews and pre-testing of questionnaire items. The POPEQ was administered as part of a postal survey of 15,422 adult outpatients attending Norwegian clinics; 6677 (43.3%) patients responded to the questionnaire. Items had low levels of missing data. Factor analysis showed that 11 widely applicable items contribute to a measure of overall experiences. Sub-dimensions include clinician interaction (six items) information (two items) and outcomes (three items). Item-total correlations ranged from 0.5 to 0.8. Cronbach's alpha and test-retest reliability estimates exceeded the criterion of 0.7; the majority were over 0.8 and total scores over 0.9. Construct validity was supported by the results of 128 tests. The POPEQ includes important aspects of patient experience for psychiatric outpatients and has excellent evidence for reliability and construct validity. The instrument is recommended for the measurement of psychiatric outpatient experiences.

[Is there a relationship between hospital size and patient experience?]. / Er det sammenheng mellom sykehusstørrelse og pasienterfaringer?

BACKGROUND: Hospitals in Norway are undergoing changes in structure and organisation. Patient experience has been selected as a national indicator of hospital quality. The objective of this study was to investigate the relationship between hospital size and patient experience. MATERIAL AND METHODS: A patient experience questionnaire was sent to a representative sample of patients after discharge from 46 somatic hospitals, which were classified by function and by annual number of patients admitted. RESULTS: 10,975 patients (50%) responded. Generally, the patients expressed a positive attitude towards their hospitals. The patient experience was significantly more positive among those discharged from small hospitals than among those discharged from medium-sized or large hospitals, especially with regard to organisation. INTERPRETATION: Our study shows that patient experience was significantly more positive among those discharged from small hospitals than from medium-sized or large hospitals, especially with regard to organisation. However, the score differences between hospitals are small and should be interpreted with caution.

[The patient experiences questionnaire PasOpp in somatic outpatient clinics]. / Pasienterfaringsinstrumentet PasOpp i somatiske poliklinikker.

BACKGROUND: The purpose of this paper is to describe the development and evaluation of the patient experiences questionnaire (PEQ) for somatic outpatients. MATERIAL AND METHODS: Literature review, focus groups, interviews with patients and clinicians, and pre-testing of questionnaire items were used. A postal survey among 12,525 patients of 23 outpatient clinics for adults in two Norwegian health regions was conducted during 2003. RESULTS: 6824 (55%) patients responded to the questionnaire. Low levels of missing data suggest that the questionnaire is acceptable to patients. Factor analysis of items deemed applicable to all patients produced three scales: clinic access (2 items), communication (6 items) and organisation (4 items). The remaining items contributed to the hypothesised scales of information (6 items), hospital standards (3 items), and pre-visit experience (3 items). With the exception of the pre-visit experience scale, the levels of Cronbach's alpha met the criterion of 0.7. The great majority of the results of validity testing were as hypothesised. Correlations between the scales ranged from 0.30 (clinic access and information) to 0.72 (communication and information). As hypothesised, scores were significantly related to responses to questions about overall satisfaction, general health, and age. INTERPRETATION: The outpatient experience questionnaire is a self-administered instrument that includes the most important aspects of experience from the outpatients' perspective. Three core dimensions for use in a shortened version have been identified.