Reliability and validity of the Patient Experiences Questionnaire for Interdisciplinary Treatment for Substance Dependence - Continuous Electronic Measurement (PEQ-ITSD - CEM).

BACKGROUND: Inpatient experiences with interdisciplinary treatment for substance dependence and mental health care are measured using continuous electronic measurements in Norway. Major changes in data collection from cross-sectional surveys to continuous measurements necessitated the revalidation of the instrument. The main purpose of the present study was to determine the psychometric properties of the Patient Experiences Questionnaire for Interdisciplinary Treatment for Substance Dependence - Continuous Electronic Measurement (PEQ-ITSD - CEM). We also aimed to develop a short version of this tool, since completing the original version can be burdensome for some patients. METHODS: The study included adult inpatients (aged ≥ 16 years) who received substance-dependence treatment at 102 different sections in Norway during 2020-2022 (n = 2,850). Factor structure and item performance were assessed. A short version was developed based on the psychometric testing results that included item response theory analysis. RESULTS: The PEQ-ITSD - CEM comprised three empirically based scales with good internal consistency, reliability and validity, which covers treatment and personnel (14 items), milieu (6 items) and outcome (5 items). The results supported a seven-item short version, with three items selected for the treatment and personnel scale, two items for the milieu scale and two items for the outcome scale. CONCLUSIONS: The PEQ-ITSD - CEM can be recommended for future assessments of patient experiences with interdisciplinary treatment for substance dependence in Norway and in other countries with similar healthcare systems. This short-form version can be applied when respondent burden is a crucial issue.

Feedback reports to the general practitioner (GP) on the patients' experiences: are GPs interested, and is this interest associated with GP factors and patient experience scores?

BACKGROUND: Patient experience feedback is key in patient centred health systems, but empirical evidence of general practitioner (GP) interest in it is sparse. We aimed to: (i) quantitatively estimate the level of GP interest for feedback reports on patient experience; (ii) explore determinants of such interest; and (iii) examine potential association between a priori interest and patient experience. METHODS: The patient experience survey included maximum 300 randomly selected patients for each of 50 randomly selected GPs (response rate 41.4%, n = 5,623). GPs were sent a postal letter offering feedback reports and were grouped according to their replies: (i) interested in the report; (ii) not interested. Associations between interest and GP variables were assessed with Chi-square tests and multivariate logistic regression, while associations between interest and scores for 5 patient experiences scales were assessed with multilevel regression models. RESULTS: About half (n = 21; 45.7%) of the GPs showed interest in the report by asking to receive the report. The only GP variable associated with a priori interest was being a specialist in general practice (58.6% vs. 23.5% for those without) (P = 0.021). Interest was significantly associated with the practice patient experience scale (4.1 higher score compared with those not interested, P = 0.048). Interest in the report had small and nonsignificant associations with the remaining patient experience scales. CONCLUSIONS: Almost half of the GPs, and almost 3 in 5 of specialists in general practice, were interested in receiving a GP-specific feedback report on patient experiences. Interest in the report was generally not related to patient experience scores.

Web-Based Public Ratings of General Practitioners in Norway: Validation Study.

BACKGROUND: Understanding the complex relationships among multiple strategies for gathering users' perspectives in the evaluation of the performance of services is crucial for the interpretation of user-reported measures. OBJECTIVE: The main objectives were to (1) evaluate the psychometric performance of an 11-item web-based questionnaire of ratings of general practitioners (GPs) currently used in Norway (Legelisten.no) and (2) assess the association between web-based and survey-based patient experience indicators. METHODS: We included all published ratings on GPs and practices on Legelisten.no in the period of May 5, 2012, to December 15, 2021 (N=76,521). The questionnaire consists of 1 mandatory item and 10 voluntary items with 5 response categories (1 to 5 stars), alongside an open-ended review question and background variables. Questionnaire dimensionality and internal consistency were assessed with Cronbach α, exploratory factor, and item response theory analyses, and a priori hypotheses were developed for assessing construct validity (chi-square analysis). We calculated Spearman correlations between web-based ratings and reference patient experience indicators based on survey data using the patient experiences with the GP questionnaire (n=5623 respondents for a random sample of 50 GPs). RESULTS: Web-based raters were predominantly women (n=32,074, 64.0%), in the age range of 20-50 years (n=35,113, 74.6%), and reporting 5 or fewer consultations with the GP each year (n=28,798, 64.5%). Ratings were missing for 18.9% (n=14,500) to 27.4% (n=20,960) of nonmandatory items. A total of 4 of 11 rating items showed a U-shaped distribution, with >60% reporting 5 stars. Factor analysis and internal consistency testing identified 2 rating scales: "GP" (5 items; α=.98) and "practice" (6 items; α=.85). Some associations were not consistent with a priori hypotheses and allowed only partial confirmation of the construct validity of ratings. Item response theory analysis results were adequate for the "practice" scale but not for the "GP" scale, with items with inflated discrimination (>5) distributed over a narrow interval of the scale. The correlations between the web-based ratings GP scale and GP reference indicators ranged from 0.34 (P=.021) to 0.44 (P=.002), while the correlation between the web-based ratings practice scale and reference indicators ranged from 0.17 (not significant) to 0.49 (P<.001). The strongest correlations between web-based and survey scores were found for items measuring practice-related experiences: phone availability (ρ=0.51), waiting time in the office (ρ=0.62), other staff (ρ=0.54-0.58; P<.001). CONCLUSIONS: The practice scale of the web-based ratings has adequate psychometric performance, while the GP suffers from important limitations. The associations with survey-based patient experience indicators were accordingly mostly weak to modest. Our study underlines the importance of interpreting web-based ratings with caution and the need to further develop rating sites.

Reliability and validity of the Psychiatric Inpatient Patient Experience Questionnaire - Continuous Electronic Measurement (PIPEQ-CEM).

BACKGROUND: The increasing emphasis on patient-centred care has accelerated the demand for high-quality assessment instruments, but the development and application of measures of the quality of care provided for mental health have lagged behind other areas of medicine. The main objective of this study was to determine the psychometric properties of the Psychiatric Inpatient Patient Experience Questionnaire - Continuous Electronic Measurement (PIPEQ-CEM), which consists of large-scale measurements from a Norwegian population. The change from cross-sectional surveys to continuous measurements necessitated further validation of the instrument. The secondary objective was to develop a short version of the PIPEQ-CEM. METHODS: The data included responses from the first year of continuous measurement, and included adult inpatients (age ≥ 18 years) who received specialized mental healthcare from 191 different sections in Norway (n = 3,249). Missing data, ceiling effects, factor structure and internal consistency levels were assessed. The short scale was developed by exploring missing items, ceiling effects, results from exploratory factor analysis (EFA) and item performance from item response theory (IRT) analyses. RESULTS: Psychometric testing supported previous results and illustrated that the PIPEQ-CEM comprises three empirically based scales with good internal consistency, reliability and validity, and covers structure and facilities, patient-centred interactions, and outcomes. A seven-item short form was developed, which provides an efficient approach for brief yet comprehensive measurements that can be applied in the future. CONCLUSION: The PIPEQ-CEM can be recommended for use in future national surveys that assess patient experience with inpatient psychiatric care in Norway and in other countries with similar healthcare systems. The short form can be applied where respondent burden and cognitive load are crucial issues.

Patient experiences with general practitioners: psychometric performance of the generic PEQ-GP instrument among patients with chronic conditions.

BACKGROUND: Most generic patient experience instruments have not been validated specifically for persons with chronic health problems, even though they are the dominant user of GPs/family physicians. OBJECTIVES: To assess the psychometric properties of the generic Patient Experiences with GP Questionnaire (PEQ-GP) instrument (five scales: assessment of GP, coordination, patient enablement, accessibility, and practice) in persons with chronic conditions, and to develop a short version to maximize response rates and minimize respondent fatigue in future applications. METHODS: Secondary analysis of data from a national survey of patient experiences with general practitioners in 2018-2019 (response rate: 42.6%). The psychometric properties of PEQ-GP were assessed with exploratory factor analysis and Cronbach's alpha, supplemented with confirmatory factor analysis (CFA) and item response theory (IRT). A short version was constructed and evaluated based on item performance. RESULTS: Nine hundred and seventy persons reported a chronic condition(s), the most frequent being "musculoskeletal, arthritis, other back and joints" (n = 473, 48.8%). Factor analysis identified three scales with adequate psychometric results: GP (15 items; Cronbach's alpha: 0.96), practice (3 items; Cronbach's alpha: 0.87), and accessibility (2 items; Cronbach's alpha: 0.77). Evaluation of item performance identified a 7-item short version, including a 5-item GP scale with scores with strong concordance with the 15-item scale (Intraclass Correlation Coefficient: 0.97, P < 0.001). CONCLUSIONS: The generic PEQ-GP exhibits adequate psychometric performance for persons with chronic conditions. Three empirically derived PEQ-GP scales cover evaluation of the GP, accessibility, and practice. The 7-item short form minimize respondent burden, but further validation work is warranted before large-scale use.

Are parents' geographical origin associated with their evaluation of child and adolescent mental health services? Results from a national survey in Norway.

The objective was to assess the association between parents' geographical origin and their evaluation of outpatient child and adolescent mental health services (CAMHS). Data were collected in a national parent's experience survey of all outpatient CAMHS in Norway in 2017. Following exclusions, 16,143 parents were part of the study, of which 5932 responded (36.1%). Diagnosis and global psychosocial functional level were collected from the National Patient Register. Multilevel regression was used to assess the association between parents' geographical origin and parent evaluation of the outpatient CAMHS on ten indicators. Sentiment and content analysis was conducted on open-ended comments from parents. The estimated regression coefficients showed that parents born in Eastern Europe scored the services significantly poorer than parents born in Norway on outcome of treatment (- 7.73, p < 0.01), general satisfaction (- 5.53, p < 0.05), ease of getting in contact with health personnel outside of scheduled appointments (- 17.04, p < 0.001), and knowledge of the services that the child has received at the service (- 10.63, p < 0.001). Parents born in Asia/Africa/South America scored the services similar as Norwegian parents on eight of ten indicators, better on one (waiting time) and poorer on one (ease of getting in contact). Sentiment analysis showed that 54% of the comments from parents born in Eastern Europe were negative, compared to 42% for the Norwegian group and 36% for Asia/Africa/South America. The parents' evaluation of the outpatient CAMHS were partly associated with their geographical origin, with parents born in Eastern Europe reporting poorer experiences than parents born in Norway.

Patient experiences with general practice in Norway: a comparison of immigrant groups and the majority population following a national survey.

BACKGROUND: Patient experience is an important indicator of quality of health care. In Norway, little is known about the quality of health care for immigrants. The aim of this study was to compare patient-reported experiences with general practice between the Norwegian-born population and immigrant groups. METHODS: We performed secondary analyses of data from a national survey on patient experiences with general practice, including assessments of general practitioners (GPs) and their GP offices. The survey was carried out in Norway in 2018-19. The total number of respondents was 2029, with a response rate of 42.6%. Region of birth was available for 1981 participants, and these were included in the analyses ("Norway" (N = 1756), "Asia, Africa or South America" (N = 95), "Eastern Europe" (N = 70) and "Western Europe, North America or Oceania" (N = 60)). Five indicators of patient experiences were used as dependent variables in bivariate and multivariate analyses, with region of birth as the main exposure variable and other background variables about the patient as adjustment variables: "the GP" (measures related to communication and competency), "auxiliary staff" (politeness, competency, organization), "accessibility" (waiting times), "coordination" (with other services) and `enablement` (GP facilitates coping with/understanding illness). RESULTS: Immigrants as a whole reported poorer experiences with general practice than the majority population, with significantly poorer scores on four of five patient experience indicators. Patients from Asia/Africa/South America reported poorer experiences than those from Norway on the indicators "GP", "auxiliary staff", "accessibility" and "coordination": on a scale from 0 to 100 where 100 is the best, the difference ranged from 7.8 (GP) to 20.3 (accessibility). Patients from Eastern Europe reported lower scores on "GP" and patients from Western Europe/North America/Oceania reported lower scores on "auxiliary staff". These associations were still significant after adjustment for sex, age, self-rated physical and mental health, number of contacts with the GP and education. CONCLUSIONS: For countries with a substantial proportion of foreign-born patients in the health system, immigrant background is an important parameter in quality improvement work. Immigrant background is also an important parameter in health service research.

Patient-reported experiences with general practitioners: a randomised study of mail and web-based approaches following a national survey.

OBJECTIVE: The standard data-collection procedure in the Norwegian national patient experience survey programme is post-discharge mail surveys, which include a pen-and-paper questionnaire with the option to answer electronically. A purely electronic protocol has not previously been explored in Norway. The aim of this study was to compare response rates, background characteristics, data quality and main study results for a survey of patient experiences with general practitioners (GPs) administered by the standard mail data-collection procedure and a web-based approach. DESIGN: Cross-sectional survey. SETTING: GP offices in Norway. PARTICIPANTS: The sample consisted of 6999 patients aged 16 years and older registered with a GP in November 2018. INTERVENTION: Based on a three-stage sampling design, 6999 patients of GPs aged 16 or older were randomised to one of two survey administration protocols: Group A, who were mailed an invitation with both a pen-and-paper including an electronic response option (n=4999) and Group B, who received an email invitation with electronic response option (n=2000). MAIN OUTCOME MEASURES: Response rates, background characteristics, data quality and main study results. RESULTS: The response rate was markedly higher for the mail survey (42.6%) than for the web-based survey (18.3%). A few of the background variables differed significantly between the two groups, but the data quality and patient-reported experiences were similar. CONCLUSIONS: Web-based surveys are faster and less expensive than standard mail surveys, but their low response rates and coverage problems threaten their usefulness and legitimacy. Initiatives to increase response rates for web-based data collection and strategies for tailoring data collection to different groups should be key elements in future research.

The value of Facebook in nation-wide hospital quality assessment: a national mixed-methods study in Norway.

OBJECTIVES: The objective was to assess the possibility of using a combination of official and unofficial Facebook ratings and comments as a basis for nation-wide hospital quality assessments in Norway. METHODS: All hospitals from a national cross-sectional patient experience survey in 2015 were matched with corresponding Facebook ratings. Facebook ratings were correlated with both case-mix adjusted and unadjusted patient-reported experience scores, with separate analysis for hospitals with official site ratings and hospitals with unofficial site ratings. Facebook ratings were also correlated with patient-reported incident scores, hospital size, 30-day mortality and 30-day readmission. Facebook comments from 20 randomly selected hospitals were analysed, contrasting the content and sentiments of official versus unofficial Facebook pages. RESULTS: Facebook ratings were significantly correlated with most patient-reported indicators, with the highest correlations relating to unadjusted scores for organisation (0.60, p<0.000) and nursing services (0.57, p<0.000). Facebook ratings were significantly correlated with hospital size (-0.40, p=0.003) and 30-day mortality (0.31, p=0.040). Sentiment analysis showed that 84.7% of the comments from unofficial Facebook sites included neutral comments that did not give any specific description of experiences of the quality of care at the hospital. Content analysis identified common themes on official and unofficial Facebook pages. CONCLUSIONS: Facebook ratings were associated with patient-reported indicators, hospital size, and 30-day mortality. Qualitative comments from official Facebook are more relevant for hospital evaluation than unofficial sites. More research is needed on using Facebook ratings as a standalone indicator of patient experiences in national quality measurement, and such ratings should be reported together with research-based patient experience indicators and with explicit criteria for the inclusion of unofficial sites.

Associations between adolescent experiences, parent experiences and HbA1c: results following two surveys based on the Norwegian Childhood Diabetes Registry (NCDR).

OBJECTIVE: The aim of the current study was to determine the association between the experiences of adolescents and their parents with paediatric diabetes care at hospital outpatient departments and the association between these experiences and the Hemoglobin A1c (HbA1c) levels of adolescents. DESIGN: Cross-sectional survey. SETTING: Paediatric diabetes care at hospital outpatient departments in Norway. PARTICIPANTS: Parents of all outpatients registered in the Norwegian Childhood Diabetes Registry and patients in the same registry aged 12-17 years. INTERVENTION: 1399 parents participated in a national pilot survey and 335 patients aged 12-17 years from the four largest paediatric outpatient departments in Norway responded in another pilot study. 181 paired parental and patient questionnaires were analysed. MAIN OUTCOME MEASURES: The correlations between single items, indicator scores and overall scores were explored, as was that between indicator scores and HbA1c levels. RESULTS: There was a moderate but significant correlation between the responses of the patients and parents. For 40 of the 42 associations the correlations were significant, ranging from 0.16 to 0.42. A weak but significant negative correlation was found between the indicator scores of parents and the HbA1c levels of the adolescents. The strongest correlations were between HbA1c level and nurse contact and organisation, both with a correlation coefficient of 0.21 (p<0.01). There was no significant correlation between HbA1c level and patient indicator scores. CONCLUSIONS: These results highlight the need to obtain information from both parents and adolescents, and indicate that the views of adolescents are not always mirrored by their parents. Three of the seven parent experience indicators were significantly related to the HbA1c levels of adolescents, but replication in future research with larger sample sizes is warranted.

Patient experiences with interdisciplinary treatment for substance dependence: an assessment of quality indicators based on two national surveys in Norway.

Purpose: The quality of health care is often measured using quality indicators, which can be utilized to compare the performance of health-care providers. Conducting comparisons in a meaningful and fair way requires the quality indicators to be adjusted for patient characteristics and other individual-level factors. The aims of the study were to develop and test a case-mix adjustment model for quality indicators based on patient-experience surveys among inpatients receiving interdisciplinary treatment for substance dependence, and to establish whether the quality indicators discriminate between health care providers. Patients and methods: Data were collected through two national surveys involving inpatients receiving residential treatment in Norway in 2013 and 2014. The same questionnaire was used in both surveys, and comprised three patient-experience scales. The scales are reported as national quality indicators, and associations between the scales and patient characteristics were tested through multilevel modeling to establish a case-mix model. The intraclass correlation coefficient was computed to assess the amount of variation at the hospital-trust level. Results: The intraclass correlation coefficient for the patient-reported experience scales varied from 2.3% for "treatment and personnel" to 8.1% for "milieu". Multivariate multilevel regression analyses showed that alcohol reported as the most frequently used substance, gender and age were significantly associated with two of the three scales. The length of stay at the institution, pressure to be admitted for treatment, and self-perceived health were significantly related to all three scales. Explained variance at the individual level was approximately 7% for all three scales. Conclusion: This study identified several important case-mix variables for the patient-based quality indicators and systematic variations at the hospital-trust level. Future research should assess the association between patient-based quality indicators and other quality indicators, and the predictive validity of patient-experience indicators based on on-site measurements.

The Adolescent Patient Experiences of Diabetes Care Questionnaire (APEQ-DC): Reliability and Validity in a Study Based on Data from the Norwegian Childhood Diabetes Registry.

PURPOSE: Patient-reported experiences are a key source of information on quality in health care. Most patient experience surveys only include adults' assessments including parent or proxy surveys in child health care settings. The aim of this study was to determine the psychometric properties of the Adolescent Patient Experiences of Diabetes Care Questionnaire, a new instrument developed to measure adolescent experiences of paediatric diabetes care at hospital outpatient departments in Norway. PATIENTS AND METHODS: The questionnaire was developed based on a literature review, qualitative interviews with adolescents, expert-group consultations, pretesting of the questionnaire and a pilot study. The pilot study involved adolescents aged 12-17 years with type 1 diabetes, sampled from the four largest paediatric outpatient departments in Norway. We assessed the levels of missing data, ceiling effects, factor structure, internal consistency, item discriminant validity and construct validity. RESULTS: The pilot study included responses from 335 (54%) patients. Low proportions of missing or "not applicable" responses were found for 17 of the 19 items, and 14 of these 19 items were below the ceiling-effect criterion. Five indicators were identified: consultation, information on food and physical activity/exercise, nurse contact, doctor contact and outcome. All except one indicator met the criterion of 0.7 for Cronbach's alpha. Each of the single items had a stronger correlation with its hypothesized indicator than with any of the other indicators. The construct validity of the instrument was supported by 38 out of 45 significant associations. CONCLUSION: The content validity of the instrument was secured by a rigorous development process. Psychometric testing produced good evidence for data quality, internal consistency and construct validity. Further research is needed to assess the usefulness of the Adolescent Patient Experiences of Diabetes Care Questionnaire as a basis for quality indicators.

Inpatients' assessment of outcome at psychiatric institutions: an analysis of predictors following a national cross-sectional survey in Norway.

OBJECTIVES: The objective was to assess the importance of different types of predictors for patient-reported outcome, both background factors at the patient level and healthcare predictors related to structure and processes of healthcare. DESIGN: Cross-sectional patient experience survey. SETTING: All 280 secondary care institutions in Norway providing inpatient care for adult psychiatric patients. PARTICIPANTS: 1683 inpatients responded to the questionnaire on-site (73.4%). PRIMARY OUTCOME MEASURES: The outcome scale of the Psychiatric Inpatient Patient Experience Questionnaire-On-Site was the primary dependent variable. The scale consists of five items relating to overall patient satisfaction, benefit of treatment and patient enablement. Regressions were used to assess predictors, for all patients and for five different patient groups reported by the patients including anxiety/depression, drug-related problems and eating disorders. RESULTS: Multilevel linear regression for all patients showed that background factors related to overall current state, self-perceived mental health before admission and admission type were the most important predictors for patient-assessed outcome. Poor current state was associated with poor assessment of outcome (estimate: 8.64, p<0.001), poor health before admission was associated with better outcome (estimate: -6.89, p<0.001) and patients with urgent admission had poorer scores on the outcome scale (estimate: 4.40, p<0.001). A range of structure and healthcare variables were related to patient-assessed outcome, the most important being clinicians/personnel understanding your situation, treatment adjusted to your situation and adequate information about mental health condition. CONCLUSIONS: Self-perceived mental health before admission, current overall state and type of admission were the most important background factors for patient-assessed outcome. The most important structure and process variables were related to patient-centred interaction. The background factors should be considered in case-mix adjustments of quality indicators, while the process variables could be used as focus areas in work aiming to improve patients' assessment of outcome.

Parent experiences of diabetes care questionnaire (PEQ-DC): reliability and validity following a national survey in Norway.

BACKGROUND: Patient experiences are acknowledged as an important aspect of health care quality but no validated instruments have been identified for the measurement of either parent or patient experiences with outpatient paediatric diabetes care. The aim of the current study was to assess the psychometric properties of a new instrument developed to measure parent experiences of paediatric diabetes care at hospital outpatient departments in Norway. METHODS: The development of the questionnaire was based on a literature review of existing questionnaires, qualitative interviews with both parents and children/adolescents, expert-group consultations, pretesting of the questionnaire and a pilot study. The national pilot study included parents of 2606 children/adolescents aged 0-17 years with Type 1 Diabetes registered in The Norwegian Childhood Diabetes Registry, a nationwide, population-based registry. Levels of missing data, ceiling effects, factor structure, internal consistency, item discriminant validity and construct validity were assessed. RESULTS: A total of 2606 patients were included in the survey, but 80 were excluded due to incorrect addresses. 1399 (55%) parents responded to the questionnaire. Low levels of missing or "not applicable" responses were found for 31 of the 35 items (< 10%), and 27 of 35 items were below the ceiling-effect criterion. Psychometric testing and theoretical considerations identified six scales: Consultation (six items), organisation (five items), equipment (three items), nurse contact (four items), doctor contact (four items) and outcome (five items). All six scales met the 0.7 criterion for Cronbach's alpha (range: 0.71-0.90). As expected, each item had a higher correlation with its hypothesised scale than with any of the other five scales. The construct validity of the Parent Experiences of Diabetes Care Questionnaire (PEQ-DC) was supported by 17 out of 18 associations with variables expected to be related to parent experiences. CONCLUSION: The psychometric testing of the PEQ-DC showed good evidence for data quality, internal consistency and construct validity. The instrument includes important aspects of diabetes care at paediatric outpatient departments from the perspective of the parent. The content validity of the PEQ-DC was secured by a rigorous development process, and the instrument was tested following a national survey in Norway, securing generalisability across Norway.

A randomized comparison of three data collection models for the measurement of parent experiences with diabetes outpatient care.

BACKGROUND: The objective of this study was to compare three data collection methods for the measurement of parent experiences with hospital outpatient care for child and adolescent diabetes, based on a randomised national trial in Norway involving both pen-and-paper and electronic response options. METHODS: The sample frame was patients registered in the Norwegian Childhood Diabetes Registry. Parents of patients were randomised into the following groups (n = 2606): group A, who were posted questionnaires with only a pen-and-paper response option (n = 859); group B, who were posted questionnaires with both an electronic and a pen-and-paper response option (n = 886); and group C, who were posted questionnaires with only an electronic response option (n = 861). The three groups were compared on response rate, background variables about respondents, main study results and survey costs. Statistical analysis included logistic regression to test group differences in response probabilities and multilevel linear regression to test differences in parent experiences. RESULTS: The response rate was 61.8% for group A, 62.4% for group B and 41.6% for group C. The probability of answering was significantly higher for group A (OR = 2.3, p < 0.001) and B (OR = 2.3, p < 0.001) compared to group C. Respondent age, gender, education, living with the child and the degree of participation in consultations did not differ significantly between the three groups. Group differences in parent-reported experiences were small, varying from 1.0 (equipment and doctor contact) to 2.4 (outcome), on a scale from 0 to 100. Only one of 18 group differences was significant: the mixed group had significantly higher score than the electronic group on the organization scale (p < 0.05). The total cost of the electronic model was less than half the cost of the other models, and cost per response was 5.1 euros for the electronic model compared to 8.2 euros for group A and 7.6 euros for group B. CONCLUSIONS: The models with pen-and paper questionnaire included had more than 20% higher response rate than the model with an electronic-only response option. Background variables and parent-reported experiences were similar between the three groups, and the electronic model was the more cost-effective model.

The Norwegian patient experiences with GP questionnaire (PEQ-GP): reliability and construct validity following a national survey.

OBJECTIVES: The aim of this study was to test the reliability and validity of a new questionnaire for measuring patient experiences with general practitioners (PEQ-GP) following a national survey. SETTING: Postal survey among patients on any of 500 GPs patient lists in Norway. GPs were stratified by practice size and geographical criteria. PARTICIPANTS: 4964 patients who had at least one consultation with their regular GP in the foregoing 12 months were included in the study. The patients were randomly selected after the selection of GPs. 2377 patients (49%) responded to the survey. PRIMARY AND SECONDARY OUTCOME MEASURES: The items were assessed for missing data and ceiling effects. Factor structure was assessed using exploratory factor analyses. Reliability was tested with item-total correlation, Cronbach's alpha and test-retest correlations. Item discriminant validity was tested by correlating items with all scales. Construct validity was assessed through associations of scale scores with health status, the patients' general satisfaction with the services, whether the patient had been incorrectly treated by the GP and whether the patient would recommend the GP to others. RESULTS: Item missing varied from 1.0% to 3.1%, while ceiling effects varied from 16.1% to 45.9%. The factor analyses identified three factors. Reliability statistics for scales based on these three factors, and two theoretically derived scales, showed item-total correlations ranging from 0.63 to 0.85 and Cronbach's alpha values from 0.77 to 0.93. Test-retest correlation for the five scales varied from 0.72 to 0.88. All scales had the expected association with other variables. CONCLUSIONS: The PEQ-GP has good evidence for data quality, internal consistency and construct validity. The PEQ-GP is recommended for use in local, regional and national surveys in Norway, but further studies are needed to assess the instrument's ability to detect differences over time and between different GPs.

Patient experiences questionnaire for interdisciplinary treatment for substance dependence (PEQ-ITSD): reliability and validity following a national survey in Norway.

BACKGROUND: Patient experiences are an important aspect of health care quality, but there is a lack of validated instruments for their measurement in the substance dependence literature. A new questionnaire to measure inpatients' experiences of interdisciplinary treatment for substance dependence has been developed in Norway. The aim of this study was to psychometrically test the new questionnaire, using data from a national survey in 2013. METHODS: The questionnaire was developed based on a literature review, qualitative interviews with patients, expert group discussions and pretesting. Data were collected in a national survey covering all residential facilities with inpatients in treatment for substance dependence in 2013. Data quality and psychometric properties were assessed, including ceiling effects, item missing, exploratory factor analysis, and tests of internal consistency reliability, test-retest reliability and construct validity. RESULTS: The sample included 978 inpatients present at 98 residential institutions. After correcting for excluded patients (n = 175), the response rate was 91.4%. 28 out of 33 items had less than 20.5% of missing data or replies in the "not applicable" category. All but one item met the ceiling effect criterion of less than 50.0% of the responses in the most favorable category. Exploratory factor analysis resulted in three scales: "treatment and personnel", "milieu" and "outcome". All scales showed satisfactory internal consistency reliability (Cronbach's alpha ranged from 0.75-0.91) and test-retest reliability (ICC ranged from 0.82-0.85). 17 of 18 significant associations between single variables and the scales supported construct validity of the PEQ-ITSD. CONCLUSION: The content validity of the PEQ-ITSD was secured by a literature review, consultations with an expert group and qualitative interviews with patients. The PEQ-ITSD was used in a national survey in Norway in 2013 and psychometric testing showed that the instrument had satisfactory internal consistency reliability and construct validity.

The Universal Patient Centeredness Questionnaire: scaling approaches to reduce positive skew.

PURPOSE: Surveys of patients' experiences typically show results that are indicative of positive experiences. Unbalanced response scales have reduced positive skew for responses to items within the Universal Patient Centeredness Questionnaire (UPC-Q). The objective of this study was to compare the unbalanced response scale with another unbalanced approach to scaling to assess whether the positive skew might be further reduced. PATIENTS AND METHODS: The UPC-Q was included in a patient experience survey conducted at the ward level at six hospitals in Norway in 2015. The postal survey included two reminders to nonrespondents. For patients in the first month of inclusion, UPC-Q items had standard scaling: poor, fairly good, good, very good, and excellent. For patients in the second month, the scaling was more positive: poor, good, very good, exceptionally good, and excellent. The effect of scaling on UPC-Q scores was tested with independent samples t-tests and multilevel linear regression analysis, the latter controlling for the hierarchical structure of data and known predictors of patient-reported experiences. RESULTS: The response rate was 54.6% (n=4,970). Significantly lower scores were found for all items of the more positively worded scale: UPC-Q total score difference was 7.9 (P<0.001), on a scale from 0 to 100 where 100 is the best possible score. Differences between the four items of the UPC-Q ranged from 7.1 (P<0.001) to 10.4 (P<0.001). Multivariate multilevel regression analysis confirmed the difference between the response groups, after controlling for other background variables; UPC-Q total score difference estimate was 8.3 (P<0.001). CONCLUSION: The more positively worded scaling significantly lowered the mean scores, potentially increasing the sensitivity of the UPC-Q to identify differences over time and between providers. However, none of the groups exhibited large positive skew and ceiling effects, implying that such effects might not be a big measurement problem for either scaling format. We recommend using the standard scaling in surveys producing external indicators for inter-provider comparisons. The more positively worded scaling has greater relevance for local measurement work where the results of patient experience surveys have shown large positive skew, and intra-provider comparison is the primary goal.

The Universal Patient Centeredness Questionnaire: reliability and validity of a one-page questionnaire following surveys in three patient populations.

BACKGROUND: This study developed and tested the reliability and validity of the Universal Patient Centeredness Questionnaire (UPC-Q). METHODS: The UPC-Q developed in this study has three parts: 1) the aspects that patients consider the most important when receiving a relevant health care service, rating the health care services on these aspects and their prioritization, 2) the overall experiences of patients using the relevant health care service, and 3) suggestions for improvements. The UPC-Q was tested in four different patient-experience surveys in 2015, including psychiatric inpatients (n=109), general practitioner (GP) patients (n=1,059), and inpatients from two hospital samples (n=973, n=599). The UPC-Q was tested for item completeness and ceiling effects, while the UPC-Q scale consisting of the first part of the UPC-Q was tested for internal consistency reliability and construct validity. RESULTS: The percentage of patients rating at least one aspect was 70.6% for psychiatric inpatients, 77.6% for hospital inpatients, and 90.6% for GP patients, while 88.9% of the psychiatric inpatients, 93.1% of the hospital inpatients, and 95.3% of the GP patients were able to prioritize the aspects. The internal consistency reliability of the UPC-Q scale was acceptable in all samples (Cronbach's alpha >0.7), and construct validity was supported by 20 of 21 significant associations between the UPC-Q and related variables. The UPC-Q total score was skewed toward positive evaluations, but the ceiling effect was smaller for an unbalanced response scale than for a balanced scale. CONCLUSION: The UPC-Q includes ratings of what is most important for individual patients, while at the same time providing data for improving the quality of health care and making it possible to monitor trends within and across patient populations. This study included psychiatric inpatients, hospital inpatients, and GP patients, and found that the UPC-Q performed well in terms of acceptance, internal consistency reliability, and construct validity.