Modelling variance in the multidimensional Health Literacy Questionnaire: Does a General Health Literacy factor account for observed interscale correlations?

Objectives: Self-report measures of health literacy frequently encompass a group of conceptually distinct but related domains scored as either a single scale or separately for each domain. Psychometric studies of the 44-item Health Literacy Questionnaire, scored as nine separate scales, typically report the results of correlated factors confirmatory factor analyses as validity evidence. However, some scales are quite strongly correlated, raising the question of whether there is sufficient discriminant validity to warrant separate scoring. This psychometric study aims to contrast the results of fitting higher-order and bifactor models as alternative options to illuminate the issue. Methods: Correlated factors, higher-order and bifactor confirmatory factor analysis models were fitted to an Australian sample of responses to the Health Literacy Questionnaire (N = 813) using Bayesian confirmatory factor analysis methods. Results: All models representing a nine-factor structure for the Health Literacy Questionnaire fitted well. The correlated factors model replicated previous findings, showing inter-factor correlations between 0.19 and 0.93. A higher-order model showed relatively high loadings of all nine first-order factors on the second-order factor with particularly high loadings (⩾0.97) for three. Two bifactor models showed that the majority of Health Literacy Questionnaire items were multifactorial, each containing systematic variance from both a General Health Literacy factor and a domain-specific factor. Seven items from four scales were identified as strongly associated with the General Health Literacy factor, with item content suggesting that this factor indexes a broad sense of agency and efficacy in interacting with health-related information and healthcare providers. Conclusion: Contrasting correlated factors, higher-order and bifactor models fitted to the Health Literacy Questionnaire suggest that constituent items in self-report health literacy questionnaires might be anticipated to represent at least two sources of reliable and substantive common factor variance: variance associated with General Health Literacy and variance associated with a more specific domain, suggesting that items may be 'irreducibly' heterogeneous. Implications for test development and validation practice are discussed.

eHealth Literacy of Australian Undergraduate Health Profession Students: A Descriptive Study.

Rapid growth in digital health technologies has increased demand for eHealth literacy of all stakeholders within health and social care environments. The digital future of health care services requires the next generation of health professionals to be well-prepared to confidently provide high-quality and safe health care. The aim of this study was to explore the eHealth literacy of undergraduate health profession students to inform undergraduate curriculum development to promote work-readiness. A cross-sectional survey was undertaken at an Australian university using the seven-domain eHealth Literacy Questionnaire (eHLQ), with 610 students participating. A one-way Multivariate Analysis of Variance (MANOVA) with follow-up univariate analysis (ANOVA) was used to determine if there were differences in eHLQ scores across 11 sociodemographic variables. Students generally had good knowledge of health (Scale 2); however, they had concerns over the security of online health data (Scale 4). There were also significant differences in age and ownership of digital devices. Students who were younger reported higher scores across all seven eHLQ scales than older students. This research provided an understanding of eHealth literacy of health profession students and revealed sub-groups that have lower eHealth literacy, suggesting that digital health skills should be integrated into university curriculums, especially related to practice-based digital applications with special focus to address privacy and security concerns. Preparation of health profession students so they can efficiently address their own needs, and the needs of others, is recommended to minimise the digital divide within health and social care environments.

Cultural Adaptation and Validity Testing of the Portuguese Version of the Health Literacy Questionnaire (HLQ).

BACKGROUND: Health literacy is considered a determinant of self-management behaviors and health outcomes among people with diabetes. The assessment of health literacy is central to understanding the health needs of a population. This study aimed to adapt the Health Literacy Questionnaire (HLQ) to the Portuguese context and to examine the psychometric properties of a population of people with diabetes. METHODS: Data were collected using a self-administrated questionnaire from 453 people with diabetes in a specialized diabetes care unit. Analysis included item difficulty level, composite scale reliability, and confirmatory factor analysis (CFA). RESULTS: The HLQ showed that the items were easily understood by participants. Composite reliability ranged from 0.74 to 0.83. A nine-factor CFA model was fitted to the 44 items. Given the very restricted model, the fit was quite satisfactory [χ2wlsmv = 2147.3 (df = 866), p = 0.001; CFI = 0.931, TLI = 0.925, RMSEA = 0.057 (90% C.I. 0.054-0.060), and WRMR = 1.528]. CONCLUSION: The Portuguese version of the HLQ has shown satisfactory psychometric properties across its nine separate scales in people with diabetes. Given the strong observed properties of the HLQ across cultures, languages, and diseases, the HLQ is likely to be a useful tool in a range of Portuguese settings.

Use of the English Health Literacy Questionnaire (HLQ) with Health Science University Students in Nepal: A Validity Testing Study.

Research evidence shows that health literacy development is a key factor influencing non-communicable diseases care and patient outcomes. Healthcare professionals with strong health literacy skills are essential for providing quality care. We aimed to report the validation testing of the Health Literacy Questionnaire (HLQ) among health professional students in Nepal. A cross-sectional study was conducted with 419 health sciences students using the HLQ in Nepal. Validation testing and reporting were conducted using five sources outlined by 'the 2014 Standards for Educational and Psychological Testing'. The average difficulty was lowest (17.4%) for Scale 4. Social support for health, and highest (51.9%) for Scale 6. Ability to actively engage with healthcare providers. One factor Confirmatory Factor Analysis (CFA) model showed a good fit for Scale 2, Scale 7 and Scale 9 and a reasonable fit for Scale 3 and Scale 4. The restricted nine-factor CFA model showed a satisfactory level of fit. The use of HLQ is seen to be meaningful in Nepal and warrants translation into native Nepali and other dominant local languages with careful consideration of cultural appropriateness using cognitive interviews.

Validity Evidence of the eHealth Literacy Questionnaire (eHLQ) Part 2: Mixed Methods Approach to Evaluate Test Content, Response Process, and Internal Structure in the Australian Community Health Setting.

BACKGROUND: Digital technologies have changed how we manage our health, and eHealth literacy is needed to engage with health technologies. Any eHealth strategy would be ineffective if users' eHealth literacy needs are not addressed. A robust measure of eHealth literacy is essential for understanding these needs. On the basis of the eHealth Literacy Framework, which identified 7 dimensions of eHealth literacy, the eHealth Literacy Questionnaire (eHLQ) was developed. The tool has demonstrated robust psychometric properties in the Danish setting, but validity testing should be an ongoing and accumulative process. OBJECTIVE: This study aims to evaluate validity evidence based on test content, response process, and internal structure of the eHLQ in the Australian community health setting. METHODS: A mixed methods approach was used with cognitive interviewing conducted to examine evidence on test content and response process, whereas a cross-sectional survey was undertaken for evidence on internal structure. Data were collected at 3 diverse community health sites in Victoria, Australia. Psychometric testing included both the classical test theory and item response theory approaches. Methods included Bayesian structural equation modeling for confirmatory factor analysis, internal consistency and test-retest for reliability, and the Bayesian multiple-indicators, multiple-causes model for testing of differential item functioning. RESULTS: Cognitive interviewing identified only 1 confusing term, which was clarified. All items were easy to read and understood as intended. A total of 525 questionnaires were included for psychometric analysis. All scales were homogenous with composite scale reliability ranging from 0.73 to 0.90. The intraclass correlation coefficient for test-retest reliability for the 7 scales ranged from 0.72 to 0.95. A 7-factor Bayesian structural equation modeling using small variance priors for cross-loadings and residual covariances was fitted to the data, and the model of interest produced a satisfactory fit (posterior productive P=.49, 95% CI for the difference between observed and replicated chi-square values -101.40 to 108.83, prior-posterior productive P=.92). All items loaded on the relevant factor, with loadings ranging from 0.36 to 0.94. No significant cross-loading was found. There was no evidence of differential item functioning for administration format, site area, and health setting. However, discriminant validity was not well established for scales 1, 3, 5, 6, and 7. Item response theory analysis found that all items provided precise information at different trait levels, except for 1 item. All items demonstrated different sensitivity to different trait levels and represented a range of difficulty levels. CONCLUSIONS: The evidence suggests that the eHLQ is a tool with robust psychometric properties and further investigation of discriminant validity is recommended. It is ready to be used to identify eHealth literacy strengths and challenges and assist the development of digital health interventions to ensure that people with limited digital access and skills are not left behind.

Validity Evidence Based on Relations to Other Variables of the eHealth Literacy Questionnaire (eHLQ): Bayesian Approach to Test for Known-Groups Validity.

BACKGROUND: As health resources and services are increasingly delivered through digital platforms, eHealth literacy is becoming a set of essential capabilities to improve consumer health in the digital era. To understand eHealth literacy needs, a meaningful measure is required. Strong initial evidence for the reliability and construct validity of inferences drawn from the eHealth Literacy Questionnaire (eHLQ) was obtained during its development in Denmark, but validity testing for varying purposes is an ongoing and cumulative process. OBJECTIVE: This study aims to examine validity evidence based on relations to other variables-using data collected with the known-groups approach-to further explore if the eHLQ is a robust tool to understand eHealth literacy needs in different contexts. A priori hypotheses are set for the expected score differences among age, sex, education, and information and communication technology (ICT) use for each of the 7 eHealth literacy constructs represented by the 7 eHLQ scales. METHODS: A Bayesian mediated multiple indicators multiple causes model approach was used to simultaneously identify group differences and test measurement invariance through differential item functioning across the groups, with ICT use as a mediator. A sample size of 500 participants was estimated. Data were collected at 3 diverse health sites in Australia. RESULTS: Responses from 525 participants were included for analysis. Being older was significantly related to lower scores in 4 eHLQ scales, with 3. Ability to actively engage with digital services having the strongest effect (total effect -0.37; P<.001), followed by 1. Using technology to process health information (total effect -0.32; P<.001), 5. Motivated to engage with digital services (total effect -0.21; P=.01), and 7. Digital services that suit individual needs (total effect -0.21; P=.02). However, the effects were only partially mediated by ICT use. Higher education was associated with higher scores in 1. Using technology to process health information (total effect 0.22; P=.01) and 3. Ability to actively engage with digital services (total effect 0.25; P<.001), with the effects mostly mediated by ICT use. Higher ICT use was related to higher scores in all scales except 2. Understanding health concepts and language and 4. Feel safe and in control. Either no or ignorable cases of differential item functioning were found across the 4 groups. CONCLUSIONS: By using a Bayesian mediated multiple indicators multiple causes model, this study provides supportive validity evidence for the eHLQ based on relations to other variables as well as established evidence regarding internal structure related to measurement invariance across the groups for the 7 scales in the Australian community health context. This study also demonstrates that the eHLQ can be used to gain valuable insights into people's eHealth literacy needs to help optimize access and use of digital health and promote health equity.

Validity arguments for patient-reported outcomes: justifying the intended interpretation and use of data.

BACKGROUND: Contrary to common usage in the health sciences, the term "valid" refers not to the properties of a measurement instrument but to the extent to which data-derived inferences are appropriate, meaningful, and useful for intended decision making. The aim of this study was to determine how validity testing theory (the Standards for Educational and Psychological Testing) and methodology (Kane's argument-based approach to validation) from education and psychology can be applied to validation practices for patient-reported outcomes that are measured by instruments that assess theoretical constructs in health. METHODS: The Health Literacy Questionnaire (HLQ) was used as an example of a theory-based self-report assessment for the purposes of this study. Kane's five inferences (scoring, generalisation, extrapolation, theory-based interpretation, and implications) for theoretical constructs were applied to the general interpretive argument for the HLQ. Existing validity evidence for the HLQ was identified and collated (as per the Standards recommendation) through a literature review and mapped to the five inferences. Evaluation of the evidence was not within the scope of this study. RESULTS: The general HLQ interpretive argument was built to demonstrate Kane's five inferences (and associated warrants and assumptions) for theoretical constructs, and which connect raw data to the intended interpretation and use of the data. The literature review identified 11 HLQ articles from which 57 sources of validity evidence were extracted and mapped to the general interpretive argument. CONCLUSIONS: Kane's five inferences and associated warrants and assumptions were demonstrated in relation to the HLQ. However, the process developed in this study is likely to be suitable for validation planning for other measurement instruments. Systematic and transparent validation planning and the generation (or, as in this study, collation) of relevant validity evidence supports developers and users of PRO instruments to determine the extent to which inferences about data are appropriate, meaningful and useful (i.e., valid) for intended decisions about the health and care of individuals, groups and populations.

Co-designing eHealth and Equity Solutions: Application of the Ophelia (Optimizing Health Literacy and Access) Process.

Background: The unequal access, challenges and outcomes related to using technology have created the digital divide, which leads to health inequalities. The aim of this study was to apply the Ophelia (Optimizing Health Literacy and Access) process, a widely used systematic approach to whole of community co-design, to the digital context to generate solutions to improve health and equity outcomes. Methods: This was a mixed method study. A cross-sectional survey was undertaken at 3 health organizations in Victoria, Australia using the eHealth Literacy Questionnaire (eHLQ) as a needs assessment tool. Cluster analysis was conducted to identify subgroups with varying eHealth literacy needs. These data, combined with semi-structured interviews with clients, were used to generate vignettes representing different eHealth literacy profiles. The vignettes were presented at co-design workshops with clients and health professionals to generate solutions for digital health services improvement. Expert validation and proof-of-concept testing was explored through mapping the process against Ophelia guiding principles. Results: The cluster analyses identified 8 to 9 clusters with different profiles of eHealth literacy needs, with 4 to 6 vignettes developed to represent the eHealth literacy strengths and weaknesses of clients at each of the 3 sites. A total of 32, 43, and 32 solutions across 10 strategies were co-created based on ideas grounded in local expertise and experiences. Apart from digital solutions, non-digital solutions were frequently recommended as a strategy to address eHealth literacy needs. Expert validation identified at least half of the ideas were very important and feasible, while most of the guiding principles of the Ophelia process were successfully applied. Conclusion: By harnessing collective creativity through co-design, the Ophelia process has been shown to assist the development of solutions with the potential to improve health and equity outcomes in the digital context. Implementation of the solutions is needed to provide further evidence of the impact of the process. The suggested inclusion of non-digital solutions revealed through the co-design process reminds health organizations and policymakers that solutions should be flexible enough to suit individual needs. As such, taking a co-design approach to digital health initiatives will assist in preventing the widening of health inequalities.

Applying the Electronic Health Literacy Lens: Systematic Review of Electronic Health Interventions Targeted at Socially Disadvantaged Groups.

BACKGROUND: Electronic health (eHealth) has the potential to improve health outcomes. However, eHealth systems need to match the eHealth literacy needs of users to be equitably adopted. Socially disadvantaged groups have lower access and skills to use technologies and are at risk of being digitally marginalized, leading to the potential widening of health disparities. OBJECTIVE: This systematic review aims to explore the role of eHealth literacy and user involvement in developing eHealth interventions targeted at socially disadvantaged groups. METHODS: A systematic search was conducted across 10 databases for eHealth interventions targeted at older adults, ethnic minority groups, low-income groups, low-literacy groups, and rural communities. The eHealth Literacy Framework was used to examine the eHealth literacy components of reviewed interventions. The results were analyzed using narrative synthesis. RESULTS: A total of 51 studies reporting on the results of 48 interventions were evaluated. Most studies were targeted at older adults and ethnic minorities, with only 2 studies focusing on low-literacy groups. eHealth literacy was not considered in the development of any of the studies, and no eHealth literacy assessment was conducted. User involvement in designing interventions was limited, and eHealth intervention developmental frameworks were rarely used. Strategies to assist users in engaging with technical systems were seldom included in the interventions, and accessibility features were limited. The results of the included studies also provided inconclusive evidence on the effectiveness of eHealth interventions. CONCLUSIONS: The findings highlight that eHealth literacy is generally overlooked in developing eHealth interventions targeted at socially disadvantaged groups, whereas evidence about the effectiveness of such interventions is limited. To ensure equal access and inclusiveness in the age of eHealth, eHealth literacy of disadvantaged groups needs to be addressed to help avoid a digital divide. This will assist the realization of recent technological advancements and, importantly, improve health equity.

Questionnaire validation practice within a theoretical framework: a systematic descriptive literature review of health literacy assessments.

OBJECTIVE: Validity refers to the extent to which evidence and theory support the adequacy and appropriateness of inferences based on score interpretations. The health sector is lacking a theoretically-driven framework for the development, testing and use of health assessments. This study used the Standards for Educational and Psychological Testing framework of five sources of validity evidence to assess the types of evidence reported for health literacy assessments, and to identify studies that referred to a theoretical validity testing framework. METHODS: A systematic descriptive literature review investigated methods and results in health literacy assessment development, application and validity testing studies. Electronic searches were conducted in EBSCOhost, Embase, Open Access Theses and Dissertations and ProQuest Dissertations. Data were coded to the Standards' five sources of validity evidence, and for reference to a validity testing framework. RESULTS: Coding on 46 studies resulted in 195 instances of validity evidence across the five sources. Only nine studies directly or indirectly referenced a validity testing framework. Evidence based on relations to other variables is most frequently reported. CONCLUSIONS: The health and health equity of individuals and populations are increasingly dependent on decisions based on data collected through health assessments. An evidence-based theoretical framework provides structure and coherence to existing evidence and stipulates where further evidence is required to evaluate the extent to which data are valid for an intended purpose. This review demonstrates the use of the Standards' theoretical validity testing framework to evaluate sources of evidence reported for health literacy assessments. Findings indicate that theoretical validity testing frameworks are rarely used to collate and evaluate evidence in validation practice for health literacy assessments. Use of the Standards' theoretical validity testing framework would improve evaluation of the evidence for inferences derived from health assessment data on which public health and health equity decisions are based.

Evaluation of the Health Education Impact Questionnaire (heiQ), a Self-Management Skill Assessment Tool, in Italian Chronic Patients.

BACKGROUND: The Health Education Impact Questionnaire (heiQ) aims to evaluate eight self-management skills in people with chronic conditions. Knowledge about the relations between these self-management skills and different quality of life (QoL) outcomes has received little attention. It is also important to provide further evidence on its properties in non-English healthcare contexts, as the questionnaire is being used in cross-cultural research. Furthermore, in the Italian healthcare context, the relationship between the medical staff and the patients remains asymmetrical, with the latter having the role of passive recipients of medical prescriptions and services. The current study provided further evidence about the psychometric properties of the heiQ among Italian people with chronic conditions, specifically by assessing the factor structure, reliability, convergent/divergent and criterion validity (ie, the specific contribution of each of the self-management skills to QoL outcomes). METHODS: Two hundred ninety-nine individuals with a chronic condition (mean age = 61.4 years, 50% females) completed the heiQ and the Medical Outcomes Study-Short Form (MOS SF-36). Confirmatory factor analyses, Composite Reliability Indices (CRI), bivariate correlations and linear regression analyses were computed. RESULTS: A model with 8 correlated factors showed good fit, in a similar way to previous studies. CRI values were acceptable to good for all the subscales. Associations between some of the heiQ subscales and some of the MOS SF-36 subscales supported criterion validity. In particular, it was confirmed by the moderate associations between the constructive attitudes and approaches subscale and the MOS SF-36 vitality and perceived mental health and by the moderate correlations between the health directed activities subscale and the MOS SF-36 Vitality. In linear regressions, higher emotional distress predicted higher physical and mental QoL, while higher mental QoL was also associated with stronger constructive attitudes and approaches. CONCLUSION: The heiQ has robust properties across translations and it can also be used routinely in Italian healthcare contexts. The evidence that all the other self-management skills did not predict either physical or mental QoL could suggest that the self-management model of chronic diseases is still not sufficiently developed in the Italian context, where patients are passive recipients of medical instructions.

Translation method is validity evidence for construct equivalence: analysis of secondary data routinely collected during translations of the Health Literacy Questionnaire (HLQ).

BACKGROUND: Cross-cultural research with patient-reported outcomes measures (PROMs) assumes that the PROM in the target language will measure the same construct in the same way as the PROM in the source language. Yet translation methods are rarely used to qualitatively maximise construct equivalence or to describe the intents of each item to support common understanding within translation teams. This study aimed to systematically investigate the utility of the Translation Integrity Procedure (TIP), in particular the use of item intent descriptions, to maximise construct equivalence during the translation process, and to demonstrate how documented data from the TIP contributes evidence to a validity argument for construct equivalence between translated and source language PROMs. METHODS: Analysis of secondary data was conducted on routinely collected data in TIP Management Grids of translations (n = 9) of the Health Literacy Questionnaire (HLQ) that took place between August 2014 and August 2015: Arabic, Czech, French (Canada), French (France), Hindi, Indonesian, Slovak, Somali and Spanish (Argentina). Two researchers initially independently deductively coded the data to nine common types of translation errors. Round two of coding included an identified 10th code. Coded data were compared for discrepancies, and checked when needed with a third researcher for final code allocation. RESULTS: Across the nine translations, 259 changes were made to provisional forward translations and were coded into 10 types of errors. Most frequently coded errors were Complex word or phrase (n = 99), Semantic (n = 54) and Grammar (n = 27). Errors coded least frequently were Cultural errors (n = 7) and Printed errors (n = 5). CONCLUSIONS: To advance PROM validation practice, this study investigated a documented translation method that includes the careful specification of descriptions of item intents. Assumptions that translated PROMs have construct equivalence between linguistic contexts can be incorrect due to errors in translation. Of particular concern was the use of high level complex words by translators, which, if undetected, could cause flawed interpretation of data from people with low literacy. Item intent descriptions can support translations to maximise construct equivalence, and documented translation data can contribute evidence to justify score interpretation and use of translated PROMS in new linguistic contexts.

Italian validation of the health education impact questionnaire (heiQ) in people with chronic conditions.

BACKGROUND: The health education impact Questionnaire (heiQ) measures eight self-management skills in people with chronic conditions. It seems to be important to provide cross-cultural evidence on its properties in non-English healthcare contexts. The present study assessed the psychometric properties of the heiQ in Italian adults with chronic conditions. METHODS: Two hundred ninety-nine individuals with a chronic condition (mean age = 61.4 years, 50.16% females) completed the heiQ and the Medical Outcomes Study-Short Form (SF-36). Confirmatory factor analyses, Composite Reliability Indices, and bivariate correlations were performed. RESULTS: Structural validity based on 8 correlated factors with good fit was in line with previous research. Concurrent validity was confirmed, as shown by moderate associations between the scores on the Constructive attitudes and approaches, Self-monitoring and insight, Health directed activities, Social integration and support, and Emotional distress subscales and the scores on SF-36 Physical functioning, General health perceptions, Vitality, Social functioning, Perceived mental health and Role limitations due to physical and emotional problems subscales. CONCLUSIONS: The Italian heiQ has strong properties and it can be used routinely also in the Italian healthcare services.

Questionnaire validation practice: a protocol for a systematic descriptive literature review of health literacy assessments.

INTRODUCTION: Contemporary validity testing theory holds that validity lies in the extent to which a proposed interpretation and use of test scores is justified, the evidence for which is dependent on both quantitative and qualitative research methods. Despite this, we hypothesise that development and validation studies for assessments in the field of health primarily report a limited range of statistical properties, and that a systematic theoretical framework for validity testing is rarely applied. Using health literacy assessments as an exemplar, this paper outlines a protocol for a systematic descriptive literature review about types of validity evidence being reported and if the evidence is reported within a theoretical framework. METHODS AND ANALYSIS: A systematic descriptive literature review of qualitative and quantitative research will be used to investigate the scope of validation practice in the rapidly growing field of health literacy assessment. This review method employs a frequency analysis to reveal potentially interpretable patterns of phenomena in a research area; in this study, patterns in types of validity evidence reported, as assessed against the criteria of the 2014 Standards for Educational and Psychological Testing, and in the number of studies using a theoretical validity testing framework. The search process will be consistent with the Preferred Reporting Items for Systematic Reviews and Meta-analyses statement. Outcomes of the review will describe patterns in reported validity evidence, methods used to generate the evidence and theoretical frameworks underpinning validation practice and claims. This review will inform a theoretical basis for future development and validity testing of health assessments in general. ETHICS AND DISSEMINATION: Ethics approval is not required for this systematic review because only published research will be examined. Dissemination of the review findings will be through publication in a peer-reviewed journal, at conference presentations and in the lead author's doctoral thesis.

Development of the Multidimensional Readiness and Enablement Index for Health Technology (READHY) Tool to Measure Individuals' Health Technology Readiness: Initial Testing in a Cancer Rehabilitation Setting.

BACKGROUND: The increasing digitization of health care services with enhanced access to fast internet connections, along with wide use of smartphones, offers the opportunity to get health advice or treatment remotely. For service providers, it is important to consider how consumers can take full advantage of available services and how this can create an enabling environment. However, it is important to consider the digital context and the attributes of current and future users, such as their readiness (ie, knowledge, skills, and attitudes, including trust and motivation). OBJECTIVE: The objective of this study was to evaluate how the eHealth Literacy Questionnaire (eHLQ) combined with selected dimensions from the Health Education Impact Questionnaire (heiQ) and the Health Literacy Questionnaire (HLQ) can be used together as an instrument to characterize an individual's level of health technology readiness and explore how the generated data can be used to create health technology readiness profiles of potential users of health technologies and digital health services. METHODS: We administered the instrument and sociodemographic questions to a population of 305 patients with a recent cancer diagnosis referred to rehabilitation in a setting that plans to introduce various technologies to assist the individuals. We evaluated properties of the Readiness and Enablement Index for Health Technology (READHY) instrument using confirmatory factor analysis, convergent and discriminant validity analysis, and exploratory factor analysis. To identify different health technology readiness profiles in the population, we further analyzed the data using hierarchical and k-means cluster analysis. RESULTS: The confirmatory factor analysis found a suitable fit for the 13 factors with only 1 cross-loading of 1 item between 2 dimensions. The convergent and discriminant validity analysis revealed many factor correlations, suggesting that, in this population, a more parsimonious model might be achieved. Exploratory factor analysis pointed to 5 to 6 constructs based on aggregates of the existing dimensions. The results were not satisfactory, so we performed an 8-factor confirmatory factor analysis, resulting in a good fit with only 1 item cross-loading between 2 dimensions. Cluster analysis showed that data from the READHY instrument can be clustered to create meaningful health technology readiness profiles of users. CONCLUSIONS: The 13 dimensions from heiQ, HLQ, and eHLQ can be used in combination to describe a user's health technology readiness level and degree of enablement. Further studies in other populations are needed to understand whether the associations between dimensions are consistent and the number of dimensions can be reduced.

Characterisation of health literacy strengths and weaknesses among people at metabolic and cardiovascular risk: Validity testing of the Health Literacy Questionnaire.

OBJECTIVES: Health literacy refers to the ability of individuals to gain access to, use, and understand health information and services in order to maintain a good health. The assessment of health literacy profiles in a population is potentially crucial to respond to health needs. The Health Literacy Questionnaire explores nine dimensions of health literacy and has been shown to display robust psychometric properties. The aim was to test the validity of the multidimensional Health Literacy Questionnaire and to describe the health literacy profiles in a French population at risk of cardiovascular disease. METHODS: Data were collected using self-administered questionnaires from 175 participants attending health education and support programmes in local associations of patients in Paris. Analysis included scale reliability, confirmatory factor analysis, and health literacy profiles via descriptive statistics. RESULTS: In confirmatory factor analysis, the nine-factor structure was close to the original Health Literacy Questionnaire. A nine-factor confirmatory factor analysis model was fitted to the 44 items with no cross-loadings or correlated residuals allowed. Given the restricted nature of the model, the fit was satisfactory: χ2 WLSMV(866 df) = 1383.81, p = 0.0000, comparative fit index = 0.925, Tucker-Lewis index = 0.918, root mean square error of approximation = 0.058, weighted root mean square residual = 1.175. Composite reliability ranged from 0.77 to 0.91. Among the 9 scales of the Health Literacy Questionnaire, the highest scores were found for scale 1 'Feeling understood and supported by healthcare professionals' and scale 9 'Understand health information enough to know what to do' and the lowest for scale 2 'Having sufficient information to manage my health' and scale 7 'Navigating the healthcare system'. CONCLUSION: The French version of the Health Literacy Questionnaire was shown to be psychometrically robust with good reliability. In the context of France, the 9 scales of Health Literacy Questionnaire allow a thorough assessment of health literacy strengths and weaknesses to respond to health literacy needs and improve the accessibility of health information and services.

A Multidimensional Tool Based on the eHealth Literacy Framework: Development and Initial Validity Testing of the eHealth Literacy Questionnaire (eHLQ).

BACKGROUND: For people to be able to access, understand, and benefit from the increasing digitalization of health services, it is critical that services are provided in a way that meets the user's needs, resources, and competence. OBJECTIVE: The objective of the study was to develop a questionnaire that captures the 7-dimensional eHealth Literacy Framework (eHLF). METHODS: Draft items were created in parallel in English and Danish. The items were generated from 450 statements collected during the conceptual development of eHLF. In all, 57 items (7 to 9 items per scale) were generated and adjusted after cognitive testing. Items were tested in 475 people recruited from settings in which the scale was intended to be used (community and health care settings) and including people with a range of chronic conditions. Measurement properties were assessed using approaches from item response theory (IRT) and classical test theory (CTT) such as confirmatory factor analysis (CFA) and reliability using composite scale reliability (CSR); potential bias due to age and sex was evaluated using differential item functioning (DIF). RESULTS: CFA confirmed the presence of the 7 a priori dimensions of eHLF. Following item analysis, a 35-item 7-scale questionnaire was constructed, covering (1) using technology to process health information (5 items, CSR=.84), (2) understanding of health concepts and language (5 items, CSR=.75), (3) ability to actively engage with digital services (5 items, CSR=.86), (4) feel safe and in control (5 items, CSR=.87), (5) motivated to engage with digital services (5 items, CSR=.84), (6) access to digital services that work (6 items, CSR=.77), and (7) digital services that suit individual needs (4 items, CSR=.85). A 7-factor CFA model, using small-variance priors for cross-loadings and residual correlations, had a satisfactory fit (posterior productive P value: .27, 95% CI for the difference between the observed and replicated chi-square values: -63.7 to 133.8). The CFA showed that all items loaded strongly on their respective factors. The IRT analysis showed that no items were found to have disordered thresholds. For most scales, discriminant validity was acceptable; however, 2 pairs of dimensions were highly correlated; dimensions 1 and 5 (r=.95), and dimensions 6 and 7 (r=.96). All dimensions were retained because of strong content differentiation and potential causal relationships between these dimensions. There is no evidence of DIF. CONCLUSIONS: The eHealth Literacy Questionnaire (eHLQ) is a multidimensional tool based on a well-defined a priori eHLF framework with robust properties. It has satisfactory evidence of construct validity and reliable measurement across a broad range of concepts (using both CTT and IRT traditions) in various groups. It is designed to be used to understand and evaluate people's interaction with digital health services.

Application of validity theory and methodology to patient-reported outcome measures (PROMs): building an argument for validity.

BACKGROUND: Data from subjective patient-reported outcome measures (PROMs) are now being used in the health sector to make or support decisions about individuals, groups and populations. Contemporary validity theorists define validity not as a statistical property of the test but as the extent to which empirical evidence supports the interpretation of test scores for an intended use. However, validity testing theory and methodology are rarely evident in the PROM validation literature. Application of this theory and methodology would provide structure for comprehensive validation planning to support improved PROM development and sound arguments for the validity of PROM score interpretation and use in each new context. OBJECTIVE: This paper proposes the application of contemporary validity theory and methodology to PROM validity testing. ILLUSTRATIVE EXAMPLE: The validity testing principles will be applied to a hypothetical case study with a focus on the interpretation and use of scores from a translated PROM that measures health literacy (the Health Literacy Questionnaire or HLQ). DISCUSSION: Although robust psychometric properties of a PROM are a pre-condition to its use, a PROM's validity lies in the sound argument that a network of empirical evidence supports the intended interpretation and use of PROM scores for decision making in a particular context. The health sector is yet to apply contemporary theory and methodology to PROM development and validation. The theoretical and methodological processes in this paper are offered as an advancement of the theory and practice of PROM validity testing in the health sector.

Percentile ranks and benchmark estimates of change for the Health Education Impact Questionnaire: Normative data from an Australian sample.

OBJECTIVE: Participant self-report data play an essential role in the evaluation of health education activities, programmes and policies. When questionnaire items do not have a clear mapping to a performance-based continuum, percentile norms are useful for communicating individual test results to users. Similarly, when assessing programme impact, the comparison of effect sizes for group differences or baseline to follow-up change with effect sizes observed in relevant normative data provides more directly useful information compared with statistical tests of mean differences and the evaluation of effect sizes for substantive significance using universal rule-of-thumb such as those for Cohen's 'd'. This article aims to assist managers, programme staff and clinicians of healthcare organisations who use the Health Education Impact Questionnaire interpret their results using percentile norms for individual baseline and follow-up scores together with group effect sizes for change across the duration of typical chronic disease self-management and support programme. METHODS: Percentile norms for individual Health Education Impact Questionnaire scale scores and effect sizes for group change were calculated using freely available software for each of the eight Health Education Impact Questionnaire scales. Data used were archived responses of 2157 participants of chronic disease self-management programmes conducted by a wide range of organisations in Australia between July 2007 and March 2013. RESULTS: Tables of percentile norms and three possible effect size benchmarks for baseline to follow-up change are provided together with two worked examples to assist interpretation. CONCLUSION: While the norms and benchmarks presented will be particularly relevant for Australian organisations and others using the English-language version of the Health Education Impact Questionnaire, they will also be useful for translated versions as a guide to the sensitivity of the scales and the extent of the changes that might be anticipated from attendance at a typical chronic disease self-management or health education programme.

The Health Literacy Questionnaire (HLQ) at the patient-clinician interface: a qualitative study of what patients and clinicians mean by their HLQ scores.

BACKGROUND: The Health Literacy Questionnaire (HLQ) has nine scales that each measure an aspect of the multidimensional construct of health literacy. All scales have good psychometric properties. However, it is the interpretations of data within contexts that must be proven valid, not just the psychometric properties of a measurement instrument. The purpose of this study was to establish the extent of concordance and discordance between individual patient and clinician interpretations of HLQ data in the context of complex case management. METHODS: Sixteen patients with complex needs completed the HLQ and were interviewed to discuss the reasons for their answers. Also, the clinicians of each of these patients completed the HLQ about their patient, and were interviewed to discuss the reasons for their answers. Thematic analysis of HLQ scores and interview data determined the extent of concordance between patient and clinician HLQ responses, and the reasons for discordance. RESULTS: Highest concordance (80%) between patient and clinician item-response pairs was seen in Scale 1 and highest discordance (56%) was seen in Scale 6. Four themes were identified to explain discordance: 1) Technical or literal meaning of specific words; 2) Patients' changing or evolving circumstances; 3) Different expectations and criteria for assigning HLQ scores; and 4) Different perspectives about a patient's reliance on healthcare providers. CONCLUSION: This study shows that the HLQ can act as an adjunct to clinical practice to help clinicians understand a patient's health literacy challenges and strengths early in a clinical encounter. Importantly, clinicians can use the HLQ to detect differences between their own perspectives about a patient's health literacy and the patient's perspective, and to initiate discussion to explore this. Provision of training to better detect these differences may assist clinicians to provide improved care. The outcomes of this study contribute to the growing body of international validation evidence about the use of the HLQ in different contexts. More specifically, this study has shown that the HLQ has measurement veracity at the patient and clinician level and may support clinicians to understand patients' health literacy and enable a deeper engagement with healthcare services.