[Assessing the Benefit of Palliative Care in Germany - Perspective of Patients, Relatives and Professionals]. / Den Nutzen von Palliativversorgung bewerten ­ Was muss aus Patienten-, Angehörigen- und Versorgersicht beachtet werden?

BACKGROUND: The evaluation of complex interventions such as palliative care is challenging. Methods guidance such as the guidance documents of the Integrated Health Technology Assessment for the Evaluation of Complex Technologies (INTEGRATE-HTA) project help address specific challenges. The INTEGRATE-guidance was developed cooperatively by various international stakeholders and it was applied in a case study on palliative care. The presented study was part of the INTEGRATE-HTA project. ObjectivesThe objective was to identify important assessment aspects of palliative care in Germany from the perspective of patients, relatives and professional providers. METHODS: Applying the structured consensus method of the Improved Nominal Group Technique, we conducted interviews with two focus groups - one with relatives and one with professionals. Additionally, we conducted 4 interviews with patients. We used an open coding procedure to analyze the data and a dialogical approach to validate the results. RESULTS: 8 assessment aspects were found to be important. These are the definition and legitimacy of the term benefit, understanding of palliative care, patient-centered and holistic care approach, access to palliative care, continuity and flexibility of care, education of palliative care providers, and multidisciplinary approach. A central point was the need to address the intervention in its complexity and to include interactions between the different components. CONCLUSIONS: To be able to integrate various models and different perspectives of palliative care, it is important to have the involvement of different stakeholders. This also enhances the incorporation of important aspects during the development of assessment instruments.

Methods Assessing Sociocultural Aspects of Health Technologies: Results of a Literature Review.

OBJECTIVES: Social and cultural aspects are rarely assessed in health technology assessments (HTA), despite being part of most HTA definitions. One hypothesis for the reason why they are hardly considered in HTA is that we lack relevant assessment methods. Accordingly, this review aims at providing an overview of methodological approaches to address social and cultural aspects related to health technologies in HTA. METHODS: We conducted a comprehensive literature search by searching fourteen databases and a hand-search of two pertinent journals. Additionally, we sent a query to all member agencies of the International Network of Agencies for Health Technology Assessment (INAHTA) asking them for methods they use to assess social and cultural aspects. RESULTS: A total of 125 publications met our inclusion criteria. We grouped the methodological approaches into checklists for experts, literature reviews, stakeholder participatory approaches, primary data collection methods, and combinations of methodological approaches. CONCLUSIONS: There is a wide variety of methods available for assessing social and cultural aspects of health technologies, some of which have been applied in HTA. The presented overview of the different approaches and their merits can facilitate the assessment of these aspects, and improve the knowledge regarding (potential) success and failure of the implementation of a health technology.

Structured methodology review identified seven (RETREAT) criteria for selecting qualitative evidence synthesis approaches.

OBJECTIVE: To compare and contrast different methods of qualitative evidence synthesis (QES) against criteria identified from the literature and to map their attributes to inform selection of the most appropriate QES method to answer research questions addressed by qualitative research. STUDY DESIGN AND SETTING: Electronic databases, citation searching, and a study register were used to identify studies reporting QES methods. Attributes compiled from 26 methodological papers (2001-2014) were used as a framework for data extraction. Data were extracted into summary tables by one reviewer and then considered within the author team. RESULTS: We identified seven considerations determining choice of methods from the methodological literature, encapsulated within the mnemonic Review question-Epistemology-Time/Timescale-Resources-Expertise-Audience and purpose-Type of data. We mapped 15 different published QES methods against these seven criteria. The final framework focuses on stand-alone QES methods but may also hold potential when integrating quantitative and qualitative data. CONCLUSION: These findings offer a contemporary perspective as a conceptual basis for future empirical investigation of the advantages and disadvantages of different methods of QES. It is hoped that this will inform appropriate selection of QES approaches.

Towards a taxonomy of logic models in systematic reviews and health technology assessments: A priori, staged, and iterative approaches.

The complexity associated with how interventions result-or fail to result-in outcomes and how context matters is increasingly recognised. Logic models provide an important tool for handling complexity, with contrasting uses in programme evaluation and evidence synthesis. To reconcile these, we developed an approach that combines the strengths of both traditions, propose a taxonomy of logic models, and provide guidance on how to choose between approaches and types of logic models in systematic reviews and health technology assessments (HTA). The taxonomy distinguishes 3 approaches (a priori, staged, and iterative) and 2 types (systems-based and process-orientated) of logic models. An a priori logic model is specified at the start of the systematic review/HTA and remains unchanged. With a staged logic model, the reviewer prespecifies several points, at which major data inputs require a subsequent version. An iterative logic model is continuously modified throughout the systematic review/HTA process. System-based logic models describe the system, in which the interaction between participants, intervention, and context takes place; process-orientated models display the causal pathways leading from the intervention to multiple outcomes. The proposed taxonomy of logic models offers an improved understanding of the advantages and limitations of logic models across the spectrum from a priori to fully iterative approaches. Choice of logic model should be informed by scope of evidence synthesis, presence/absence of clearly defined population, intervention, comparison, outcome (PICO) elements, and feasibility considerations. Applications across distinct interventions and methodological approaches will deliver good practice case studies and offer further insights on the choice and implementation of logic modelling approaches.

TAKING PATIENT HETEROGENEITY AND PREFERENCES INTO ACCOUNT IN HEALTH TECHNOLOGY ASSESSMENTS.

OBJECTIVES: The INTEGRATE-HTA project provided methodology to evaluate complex technologies. This study provides guidance on how to retrieve and critically appraise available evidence on moderators and predictors of treatment effects and on patient preferences for treatment outcomes as a source of complexity. METHODS: Search filters for PubMed were developed by hand-searching a large volume of articles reporting on relevant aspects. Search terms were retrieved from selected papers and algorithmically combined to find the optimal combination of search terms. For the development of the appraisal checklists literature was searched in PubMed and Google Scholar together with citation chasing. For the CHecklist for the Appraisal of Moderators and Predictors (CHAMP) a Delphi procedure was used to value a set of eligible appraisal criteria retrieved from the literature. RESULTS: Search filters were developed optimized for different accuracy measures. The final version of CHAMP consists of a seventeen questions covering the design, analysis, results and transferability of results of moderator and predictor analysis. The final checklist for appraisal of literature on patient preferences for treatment outcomes consist of six questions meant to help the user to identify relevant quality issues together with a guidance toward existing tools concerning the appraisal of specific preference elicitation methods. CONCLUSIONS: Incorporating knowledge on subgroups for whom a specific treatment will produce more benefit holds the promise of better targeting and, ultimately, enhancing overall effectiveness and efficiency of healthcare technology. Finally, incorporating information on preferences for treatment outcomes will foster health technology assessment that addresses outcomes that are important to patients.

STAKEHOLDER INVOLVEMENT THROUGHOUT HEALTH TECHNOLOGY ASSESSMENT: AN EXAMPLE FROM PALLIATIVE CARE.

OBJECTIVES: Internationally, funders require stakeholder involvement throughout health technology assessment (HTA). We report successes, challenges, and lessons learned from extensive stakeholder involvement throughout a palliative care case study that demonstrates new concepts and methods for HTA. METHODS: A 5-step "INTEGRATE-HTA Model" developed within the INTEGRATE-HTA project guided the case study. Using convenience or purposive sampling or directly / indirectly identifying and approaching individuals / groups, stakeholders participated in qualitative research or consultation meetings. During scoping, 132 stakeholders, aged ≥ 18 years in seven countries (England, Italy, Germany, The Netherlands, Norway, Lithuania, and Poland), highlighted key issues in palliative care that assisted identification of the intervention and comparator. Subsequently stakeholders in four countries participated in face-face, telephone and / or video Skype meetings to inform evidence collection and / or review assessment results. An applicability assessment to identify contextual and implementation barriers and enablers for the case study findings involved twelve professionals in the three countries. Finally, thirteen stakeholders participated in a mock decision-making meeting in England. RESULTS: Views about the best methods of stakeholder involvement vary internationally. Stakeholders make valuable contributions in all stages of HTA; assisting decision making about interventions, comparators, research questions; providing evidence and insights into findings, gap analyses and applicability assessments. Key challenges exist regarding inclusivity, time, and resource use. CONCLUSION: Stakeholder involvement is feasible and worthwhile throughout HTA, sometimes providing unique insights. Various methods can be used to include stakeholders, although challenges exist. Recognition of stakeholder expertise and further guidance about stakeholder consultation methods is needed.

COMPREHENSIVE ASSESSMENT OF COMPLEX TECHNOLOGIES: INTEGRATING VARIOUS ASPECTS IN HEALTH TECHNOLOGY ASSESSMENT.

OBJECTIVES: Despite recent development of health technology assessment (HTA) methods, there are still methodological gaps for the assessment of complex health technologies. The INTEGRATE-HTA guidance for effectiveness, economic, ethical, socio-cultural, and legal aspects, deals with challenges when assessing complex technologies, such as heterogeneous study designs, multiple stakeholder perspectives, and unpredictable outcomes. The objective of this article is to outline this guidance and describe the added value of integrating these assessment aspects. METHODS: Different methods were used to develop the various parts of the guidance, but all draw on existing, published knowledge and were supported by stakeholder involvement. The guidance was modified after application in a case study and in response to feedback from internal and external reviewers. RESULTS: The guidance consists of five parts, addressing five core aspects of HTA, all presenting stepwise approaches based on the assessment of complexity, context, and stakeholder involvement. The guidance on effectiveness, health economics and ethics aspects focus on helping users choose appropriate, or further develop, existing methods. The recommendations are based on existing methods' applicability for dealing with problems arising with complex interventions. The guidance offers new frameworks to identify socio-cultural and legal issues, along with overviews of relevant methods and sources. CONCLUSIONS: The INTEGRATE-HTA guidance outlines a wide range of methods and facilitates appropriate choices among them. The guidance enables understanding of how complexity matters for HTA and brings together assessments from disciplines, such as epidemiology, economics, ethics, law, and social theory. This indicates relevance for a broad range of technologies.

AN INTEGRATED PERSPECTIVE ON THE ASSESSMENT OF TECHNOLOGIES: INTEGRATE-HTA.

OBJECTIVES: Current health technology assessment (HTA) is not well equipped to assess complex technologies as insufficient attention is being paid to the diversity in patient characteristics and preferences, context, and implementation. Strategies to integrate these and several other aspects, such as ethical considerations, in a comprehensive assessment are missing. The aim of the European research project INTEGRATE-HTA was to develop a model for an integrated HTA of complex technologies. METHODS: A multi-method, four-stage approach guided the development of the INTEGRATE-HTA Model: (i) definition of the different dimensions of information to be integrated, (ii) literature review of existing methods for integration, (iii) adjustment of concepts and methods for assessing distinct aspects of complex technologies in the frame of an integrated process, and (iv) application of the model in a case study and subsequent revisions. RESULTS: The INTEGRATE-HTA Model consists of five steps, each involving stakeholders: (i) definition of the technology and the objective of the HTA; (ii) development of a logic model to provide a structured overview of the technology and the system in which it is embedded; (iii) evidence assessment on effectiveness, economic, ethical, legal, and socio-cultural aspects, taking variability of participants, context, implementation issues, and their interactions into account; (iv) populating the logic model with the data generated in step 3; (v) structured process of decision-making. CONCLUSIONS: The INTEGRATE-HTA Model provides a structured process for integrated HTAs of complex technologies. Stakeholder involvement in all steps is essential as a means of ensuring relevance and meaningful interpretation of the evidence.

Making sense of complexity in context and implementation: the Context and Implementation of Complex Interventions (CICI) framework.

BACKGROUND: The effectiveness of complex interventions, as well as their success in reaching relevant populations, is critically influenced by their implementation in a given context. Current conceptual frameworks often fail to address context and implementation in an integrated way and, where addressed, they tend to focus on organisational context and are mostly concerned with specific health fields. Our objective was to develop a framework to facilitate the structured and comprehensive conceptualisation and assessment of context and implementation of complex interventions. METHODS: The Context and Implementation of Complex Interventions (CICI) framework was developed in an iterative manner and underwent extensive application. An initial framework based on a scoping review was tested in rapid assessments, revealing inconsistencies with respect to the underlying concepts. Thus, pragmatic utility concept analysis was undertaken to advance the concepts of context and implementation. Based on these findings, the framework was revised and applied in several systematic reviews, one health technology assessment (HTA) and one applicability assessment of very different complex interventions. Lessons learnt from these applications and from peer review were incorporated, resulting in the CICI framework. RESULTS: The CICI framework comprises three dimensions-context, implementation and setting-which interact with one another and with the intervention dimension. Context comprises seven domains (i.e., geographical, epidemiological, socio-cultural, socio-economic, ethical, legal, political); implementation consists of five domains (i.e., implementation theory, process, strategies, agents and outcomes); setting refers to the specific physical location, in which the intervention is put into practise. The intervention and the way it is implemented in a given setting and context can occur on a micro, meso and macro level. Tools to operationalise the framework comprise a checklist, data extraction tools for qualitative and quantitative reviews and a consultation guide for applicability assessments. CONCLUSIONS: The CICI framework addresses and graphically presents context, implementation and setting in an integrated way. It aims at simplifying and structuring complexity in order to advance our understanding of whether and how interventions work. The framework can be applied in systematic reviews and HTA as well as primary research and facilitate communication among teams of researchers and with various stakeholders.

Lay and professional stakeholder involvement in scoping palliative care issues: Methods used in seven European countries.

BACKGROUND: Stakeholders are people with an interest in a topic. Internationally, stakeholder involvement in palliative care research and health technology assessment requires development. Stakeholder involvement adds value throughout research (from prioritising topics to disseminating findings). Philosophies and understandings about the best ways to involve stakeholders in research differ internationally. Stakeholder involvement took place in seven countries (England, Germany, Italy, Lithuania, the Netherlands, Norway and Poland). Findings informed a project that developed concepts and methods for health technology assessment and applied these to evaluate models of palliative care service delivery. AIMS: To report on stakeholder involvement in the INTEGRATE-HTA project and how issues identified informed project development. DESIGN: Using stakeholder consultation or a qualitative research design, as appropriate locally, stakeholders in seven countries acted as 'advisors' to aid researchers' decision making. Thematic analysis was used to identify key issues across countries. SETTING/PARTICIPANTS: A total of 132 stakeholders (82 professionals and 50 'lay' people) aged ⩾18 participated in individual face-to-face or telephone interviews, consultation meetings or focus groups. RESULTS: Different stakeholder involvement methods were used successfully to identify key issues in palliative care. A total of 23 issues common to three or more countries informed decisions about the intervention and comparator of interest, sub questions and specific assessments within the health technology assessment. CONCLUSION: Stakeholders, including patients and families undergoing palliative care, can inform project decision making using various involvement methods according to the local context. Researchers should consider local understandings about stakeholder involvement as views of appropriate and feasible methods vary. Methods for stakeholder involvement, especially consultation, need further development.

The development of PubMed search strategies for patient preferences for treatment outcomes.

BACKGROUND: The importance of respecting patients' preferences when making treatment decisions is increasingly recognized. Efficiently retrieving papers from the scientific literature reporting on the presence and nature of such preferences can help to achieve this goal. The objective of this study was to create a search filter for PubMed to help retrieve evidence on patient preferences for treatment outcomes. METHODS: A total of 27 journals were hand-searched for articles on patient preferences for treatment outcomes published in 2011. Selected articles served as a reference set. To develop optimal search strategies to retrieve this set, all articles in the reference set were randomly split into a development and a validation set. MeSH-terms and keywords retrieved using PubReMiner were tested individually and as combinations in PubMed and evaluated for retrieval performance (e.g. sensitivity (Se) and specificity (Sp)). RESULTS: Of 8238 articles, 22 were considered to report empirical evidence on patient preferences for specific treatment outcomes. The best search filters reached Se of 100 % [95 % CI 100-100] with Sp of 95 % [94-95 %] and Sp of 97 % [97-98 %] with 75 % Se [74-76 %]. In the validation set these queries reached values of Se of 90 % [89-91 %] with Sp 94 % [93-95 %] and Se of 80 % [79-81 %] with Sp of 97 % [96-96 %], respectively. CONCLUSIONS: Narrow and broad search queries were developed which can help in retrieving literature on patient preferences for treatment outcomes. Identifying such evidence may in turn enhance the incorporation of patient preferences in clinical decision making and health technology assessment.

Ethical analysis in HTA of complex health interventions.

BACKGROUND: In the field of health technology assessment (HTA), there are several approaches that can be used for ethical analysis. However, there is a scarcity of literature that critically evaluates and compares the strength and weaknesses of these approaches when they are applied in practice. In this paper, we analyse the applicability of some selected approaches for addressing ethical issues in HTA in the field of complex health interventions. Complex health interventions have been the focus of methodological attention in HTA. However, the potential methodological challenges for ethical analysis are as yet unknown. METHODS: Six of the most frequently described and applied ethical approaches in HTA were critically assessed against a set of five characteristics of complex health interventions: multiple and changing perspectives, indeterminate phenomena, uncertain causality, unpredictable outcomes, and ethical complexity. The assessments are based on literature and the authors' experiences of developing, applying and assessing the approaches. RESULTS: The Interactive, participatory HTA approach is by its nature and flexibility, applicable across most complexity characteristics. Wide Reflective Equilibrium is also flexible and its openness to different perspectives makes it better suited for complex health interventions than more rigid conventional approaches, such as Principlism and Casuistry. Approaches developed for HTA purposes are fairly applicable for complex health interventions, which one could expect because they include various ethical perspectives, such as the HTA Core Model® and the Socratic approach. CONCLUSION: This study shows how the applicability for addressing ethical issues in HTA of complex health interventions differs between the selected ethical approaches. Knowledge about these differences may be helpful when choosing and applying an approach for ethical analyses in HTA. We believe that the study contributes to increasing awareness and interest of the ethical aspects of complex health interventions in general.

Context and implementation: A concept analysis towards conceptual maturity.

Context and implementation of health interventions have received increasing attention over the past decade, in particular with respect to their influence on the effectiveness and reach of complex interventions. The underlying concepts are both considered partially mature, limiting their operationalization in research and practice. We conducted systematic literature searches and pragmatic utility (PU) concept analyses to provide a state-of-the-art assessment of the concepts of "context" and "implementation" in the health sciences to create a common understanding for their use within systematic reviews and HTA. We performed two separate searches, one for context (EMBASE, MEDLINE) and the other for implementation (Google Scholar) to identify relevant models, theories and frameworks. 17 publications on context and 35 articles on implementation met our inclusion criteria. PU concept analysis comprises three guiding principles: selection of the literature, organization and structuring of the literature, and asking analytic questions of the literature. Both concepts were analyzed according to four features of conceptual maturity, i.e., consensual definitions, clear characteristics, fully described preconditions and outcomes, and delineated boundaries. Context and implementation are highly intertwined, with both concepts influencing and interacting with each other. Context is defined as a set of characteristics and circumstances that surround the implementation effort. Implementation is conceptualized as a planned and deliberately initiated effort with the intention to put an intervention into practice. The concept of implementation presents largely consensual definitions and relatively well-defined boundaries, while distinguishing features, preconditions and outcomes are not yet fully articulated. In contrast, definitions of context vary widely, and boundaries with neighbouring concepts, such as setting and environment, are blurred; characteristics, preconditions and outcomes are ill-defined. Therefore, the maturity of both concepts should be further improved to facilitate operationalization in systematic reviews and HTAs.