Who can you trust? A review of free online sources of "trustworthy" information about treatment effects for patients and the public.

BACKGROUND: Information about effects of treatments based on unsystematic reviews of research evidence may be misleading. However, finding trustworthy information about the effects of treatments based on systematic reviews, which is accessible to patients and the public can be difficult. The objectives of this study were to identify and evaluate free sources of health information for patients and the public that provide information about effects of treatments based on systematic reviews. METHODS: We reviewed websites that we and our colleagues knew of, searched for government sponsored health information websites, and searched for online sources of health information that provide evidence-based information. To be included in our review, a website had to be available in English, freely accessible, and intended for patients and the public. In addition, it had to have a broad scope, not limited to specific conditions or types of treatments. It had to include a description of how the information is prepared and the description had to include a statement about using systematic reviews. We compared the included websites by searching for information about the effects of eight treatments. RESULTS: Three websites met our inclusion criteria: Cochrane Evidence, Informed Health, and PubMed Health. The first two websites produce content, whereas PubMed Health aggregated content. A fourth website that met our inclusion criteria, CureFacts, was under development. Cochrane Evidence provides plain language summaries of Cochrane Reviews (i.e. summaries that are intended for patients and the public). They are translated to several other languages. No information besides treatment effects is provided. Informed Health provides information about treatment effects together with other information for a wide range of topics. PubMed Health was discontinued in October 2018. It included a large number of systematic reviews of treatment effects with plain language summaries for Cochrane Reviews and some other reviews. None of the three websites included links to ongoing trials, and information about treatment effects was not reported consistently on any of the websites. CONCLUSION: It is possible for patients and the public to access trustworthy information about the effects of treatments using the two of the websites included in this review.

A comparative evaluation of PDQ-Evidence.

BACKGROUND: A strategy for minimising the time and obstacles to accessing systematic reviews of health system evidence is to collect them in a freely available database and make them easy to find through a simple 'Google-style' search interface. PDQ-Evidence was developed in this way. The objective of this study was to compare PDQ-Evidence to six other databases, namely Cochrane Library, EVIPNet VHL, Google Scholar, Health Systems Evidence, PubMed and Trip. METHODS: We recruited healthcare policy-makers, managers and health researchers in low-, middle- and high-income countries. Participants selected one of six pre-determined questions. They searched for a systematic review that addressed the chosen question and one question of their own in PDQ-Evidence and in two of the other six databases which they would normally have searched. We randomly allocated participants to search PDQ-Evidence first or to search the two other databases first. The primary outcomes were whether a systematic review was found and the time taken to find it. Secondary outcomes were perceived ease of use and perceived time spent searching. We asked open-ended questions about PDQ-Evidence, including likes, dislikes, challenges and suggestions for improvements. RESULTS: A total of 89 people from 21 countries completed the study; 83 were included in the primary analyses and 6 were excluded because of data errors that could not be corrected. Most participants chose PubMed and Cochrane Library as the other two databases. Participants were more likely to find a systematic review using PDQ-Evidence than using Cochrane Library or PubMed for the pre-defined questions. For their own questions, this difference was not found. Overall, it took slightly less time to find a systematic review using PDQ-Evidence. Participants perceived that it took less time, and most participants perceived PDQ-Evidence to be slightly easier to use than the two other databases. However, there were conflicting views about the design of PDQ-Evidence. CONCLUSIONS: PDQ-Evidence is at least as efficient as other databases for finding health system evidence. However, using PDQ-Evidence is not intuitive for some people. TRIAL REGISTRATION: The trial was prospectively registered in the ISRCTN registry 17 April 2015. Registration number: ISRCTN12742235 .

Governance arrangements for health systems in low-income countries: an overview of systematic reviews.

BACKGROUND: Governance arrangements include changes in rules or processes that determine authority and accountability for health policies, organisations, commercial products and health professionals, as well as the involvement of stakeholders in decision-making. Changes in governance arrangements can affect health and related goals in numerous ways, generally through changes in authority, accountability, openness, participation and coherence. A broad overview of the findings of systematic reviews can help policymakers, their technical support staff and other stakeholders to identify strategies for addressing problems and improving the governance of their health systems. OBJECTIVES: To provide an overview of the available evidence from up-to-date systematic reviews about the effects of governance arrangements for health systems in low-income countries. Secondary objectives include identifying needs and priorities for future evaluations and systematic reviews on governance arrangements and informing refinements of the framework for governance arrangements outlined in the overview. METHODS: We searched Health Systems Evidence in November 2010 and PDQ Evidence up to 17 December 2016 for systematic reviews. We did not apply any date, language or publication status limitations in the searches. We included well-conducted systematic reviews of studies that assessed the effects of governance arrangements on patient outcomes (health and health behaviours), the quality or utilisation of healthcare services, resource use (health expenditures, healthcare provider costs, out-of-pocket payments, cost-effectiveness), healthcare provider outcomes (such as sick leave), or social outcomes (such as poverty, employment) and that were published after April 2005. We excluded reviews with limitations that were important enough to compromise the reliability of the findings of the review. Two overview authors independently screened reviews, extracted data and assessed the certainty of evidence using GRADE. We prepared SUPPORT Summaries for eligible reviews, including key messages, 'Summary of findings' tables (using GRADE to assess the certainty of the evidence) and assessments of the relevance of findings to low-income countries. MAIN RESULTS: We identified 7272 systematic reviews and included 21 of them in this overview (19 primary reviews and 2 supplementary reviews). We focus here on the results of the 19 primary reviews, one of which had important methodological limitations. The other 18 were reliable (with only minor limitations).We grouped the governance arrangements addressed in the reviews into five categories: authority and accountability for health policies (three reviews); authority and accountability for organisations (two reviews); authority and accountability for commercial products (three reviews); authority and accountability for health professionals (seven reviews); and stakeholder involvement (four reviews).Overall, we found desirable effects for the following interventions on at least one outcome, with moderate- or high-certainty evidence and no moderate- or high-certainty evidence of undesirable effects. Decision-making about what is covered by health insurance- Placing restrictions on the medicines reimbursed by health insurance systems probably decreases the use of and spending on these medicines (moderate-certainty evidence). Stakeholder participation in policy and organisational decisions- Participatory learning and action groups for women probably improve newborn survival (moderate-certainty evidence).- Consumer involvement in preparing patient information probably improves the quality of the information and patient knowledge (moderate-certainty evidence). Disclosing performance information to patients and the public- Disclosing performance data on hospital quality to the public probably encourages hospitals to implement quality improvement activities (moderate-certainty evidence).- Disclosing performance data on individual healthcare providers to the public probably leads people to select providers that have better quality ratings (moderate-certainty evidence). AUTHORS' CONCLUSIONS: Investigators have evaluated a wide range of governance arrangements that are relevant for low-income countries using sound systematic review methods. These strategies have been targeted at different levels in health systems, and studies have assessed a range of outcomes. Moderate-certainty evidence shows desirable effects (with no undesirable effects) for some interventions. However, there are important gaps in the availability of systematic reviews and primary studies for the all of the main categories of governance arrangements.

Implementation strategies for health systems in low-income countries: an overview of systematic reviews.

BACKGROUND: A key function of health systems is implementing interventions to improve health, but coverage of essential health interventions remains low in low-income countries. Implementing interventions can be challenging, particularly if it entails complex changes in clinical routines; in collaborative patterns among different healthcare providers and disciplines; in the behaviour of providers, patients or other stakeholders; or in the organisation of care. Decision-makers may use a range of strategies to implement health interventions, and these choices should be based on evidence of the strategies' effectiveness. OBJECTIVES: To provide an overview of the available evidence from up-to-date systematic reviews about the effects of implementation strategies for health systems in low-income countries. Secondary objectives include identifying needs and priorities for future evaluations and systematic reviews on alternative implementation strategies and informing refinements of the framework for implementation strategies presented in the overview. METHODS: We searched Health Systems Evidence in November 2010 and PDQ-Evidence up to December 2016 for systematic reviews. We did not apply any date, language or publication status limitations in the searches. We included well-conducted systematic reviews of studies that assessed the effects of implementation strategies on professional practice and patient outcomes and that were published after April 2005. We excluded reviews with limitations important enough to compromise the reliability of the review findings. Two overview authors independently screened reviews, extracted data and assessed the certainty of evidence using GRADE. We prepared SUPPORT Summaries for eligible reviews, including key messages, 'Summary of findings' tables (using GRADE to assess the certainty of the evidence) and assessments of the relevance of findings to low-income countries. MAIN RESULTS: We identified 7272 systematic reviews and included 39 of them in this overview. An additional four reviews provided supplementary information. Of the 39 reviews, 32 had only minor limitations and 7 had important methodological limitations. Most studies in the reviews were from high-income countries. There were no studies from low-income countries in eight reviews.Implementation strategies addressed in the reviews were grouped into four categories - strategies targeting:1. healthcare organisations (e.g. strategies to change organisational culture; 1 review);2. healthcare workers by type of intervention (e.g. printed educational materials; 14 reviews);3. healthcare workers to address a specific problem (e.g. unnecessary antibiotic prescription; 9 reviews);4. healthcare recipients (e.g. medication adherence; 15 reviews).Overall, we found the following interventions to have desirable effects on at least one outcome with moderate- or high-certainty evidence and no moderate- or high-certainty evidence of undesirable effects.1.Strategies targeted at healthcare workers: educational meetings, nutrition training of health workers, educational outreach, practice facilitation, local opinion leaders, audit and feedback, and tailored interventions.2.Strategies targeted at healthcare workers for specific types of problems: training healthcare workers to be more patient-centred in clinical consultations, use of birth kits, strategies such as clinician education and patient education to reduce antibiotic prescribing in ambulatory care settings, and in-service neonatal emergency care training.3. Strategies targeted at healthcare recipients: mass media interventions to increase uptake of HIV testing; intensive self-management and adherence, intensive disease management programmes to improve health literacy; behavioural interventions and mobile phone text messages for adherence to antiretroviral therapy; a one time incentive to start or continue tuberculosis prophylaxis; default reminders for patients being treated for active tuberculosis; use of sectioned polythene bags for adherence to malaria medication; community-based health education, and reminders and recall strategies to increase vaccination uptake; interventions to increase uptake of cervical screening (invitations, education, counselling, access to health promotion nurse and intensive recruitment); health insurance information and application support. AUTHORS' CONCLUSIONS: Reliable systematic reviews have evaluated a wide range of strategies for implementing evidence-based interventions in low-income countries. Most of the available evidence is focused on strategies targeted at healthcare workers and healthcare recipients and relates to process-based outcomes. Evidence of the effects of strategies targeting healthcare organisations is scarce.

Delivery arrangements for health systems in low-income countries: an overview of systematic reviews.

BACKGROUND: Delivery arrangements include changes in who receives care and when, who provides care, the working conditions of those who provide care, coordination of care amongst different providers, where care is provided, the use of information and communication technology to deliver care, and quality and safety systems. How services are delivered can have impacts on the effectiveness, efficiency and equity of health systems. This broad overview of the findings of systematic reviews can help policymakers and other stakeholders identify strategies for addressing problems and improve the delivery of services. OBJECTIVES: To provide an overview of the available evidence from up-to-date systematic reviews about the effects of delivery arrangements for health systems in low-income countries. Secondary objectives include identifying needs and priorities for future evaluations and systematic reviews on delivery arrangements and informing refinements of the framework for delivery arrangements outlined in the review. METHODS: We searched Health Systems Evidence in November 2010 and PDQ-Evidence up to 17 December 2016 for systematic reviews. We did not apply any date, language or publication status limitations in the searches. We included well-conducted systematic reviews of studies that assessed the effects of delivery arrangements on patient outcomes (health and health behaviours), the quality or utilisation of healthcare services, resource use, healthcare provider outcomes (such as sick leave), or social outcomes (such as poverty or employment) and that were published after April 2005. We excluded reviews with limitations important enough to compromise the reliability of the findings. Two overview authors independently screened reviews, extracted data, and assessed the certainty of evidence using GRADE. We prepared SUPPORT Summaries for eligible reviews, including key messages, 'Summary of findings' tables (using GRADE to assess the certainty of the evidence), and assessments of the relevance of findings to low-income countries. MAIN RESULTS: We identified 7272 systematic reviews and included 51 of them in this overview. We judged 6 of the 51 reviews to have important methodological limitations and the other 45 to have only minor limitations. We grouped delivery arrangements into eight categories. Some reviews provided more than one comparison and were in more than one category. Across these categories, the following intervention were effective; that is, they have desirable effects on at least one outcome with moderate- or high-certainty evidence and no moderate- or high-certainty evidence of undesirable effects. Who receives care and when: queuing strategies and antenatal care to groups of mothers. Who provides care: lay health workers for caring for people with hypertension, lay health workers to deliver care for mothers and children or infectious diseases, lay health workers to deliver community-based neonatal care packages, midlevel health professionals for abortion care, social support to pregnant women at risk, midwife-led care for childbearing women, non-specialist providers in mental health and neurology, and physician-nurse substitution. Coordination of care: hospital clinical pathways, case management for people living with HIV and AIDS, interactive communication between primary care doctors and specialists, hospital discharge planning, adding a service to an existing service and integrating delivery models, referral from primary to secondary care, physician-led versus nurse-led triage in emergency departments, and team midwifery. Where care is provided: high-volume institutions, home-based care (with or without multidisciplinary team) for people living with HIV and AIDS, home-based management of malaria, home care for children with acute physical conditions, community-based interventions for childhood diarrhoea and pneumonia, out-of-facility HIV and reproductive health services for youth, and decentralised HIV care. Information and communication technology: mobile phone messaging for patients with long-term illnesses, mobile phone messaging reminders for attendance at healthcare appointments, mobile phone messaging to promote adherence to antiretroviral therapy, women carrying their own case notes in pregnancy, interventions to improve childhood vaccination. Quality and safety systems: decision support with clinical information systems for people living with HIV/AIDS. Complex interventions (cutting across delivery categories and other health system arrangements): emergency obstetric referral interventions. AUTHORS' CONCLUSIONS: A wide range of strategies have been evaluated for improving delivery arrangements in low-income countries, using sound systematic review methods in both Cochrane and non-Cochrane reviews. These reviews have assessed a range of outcomes. Most of the available evidence focuses on who provides care, where care is provided and coordination of care. For all the main categories of delivery arrangements, we identified gaps in primary research related to uncertainty about the applicability of the evidence to low-income countries, low- or very low-certainty evidence or a lack of studies.

Financial arrangements for health systems in low-income countries: an overview of systematic reviews.

BACKGROUND: One target of the Sustainable Development Goals is to achieve "universal health coverage, including financial risk protection, access to quality essential health-care services and access to safe, effective, quality and affordable essential medicines and vaccines for all". A fundamental concern of governments in striving for this goal is how to finance such a health system. This concern is very relevant for low-income countries. OBJECTIVES: To provide an overview of the evidence from up-to-date systematic reviews about the effects of financial arrangements for health systems in low-income countries. Secondary objectives include identifying needs and priorities for future evaluations and systematic reviews on financial arrangements, and informing refinements in the framework for financial arrangements presented in the overview. METHODS: We searched Health Systems Evidence in November 2010 and PDQ-Evidence up to 17 December 2016 for systematic reviews. We did not apply any date, language, or publication status limitations in the searches. We included well-conducted systematic reviews of studies that assessed the effects of financial arrangements on patient outcomes (health and health behaviours), the quality or utilisation of healthcare services, resource use, healthcare provider outcomes (such as sick leave), or social outcomes (such as poverty, employment, or financial burden of patients, e.g. out-of-pocket payment, catastrophic disease expenditure) and that were published after April 2005. We excluded reviews with limitations important enough to compromise the reliability of the findings. Two overview authors independently screened reviews, extracted data, and assessed the certainty of evidence using GRADE. We prepared SUPPORT Summaries for eligible reviews, including key messages, 'Summary of findings' tables (using GRADE to assess the certainty of the evidence), and assessments of the relevance of findings to low-income countries. MAIN RESULTS: We identified 7272 reviews and included 15 in this overview, on: collection of funds (2 reviews), insurance schemes (1 review), purchasing of services (1 review), recipient incentives (6 reviews), and provider incentives (5 reviews). The reviews were published between 2008 and 2015; focused on 13 subcategories; and reported results from 276 studies: 115 (42%) randomised trials, 11 (4%) non-randomised trials, 23 (8%) controlled before-after studies, 51 (19%) interrupted time series, 9 (3%) repeated measures, and 67 (24%) other non-randomised studies. Forty-three per cent (119/276) of the studies included in the reviews took place in low- and middle-income countries. Collection of funds: the effects of changes in user fees on utilisation and equity are uncertain (very low-certainty evidence). It is also uncertain whether aid delivered under the Paris Principles (ownership, alignment, harmonisation, managing for results, and mutual accountability) improves health outcomes compared to aid delivered without conforming to those principles (very low-certainty evidence). Insurance schemes: community-based health insurance may increase service utilisation (low-certainty evidence), but the effects on health outcomes are uncertain (very low-certainty evidence). It is uncertain whether social health insurance improves utilisation of health services or health outcomes (very low-certainty evidence). Purchasing of services: it is uncertain whether increasing salaries of public sector healthcare workers improves the quantity or quality of their work (very low-certainty evidence). Recipient incentives: recipient incentives may improve adherence to long-term treatments (low-certainty evidence), but it is uncertain whether they improve patient outcomes. One-time recipient incentives probably improve patient return for start or continuation of treatment (moderate-certainty evidence) and may improve return for tuberculosis test readings (low-certainty evidence). However, incentives may not improve completion of tuberculosis prophylaxis, and it is uncertain whether they improve completion of treatment for active tuberculosis. Conditional cash transfer programmes probably lead to an increase in service utilisation (moderate-certainty evidence), but their effects on health outcomes are uncertain. Vouchers may improve health service utilisation (low-certainty evidence), but the effects on health outcomes are uncertain (very low-certainty evidence). Introducing a restrictive cap may decrease use of medicines for symptomatic conditions and overall use of medicines, may decrease insurers' expenditures on medicines (low-certainty evidence), and has uncertain effects on emergency department use, hospitalisations, and use of outpatient care (very low-certainty evidence). Reference pricing, maximum pricing, and index pricing for drugs have mixed effects on drug expenditures by patients and insurers as well as the use of brand and generic drugs. Provider incentives: the effects of provider incentives are uncertain (very low-certainty evidence), including: the effects of provider incentives on the quality of care provided by primary care physicians or outpatient referrals from primary to secondary care, incentives for recruiting and retaining health professionals to serve in remote areas, and the effects of pay-for-performance on provider performance, the utilisation of services, patient outcomes, or resource use in low-income countries. AUTHORS' CONCLUSIONS: Research based on sound systematic review methods has evaluated numerous financial arrangements relevant to low-income countries, targeting different levels of the health systems and assessing diverse outcomes. However, included reviews rarely reported social outcomes, resource use, equity impacts, or undesirable effects. We also identified gaps in primary research because of uncertainty about applicability of the evidence to low-income countries. Financial arrangements for which the effects are uncertain include external funding (aid), caps and co-payments, pay-for-performance, and provider incentives. Further studies evaluating the effects of these arrangements are needed in low-income countries. Systematic reviews should include all outcomes that are relevant to decision-makers and to people affected by changes in financial arrangements.

Risks to patient safety associated with implementation of electronic applications for medication management in ambulatory care--a systematic review.

BACKGROUND: The objective was to find evidence to substantiate assertions that electronic applications for medication management in ambulatory care (electronic prescribing, clinical decision support (CDSS), electronic health record, and computer generated paper prescriptions), while intended to reduce prescribing errors, can themselves result in errors that might harm patients or increase risks to patient safety. METHODS: Because a scoping search for adverse events in randomized controlled trials (RCTs) yielded few relevant results, we systematically searched nine databases, including MEDLINE, EMBASE, and The Cochrane Database of Systematic Reviews for systematic reviews and studies of a wide variety of designs that reported on implementation of the interventions. Studies that had safety and adverse events as outcomes, monitored for them, reported anecdotally adverse events or other events that might indicate a threat to patient safety were included. RESULTS: We found no systematic reviews that examined adverse events or patient harm caused by organizational interventions. Of the 4056 titles and abstracts screened, 176 full-text articles were assessed for inclusion. Sixty-one studies with appropriate interventions, settings and participants but without patient safety, adverse event outcomes or monitoring for risks were excluded, along with 77 other non-eligible studies. Eighteen randomized controlled trials (RCTs), 5 non-randomized controlled trials (non-R,CTs) and 15 observational studies were included. The most common electronic intervention studied was CDSS and the most frequent clinical area was cardio-vascular, including anti-coagulants. No RCTS or non-R,CTS reported adverse event. Adverse events reported in observational studies occurred less frequently after implementation of CDSS. One RCT and one observational study reported an increase in problematic prescriptions with electronic prescribing CONCLUSIONS: The safety implications of electronic medication management in ambulatory care have not been established with results from studies included in this systematic review. Only a minority of studies that investigated these interventions included threats to patients' safety as outcomes or monitored for adverse events. It is therefore not surprising that we found little evidence to substantiate fears of new risks to patient safety with their implementation. More research is needed to focus on the draw-backs and negative outcomes that implementation of these interventions might introduce.

Parental perceptions of clinical research in the pediatric emergency department.

BACKGROUND: There is a paucity of information about parental perceptions of clinical research in children, particularly in the emergency department (ED) setting. METHODS: Parents accompanying their child to the ED completed a self-administered survey gauging perceptions of research and willingness to enroll a child in a clinical research study. Factor analysis was used to correlate survey responses into domains representing parents' feeling about participation in a research study. Logistic regression was used to assess the predictors of caregivers' amenability to research participation for their child. RESULTS: Three hundred eighty-eight parents were enrolled. Most subjects were willing to enroll their child in a study involving follow-up after ED care (87%) and collection of a urine or saliva sample (79% and 81%, respectively) and extant blood (69%). Fewer were amenable to studies that involve an investigational medication (26%) or additional phlebotomy (27%). Overall, more than 90% of parents felt that research was needed to help other children and was conducted in a way that is morally right, and 25% felt that research may compromise their child's confidentiality. Factor analyses yielded 3 factors that accounted for the variance across the survey questions. Patient and parent demographics, including the patient's triage acuity level, were not associated with willingness to participate in research. CONCLUSIONS: Most parents are amenable to having their child participate in a research study in the ED setting. Most parents share a sense of altruism that research is needed to help children, and this belief is predictive of willingness to participate in a research study.

Evidence-informed health policy 1 - synthesis of findings from a multi-method study of organizations that support the use of research evidence.

BACKGROUND: Organizations have been established in many countries and internationally to support the use of research evidence by producing clinical practice guidelines, undertaking health technology assessments, and/or directly supporting the use of research evidence in developing health policy on an international, national, and state or provincial level. Learning from these organizations can reduce the need to 'reinvent the wheel' and inform decisions about how best to organize support for such organizations, particularly in low- and middle-income countries (LMICs). METHODS: We undertook a multi-method study in three phases - a survey, interviews, and case descriptions that drew on site visits - and in each of the second and third phases we focused on a purposive sample of those involved in the previous phase. We used the seven main recommendations that emerged from the advice offered in the interviews to organize much of the synthesis of findings across phases and methods. We used a constant comparative method to identify themes from across phases and methods. RESULTS: Seven recommendations emerged for those involved in establishing or leading organizations that support the use of research evidence in developing health policy: 1) collaborate with other organizations; 2) establish strong links with policymakers and involve stakeholders in the work; 3) be independent and manage conflicts of interest among those involved in the work; 4) build capacity among those working in the organization; 5) use good methods and be transparent in the work; 6) start small, have a clear audience and scope, and address important questions; and 7) be attentive to implementation considerations, even if implementation is not a remit. Four recommendations emerged for the World Health Organization (WHO) and other international organizations and networks: 1) support collaborations among organizations; 2) support local adaptation efforts; 3) mobilize support; and 4) create global public goods. CONCLUSION: This synthesis of findings from a multi-method study, along with the more detailed findings from each of the three phases of the study (which are reported in the three following articles in the series), provide a strong basis on which researchers, policymakers, international organizations (and networks) like WHO can respond to the growing chorus of voices calling for efforts to support the use of research evidence in developing health policy.

Evidence-informed health policy 2 - survey of organizations that support the use of research evidence.

BACKGROUND: Previous surveys of organizations that support the development of evidence-informed health policies have focused on organizations that produce clinical practice guidelines (CPGs) or undertake health technology assessments (HTAs). Only rarely have surveys focused at least in part on units that directly support the use of research evidence in developing health policy on an international, national, and state or provincial level (i.e., government support units, or GSUs) that are in some way successful or innovative or that support the use of research evidence in low- and middle-income countries (LMICs). METHODS: We drew on many people and organizations around the world, including our project reference group, to generate a list of organizations to survey. We modified a questionnaire that had been developed originally by the Appraisal of Guidelines, Research and Evaluation in Europe (AGREE) collaboration and adapted one version of the questionnaire for organizations producing CPGs and HTAs, and another for GSUs. We sent the questionnaire by email to 176 organizations and followed up periodically with non-responders by email and telephone. RESULTS: We received completed questionnaires from 152 (86%) organizations. More than one-half of the organizations (and particularly HTA agencies) reported that examples from other countries were helpful in establishing their organization. A higher proportion of GSUs than CPG- or HTA-producing organizations involved target users in the selection of topics or the services undertaken. Most organizations have few (five or fewer) full-time equivalent (FTE) staff. More than four-fifths of organizations reported providing panels with or using systematic reviews. GSUs tended to use a wide variety of explicit valuation processes for the research evidence, but none with the frequency that organizations producing CPGs, HTAs, or both prioritized evidence by its quality. Between one-half and two-thirds of organizations do not collect data systematically about uptake, and roughly the same proportions do not systematically evaluate their usefulness or impact in other ways. CONCLUSION: The findings from our survey, the most broadly based of its kind, both extend or clarify the applicability of the messages arising from previous surveys and related documentary analyses, such as how the 'principles of evidence-based medicine dominate current guideline programs' and the importance of collaborating with other organizations. The survey also provides a description of the history, structure, processes, outputs, and perceived strengths and weaknesses of existing organizations from which those establishing or leading similar organizations can draw.

Evidence-informed health policy 3 - interviews with the directors of organizations that support the use of research evidence.

BACKGROUND: Only a small number of previous efforts to describe the experiences of organizations that produce clinical practice guidelines (CPGs), undertake health technology assessments (HTAs), or directly support the use of research evidence in developing health policy (i.e., government support units, or GSUs) have relied on interviews and then only with HTA agencies. Interviews offer the potential for capturing experiences in great depth, particularly the experiences of organizations that may be under-represented in surveys. METHODS: We purposively sampled organizations from among those who completed a questionnaire in the first phase of our three-phase study, developed and piloted a semi-structured interview guide, and conducted the interviews by telephone, audio-taped them, and took notes simultaneously. Binary or categorical responses to more structured questions were counted when possible. Themes were identified from among responses to semi-structured questions using a constant comparative method of analysis. Illustrative quotations were identified to supplement the narrative description of the themes. RESULTS: We interviewed the director (or his or her nominee) in 25 organizations, of which 12 were GSUs. Using rigorous methods that are systematic and transparent (sometimes shortened to 'being evidence-based') was the most commonly cited strength among all organizations. GSUs more consistently described their close links with policymakers as a strength, whereas organizations producing CPGs, HTAs, or both had conflicting viewpoints about such close links. With few exceptions, all types of organizations tended to focus largely on weaknesses in implementation, rather than strengths. The advice offered to those trying to establish similar organizations include: 1) collaborate with other organizations; 2) establish strong links with policymakers and stakeholders; 3) be independent and manage conflicts of interest; 4) build capacity; 5) use good methods and be transparent; 6) start small and address important questions; and 7) be attentive to implementation considerations. The advice offered to the World Health Organization (WHO) and other international organizations and networks was to foster collaborations across organizations. CONCLUSION: The findings from our interview study, the most broadly based of its kind, extend to both CPG-producing organizations and GSUs the applicability of the messages arising from previous interview studies of HTA agencies, such as to collaborate with other organizations and to be attentive to implementation considerations. Our interview study also provides a rich description of organizations supporting the use of research evidence, which can be drawn upon by those establishing or leading similar organizations in LMICs.

Evidence-informed health policy 4 - case descriptions of organizations that support the use of research evidence.

BACKGROUND: Previous efforts to produce case descriptions have typically not focused on the organizations that produce research evidence and support its use. External evaluations of such organizations have typically not been analyzed as a group to identify the lessons that have emerged across multiple evaluations. Case descriptions offer the potential for capturing the views and experiences of many individuals who are familiar with an organization, including staff, advocates, and critics. METHODS: We purposively sampled a subgroup of organizations from among those that participated in the second (interview) phase of the study and (once) from among other organizations with which we were familiar. We developed and pilot-tested a case description data collection protocol, and conducted site visits that included both interviews and documentary analyses. Themes were identified from among responses to semi-structured questions using a constant comparative method of analysis. We produced both a brief (one to two pages) written description and a video documentary for each case. RESULTS: We conducted 51 interviews as part of the eight site visits. Two organizational strengths were repeatedly cited by individuals participating in the site visits: use of an evidence-based approach (which was identified as being very time-consuming) and existence of a strong relationship between researchers and policymakers (which can be challenged by conflicts of interest). Two organizational weaknesses - a lack of resources and the presence of conflicts of interest - were repeatedly cited by individuals participating in the site visits. Participants offered two main suggestions for the World Health Organization (and other international organizations and networks): 1) mobilize one or more of government support, financial resources, and the participation of both policymakers and researchers; and 2) create knowledge-related global public goods. CONCLUSION: The findings from our case descriptions, the first of their kind, intersect in interesting ways with the messages arising from two systematic reviews of the factors that increase the prospects for research use in policymaking. Strong relationships between researchers and policymakers bodes well given such interactions appear to increase the prospects for research use. The time-consuming nature of an evidence-based approach, on the other hand, suggests the need for more efficient production processes that are 'quick and clean enough.' Our case descriptions and accompanying video documentaries provide a rich description of organizations supporting the use of research evidence, which can be drawn upon by those establishing or leading similar organizations, particularly in low- and middle-income countries.

Translating research into policy and practice in developing countries: a case study of magnesium sulphate for pre-eclampsia.

BACKGROUND: The evidence base for improving reproductive health continues to grow. However, concerns remain that the translation of this evidence into appropriate policies is partial and slow. Little is known about the factors affecting the use of evidence by policy makers and clinicians, particularly in developing countries. The objective of this study was to examine the factors that might affect the translation of randomised controlled trial (RCT) findings into policies and practice in developing countries. METHODS: The recent publication of an important RCT on the use of magnesium sulphate to treat pre-eclampsia provided an opportunity to explore how research findings might be translated into policy. A range of research methods, including a survey, group interview and observations with RCT collaborators and a survey of WHO drug information officers, regulatory officials and obstetricians in 12 countries, were undertaken to identify barriers and facilitators to knowledge translation. RESULTS: It proved difficult to obtain reliable data regarding the availability and use of commonly used drugs in many countries. The perceived barriers to implementing RCT findings regarding the use of magnesium sulphate for pre-eclampsia include drug licensing and availability; inadequate and poorly implemented clinical guidelines; and lack of political support for policy change. However, there were significant regional and national differences in the importance of specific barriers. CONCLUSION: The policy changes needed to ensure widespread availability and use of magnesium sulphate are variable and complex. Difficulties in obtaining information on availability and use are combined with the wide range of barriers across settings, including a lack of support from policy makers. This makes it difficult to envisage any single intervention strategy that might be used to promote the uptake of research findings on magnesium sulphate into policy across the study settings. The publication of important trials may therefore not have the impacts on health care that researchers hope for.

Portals to Wonderland: health portals lead to confusing information about the effects of health care.

BACKGROUND: The Internet offers a seemingly endless amount of health information of varying quality. Health portals, which provide entry points to quality-controlled collections of websites, have been hailed as a solution to this problem. The objective of this study is to assess the extent to which government-run health portals provide access to relevant, valid and understandable information about the effects of health care. METHODS: We selected eight clinically relevant questions for which there was a systematic review, searched four portals for answers, and compared the answers we found to the results of the systematic reviews. RESULTS: Our searches resulted in 3400 hits, 155 of which mentioned both the condition and the intervention in one of the eight questions. Sixty-three of the 155 web pages did not give any information about the effect of the intervention. Seventy-seven qualitatively described the effects of the intervention. Twenty-six of these had information that was too unclear to be categorised; 15 were not consistent with the systematic review; and 36 were consistent with the review, but usually did not mention what happens without the intervention, what outcomes have been measured or when they were measured. Fifteen web pages quantitatively described effects. Four of these were abstracts from the systematic review, nine had information that was incomplete and potentially misleading because of a lack of information about people not receiving the intervention and the length of follow-up; one had information that was consistent with the review, but only referred to three trials whereas the review included six; and one was consistent with the review. CONCLUSION: Information accessible through health portals is unlikely to be based on systematic reviews and is often unclear, incomplete and misleading. Portals are only as good as the websites they lead to. Investments in national health portals are unlikely to benefit consumers without investments in the production and maintenance of relevant, valid and understandable information to which the portals lead.