User Testing to Improve Retrieval and Comprehension of Information in Guidelines to Improve Medicines Safety.

OBJECTIVE: The aim of the study was to investigate the effectiveness of user testing for improving healthcare professionals' retrieval and comprehension of information in medicines guidelines. METHODS: The United Kingdom's Injectable Medicines Guide was selected as a case study. This gives guidance to nurses on preparing and administering intravenous medicines on hospital wards, in line with standard UK practice. Three rounds of user testing were completed with 10 hospital nurses per round, using the Injectable Medicines Guide for voriconazole and aminophylline. Participants used the guidelines to answer 17 questions related to the administration of these medicines. Answers were scored for "finding" and "understanding" the required information. Semistructured interviews explored participants' opinions of guideline content, design, and wording, with responses analyzed thematically. The guidelines were revised between rounds. RESULTS: In round 1, 8 of 17 questions were answered correctly by all participants. Participants had difficulty with dose, dilution, administration rate, and adverse effects questions. Revisions included new subsections and increased calculation support. In round 2, 14 of 17 questions were answered correctly by all participants. Difficulty persisted with dose and administration rate questions and further revisions made. In round 3, 15 of 17 questions were answered correctly by all participants. Across all rounds, participants considered appropriate subheadings and information order as important for fast location of information. Specific, detailed, and practical instructions were perceived as important to improve understandability and usefulness. CONCLUSIONS: Key information in medicines guidelines may not be found and/or understood by healthcare professionals. User testing increased information retrieval and comprehension and could have an important role in improving the safety of medicines use.

Costs and Cost-Effectiveness of User-Testing of Health Professionals' Guidelines to Reduce the Frequency of Intravenous Medicines Administration Errors by Nurses in the United Kingdom: A Probabilistic Model Based on Voriconazole Administration.

AIM: In the UK, injectable medicines are often prepared and administered by nurses following the Injectable Medicines Guide (IMG). Our earlier study confirmed a higher frequency of correct administration with user-tested versus standard IMG guidelines. This current study aimed to model the cost-effectiveness of user-testing. METHODS: The costs and cost-effectiveness of user-testing were explored by modifying an existing probabilistic decision-analytic model. The adapted model considered administration of intravenous voriconazole to hospital inpatients by nurses. It included 11 error types, their probability of detection and level of harm. Model inputs (including costs) were derived from our previous study and other published data. Monte Carlo simulation using 20,000 samples (sufficient for convergence) was performed with a 5-year time horizon from the perspective of the 121 NHS trusts and health boards that use the IMG. Sensitivity analyses were undertaken for the risk of a medication error and other sources of uncertainty. RESULTS: The net monetary benefit at £20,000/quality-adjusted life year was £3,190,064 (95% credible interval (CrI): -346,709 to 8,480,665), favouring user-testing with a 96% chance of cost-effectiveness. Incremental cost-savings were £240,943 (95% CrI 43,527-491,576), also favouring user-tested guidelines with a 99% chance of cost-saving. The total user testing cost was £6317 (95% CrI 6012-6627). These findings were robust to assumptions about a range of input parameters, but greater uncertainty was seen with a lower medication error risk. CONCLUSIONS: User-testing of injectable medicines guidelines is a low-cost intervention that is highly likely to be cost-effective, especially for high-risk medicines.

User-testing guidelines to improve the safety of intravenous medicines administration: a randomised in situ simulation study.

BACKGROUND: User-testing and subsequent modification of clinical guidelines increases health professionals' information retrieval and comprehension. No study has investigated whether this results in safer care. OBJECTIVE: To compare the frequency of medication errors when administering an intravenous medicine using the current National Health Service Injectable Medicines Guide (IMG) versus an IMG version revised with user-testing. METHOD: Single-blind, randomised parallel group in situ simulation. Participants were on-duty nurses/midwives who regularly prepared intravenous medicines. Using a training manikin in their clinical area, participants administered a voriconazole infusion, a high-risk medicine requiring several steps to prepare. They were randomised to use current IMG guidelines or IMG guidelines revised with user-testing. Direct observation was used to time the simulation and identify errors. Participant confidence was measured using a validated instrument. The primary outcome was the percentage of simulations with at least one moderate-severe IMG-related error, with error severity classified by an expert panel. RESULTS: In total, 133 participants were randomised to current guidelines and 140 to user-tested guidelines. Fewer moderate-severe IMG-related errors occurred with the user-tested guidelines (n=68, 49%) compared with current guidelines (n=79, 59%), but this difference was not statistically significant (risk ratio: 0.82; 95% CI 0.66 to 1.02). Significantly more simulations were completed without any IMG-related errors with the user-tested guidelines (n=67, 48%) compared with current guidelines (n=26, 20%) (risk ratio: 2.46; 95% CI 1.68 to 3.60). Median simulation completion time was 1.6 min (95% CI 0.2 to 3.0) less with the user-tested guidelines. Participants who used user-tested guidelines reported greater confidence. CONCLUSION: User-testing injectable medicines guidelines reduces the number of errors and the time taken to prepare and administer intravenous medicines, while increasing staff confidence. TRIAL REGISTRATION NUMBER: researchregistry5275.

Large effects of brief meditation intervention on EEG spectra in meditation novices.

This study investigated the impact of a brief meditation workshop on a sample of 223 novice meditators. Participants attended a three-day workshop comprising daily guided seated meditation sessions using music without vocals that focused on various emotional states and intentions (open focus). Based on the theory of integrative consciousness, it was hypothesized that altered states of consciousness would be experienced by participants during the meditation intervention as assessed using electroencephalogram (EEG). Brainwave power bands patterns were measured throughout the meditation training workshop, producing a total of 5616 EEG scans. Changes in conscious states were analysed using pre-meditation and post-meditation session measures of delta through to gamma oscillations. Results suggested the meditation intervention had large varying effects on EEG spectra (up to 50 % increase and 24 % decrease), and the speed of change from pre-meditation to post-meditation state of the EEG co-spectra was significant (with 0.76 probability of entering end-meditation state within the first minute). There was a main 5 % decrease in delta power (95 % HDI = [-0.07, -0.03]); a global increase in theta power of 29 % (95 % HDI = [0.27, 0.33]); a global increase of 16 % (95 % HDI = [0.13, 0.19]) in alpha power; a main effect of condition, with global beta power increasing by 17 % (95 % HDI = [0.15, 0.19]); and an 11 % increase (95 % HDI = [0.08, 0.14]) in gamma power from pre-meditation to end-meditation. Findings provided preliminary support for brief meditation in altering states of consciousness in novice meditators. Future clinical examination of meditation was recommended as an intervention for mental health conditions particularly associated with hippocampal impairments.

What has been the impact of the Traditional Herbal Registration (THR) scheme in the UK on information provided with herbal products bought over the counter?

BACKGROUND: In 2011 there was a strengthening of European Union (EU) legislation on the licencing of herbal products which, in the UK, resulted in the introduction of the Traditional Herbal Registration (THR) scheme. This scheme sets out standards for the safety and quality of herbal medicines and includes the provision of information to the customer on the safe use of the product. The aim of this study is to replicate a survey undertaken in 2011, prior to the implementation of the THR scheme, and evaluate the impact of this scheme on the information provided with herbal products bought over-the-counter. METHODS: We undertook a survey on 5 herbal products commonly available over-the-counter (St John's wort, echinacea, Ginkgo biloba, Asian ginseng, garlic). The information was searched for key safety messages identified by the National Center for Complementary and Integrative Health (NCCIH). We also explored the presence of risk of harm information. RESULTS: We recorded a rise in the number of products registered with the THR scheme (37% in 2016 compared to 7% in 2011). We also identified a reduction in the number of products that did not contain key safety information (75% in 2011 compared to 20% of products obtained in 2016). Risk of harm information was only communicated in products containing a PIL. We identified more products containing frequency of risk of harm information but this was not statistically significant. CONCLUSION: The introduction of the THR scheme appears to be associated with an increase in the provision of information about key safety messages on the safe use of herbal products. However, it is important to note that at least half of the products on the market that are not included in the THR scheme do not contain any information about their safe use; this includes information about precautions, interactions and side effects. The use of NCCIH herbal monographs replicated the methods used in the previous study; we recognise that the use of a different resource might effect the appraisal of the information provided. We also acknowledge that surveying presence of information does not assure that the latter is effectively communicated to patients, for which a close textual analysis would be required. While it is promising that more information is available after the introduction of the THR scheme, the public needs to be informed about ways to optimise safe use of all herbal products.

Clinical Trial Results Summary for Laypersons: A User Testing Study.

OBJECTIVE: To apply "user testing" to maximize readability and acceptability of a Clinical Trial Results Laypersons Summary-a new European requirement. METHODS: "User testing" (using questionnaire and semistructured interview) assessed whether people could find and understand key points. Findings were used to improve content and design, prior to retesting. Participants had a range of levels of health literacy and there was a higher education group. Participants accessed the summary on screen. In round 1 we tested 12 points of information. In round 2 a revised summary addressing round 1 findings was tested, leading to a third final version. RESULTS: In round 1, 2 of 12 points of information did not reach the target and interviews raised further format and content issues (some distracting technical explanations and inability to find or understand the 2 main study purposes). These findings informed revisions for the version tested in round 2, with 2 different points not reaching the target (inclusion criteria relating to duration of seasonal allergies and how researchers found out about participants' symptoms). Identified problems in both rounds were addressed and reflected in the final version. Despite improvements, participants did not consistently understand that summaries were intended for the public, or to only interpret results of single trials in the context of additional trials. All readers, including those with higher education, found the clear and straightforward language acceptable. CONCLUSIONS: Applying "user testing" resulted in a largely health-literate summary suitable for people across a range of backgrounds.

Community IntraVenous Antibiotic Study (CIVAS): protocol for an evaluation of patient preferences for and cost-effectiveness of community intravenous antibiotic services.

INTRODUCTION: Outpatient parenteral antimicrobial therapy (OPAT) is used to treat a wide range of infections, and is common practice in countries such as the USA and Australia. In the UK, national guidelines (standards of care) for OPAT services have been developed to act as a benchmark for clinical monitoring and quality. However, the availability of OPAT services in the UK is still patchy and until quite recently was available only in specialist centres. Over time, National Health Service (NHS) Trusts have developed OPAT services in response to local needs, which has resulted in different service configurations and models of care. However, there has been no robust examination comparing the cost-effectiveness of each service type, or any systematic examination of patient preferences for services on which to base any business case decision. METHODS AND ANALYSIS: The study will use a mixed methods approach, to evaluate patient preferences for and the cost-effectiveness of OPAT service models. The study includes seven NHS Trusts located in four counties. There are five inter-related work packages: a systematic review of the published research on the safety, efficacy and cost-effectiveness of intravenous antibiotic delivery services; a qualitative study to explore existing OPAT services and perceived barriers to future development; an economic model to estimate the comparative value of four different community intravenous antibiotic services; a discrete choice experiment to assess patient preferences for services, and an expert panel to agree which service models may constitute the optimal service model(s) of community intravenous antibiotics delivery. ETHICS AND DISSEMINATION: The study has been approved by the NRES Committee, South West-Frenchay using the Proportionate Review Service (ref 13/SW/0060). The results of the study will be disseminated at national and international conferences, and in international journals.

Worlds apart? An exploration of prescribing and medicine-taking decisions by patients, GPs and local policy makers.

UNLABELLED: Current healthcare policy in the UK has been shaped by two major forces; increasing accountability to evidence-based standards and increasing patient involvement. Shared decision-making brings the patient into prescribing decisions, and guidelines introduce a third decision-maker, the policy maker, into the doctor-patient consultation. This study explored the decision-making processes used by patients and GPs in comparison to local policy makers. METHOD: Qualitative interviews with 8 GPs, 14 patients and 2 PCT Prescribing Advisers, followed by quantitative questionnaires completed by 305 GPs and 533 patients. RESULTS: Patients made individual medicine-taking decisions based on experience, personal financial and human cost, trust and the relational aspects of their interactions with doctors over time. In contrast local implementation of prescribing guidelines was based on consideration of financial costs, efficacy and risks, based on objective clinical evidence at a population level. GPs adopted a mid-position between these two polar views. Guidelines are written from a different perspective to the worldview of patients, and they tend to downplay the criteria most important to patients. This has the potential to have a harmful effect on patients' medicine-taking and adherence. Paradoxically, enforcing the use of guidelines could inhibit the achievement of guideline targets.

User testing in developing patient medication information in Europe.

In Europe, patient leaflets are supplied as an insert inside all medicine packs. Manufacturers must produce such leaflets according to a template and undertake consultation with target patient groups to ensure it is legible, clear and easy to use, before a licence will be granted. The consultation is usually completed through 'user testing', involving individual interviews with lay participants, with both quantitative and qualitative components. After testing, good practice is applied to address shortcomings identified, and testing repeated. The process is described using the example of a patient booklet for lithium therapy. Such testing could benefit any future US process for patient medication information.

Developing and testing a patient information booklet for thrombolysis used in acute stroke.

Objective Thrombolysis decreases the chance of post-stroke dependence, although its use carries significant risk, notably of intra-cerebral haemorrhage. Patients (and families) face an important risk/benefit decision before consenting. We drafted a patient information booklet for this purpose, and applied performance-based readability testing with the aim that the most important information in the booklet could be found and understood. Methods The booklet was developed with reference to best practice in information writing and design. We User-Tested its performance on 56 people without prior experience of stroke. After reading the booklet they were asked to find and explain 15 pieces of information. The booklet was revised according to its performance and re-tested, until each item was found and understood by at least 80% of participants. Key findings The four-page information booklet contained approximately 900 words, organised into six sections. A risk-palette graphic showed the chance of positive and negative outcomes. The booklet was tested on four participant cohorts and revised, including more bold text, re-wording, changing the title and changing the graphic to a coloured bar chart. Testing the final version on the fourth cohort of 20 people showed that each of the 15 tested items of information met the target of at least 80% participants being able to find and understand it. Conclusions The use of information design and User Testing produced a booklet that is understandable by people with no prior experience of stroke. User Testing is an inexpensive and quick method to ensure that information intended for patients is usable.

The recording of drug sensitivities for older people living in care homes.

AIMS: The aims of this study were to determine the recording of drug sensitivities of elderly care home residents, to describe the nature of sensitivities and to identify and describe discrepancies in the documentation of drug sensitivity status in general practices, pharmacies and care homes. METHODS: A random sample of residents within a purposive sample of care homes (nursing and residential) was selected. A clinical pharmacist inspected the GP medical record, the medicines administration record, and the care home record for each resident to identify drug sensitivities and discrepancies between records and to describe the nature of the recorded sensitivities. RESULTS: The records of 121 residents in 31 care homes were studied. Thirty-one (26%) residents had at least one documented drug sensitivity in one of the sources inspected, with 48 sensitivities in total recorded. There was no description of the nature of the sensitivities recorded in 39/48 (81%) cases. The number of sensitivities recorded on the medicines administration record, care home record and the GP record were 3 (6%), 29 (60%) and 35 (73%), respectively. Only two sensitivities were simultaneously recorded on all three records. CONCLUSIONS: It was of concern that over 90% of drug sensitivities were not recorded on the medicines administration record which is the final checking document when administering medication. The reason for this was that the dispensing pharmacy was responsible for generating the medicines administration record; however, drug sensitivity status is seldom shared between the GP and the dispensing pharmacy. Printing sensitivities on prescriptions would help to resolve this.

Care homes' use of medicines study: prevalence, causes and potential harm of medication errors in care homes for older people.

INTRODUCTION: Care home residents are at particular risk from medication errors, and our objective was to determine the prevalence and potential harm of prescribing, monitoring, dispensing and administration errors in UK care homes, and to identify their causes. METHODS: A prospective study of a random sample of residents within a purposive sample of homes in three areas. Errors were identified by patient interview, note review, observation of practice and examination of dispensed items. Causes were understood by observation and from theoretically framed interviews with home staff, doctors and pharmacists. Potential harm from errors was assessed by expert judgement. RESULTS: The 256 residents recruited in 55 homes were taking a mean of 8.0 medicines. One hundred and seventy-eight (69.5%) of residents had one or more errors. The mean number per resident was 1.9 errors. The mean potential harm from prescribing, monitoring, administration and dispensing errors was 2.6, 3.7, 2.1 and 2.0 (0 = no harm, 10 = death), respectively. Contributing factors from the 89 interviews included doctors who were not accessible, did not know the residents and lacked information in homes when prescribing; home staff's high workload, lack of medicines training and drug round interruptions; lack of team work among home, practice and pharmacy; inefficient ordering systems; inaccurate medicine records and prevalence of verbal communication; and difficult to fill (and check) medication administration systems. CONCLUSIONS: That two thirds of residents were exposed to one or more medication errors is of concern. The will to improve exists, but there is a lack of overall responsibility. Action is required from all concerned.

Performance-based readability testing of participant materials for a phase I trial: TGN1412.

BACKGROUND: Concern has been expressed about the process of consent to clinical trials, particularly in phase I "first-in-man" trials. Trial participant information sheets are often lengthy and technical. Content-based readability testing of sheets, which is often required to obtain research ethics approval for trials in the USA, is limited and cannot indicate how information will perform. METHODS: An independent-groups design was used to study the user-testing performance of the participant information sheet from the phase I TGN1412 trial. Members of the public were asked to read it, then find and demonstrate understanding of 21 key aspects of the trial. The participant information sheet was then rewritten, redesigned and tested on 20 members of the public, using the same 21-item questionnaire. RESULTS: On the original TGN1412 participant information sheet, participants could not find answers and some of the found information was not understood. Six of 21 questions, including those relating to placebo, follow-up visits and the emergency phone number, were found by eight or fewer of 10 participants. The revised information sheet performed better, with the answers to 17 of 21 questions found and understood by all 20 participants. CONCLUSIONS: Tests showed that the TGN1412 participant information sheet may not inform participants adequately for consent. Revising its content and design led to significant improvements. Writers of materials for trial participants should take account of good practice in information design. Performance-based user testing may be a useful method to indicate strengths and weaknesses in trial materials.

Performance-based readability testing of participant information for a Phase 3 IVF trial.

BACKGROUND: Studies suggest that the process of patient consent to clinical trials is sub-optimal. Participant information sheets are important but can be technical and lengthy documents. Performance-based readability testing is an established means of assessing patient information, and this study aimed to test its application to participant information for a Phase 3 trial. METHODS: An independent groups design was used to study the User Testing performance of the participant information sheet from the Phase 3 'Poor Responders' trial of In Vitro Fertilisation (IVF). 20 members of the public were asked to read it, then find and demonstrate understanding of 21 key aspects of the trial. The participant information sheet was then re-written and re-designed, and tested on 20 members of the public, using the same 21 item questionnaire. RESULTS: The original participant information sheet performed well in some places. Participants could not find some answers and some of the found information was not understood. In total there were 30 instances of information being not found or not understood. Answers to three questions were found but not understood by many of the participants, these related to aspects of the drug timing, Follicle Stimulating Hormone and compensation. Only two of the 20 participants could find and show understanding of all question items when using the original sheet. The revised sheet performed generally better, with 17 instances of information being not found or not understood, although the number of 'not found' items increased. Half of the 20 participants could find and show understanding of all question items when using the revised sheet. When asked to compare the versions of the sheet, almost all participants preferred the revised version. CONCLUSION: The original participant information sheet may not have enabled patients fully to give valid consent. Participants seeing the revised sheet were better able to understand the trial. Those who write information for trial participants should take account of good practice in information design. Performance-based User Testing may be a useful method to indicate strengths and weaknesses in trial information.

Written information about individual medicines for consumers.

BACKGROUND: Medicines are the most common intervention in most health services. As with all treatments, those taking medicines need sufficient information: to enable them to take and use the medicines effectively, to understand the potential harms and benefits, and to allow them to make an informed decision about taking them. Written medicines information, such as a leaflet or provided via the Internet, is an intervention that may meet these purposes. OBJECTIVES: To assess the effects of providing written information about individual medicines on relevant patient outcomes (knowledge, attitudes, behaviours and health outcomes) in relation to prescribed and over-the-counter medicines. SEARCH STRATEGY: We searched MEDLINE, EMBASE, CINAHL, The Cochrane Library, PsycINFO and other databases to March 2007. We handsearched five journals' tables of contents, and the reference lists of included studies, and contacted experts in the field. SELECTION CRITERIA: Randomised controlled trials (RCTs) of medicine users, comparing written medicines information with no written medicines information; or trials that compared two or more styles of written medicines information. We only included trials that measured a knowledge, attitudinal or behavioural outcome. There were no language restrictions. DATA COLLECTION AND ANALYSIS: Two review authors independently extracted data relating to the interventions, methods of the trials, and outcome measures; and reconciled differences by discussion. Heterogeneity of interventions and outcomes measured meant that data synthesis was not possible. The results are presented in narrative and tabular format. MAIN RESULTS: We included 25 RCTs involving 4788 participants. Six of twelve trials showed that written information significantly improved knowledge about a medicine, compared with no written information. The inability to combine results means we cannot conclude whether written information was effective for increasing knowledge. The results for attitudinal and behavioural outcomes were mixed. No studies showed an adverse effect of medicines information. AUTHORS' CONCLUSIONS: The combined evidence was not strong enough to say whether written medicines information is effective in changing knowledge, attitudes and behaviours related to medicine taking. There is some evidence that written information can improve knowledge. The trials were generally of poor quality, which reduces confidence in the results. Trials examining the effects of written information need to be better designed and use consistent and validated outcome measures. Trials should evaluate internet-based medicines information. It is imperative that written medicines information be based on best practice for its information design and content, which could improve its effectiveness in helping people to use medicines appropriately.

Perceived risk of medicine side effects in users of a patient information website: a study of the use of verbal descriptors, percentages and natural frequencies.

OBJECTIVES: Research into the provision of patient information has demonstrated that, under certain conditions, patients overestimate the risks of medicine side effects. Gigerenzer and Edwards (2004) argue that 'natural frequencies' are a less confusing way of expressing risk information. Two experiments with users of an existing high profile patient information website, investigate the effectiveness of presenting medicine side-effect risk information in different forms. DESIGN: In both experiments participants were randomly allocated to one of the three conditions for representing risk information (a form of 'natural frequency', percentages and verbal descriptors). METHOD: Participants were recruited from users of the Cancer Research UK patient information website (Cancer Help UK). In Experiment 1, participants (N=148) were asked to imagine that they had to take a chemotherapy drug (Taxol) and were asked to estimate the risks of two side-effects occurring. In Experiment 2 participants (N=137) estimated the risk for three different side-effects occurring with the painkiller ibuprofen. RESULTS: In both experiments, verbal descriptions led to significantly higher estimations of risk compared to the other two formats. There was some evidence that people given information as frequencies were more accurate in their estimates than those given percentage information. CONCLUSIONS: These findings provide partial support for the advantages of a form of 'natural frequencies' when presenting side-effects to patients. They also add weight to the growing body of research highlighting the deficiencies in using verbal descriptions of side-effect risk alone.