A methodological study to compare survey-based and observation-based evaluations of organisational and safety cultures and then compare both approaches with markers of the quality of care.

BACKGROUND: Patient safety concerns have focused attention on organisational and safety cultures, in turn directing attention to the measurement of organisational and safety climates. OBJECTIVES: First, to compare levels of agreement between survey- and observation-based measures of organisational and safety climates/cultures and to compare both measures with criterion-based audits of the quality of care, using evidence-based markers drawn from national care standards relating to six common clinical conditions. (This required development of an observation-based instrument.) Second, to examine whether observation-based evaluations could replace or augment survey measurements to mitigate concerns about declining response rates and increasing social desirability bias. Third, to examine mediating factors in safety and organisational climate scores. DESIGN: The study had three strands: (A) a postal questionnaire survey to elicit staff perceptions of organisational and safety climates, using six prevalidated scales; (B) semistructured non-participant observation of clinical teams; and (C) a retrospective criterion-based audit carried out by non-clinical auditors to minimise hindsight bias. Standardised summary scores were created for each strand, and pairs of measurements were compared (strand A with strand B, strand A with strand C, and strand B with strand C) using Bland-Altman plots to evaluate agreement. Correlations were also examined. Multilevel modelling of Strand A scores explored mediating factors. SETTING AND PARTICIPANTS: Eight emergency departments and eight maternity units in England, UK. INTERVENTIONS: None. MAIN OUTCOME MEASURES: Examination of feasibility, correlation and agreement. RESULTS: Strand A: the overall response rate was 27.6%, whereas site-specific rates ranged from 9% to 47%. We identified more mediating factors than previous studies; variable response rates had little effect on the results. Organisational and safety climate scores were strongly correlated (r = 0.845) and exhibited good agreement [standard deviation (SD) differences 0.449; 14 (88%) within ± 0.5; one large difference]. Two commonly used scales had high levels of positive responses, suggesting positive climates or social desirability bias. Strand B: scoring on a four-point scale was feasible. Observational evaluation of teamwork culture was good but too limited for evaluating organisational culture. Strand C: a total of 359-399 cases were audited per condition. The results varied widely between different markers for the same condition, so selection matters. Each site performed well on some markers but not others, with few consistent patterns. Some national guidelines were contested. Comparisons: the comparison of safety climate (survey) and teamwork culture (observation) revealed a moderately low correlation (r = 0.316) and good agreement [SD differences 1.082; 7 (44%) within ±0.5; one large difference]. The comparison of safety climate (survey) and performance (audit) revealed lower correlation (r = 0.150, i.e. relationship not linear) but reasonably good agreement [SD differences 0.992; 9 (56%) within ± 0.5; two large differences]. Comparisons between performance (audit) and both organisational climate (survey) and teamwork culture (observation) showed negligible correlations (< 0.1) but moderately good agreement [SD differences 1.058 and 1.241; 6 (38%) and 7 (44%) within ± 0.5; each with two large differences (at different sites)]. Field notes illuminated large differences. CONCLUSIONS: Climate scores from staff surveys are not unduly affected by survey response rates, but increasing use risks social desirability bias. Safety climate provides a partial indicator of performance, but qualitative data are needed to understand discrepant results. Safety climate (surveys) and, to a lesser degree, teamwork culture (observations) are better indicators of performance than organisational climate (surveys) or attempts to evaluate organisational culture from time-limited observations. Scoring unobtrusive, time-limited observations to evaluate teamwork culture is feasible, but the instrument developed for this study needs further testing. A refined observation-based measure would be useful to augment or replace surveys. FUNDING: The National Institute for Health Research Health Technology Assessment programme.

Interprofessional education: effects on professional practice and health care outcomes.

BACKGROUND: Patient care is a complex activity which demands that health and social care professionals work together in an effective manner. The evidence suggests, however, that these professionals do not collaborate well together. Interprofessional education (IPE) offers a possible way to improve collaboration and patient care. OBJECTIVES: To assess the effectiveness of IPE interventions compared to education interventions in which the same health and social care professionals learn separately from one another; and to assess the effectiveness of IPE interventions compared to no education intervention. SEARCH STRATEGY: We searched the Cochrane Effective Practice and Organisation of Care Group specialised register, MEDLINE and CINAHL, for the years 1999 to 2006. We also handsearched the Journal of Interprofessional Care (1999 to 2006), reference lists of the six included studies and leading IPE books, IPE conference proceedings, and websites of IPE organisations. SELECTION CRITERIA: Randomised controlled trials (RCTs), controlled before and after (CBA) studies and interrupted time series (ITS) studies of IPE interventions that reported objectively measured or self reported (validated instrument) patient/client and/or healthcare process outcomes. DATA COLLECTION AND ANALYSIS: Two reviewers independently assessed the eligibility of potentially relevant studies, and extracted data from, and assessed study quality of, included studies. A meta-analysis of study outcomes was not possible given the small number of included studies and the heterogeneity in methodological designs and outcome measures. Consequently, the results are presented in a narrative format. MAIN RESULTS: We included six studies (four RCTs and two CBA studies). Four of these studies indicated that IPE produced positive outcomes in the following areas: emergency department culture and patient satisfaction; collaborative team behaviour and reduction of clinical error rates for emergency department teams; management of care delivered to domestic violence victims; and mental health practitioner competencies related to the delivery of patient care. In addition, two of the six studies reported mixed outcomes (positive and neutral) and two studies reported that the IPE interventions had no impact on either professional practice or patient care. AUTHORS' CONCLUSIONS: This updated review found six studies that met the inclusion criteria, in contrast to our first review that found no eligible studies. Although these studies reported some positive outcomes, due to the small number of studies, the heterogeneity of interventions, and the methodological limitations, it is not possible to draw generalisable inferences about the key elements of IPE and its effectiveness. More rigorous IPE studies (i.e. those employing RCTs, CBA or ITS designs with rigorous randomisation procedures, better allocation concealment, larger sample sizes, and more appropriate control groups) are needed to provide better evidence of the impact of IPE on professional practice and healthcare outcomes. These studies should also include data collection strategies that provide insight into how IPE affects changes in health care processes and patient outcomes.

A best evidence systematic review of interprofessional education: BEME Guide no. 9.

BACKGROUND AND REVIEW CONTEXT: Evidence to support the proposition that learning together will help practitioners and agencies work better together remains limited and thinly spread. This review identified, collated, analysed and synthesised the best available contemporary evidence from 21 of the strongest evaluations of IPE to inform the above proposition. In this way we sought to help shape future interprofessional education and maximize the potential for interprofessional learning to contribute to collaborative practice and better care. OBJECTIVES OF THE REVIEW: To identify and review the strongest evaluations of IPE. To classify the outcomes of IPE and note the influence of context on particular outcomes. To develop a narrative about the mechanisms that underpin and inform positive and negative outcomes of IPE. SEARCH STRATEGY: Bibliographic database searches as follows: Medline 1966-2003, CINAHL 1982-2001, BEI 1964-2001, ASSIA 1990-2003 which produced 10,495 abstracts. Subsequently, 884 full papers were obtained and scrutinized. In addition, hand searching (2003-5 issues) of 21 journals known to have published two or more higher quality studies from a previous review. TOPIC DEFINITION AND INCLUSION CRITERIA: Peer-reviewed papers and reports included in the review had to be formal educational initiatives attended by at least two of the many professional groups from health and social care, with the objective of improving care; and learning with, from and about each other. DATA COLLECTION, ANALYSIS AND SYNTHESIS: Standard systematic review procedures were applied for sifting abstracts, scrutinizing full papers and abstracting data. Two members of the team checked each abstract to decide whether the full paper should be read. A third member was consulted over any discrepancies. Similarly, each full paper was read by at least two members of the team and agreement sought before passing it to one member of the team (SR) for data abstraction. Other members of the team checked 10% of the abstraction records. Coding into a Statistical Package for Social Scientists (SPSS) data base led to collection of different outcome measures used in the primary studies via the common metric of an adapted Kirkpatrick's four-level model of educational outcomes. Additionally, a narrative synthesis was built after analysis of primary data with the 3-P model (presage-process-product) of education development and delivery. HEADLINE RESULTS: Government calls for enhanced collaboration amongst practitioners frequently leads to IPE that is then developed and delivered by educators, practitioners or service managers. Staff development is a key influence on the effectiveness of IPE for learners who all have unique values about themselves and others. Authenticity and customization of IPE are important mechanisms for positive outcomes of IPE. Interprofessional education is generally well received, enabling knowledge and skills necessary for collaborative working to be learnt; it is less able to positively influence attitudes and perceptions towards others in the service delivery team. In the context of quality improvement initiatives interprofessional education is frequently used as a mechanism to enhance the development of practice and improvement of services.

Re-examining the evaluation of interprofessional education for community mental health teams with a different lens: understanding presage, process and product factors.

This paper revisits the formative evaluation of a pilot project that offered in-service interprofessional education (IPE), which is designed to enhance the collaborative practice, to two UK community mental health teams (CMHTs). While the IPE was well received and resulted in some improvements in team functioning, wider successes were elusive. Specifically, collaborative action plans were not implemented, and the pilot programme was ultimately not rolled out to other CMHTs. The purpose of this paper is to test the usefulness of the presage-process-product (3P) framework for analysis as a means to untangle the complex web of factors that promoted and inhibited success in this initiative. The framework, which captures key features of the initiative as a dynamic system, proved effective, yielding new insights, making connections clearer and highlighting the critical importance of presage. We argue that use of the 3P model during the development of in-service IPE could ensure that planning oversights are minimized, thereby improving outcomes.

Sustaining interprofessional collaboration.

There has been much written about new interprofessional collaborations, rather less about established or terminated collaborations. This paper briefly rehearses well known factors that encourage and discourage interprofessional collaboration, before considering factors that may help sustain collaborations or signal that termination of the collaboration is appropriate. The Clinical Skills Initiative at St Bartholomew's in London, UK is offered as a case study of a sustained interprofessional collaboration. It will be argued that a combination of continued need to collaborate and empowerment to do so, creates favourable conditions for sustained collaboration. The composition of the interprofessional team that comprises the collaboration is also seen as key.

'Real life' clinical learning on an interprofessional training ward.

This paper describes the multi-method evaluation of an interprofessional training ward placement for medical, nursing, occupational therapy and physiotherapy students. Unique in the UK, and an extension of pioneering work in Sweden (Wahlström et al. 1997, Wahlstroöm & Sandén 1998), this interprofessional clinical placement allowed senior pre-qualifying students, under the supervision of practitioners, to plan and deliver interprofessional care for a group of orthopaedic and rheumatology patients. This responsibility enabled students to develop both their profession-specific skills in a real-world setting and the quality of their interprofessional teamwork. Student teams were supported by facilitators who led reflective sessions and acted as a resource for the students' problem-based learning. The training ward was evaluated by a multi-method approach, incorporating interviews, observations and questionnaires with students, patients and clinical staff. The evaluation findings have been grouped into a number of themes which offer an insight into the varying perspectives of training ward students, patients and staff. This paper pays particular attention to the nursing perspective of the interprofessional training ward pilot.

Provision of an electronic library at the clinical frontline: evaluation of impact on hospital medical staff.

Doctors need convenient access to the latest information if they are to meet the demands of clinical governance and evidence-based medicine. Forest Healthcare NHS Trust used its intranet to provide clinical areas with direct 24-hour access to library materials.

Learning clinical skills: an interprofessional approach.

The professions of nursing and medicine are committed to interprofessional education, in the belief that through this, patient care and satisfaction will be improved. Most initiatives involving nurses have been at post-qualification level, in primary health care, and concerning interpersonal or information management skills. Much of this collaboration has been with professions allied to medicine or social services. This paper discusses an innovative programme of shared learning in acute care, involving final year medical students and newly qualified staff nurses. The programme, developed in response to the blurring of professional roles between nurses and junior doctors, took place in our interprofessional Clinical Skills Centre. It was based around a developing patient scenario which was pertinent to the participants' area of practice. Each session was led by an experienced nurse lecturer and doctor, supported by specialist contributors. The style of learning was participative, with small interprofessional groups addressing a range of patient management issues. In this way, relevant clinical and communication skills were integrated within the context of holistic patient care. The course was well evaluated by both professional groups of participants and their managers. Subsequent research and curriculum development are leading to the expansion of this successful initiative.

Promoting health in young people.

This article discusses aspects of health that might affect young people and ways in which nurses can promote healthy living and support this group of clients during a potential period of anxiety.

Teenage health and the practice nurse: choice and opportunity for both?

Teenage health issues and the means to address them have caused increasing concern over recent years. This study investigated the involvement and training of practice nurses, the topics raised during consultations with teenagers, and the comfort of the practice nurses in dealing with these topics. Data were collected using a postal questionnaire survey (response rate 80.6%). Expansion of the role of the practice nurse in teenage health may be appropriate but needs to be supported by training.

Assessment of clinical skills: a new approach to an old problem.

The changed nature of clinical placements, with shorter in-patient episodes and staff in clinical areas carrying increasingly high workloads, has prompted changes in the teaching learning and assessment of clinical skills. The 'Bart's Nursing OSCE' is an innovative approach to the assessment of clinical skills, through the medium of simulated professional practice. The antecedents of this educational development are discussed before detailed description of the Bart's Nursing OSCE. Discussion focuses upon: the rationale, authenticity, validity and reliability of the assessment; the roles of assessors and simulated patients; feedback to and from students and other stakeholders; the educational impact of the innovation; and management issues.