Inter-observer agreement and reliability assessment for observational studies of clinical work.

Inter-observer agreement (IOA) is a key aspect of data quality in time-and-motion studies of clinical work. To date, such studies have used simple and ad hoc approaches for IOA assessment, often with minimal reporting of methodological details. The main methodological issues are how to align time-stamped task intervals that rarely have agreeing start and end times, and how to assess IOA for multiple nominal variables. We present a combination of methods that simultaneously addresses both these issues and provides a more appropriate measure by which to assess IOA for time-and-motion studies. The issue of alignment is addressed by converting task-level data into small time windows then aligning data from different observers by time. A method applicable to multivariate nominal data, the iota score, is then applied to the time-aligned data. We illustrate our approach by comparing iota scores to the mean of univariate Cohen's kappa scores through application of these measures to existing data from an observational study of emergency department physicians. While the two scores generated very similar results under certain conditions, iota was more resilient to sparse data issues. Our results suggest that iota applied to time windows considerably improves on previous methods used for IOA assessment in time-and-motion studies, and that Cohen's kappa and other univariate measures should not be considered the gold standard. Rather, there is an urgent need for ongoing explicit discussion of methodological issues and solutions to improve the ways in which data quality is assessed in time-and-motion studies in order to ensure the conclusions drawn from such studies are robust.

Emergency doctors' strategies to manage competing workload demands in an interruptive environment: An observational workflow time study.

An observational workflow time study was conducted involving doctors in the emergency department (ED) of a large Australian hospital. During 121.7 h across 58 sessions, we observed interruptive events, conceptualised as prompts, and doctors' strategies to handle those prompts (task-switching, multitasking, acknowledgement, deferral and deflection) to assess the role of multiple work system factors influencing doctors' work in the ED. Prompt rates varied vastly between work scenarios, being highest during non-verbal solo tasks. The propensity to use certain strategies also differed with task type, prompt type and location within the department, although task-switching was by far the most frequent. Communicative prompts were important in patient treatment and workload management. Clinicians appear to adjust their communication strategies in response to contextual factors in order to deliver patient care. Risk due to the interruptive nature of ED communication is potentially outweighed by the positive effects of timely information transfer and advice provision.

Dangers and uses of cross-correlation in analyzing time series in perception, performance, movement, and neuroscience: The importance of constructing transfer function autoregressive models.

Many articles on perception, performance, psychophysiology, and neuroscience seek to relate pairs of time series through assessments of their cross-correlations. Most such series are individually autocorrelated: they do not comprise independent values. Given this situation, an unfounded reliance is often placed on cross-correlation as an indicator of relationships (e.g., referent vs. response, leading vs. following). Such cross-correlations can indicate spurious relationships, because of autocorrelation. Given these dangers, we here simulated how and why such spurious conclusions can arise, to provide an approach to resolving them. We show that when multiple pairs of series are aggregated in several different ways for a cross-correlation analysis, problems remain. Finally, even a genuine cross-correlation function does not answer key motivating questions, such as whether there are likely causal relationships between the series. Thus, we illustrate how to obtain a transfer function describing such relationships, informed by any genuine cross-correlations. We illustrate the confounds and the meaningful transfer functions by two concrete examples, one each in perception and performance, together with key elements of the R software code needed. The approach involves autocorrelation functions, the establishment of stationarity, prewhitening, the determination of cross-correlation functions, the assessment of Granger causality, and autoregressive model development. Autocorrelation also limits the interpretability of other measures of possible relationships between pairs of time series, such as mutual information. We emphasize that further complexity may be required as the appropriate analysis is pursued fully, and that causal intervention experiments will likely also be needed.

Managing competing demands through task-switching and multitasking: a multi-setting observational study of 200 clinicians over 1000 hours.

OBJECTIVE: To provide a detailed characterisation of clinicians' work management strategies. DESIGN: 1002.3 h of observational data were derived from three previous studies conducted in a teaching hospital in Sydney, Australia, among emergency department (ED) doctors (n=40), ward doctors (n=57) and ward nurses (n=104). The rates of task-switching (pausing a task to handle an incoming task) and multitasking (adding a task in parallel to an existing task) were compared in each group. Random intercepts logistic regression was used to determine factors significantly associated with clinicians' use of task-switching over multitasking and to quantify variation between individual clinicians. RESULTS: Task-switching rates were higher among ED doctors (6.0 per hour) than ward staff (2.2 and 1.8 per hour for doctors and nurses, respectively) and vice versa for multitasking rates (9.2 vs 17.3 and 14.1 per hour). Clinicians' strategy use was significantly related to the nature and complexity of work and to the person they were working with. In some settings, time of day, day of the week or previous chosen strategy affected a clinician's strategy. Independent of these factors, there was significant variation between individual clinicians in their use of strategies in a given situation (ED doctors p=0.04, ward staff p=0.03). CONCLUSIONS: Despite differences in factors associated with work management strategy use among ED doctors, ward doctors and ward nurses, clinicians in all settings appeared to prioritise certain types of tasks over others. Documentation was generally given low priority in all groups, while the arrival of direct care tasks tended to be treated with high priority. These findings suggest that considerations of safety may be implicit in task-switching and multitasking decisions. Although these strategies have been cast in a negative light, future research should consider their role in optimising competing quality and efficiency demands.

Changes in nurses' work associated with computerised information systems: Opportunities for international comparative studies using the revised Work Observation Method By Activity Timing (WOMBAT).

An important step in advancing global health through informatics is to understand how systems support health professionals to deliver improved services to patients. Studies in several countries have highlighted the potential for clinical information systems to change patterns of work and communication, and in particular have raised concerns that they reduce nurses' time in direct care. However measuring the effects of systems on work is challenging and comparisons across studies have been hindered by a lack of standardised definitions and measurement tools. This paper describes the Work Observation Method by Activity Time (WOMBAT) technique version 1.0 and the ways in which the data generated can describe different aspects of health professionals' work. In 2011 a revised WOMBAT version 2.0 was developed specifically to facilitate its use by research teams in different countries. The new features provide opportunities for international comparative studies of nurses' work to be conducted.

The impact of interruptions on clinical task completion.

BACKGROUND: Interruptions and multitasking are implicated as a major cause of clinical inefficiency and error. OBJECTIVE: The aim was to measure the association between emergency doctors' rates of interruption and task completion times and rates. METHODS: The authors conducted a prospective observational time and motion study in the emergency department of a 400-bed teaching hospital. Forty doctors (91% of medical staff) were observed for 210.45 h on weekdays. The authors calculated the time on task (TOT); the relationship between TOT and interruptions; and the proportion of time in work task categories. Length-biased sampling was controlled for. RESULTS: Doctors were interrupted 6.6 times/h. 11% of all tasks were interrupted, 3.3% more than once. Doctors multitasked for 12.8% of time. The mean TOT was 1:26 min. Interruptions were associated with a significant increase in TOT. However, when length-biased sampling was accounted for, interrupted tasks were unexpectedly completed in a shorter time than uninterrupted tasks. Doctors failed to return to 18.5% (95% CI 15.9% to 21.1%) of interrupted tasks. CONCLUSIONS: It appears that in busy interrupt-driven clinical environments, clinicians reduce the time they spend on clinical tasks if they experience interruptions, and may delay or fail to return to a significant portion of interrupted tasks. Task shortening may occur because interrupted tasks are truncated to 'catch up' for lost time, which may have significant implications for patient safety.

Association of interruptions with an increased risk and severity of medication administration errors.

BACKGROUND: Interruptions have been implicated as a cause of clinical errors, yet, to our knowledge, no empirical studies of this relationship exist. We tested the hypothesis that interruptions during medication administration increase errors. METHODS: We performed an observational study of nurses preparing and administering medications in 6 wards at 2 major teaching hospitals in Sydney, Australia. Procedural failures and interruptions were recorded during direct observation. Clinical errors were identified by comparing observational data with patients' medication charts. A volunteer sample of 98 nurses (representing a participation rate of 82%) were observed preparing and administering 4271 medications to 720 patients over 505 hours from September 2006 through March 2008. Associations between procedural failures (10 indicators; eg, aseptic technique) and clinical errors (12 indicators; eg, wrong dose) and interruptions, and between interruptions and potential severity of failures and errors, were the main outcome measures. RESULTS: Each interruption was associated with a 12.1% increase in procedural failures and a 12.7% increase in clinical errors. The association between interruptions and clinical errors was independent of hospital and nurse characteristics. Interruptions occurred in 53.1% of administrations (95% confidence interval [CI], 51.6%-54.6%). Of total drug administrations, 74.4% (n = 3177) had at least 1 procedural failure (95% CI, 73.1%-75.7%). Administrations with no interruptions (n = 2005) had a procedural failure rate of 69.6% (n = 1395; 95% CI, 67.6%-71.6%), which increased to 84.6% (n = 148; 95% CI, 79.2%-89.9%) with 3 interruptions. Overall, 25.0% (n = 1067; 95% CI, 23.7%-26.3%) of administrations had at least 1 clinical error. Those with no interruptions had a rate of 25.3% (n = 507; 95% CI, 23.4%-27.2%), whereas those with 3 interruptions had a rate of 38.9% (n = 68; 95% CI, 31.6%-46.1%). Nurse experience provided no protection against making a clinical error and was associated with higher procedural failure rates. Error severity increased with interruption frequency. Without interruption, the estimated risk of a major error was 2.3%; with 4 interruptions this risk doubled to 4.7% (95% CI, 2.9%-7.4%; P < .001). CONCLUSION: Among nurses at 2 hospitals, the occurrence and frequency of interruptions were significantly associated with the incidence of procedural failures and clinical errors.

Molecular markers in prostate cancer. Part II: potential roles in management.

Predicting treatment responses in advanced prostate cancer (PCa) currently centres on prostate-specific antigen (PSA) kinetics and on being able to visualize measurable changes in imaging modalities. New molecular markers have emerged as potential diagnostic and prognostic indicators; these were summarized in Part I of this review in the Asian Journal of Andrology. A number of molecular markers are now being used to enhance PCa imaging and staging. However, management options for advanced and hormone-resistant PCa (HRPC) are limited and additional therapeutic options are needed. Molecular markers have been proposed as potential therapeutic targets using gene therapy and immunomodulation. Additionally, markers identified in early PCa and precursor lesions may offer novel targets for chemoprevention and vaccine development. This review summarizes the current advances regarding the roles of these markers in the management of PCa.

Molecular markers in prostate cancer. Part I: predicting lethality.

Assessing the lethality of 'early,' potentially organ-confined prostate cancer (PCa) is one of the central controversies in modern-day urological clinical practice. Such cases are often considered for radical 'curative' treatment, although active surveillance may be equally appropriate for many men. Moreover, the balance between judicious intervention and overtreatment can be difficult to judge. The patient's age, comorbidities, family history and philosophy of self-health care can be weighed against clinical features such as the palpability of disease, the number and percentage of biopsy cores involved with the disease, histological grade, presenting prostate-specific antigen (PSA) and possible previous PSA kinetics. For many years, scientists and physicians have sought additional molecular factors that may be predictive for disease stage, progression and lethality. Usually, claims for a 'new' unique marker fall short of true clinical value. More often than not, such molecular markers are useful only in multivariate models. This review summarizes relevant molecular markers and models reported up to and including 2008.

How comparable are road traffic crash cases in hospital admissions data and police records? An examination of data linkage rates.

OBJECTIVES: An assessment of linked data was used to investigate the scope and the extent to which hospitalisations data and police crash records represent road crashes in New South Wales (NSW). METHODS: Hospital separation records for the period 1 July 2000 to 30 June 2001, inclusive, were linked to police crash casualty records for the same period using probabilistic record linkage techniques. Multivariable logistic regression techniques were used to identify factors independently associated with the probability of record linkage. RESULTS: Of 17,552 road transport-related hospital records, 45.1% matched to police crash casualty records. When the analysis was restricted to road traffic crashes, 69.2% of the 9,178 records had a matching police crash casualty record. Multivariable analysis found the most significant factors contributing to the likelihood of linkage to be road user type, payment status and principal diagnosis of injury variables. Motor vehicle controllers, cases entitled to financial compensation and cases with a principal diagnosis of injury were significantly more likely to be linked than all other cases. CONCLUSIONS: The findings indicate that researchers and policy makers should be cautious when examining traffic crashes based on a separate analysis of the hospitalisations data and police crash records. This is particularly true for crashes involving pedestrians, cyclists and motorcyclists, and those resulting in less severe injuries. IMPLICATIONS: The findings have implications for use of both police crash records and hospital records in informing the development of strategies designed to prevent road trauma in the community.

Gene expression correlates of postinfective fatigue syndrome after infectious mononucleosis.

BACKGROUND: Infectious mononucleosis (IM) commonly triggers a protracted postinfective fatigue syndrome (PIFS) of unknown pathogenesis. METHODS: Seven subjects with PIFS with 6 or more months of disabling symptoms and 8 matched control subjects who had recovered promptly from documented IM were studied. The expression of 30,000 genes was examined in the peripheral blood by microarray analysis in 65 longitudinally collected samples. Gene expression patterns associated with PIFS were sought by correlation with symptom factor scores. RESULTS: Differential expression of 733 genes was identified when samples collected early during the illness and at the late (recovered) time point were compared. Of these genes, 234 were found to be significantly correlated with the reported severity of the fatigue symptom factor, and 180 were found to be correlated with the musculoskeletal pain symptom factor. Validation by analysis of the longitudinal expression pattern revealed 35 genes for which changes in expression were consistent with the illness course. These genes included several that are involved in signal transduction pathways, metal ion binding, and ion channel activity. CONCLUSIONS: Gene expression correlates of the cardinal symptoms of PIFS after IM have been identified. Further studies of these gene products may help to elucidate the pathogenesis of PIFS.

Can spot urine protein/creatinine ratio replace 24 h urine protein in usual clinical nephrology?

BACKGROUND: This prospective study was designed to assess whether a single urine specimen, collected at the time of clinical assessment, could be used to estimate 24 h urinary protein excretion. METHODS: The spot urinary protein/creatinine ratio (P/Cr) and 24 h urine protein and creatinine excretion were determined from 103 patients attending a nephrology clinic. Using the spot urine P/Cr, a regression equation was derived, predicting 24 h protein excretion. The performance of the regression equation was tested on a new cohort of 85 patients. Within this new cohort, agreement between the predicted and the actual 24 h urinary protein excretion was assessed using the Bland-Altman technique. The ability of spot urine P/Cr and dipstick urinalysis to predict different threshold levels of protein excretion was assessed using a series of receiver operator curves. RESULTS: A logarithmic relationship exists between the spot P/Cr and 24 h protein excretion, with a correlation of 0.92, P < 0.0001. Agreement between the actual and predicted 24 h urine protein and between spot and 24 h urine P/Cr ratios was suboptimal at higher levels of protein excretion. Discriminant spot urine P/Cr values reliably predicted protein excretion thresholds of 0.3, 0.5, 1.0 and 3.0 g/day. Urinalysis was a poor test for quantitating protein excretion. CONCLUSION: Random spot urinary P/Cr predicts actual 24 h protein excretion with reasonable accuracy in patients with lower levels of protein excretion but is unreliable in patients with high protein excretion and should not be used in the clinical setting unless 24 h urine collection is unavailable.

Optimized parameters for fluorescence-based verification of ballast water exchange by ships.

Mid-ocean ballast water exchange is mandatory for ships discharging foreign ballast in US territorial waters in order to reduce the risk of biological invasions. However, a reliable tool for determining whether the procedure took place is lacking. We investigated chromophoric dissolved organic matter (CDOM) fluorescence as a tracer of mid-ocean exchange on nine research cruises out of Asia, Europe, and the USA, focusing on challenging source conditions (high salinity, low CDOM). Using parallel factor analysis, we identified nine independent fluorescent components present in varying concentrations in the ocean and in ballast water. One component was sufficient for predicting the coastal vs oceanic source of most ballast water samples. Across nine cruises, thresholds (1.7 and 0.7 ppb quinine sulfate equivalent units) at two fixed wavelength pairs (lambda(ex)/lambda(em) = 320/414 and 370/496 nm, respectively) discriminated coastal from oceanic ballast water in > 95% of samples (N = 514). Our results suggest that single- and dual-channel fluorometers could be optimized for verifying ballast water exchange.

Syndromic surveillance for influenzalike illness in ambulatory care network.

Conventional disease surveillance mechanisms that rely on passive reporting may be too slow and insensitive to rapidly detect a large-scale infectious disease outbreak; the reporting time from a patient's initial symptoms to specific disease diagnosis takes days to weeks. To meet this need, new surveillance methods are being developed. Referred to as nontraditional or syndromic surveillance, these new systems typically rely on prediagnostic data to rapidly detect infectious disease outbreaks, such as those caused by bioterrorism. Using data from a large health maintenance organization, we discuss the development, implementation, and evaluation of a time-series syndromic surveillance detection algorithm for influenzalike illness in Minnesota.

Effects of lowering the legal BAC to 0.08 on single-vehicle-nighttime fatal traffic crashes in 19 jurisdictions.

BACKGROUND: In the past few decades, numerous policies, including those that lower legal blood alcohol concentration limits, have been enacted to reduce alcohol-impaired driving. In the US, 41 states and the District of Columbia have enacted 0.08 per se laws, which specify that if a driver's BAC is at or above 0.08, a violation has occurred even if the driver does not show signs of intoxication. OBJECTIVE: We examined effects of lowering the blood alcohol concentration limit to 0.08 per se on fatal traffic crashes in 18 states and the District of Columbia, and whether effects of the law varied by state or by baseline rates of fatal traffic crashes. METHOD: Data on fatal traffic crashes were obtained from the Fatality Analysis Reporting System, including all states that enacted 0.08 per se prior to 2001 in the contiguous United States. Effects of the 0.08 law were examined in each state separately, and the overall effect across states was examined using a mixed-model Poisson regression on single-vehicle-nighttime fatal traffic crashes. RESULTS: State-specific analyses showed that fatal traffic crashes significantly decreased in three of the 19 states following the introduction of the 0.08 law, prior to adjusting for potential confounders. The mixed-model regression showed a statistically significant 5.2% reduction in single-vehicle-nighttime fatal traffic crashes associated with the 0.08 law across all states, after adjusting for administrative license revocation, the number of Friday and Saturday nights in a month, and trends in all other types of fatal traffic crashes. Findings indicate that the effect of the 0.08 law does not vary significantly by state or baseline rate of fatal traffic crashes in a state, and no significant statistical interaction exists between 0.08 and administrative license revocation policy effects.