Use of outcome events in surgical trials: a systematic review.

BACKGROUND: Surgical trials sometimes fail to clearly identify the primary outcome events of interest. This results in trials that are diffuse and difficult to interpret. OBJECTIVE: The objective of this study was to systematically review the use of outcome events in surgical trials. DATA SOURCES: Surgical trials published between 1 January 2007 and 30 June 2010 in 26 peer-reviewed journals representing a wide range of specialty interests were used in this study. REVIEW METHODS: Copies of all potentially relevant articles were scrutinized to identify the admissible surgical trials. Two investigators experienced in health research methods used a standardized form to extract discrete information (i.e. it was an 'identifying and counting' exercise that did not require subjective evaluations). All forms were double-checked. RESULTS: Twenty-four per cent (130 out of 531) of the trials failed to declare the primary outcome events - 11% (56 out of 531) of the trials indicated the primary outcome events in the abstract, but not in the body of the article. The compliant trials used a median of three primary outcome events (interquartile range: 2-5, absolute range: 1-17), and a median of 19 statistical comparisons (interquartile range: 9-32, absolute range: 1-130). Only 2% (11 out of 531) of the trials made an adjustment for the multiple testing of statistical significance (9 of these trials declared a single primary outcome event). Composite outcome events appeared in 9% (48 out of 531) of the trials and these studies contained a median of 24 statistical comparisons. CONCLUSIONS: Many surgical trials fail to clearly define the specific outcome events of interest, and this is often accompanied by a subversive number of statistical comparisons.

Reporting of minimum clinically important differences in surgical trials.

BACKGROUND: The minimum clinically important difference (MCID) is the smallest difference in outcome between the groups that would be of clinical interest. It influences the estimates that are made to determine the required sample side. The aim of this study was to explore the reporting of the MCID in surgical trials. METHOD: Surgical trials that were published between January 1981 and December 2006 in five prestigious surgical journals were evaluated. Selected for study were trials that studied two groups and reported the main outcome event as a proportion. RESULTS: Only 21% (100/486) of the admissible surgical trials mentioned a value for the MCID when estimating the sample size. There was a trend, however, for compliance with these factors to increase during the study period. The present post-hoc calculations of the required sample size, which were based on the observed differences between the groups at the end of the study, suggested that one-third of the trials should have accrued at least fivefold the number of patients. Although reporting an estimate of the sample size was associated with the study of more patients (median sample size 145 vs 100), it was not associated with the reporting of more positive results, that is, 61% (95/155) versus 65% (214/331). CONCLUSION: There has been an improvement in the proportion of surgical trials reporting formal estimates of sample size during the last three decades. But the construct of these estimates is often suspect because of a failure to provide realistic values for the MCID.

Emergence of 'retropro' studies in the surgical literature.

BACKGROUND: The surgical published work now contains several studies that involve retrospective evaluations of prospectively collected data. These types of studies - that are referred to as 'RetroPro' studies - have the potential to be a valuable source of information about the care of surgical patients. The aim of this study was to evaluate the role of RetroPro studies in the surgical published work. METHODS: Original articles from 21 surgical journals were selected for study. Rejected from study were case reports, short communications, editorials, and surgical narratives (history, opinions and orations). Each original article was evaluated to determine the source of the study, the topic under evaluation, the sample size, the study design and the use of statistics. RESULTS: Retrospective studies accounted for 34% of the 1386 original articles. The other main components were single-group prospective studies (20%), non-clinical experimental studies (13%), RetroPro studies (10%), review articles (9%) and clinical trials (5%). Almost one-third of the RetroPro studies evaluated more than 1000 patients and 58% of them included some form of multivariable regression or an analysis of survival; that is, they attempted to model the outcome of surgical patients. CONCLUSION: RetroPro studies are now an important source of information about the outcome of surgical patients. They should be recognized as a specific form of clinical investigation.

Evolution of methodological standards in surgical trials.

BACKGROUND: The Consolidated Standards of Reporting Trials (CONSORT) Statement outlines acceptable ways of performing and reporting clinical trials. The objective of the present study was to identify evolving patterns in the methodological standards of surgical trials. METHODS: Compliance with 12 key standards from the CONSORT statement were evaluated in 490 trials published in either the ANZ Journal of Surgery or the British Journal of Surgery between January 1969 and December 2003. RESULTS: There has been an irregular but progressive improvement in the methodological standards of published trials. The criteria with the greatest improvement related to estimation of sample size, randomization, concealment of the allocated intervention, baseline comparisons, and the method of expressing outcomes. Compliance rates were <50% for three criteria during the last decade of the review, that is, concealment of the allocated intervention, blindness of assessment, and the method of expressing outcomes. CONCLUSION: The results of surgical trials need to be interpreted with care.

The measurement of wound infection after breast surgery.

Infection in breast wounds often occurs in the form of cellulitis, but the conventional criteria for wound infection are the presence of either pus or a serous discharge containing pathogens. Wound scoring systems may offer a more quantitative and clinically relevant approach when evaluating the morbidity caused by infection in wounds. The aim of this study was to develop a wound scoring system for patients undergoing breast surgery. The components of previously described wound scoring systems were measured in 218 women undergoing nonreconstructive breast surgery. Using conventional criteria, the incidence of wound infection was 5.5% (12/218). However, 22% of the patients exhibited at least one sign of wound infection, and 11% of the patients received antibiotics for cellulitis without any other indication of a wound infection. The validity of the wound scoring system was supported by its strong content validity, the presence of construct validity as evidenced by concordance with the conventional criteria (p < 0.001), and criterion validity in the form of an association between the presence of a seroma and a positive wound score (p < 0.001). It was also noted that 27% of the patients with an appreciable wound score (more than 40 points) had cellulitis but did not satisfy the conventional criteria for a wound infection. In conclusion, it is advisable to use a wound scoring system that includes cellulitis when evaluating patients who have undergone breast surgery.

Randomization in surgical trials.

BACKGROUND: There are concerns that use of the term "randomized" conveys a form of legitimacy to surgical trials that may sometimes be inappropriate. The objective of this study was to review the nature and use of randomization techniques in surgical trials. METHODS: We evaluated aspects of the randomization process in 619 surgical trials published within 10 prestigious journals between January 1990 and December 1999. RESULTS: Only 33% of the published trials (202/619) adequately described a valid randomization process. Furthermore, 78% (484/619) did not declare the use or extent of a blinding technique and almost two-thirds of the published trials failed to state how they concealed the randomization process. CONCLUSIONS: Our study indicates that many published surgical trials ignore basic aspects of the randomization process. It is difficult for surgeons to have faith in trials that fail to demonstrate an unbiased allocation of patients and ignore the need to maintain some confidentiality about the allocation of patients into groups.

Baseline comparisons in surgical trials.

INTRODUCTION: Baseline measurements are an essential component of clinical trials. They should establish that the groups involved are clinically equivalent so that any observed differences can be attributed to the intervention under evaluation. The objective of this study was to review the use of baseline comparisons in surgical trials. METHODS: A standardized format was used to evaluate 206 surgical trials published within 10 prestigious journals between January 1997 and December 1999. RESULTS: One hundred and fifty-one (73%) of the trials used the first table in the article to record baseline data. More than one-quarter of the trials declared less than five items and one-third of the trials inappropriately used 'P-values' as a measure of baseline equivalence. Only nine of the 54 multicentre studies (17%) mentioned the individual centres as possible confounding factors. DISCUSSION: Greater attention needs to be paid to baseline comparisons in surgical trials. One way of developing this type of critical ability is to use a checklist while reading a surgical trial of interest. Good resources include the CONSORT Statement by a group of prominent international biostaticians and the CLEAR (Critical Literature Evaluation and Research) Courses run by the Royal Australasian College of Surgeons.