Observations by a statistical watchdog.

During more than five decades, the author has kept a critical eye on how statistical methods are (mis-)used in medical research. Some areas are presented where serious statistical mistakes are prevalent. Two investigations with erroneous conclusions are described in detail. Situations where outside authorities have tried to mute medical researchers are also commented upon. The authors own efforts to improve the use of statistical methods and the current situation with easily accessible statistical program packages are described. Finally, the importance of continued 'statistical cleansing' is stressed.

Are degenerative spondylolisthesis and further slippage postoperatively really issues in spinal stenosis surgery?

Background and aims Opinions diverge concerning the prognostic importance of preoperative degenerative spondylolisthesis in patients with lumbar spinal stenosis, as well as the significance of further slippage post-operatively following decompression alone. However, a slip is only one among several factors related to the topic, e.g. duration and intensity of back and leg pain, pre-operative walking ability, number of levels operated and not least the experience of the surgeon. Our aim was to take all of the above-mentioned factors into consideration when analysing the patients' clinical outcome, reported as Change in back pain, Change in leg pain, Overall satisfaction and Change in walking ability, with special emphasis on the possible importance of pre- and/or post-operative degenerative spondylolisthesis. Methods We studied 200 consecutive patients, mean follow-up time 81 months (range 62-108). Before treatment and on the follow-up occasion all patients answered the SF-36 questionnaire and assessed their back and leg pain on a visual analogue scale (VAS). At follow-up the patients were asked about possible changes in back and leg pain (completely free, much better, somewhat better, unchanged, somewhat worse, much worse) and whether they were; satisfied with the outcome, in doubt or not satisfied. Before treatment and at follow-up the presence or not of degenerative spondylolisthesis was determined in the lateral view on a plain X-ray or MRI. By use of a microsurgical technique decompression was achieved in all patients by bilateral laminotomy not sparing the midline ligaments, irrespective of a degenerative spondylolisthesis or not. Eight surgeons with different surgical experience performed the operations. Four separate multivariate analyses were conducted, one for each clinical outcome. The Lasso method was used for variable selection and multiple imputation was applied to handle missing values. Results At follow-up 78.5% of the patients were completely satisfied with the outcome. Minimal clinical important difference (MCID) was achieved for 69% of the patients. Before surgery 28 patients were able to walk more than 1 km compared to 111 at follow-up. The reoperation rate at 6.8 years was 12% further decompressions and 2.5% fusions at the index level. Post-operative slippage was equally common in patients with and without a preoperative slip (around 30%). There were no notable differences in outcome in patients with and without a preoperative slip and no effect of further slippage at the index or another level post-operatively. Nor could the statistical analysis show any of the other covariates (age, gender, duration and intensity of back and leg pain, pre-operative walking ability or number of levels operated) to be of statistically significant importance for predicting the outcome. In the univariate statistical analysis differences were found between the patients of individual surgeons regarding satisfaction, pain improvement, and reoperation rates in favour of surgical experience, which were, however, not statistically significant in the multivariate analysis. Conclusions None of the covariates, including pre-operative spondylolisthesis and further slippage post-operatively, were statistically significant for predicting the clinical outcome. Implication Our results provide no evidence for adding fusion to the decompression.

Superior outcomes following cervical fusion vs. multimodal rehabilitation in a subgroup of randomized Whiplash-Associated-Disorders (WAD) patients indicating somatic pain origin-Comparison of outcome assessments made by four examiners from different disciplines.

BACKGROUND AND AIMS: Whiplash-Associated Disorders (WAD) are characterized by great variability in long-term symptoms. Patients with central neck and movement-induced stabbing pain participated in a randomized study comparing cervical fusion and multimodal rehabilitation. As reported in our previous paper, more patients treated by cervical fusion than by rehabilitation experienced pain relief. Although patient reported outcome measures are a core component of outcome evaluation, independent examiner has been recommended. Because of the heterogeneity of WAD complaints the patients in our study were examined at baseline and follow-up by four experts representing neurology, orthopedics, psychology and physical medicine. The aim was to compare the professional assessments of change both regarding the possible impact of the different examiners' perspectives on individual patient's outcome, and also on the analysis of possible outcome differences between the treatment groups. METHODS: WAD patients with long-term neck pain as the predominant symptom after a traffic accident were eligible. The neck pain origin should be in the midline and perceived as dull and aching, with sudden movement inducing midline stabbing pain. Of the 1,052 patients in contact with our team, 49 were eligible. The overall treatment effect was evaluated on a global outcome transitional scale. The criteria for the scale categories were defined by each expert's professional perspective on change in the whiplash complaints. Statistical methods that take account of the non-metric properties of ordered categorical data were used. Observed inter-expert disagreement was evaluated by the Svensson method that identifies and measures systematic group-related disagreement separately from disagreement caused by individual variation. Possible differences in the distributions of assessments on the expert-specific outcome scales between the treatment groups were analyzed by the Kruskal-Wallis test. RESULTS: The per-protocol evaluation showed that a majority of the 18 patients who underwent fusion surgery were assessed as somewhat or much better, ranging from 67% to 78% depending on the expert. Corresponding proportions of improvement in the 17 patients treated by multimodal rehabilitation ranged from 29% to 53%. The statistical analyses confirmed better outcomes in the patients treated by fusion surgery, with p-values ranging from 0.003 to 0.04. The experts' assessments of intra-patient change disagreed more or less for all patients. The analyses of the paired comparisons confirmed that these disagreements could most probably be explained by the different profession-specific operational definitions of the outcome scales rather than by individual variations in data. CONCLUSIONS: The multi-dimensional complexity of WAD-related complaints was comprehensively demonstrated by the inter-disciplinary disagreements in assessing intra-patient outcomes. The superiority of positive treatment effects in patients who underwent cervical fusion compared with multimodal rehabilitation was evident to all experts. IMPLICATIONS: The results strengthen our previous opinion that neck pain in this subgroup of WAD patients has a somatic origin. More than one examiner is recommended for multi-dimensional outcome assessments.

Clinical outcome following anterior arthrodesis in patients with presumed sacroiliac joint pain.

BACKGROUND: It has been reported that in 13-32% of patients with chronic low back pain, the pain may originate in the sacroiliac (SI) joints. When treatment of these patients with analgesics and physiotherapy has failed, a surgical solution may be discussed. Results of such surgery are often based on small series, retrospective analyses or studies using a minimal invasive technique, frequently sponsored by manufacturers. PURPOSE: To report the clinical outcome concerning pain, function and quality of life following anterior arthrodesis in patients presumed to have SI joint pain using validated questionnaires pre- and post-operatively. An additional aim was to describe the symptoms of the patients included and the preoperative investigations performed. MATERIAL AND METHODS: Over a 6 year period we treated 55 patients, all women, with a mean age of 45 years (range 28-65) and a mean pelvic pain duration of 9.1 years (range 2-30). The pain started in connection with minor trauma in seven patients, pregnancy in 20 and unspecified in 28. All patients had undergone long periods of treatment including physiotherapy, manipulation, needling, pelvic belt, massage and chiropractic without success, and 15 had been operated for various spinal diagnoses without improvement. The patients underwent thorough neurological investigation, plain X-ray and MRI of the spine and plain X-ray of the pelvis. They were investigated by seven clinical tests aimed at indicating pain from the SI joints. In addition, all patients underwent a percutaneous mechanical provocation test and extra-articular local anaesthetic blocks against the posterior part of the SI joints. Before surgery all patients answered the generic Short-Form-36 (SF-36) questionnaire, the disease specific Balanced Inventory for Spinal Disorders (BIS) questionnaire and rated their level of pelvic and leg pain (VAS, 0-100). At follow-up at a mean of 2 years 49 patients completed the same questionnaires (89%). RESULTS: At follow-up 26 patients reported a lower level of pelvic pain than before surgery, 16 the same level and six a higher level. Applying Svensson's method RPpelvic pain=0.3976, with 95% CI (0.2211, 0.5740) revealed a statistically significant systematic improvement in pelvic pain. At follow-up 28 patients reported a higher quality of life and 26 reported sleeping better than pre-operatively. In most patients the character of the pelvic pain was dull and aching, often accompanied by a stabbing component in connection with sudden movements. Referred pain down the leg/s even to the feet and toes was noted by half of the patients and 29 experienced frequency of micturition. CONCLUSIONS: It is apparent that in some patients the SI joints may cause long-term pain that can be treated by arthrodesis. We speculate that continued pain despite a healed arthrodesis may be due to persistent pain from adjacent ligaments. The next step should be a prospective randomized study comparing posterior fusion and ligament resection with non-surgical treatment. IMPLICATIONS: Anterior arthrodesis can apparently relieve pain in some patients with presumed SI joint pain. The problem is how to identify these patients within the low back pain group.

Symptoms and signs possibly indicating segmental, discogenic pain. A fusion study with 18 years of follow-up.

BACKGROUND: Only two out of the five existing randomized studies have reported better results from fusion surgery for chronic low back pain (CLBP) compared to conservative treatment. In these studies the back symptoms of the patients were described simply as "chronic low back pain". One possible reason for the modest results of surgery is the lack of a description of specified symptoms that might be related to a painful segment/disc, and patient selection may therefore be more or less a matter of chance. Previous prospective studies including facet joint injections and discography and eventually MRI have failed to identify patients with a painful segment/disc that will benefit from fusion surgery. PURPOSE: Our purpose was to analyse in detail the pre-operative symptoms and signs presented by patients who showed substantial relief from their back pain following spinal fusion surgery with the aim of possibly finding a pain pattern indicating segmental, discogenic pain. METHODS: We analysed 40 consecutive patients, mean age 41 years, with a history of disabling low back pain for a mean of 7.7 years. Before surgery the patients completed a detailed questionnaire concerning various aspects of their back pain, and findings at clinical examination were thoroughly noted. Monosegmental posterior lumbar interbody fusion without internal fixation was performed using microsurgical technique. Outcome was assessed at 1, 2 and 4 years after surgery and finally at 18 years, using self-reporting measures and assessment by an independent examiner. Assessment at 18 years applied the Balanced Inventory for Spinal Disorders Questionnaire and the Roland-Morris Disability Questionnaire. RESULTS: According to the independent observer's assessment at two years 27 of the 40 patients were much improved. Analysis of the pre-operative depiction of the back symptoms of this group revealed a rather uniform pattern, the most important being: dominating back pain originating in the midline of the spine, with a dull, aching character and stabbing pain in the same area provoked by sudden movements. Most patients in this group also had diffuse pain radiation of various extension down one or both legs and often bladder dysfunction with frequency. At clinical examination, localized interspinal tenderness was observed within the spinal area in question and the patient's back pain was provoked by pressure in that area and by tapping a neighbouring spinous process. At 18 years after surgery 19 patients assessed themselves as much improved. At that time 5 of them had pension due to age, 7 early pension, one worked full time and six patients part time. Eleven patients were re-operated due to defect bony healing. CONCLUSIONS: The results may suggest that the use of a detailed symptom analysis and clinical examination may make it possible to select a subgroup of patients within the CLBP group likely to have better outcome following fusion surgery. IMPLICATIONS: The next step would be to execute prospective studies and if our findings concerning back pain details and signs among CLPB patients can be confirmed this can provide for more accurate selection of patients suitable for fusion surgery.

A small group Whiplash-Associated-Disorders (WAD) patients with central neck pain and movement induced stabbing pain, the painful segment determined by mechanical provocation: Fusion surgery was superior to multimodal rehabilitation in a randomized trial.

BACKGROUND: The majority of patients suffering from a whiplash injury will recover, but some will have symptoms (Whiplash Associated Disorders, WAD) for years despite conservative treatment. Some of these patients perceive neck pain that might come from a motion segment, possibly the disc. In comprehensive reviews no evidence has been found that fusion operations have a positive treatment effect on neck pain in WAD patients. PURPOSE: Our aim was to evaluate the possibility of (a) selecting a subgroup of chronic WAD patients based on specified symptoms possibly indicating segmental pain, and (b) treating said segmental pain through fusion operation based on non-radiological segment localization. The hypothesis was that fusion operation in this selected subgroup of chronic WAD patients could alleviate perceived neck pain. METHODS: Eligible patients for the study had a traffic accident as the origin for their neck pain, and no previous neck symptoms. Neck pain should be the predominant symptom and the pain origin reported to be in the midline, being dull, aching in character and at sudden movements combined by a stabbing pain in the same area. Forty-nine patients with these specified symptoms were identified among a large number of chronic WAD patients. Those selected had pronounced symptoms for a median of around 50 months and had previously been investigated and fully treated within the ordinary healthcare system without success. No neurological abnormalities were to be found at clinical examination and no specific changes to be seen on X-ray and MRI. The patients were randomized to either cervical fusion operation or multimodal rehabilitation. By using a mechanical provocation test the level/s to be fused were identified. In all but one patient the surgery was performed anteriorly using microsurgical technique and a right-sided Smith-Pedersen approach and plate fixation. The multimodal rehabilitation at the Clinic of Medical Rehabilitation, Karolinska Hospital, Stockholm, included outpatient treatment for four days a week for six weeks and included treatment by physician, physiotherapists, occupational therapist, psychologists, social-service worker and nurses. Perceived change in neck pain was assessed using the Balanced Inventory for Spinal Disorders questionnaire at the 2-year-follow-up. RESULTS: Mean age of the patients was 38 and 40 years (surgery and rehabilitation groups, respectively), the most common type of accident being rear-end collision. At clinical examination muscle tenderness was not an outstanding sign. In most patients the mid-cervical region appeared to be the painful area but one patient localized the pain to C1. At follow-up 67% of the patients in the surgery group and 23% in the rehabilitation group assessed improvements in the ITT analysis. Corresponding proportions in the per protocol analysis were 83% and 12%, respectively. CONCLUSIONS: The results support the supposition that among patients with central neck pain for long periods of time following a whiplash injury there are some in whom the neck pain emanates from a motion segment, probably the disc, a situation suitable for fusion surgery. IMPLICATIONS: Thorough individual symptom evaluation in patients with chronic WAD may identify patients who will benefit from cervical fusion surgery.

Private and Non-Private Disc Herniation Patients: Do they Differ?

OBJECTIVES: In the 2006 yearly report from the Swedish National Register for Lumbar Spine Surgery it was claimed that international studies show obvious differences between private and non-private patients with regard to results from back surgery. Therefore our aim was to reveal such possible differences by comparing the two categories of patients at a private clinic. MATERIAL AND METHODS: The material comprises 1184 patients operated on for lumbar disc herniation during the period of 1987 to 2007. Basic pre-operative data were obtained from the medical records and follow-up was performed by a questionnaire around 5 years post-operatively. RESULTS: Small but statistically significant differences between private and non-private patients were seen pre-operatively regarding the proportions of a/ men and women in the samples, b/ those with physically demanding jobs, c/ those on sick leave and d/ those with lumbar pain. Over the years the admitted private patients had a decreasing mean duration of symptoms which was not seen in the non-private patients. No apparent differences (n.s.) were seen between the two categories of patients pre-operatively regarding age, presence and level of leg pain or the proportion who smoked. Post-operative improvement in leg and lumbar pain was very similar in private and non-private patients as was satisfaction with the results and the proportion of patients returning to work. CONCLUSION: Despite small pre-operative differences concerning some variables and a significant difference in symptom duration between private and non-private disc herniation patients, the final clinical results were very similar.

Thyroid-associated ophthalmopathy after treatment for Graves' hyperthyroidism with antithyroid drugs or iodine-131.

CONTEXT: Previous randomized trials have suggested an association between radioiodine treatment for Graves' hyperthyroidism and thyroid-associated ophthalmopathy (TAO). OBJECTIVES: The aim of the study was to compare the occurrence of worsening or development of TAO in patients who were treated with radioiodine or antithyroid drugs. DESIGN: We conducted a randomized trial (TT 96) with a follow-up of 4 yr. PATIENTS, SETTING, AND INTERVENTION: Patients with a recent diagnosis of Graves' hyperthyroidism were randomized to treatment with iodine-131 (163 patients) or 18 months of medical treatment (150 patients). Early substitution with T(4) was given in both groups. MAIN OUTCOME MEASURE: Worsening or development of TAO was significantly more common in the iodine-131 treatment group (63 patients; 38.7%) compared with the medical treatment group (32 patients; 21.3%) (P < 0.001). RESULTS: The risk for de novo development of TAO was greater in patients treated with iodine-131 (53 patients) than with medical treatment (23 patients). However, worsening of TAO in the 41 patients who had ophthalmopathy already before the start of treatment was not more common in the radioiodine group (10 patients) than in the medical group (nine patients). Smoking was shown to influence the risk of worsening or development of TAO, and smokers treated with radioiodine had the overall highest risk for TAO. However, in the group of smokers, worsening or development of TAO was not significantly associated with the choice of treatment for hyperthyroidism. CONCLUSIONS: Radioiodine treatment is a significant risk factor for development of TAO in Graves' hyperthyroidism. Smokers run the highest risk for worsening or development of TAO irrespective of treatment modality.

Perspective on Roseroot (Rhodiola rosea) studies.

Rhodiola rosea (roseroot) extract is a commercially successful product, primarily used to reduce the effect of fatigue on physical and mental performance. In this perspective we present our investigation of the most recent studies performed on human subjects. With a focus on the statistical methods we found considerable shortcomings in all but one of the studies that claim significant improvement from roseroot extract. Overall, the study designs have not been well explained. Experimental results have been confused and appear to be in some cases incorrect. Some of the conclusions are based on selected results and contradicting data have not been adequately taken into account. We point to other studies of higher quality performed on roseroot, several that found no significant effect and one that did. We conclude that the currently available evidence for the claimed effects is insufficient and that the effect of Rhodiola rosea is in need of further investigation before therapeutic claims can be made.

Personality characteristics influencing determinacy of day and night blood pressure and heart rate.

The evidence of emotional impacts on day and night blood pressure (BP), heart rate (HR) and non-dipping of BP remains fragmentary. Personality traits previously tested by self-reports as determinants of office, screening or mean daytime BP produced mixed results. Therefore, we hypothesized that some traits are acting together as modifiers of 24-h and day/night ambulatory BP and HR. A population sample of healthy 47-54 year-old men, n = 85, unaware of their BP status, was evaluated. In multiple regression, 25% of the 24-h systolic BP (SBP) variation was explained by independent augmenting predictors Trait Anger-Anger Expression Styles, Large and Lasting Emotions, and by an attenuating predictor Anxiety. Only Verbal Aggression increased night-time SBP much more than daytime SBP and diminished day-night dipping of SBP. Strong Large/Lasting Emotions and Indirect Aggression increased daytime SBP only. Day and night HR increased with a high Suspicious Hostility (7.4% of variation), in particular with items Distrust/Vulnerability (11.5% of variation). In conclusion, mean levels of HR and BP during a 24-h period are slightly modified by particular personality traits. Some traits appear to affect either daytime or night-time periods more. Day to night carry-over effects on SBP may occur.

Master blood pressure variation in office, by self, and in sleep recordings.

Variation inherent to each session in any recording situation has to be mastered to obtain informative mean values and differences. Variation due to time trend and unsteady consecutive readings versus inter-arm variations in blood pressure (BP) in the operator-initiated, self-initiated, and sleep recordings were studied by oscillometry. They were small in operator-initiated recordings, increased in self-recordings, and were paramount in sleep. A passive supine patient and simultaneous recordings with cuffs placed in equal horizontal positions are essential conditions for the estimation of a real mean inter-arm difference. The relative variation of pulse pressure (PP), compared with other components, was largest within each situation. Most of the systolic BP (SBP)/diastolic BP (DBP) variation in sleep was due to unequal vertical cuff distances to the heart level. Only sleep PP was independent of the vertical position of cuffs. A plot of inter-arm differences can visualize systematic errors in sequences of sleep SBP/DBP. As is well known, every doubling of variation (SD) requires quadrupling the number of independent recordings in order to retain the same size of confidence interval (CI) for a mean value.

How to control measurement variation in estimating the mean intraocular pressure reducing effect at a specific point in time.

OBJECTIVE: To clarify how to control measurement variation in estimating the mean intraocular pressure reducing effect at a specific point in time. METHODS: A set of 629 open-angle glaucoma or ocular hypertensive patients, eligible for treatment in both eyes, previously untreated, were reanalyzed at 3 or 6 months of intraocular pressure (IOP) reducing treatment. The trial design had parallel treatment groups; one group receives treatment A and the other group receives treatment B. The standard deviation (SD) of the IOP response variable was calculated based on a single eye per patient, mean of right and left eye (bilaterally identical treatment within each group) or difference between right and left eye (paired-eye difference within each group; one eye receives active treatment and the other eye receives placebo). The effect of using mean of replicated recordings on the SD of the patient group mean IOP was also studied. The results were generalized by applying a measurement variation model. RESULTS: The "paired-eye difference within each group" design was clearly most effective; the SD of the IOP response variable was 2.2 mm Hg. Using mean of the right and left eye in a "design with bilaterally identical treatment in each group" was the next best solution; the SD of the IOP was 3.0 mm Hg. The SD of the IOP was 3.2 to 3.4 mm Hg when based on a single eye per patient. The minimization of the SD gave important reduction of number of patients required to enroll in a new trial. Using a single IOP recording per patient produced similar SD of the patient group mean IOP as using mean of replicated IOP recordings when measured with Goldmann tonometry. CONCLUSION: Calculating the mean IOP by using measurements from both eyes of the patient gives important reduction of the variability.

Which endpoint: mean intraocular pressure or proportion of successful patients?

A reduction in intraocular pressure in clinical trials can be determined through the mean intraocular pressure, through the proportion of patients who have the intraocular pressure reduced to a specific target intraocular pressure, or both. Since both these possible endpoints measure the shift of 2 intraocular pressure distributions, we recommend that only one of them be tested. In general, testing the difference between mean-values is much more efficient than testing the difference between proportions. However, proportions of successful patients are valuable in showing the clinical implication of a reduction in mean intraocular pressure, particularly when evaluating a moderate pressure reduction. The effect of a small mean intraocular pressure reduction on the probability to reach the target intraocular pressure is pointed out, particularly the fact that it can be substantial even if the mean reduction is smaller than the measurement error.

["P" as in Program Package. About torturing data and significant "fishing expeditions"]. / Med P:n som i programpaket. Om datatortyr och signifikansfiskeri.

For the study of prognostic factors, the medical researchers have access to a number of advanced statistical techniques available in standard program packages. A tradition has developed where survival or time to relapse is analysed on the basis of statistical materials with few patients but a large number of possible explanatory variables. In statistical "fishing expeditions" the p-values are used to sort out potentially useful prognostic variables. Since the number of observations is small, all relevant prognostic factors do not give statistical significance. Since a large number of variables are tested there is a considerable risk for spurious significances. It is not enough to show that a prognostic factor seems to be efficient in the patient group where it was first found. The result must be verified in further studies of independent groups of similar patients.