The content of this website is intended for healthcare professionals only

Clinical trials ‘spun’ to underplay side-effects

Study reveals ‘bias’ in the way papers are presented

Jo Carlowe

Thursday, 10 January 2013

Many published cancer blished studies on studies are biased and ‘spin’ the way the findings are presented.

This is the finding from new research published this week in Annals of Oncology .

In the first study to investigate how accurately outcomes and side-effects are reported in breast cancer trials, researchers at the Princess Margaret Cancer Centre and University of Toronto, Canada, found that in a third of all trials that failed to show a statistically significant benefit for the treatment under investigation, the reports focused on other, less important outcomes in order to influence positively the interpretation of the results.

In two-thirds of the reports there was bias in the way adverse effects of the treatment were reported, with more serious side-effects (those with toxicities graded as III or IV) poorly reported. This was particularly the case in trials that showed a significant benefit for the treatment under investigation. Only 32% of articles gave details of the frequency of grade III or IV toxicities in the abstract.

The authors of the study are calling for more rigorous, unbiased reporting of trial results and for the introduction of guidelines.

Professor Ian Tannock, medical oncologist and senior scientist in the Division of Medical Oncology and Hematology at the Princess Margaret, who led the research, said: "Better and more accurate reporting is urgently needed. Journal editors and reviewers, who give their expertise on the topic, are very important in ensuring this happens. However, readers also need to critically appraise reports in order to detect potential bias. We believe guidelines are necessary to improve the reporting of both efficacy and toxicity."

Prof Tannock and his colleagues identified all randomised controlled, phase III clinical trials for breast cancer therapies that had been published between January 1995 and August 2011. Out of a total of 568 articles, 164 were eligible for inclusion in the analysis.

Trials always have a "primary endpoint" – the specific event that is measured at the end of the trial to see whether or not the given treatment works. Often it relates to overall survival: did more patients survive or live longer on the new treatment than patients on the existing standard treatment? However, there can also be "secondary endpoints"; these are additional events that are of interest to the investigators, but which the study has not been designed specifically to address. Secondary endpoints can include how much longer patients on the new treatment live without the disease progressing, spreading to other parts of the body or recurring, compared to patients on the standard treatment; what are the adverse side-effects and what is the quality of life.

Prof Tannock and his colleagues defined bias as "inappropriate reporting of the primary endpoint and toxicity, with emphasis on reporting of these outcomes in the abstract". They defined spin as "the use of words in the concluding statement of the abstract to suggest that a trial with a negative primary endpoint was positive based on some apparent benefit shown in one or more secondary endpoints".

They found that 33% trials were reported as positive, based on secondary endpoints, despite not finding a statistically significant benefit in the primary endpoint.

They found that 58% of 92 trials that showed no benefit for patients from the experimental therapy (negative primary endpoint) used secondary endpoints to suggest benefit from the treatment.

A total of 67% of papers reported adverse side-effects of the experimental therapy in a biased manner. If a trial showed a benefit for the treatment (positive primary endpoint), then toxicities were more likely to be under-reported.

Dr Francisco Vera-Badillo, clinical research fellow at the Princess Margaret, said: "We found a high incidence of biased reporting of the outcomes of clinical trials. In those with outcomes that were either negative or did not show a statistically significant benefit, spin was used frequently to influence positively the interpretation of the results, by focusing on apparent benefits from secondary endpoints.

"Where trials showed a positive outcome, the toxicities were less likely to be reported. A possible explanation for this could be that the investigators, sponsors or both, prefer to focus on the efficacy of the experimental treatment and downplay toxicity to make the results look more attractive."

Registered in England and Wales. Reg No. 2530185. c/o Wilmington plc, 5th Floor, 10 Whitechapel High Street, London E1 8QS. Reg No. 30158470