Half of clinical trial abstracts in top psychiatry journals contain ‘spin’

Author: Ingrid Torjesen
Half of clinical trial abstracts in top psychiatry journals contain ‘spin’

"Spin", where the clinical significance of a particular treatment is exaggerated without the statistics to back it up, occurs in more than half of clinical trial abstracts published in top psychology and psychiatry journals, a review* in BMJ Evidence Based Medicine has found.

The findings raise concerns about the potential impact this might be having on treatment decisions, as the evidence suggests that abstracts alone are capable of influencing doctors.

To find out how common "spin" might be in abstracts, the researchers trawled the research database PubMed for randomised controlled trials of psychiatric and behavioural treatments published between 2012 and 2017 in six top psychology and psychiatry journals.

They reviewed only those trials (116) in which the primary results had not been statistically significant, and used a previously published definition of "spin" to see how often researchers had "spun" their findings.

They found evidence of "spin" in the abstracts of more than half (65; 56%) of the published trials. This included titles (2%), results sections (21%), and conclusion sections (49%). In 17 trials (15%), "spin" was identified in both the results and conclusion sections of the abstract.

"Spin" was more common in trials that compared a particular drug/behavioural approach with a placebo intervention or usual care.

Industry funding was not associated with a greater likelihood of spinning the findings: only 10 of the 65 clinical trials in which "spin" was evident had some level of industry funding.

The researchers said: “Researchers have an ethical obligation to honestly and clearly report the results of their research. Adding spin to the abstract of an article may mislead physicians who are attempting to draw conclusions about a treatment for patients. Most physicians read only the article abstract the majority of the time.”

They added: “Those who write clinical trial manuscripts know that they have a limited amount of time and space in which to capture the attention of the reader. Positive results are more likely to be published, and many manuscript authors have turned to questionable reporting practices in order to beautify their results.”

Professor Michael Sharpe, professor of psychological medicine, University of Oxford, said that experience strongly suggests that this problem is widespread and not specific to the journals or indeed to areas of research studied here.

“These finding raise a major concern, especially as readers may draw conclusions on the basis of the abstract alone, without critically appraising the full paper. Not actually reading the paper is a major and increasing problem, even in academic circles, as everyone is deluged with information. This study tells us that trying to cope with that deluge by just reading the abstract may be a mistake.”

Professor Kevin McConway, emeritus professor of applied statistics, The Open University, said that similar studies of "spin" in medical research papers have been carried out in a range of specialist areas.

“Generally, like this new study, they found that ‘spin’ was present in alarmingly large proportions of the papers that were investigated,” he said.

“Quite often the decision to reject a paper, for a particular journal, is taken on the basis of the abstract alone. Researchers know that, and that may increase the pressure to make the abstract impressive enough to get beyond that stage. This problem of ‘spin’ is not going to be easy to fix.”

*Jellison S, Roberts W, Bowers A, et al. Evaluation of spin in abstracts of papers in psychiatry and psychology journals. BMJ Evidence-Based Medicine. Published Online First: 05 August 2019. DOI: 10.1136/bmjebm-2019-111176