The content of this website is intended for healthcare professionals only

Health professionals vary hugely in disability assessments

High variability and often low reliability in medical evaluations of work disability

Louise Prime

Thursday, 26 January 2017

Healthcare professionals vary hugely in how they assess people’s eligibility for disability benefits, even the same claimant, international research has revealed. The researchers behind the ‘disconcerting’ review,* published online today by The BMJ, found there was better reproducibility when experts used a standardised evaluation procedure, and they have called for substantial and urgent investment in research to improve assessment of disability.

The team of researchers from Switzerland, Canada and the Netherlands pointed out that about half of all disability claims – by people seeking wage-replacement benefits because they are unable to work as a result of a disabling illness or injury – are declined based on evaluations by medical professionals. They explored agreement among healthcare professionals assessing eligibility for work disability benefits, by conducting a systematic review of 23 published observational studies, conducted from 1992-2016 in insurance or research settings across the world.

They found that overall inter-rater reliability was much poorer in studies conducted in insurance settings than in those done in research settings. In insurance settings, inter-rater reliability was poor in six studies (37%) and excellent in only two (13%); whereas five of the seven (71%) studies in research settings achieved excellent inter-rater reliability.

The researchers also reported that reliability between professionals making assessments was higher when the evaluation was guided by a standardised instrument.

They pointed out that there has always been concern that patients’ own doctors might find it difficult to make impartial assessments of their ability to work – and their own findings suggested that medical experts were more likely than treating physicians to conclude that claimants were capable of working. They said: “Claimant lawyers and patients’ organisations have raised concerns that experts who are paid to assess claimants for insurers might feel pressure to render opinions that favour the referral source.”

They said their findings were “disconcerting”, and argued that we need urgent and substantial investment in research to improve the reliability of methods for evaluating disability, as well as improved knowledge of individual factors that contribute to variability in evaluation of people’s capacity to work.

They concluded: “Despite their widespread use, medical evaluations of work disability show high variability and often low reliability. Use of standardised and validated instruments to guide the process could improve reliability. There is an urgent need for high quality research, conducted in actual insurance settings, to explore promising strategies to improve agreement in evaluation of capacity to work.”

* Barth J, de Boer WEL, Busse JW, et al. Inter-rater agreement in evaluation of disability: systematic review of reproducibility studies. BMJ 2017;356:j14 DOI: 10.1136/bmj.j14

Registered in England and Wales. Reg No. 2530185. c/o Wilmington plc, 5th Floor, 10 Whitechapel High Street, London E1 8QS. Reg No. 30158470