The content of this website is intended for healthcare professionals only

Concern over huge diagnostic variability for melanoma

Diagnoses neither reproducible nor accurate for samples in middle of disease spectrum

Louise Prime

Thursday, 29 June 2017

Diagnoses spanning moderately dysplastic nevi to early stage invasive melanoma were neither reproducible nor accurate, according to the results of a large study of pathologists in the US. Authors of the study,* published today by The BMJ, have called for efforts to improve clinical practice such as using a standardised classification system, acknowledging uncertainty in pathology reports, and developing tools such as molecular markers to support pathologists’ visual assessments. They also urged doctors to share with their patients that the practice of medicine can be inherently uncertain.

A US research team said that although previous studies have suggested high levels of diagnostic disagreement among pathologists when interpreting melanocytic lesions, results have been conflicting. They set out to quantify the accuracy and reproducibility of pathologists’ diagnoses of melanocytic skin lesions. They grouped 240 skin biopsy cases – each of which had been independently reviewed by a panel of three experienced skin pathologists, who reached a consensus reference diagnosis – into sets of 36 or 48. They then randomised pathologists from 10 US states to independently interpret the same set on two occasions (phases 1 and 2), at least eight months apart.

They condensed pathologists’ interpretations into five classes: I (e.g. nevus or mild atypia); II (e.g. moderate atypia); III (e.g. severe atypia or melanoma in situ); IV (e.g. pathologic stage T1a (pT1a) early invasive melanoma); and V (e.g. ≥pT1b invasive melanoma). They assessed reproducibility by both intraobserver and interobserver concordance rates, and accuracy by concordance with three reference diagnoses.

They reported that they found highest levels of accuracy for class I mild lesions (92%) and class V high stage invasive melanoma (72%). But in contrast, interpretations for cases in the middle of the disease spectrum had noticeably lower accuracy, with fewer than half of the diagnoses in concordance with the reference diagnosis; class II moderately atypical lesions (25%); class III severely atypical lesions and melanoma in situ (40%); and class IV early stage invasive melanoma (43%).

Furthermore, pathologists’ interpretations of the same case on two occasions also lacked reproducibility for cases in the middle of the spectrum.

But in her opinion** piece linked to the study, lead author Dr Joann Elmore (a professor of medicine at the University of Washington School of Medicine in Seattle) argued that the diagnostic variability that they found does not mean that the problem lies with pathologists. She said: “I think a major factor in the variability we are seeing among pathologists is the lack of carefully developed and evaluated classification systems.”

The researchers pointed out that in usual clinical practice, pathologists might be able to review more slides, obtain second opinions from colleagues, or request additional tests before making a diagnosis. Even so, they remained concerned that diagnoses ranging from moderately atypical lesions to early stage invasive melanoma were neither accurate nor reproducible. They concluded: “Efforts to improve clinical practice should include using a standardised classification system, acknowledging uncertainty in pathology reports, and developing tools such as molecular markers to support pathologists’ visual assessments.”

Dr Elmore added: “We need to acknowledge and share with our patients that the practice of medicine is, at times, inherently uncertain. We have oversold our medical capabilities, leading patients to believe (and physicians to hope) that diagnoses can be clearly binary (i.e. either you have the disease or you don’t). We need to acknowledge that vast grey areas often exist.”

* Elmore JG, Barnhill RL, Elder DE, et al. Pathologists’ diagnosis of invasive melanoma and melanocytic proliferations: observer accuracy and reproducibility study. BMJ 2017; 357: j2813. DOI: 10.1136/bmj.j2813

** Elmore JG. Opinion: When diagnostic uncertainty hits home. BMJ, published online 28 June 2017.

Registered in England and Wales. Reg No. 2530185. c/o Wilmington plc, 5th Floor, 10 Whitechapel High Street, London E1 8QS. Reg No. 30158470