Two models, the Help with the Assessment of Adenopathy in Lung cancer (HAL) and Help with Oncologic Mediastinal Evaluation for Radiation (HOMER), were recently developed to estimate the probability of nodal disease in non-small cell lung cancer (NSCLC) patients as determined by endobronchial ultrasound-transbronchial needle aspiration (EBUS-TBNA). The objective of this study was to prospectively externally validate both models at multiple centers.Are the HAL and HOMER models valid across multiple centers?This multicenter prospective observational cohort study enrolled consecutive patients with PET-CT clinical-radiographic stage T1-3, N0-3, M0 NSCLC undergoing EBUS-TBNA staging. HOMER was used to predict the probability of N0 vs. N1 vs. N2 or N3 (N2|3) disease and HAL was used to predict the probability of N2|3 (vs N0 or N1) disease. Model discrimination was assessed using the area under the receiver operating characteristics curve (ROC-AUC) while calibration was assessed using the Brier score, calibration plots, and the Hosmer-Lemeshow test.Thirteen centers enrolled 1,799 patients. HAL and HOMER demonstrated good discrimination: HAL ROC-AUC=0.873 (95%CI 0.856-0.891) and HOMER ROC-AUC=0.837 (95%CI 0.814-0.859) for predicting N1 disease or higher (N1|2|3) and 0.876 (95%CI 0.855-0.897) for predicting N2|3 disease. Brier scores were 0.117 and 0.349 respectively. Calibration plots demonstrated good calibration for both models. For HAL the difference between forecast and observed probability of N2|3 disease was +0.012; for HOMER the difference for N1|2|3 was -0.018 and for N2|3 was +0.002. The Hosmer-Lemeshow test was significant for both models (p=0.034 and 0.002) indicating a small but statistically significant calibration error.HAL and HOMER demonstrated good discrimination and calibration in multiple centers. Although calibration error was present, the magnitude of the error is small, such that the models are informative.
Gabriela Martinez-Zayas, Francisco A Almeida, Lonny Yarmus, Daniel Steinfort, Donald R Lazarus, Michael J Simoff, Timothy Saettele, Septimiu Murgu, Tarek Dammad, D Kevin Duong, Lakshmi Mudambi, Joshua J Filner, Sofia Molina, Carlos Aravena, Jeffrey Thiboutot, Asha Bonney, Adriana M Rueda, Labib G Debiane, D Kyle Hogarth, Harmeet Bedi, Mark Deffebach, Ala-Eddin S Sagar, Joseph Cicenia, Diana H Yu, Avi Cohen, Laura Frye, Horiana B Grosu, Thomas Gildea, David Feller-Kopman, Roberto F Casal, Michael Machuzak, Muhammad H Arain, Sonali Sethi, George A Eapen, Louis Lam, Carlos A Jimenez, Manuel Ribeiro, Laila Z Noor, Atul Mehta, Juhee Song, Humberto Choi, Junsheng Ma, Liang Li, David E Ost