Prognostic Models for Breast Cancer: A Systematic Review and Validation Using New Zealand Data

Abstract

Background Breast cancer is the most common cancer in women worldwide, with a great diversity in outcomes among individual patients. The ability to accurately predict a breast cancer outcome is important to patients, physicians, researchers, and policy makers. Many models have been developed and tested in several populations. They vary in terms of predictors, outcomes, methods of development and/or validation, and performance in different settings. However, there has been a limited number of systematic reviews of prognostic models for breast cancer. It is suggested that, unless a model is only to be applied to the population from which it is derived from, it should be validated before being applied to a new population. Although breast cancer is the number one cancer in New Zealand women, and Māori have one of the highest incidences of breast cancer in the world, there has been a limited number of validation studies in New Zealand patients, and importantly, no validation in Māori patients. Aims This thesis aims: • to review published prognostic models for breast cancer outcomes, including mortality and recurrence, and • to test the performance of two models, the Nottingham Prognostic Index (NPI) and PREDICT version 2.0, in New Zealand patients, especially in the subgroup of Māori patients. Methods • Systematic review We conducted a systematic search in EMBASE, PUBMED, Web of Science, COCHRANE, and some specific breast cancer and oncology websites, supplemented by a manual search in the bibliographies of the included studies, to identify original studies that were published prior to 1st January 2017, and presented the development and/or validation of models based mainly on clinico-pathological factors to predict mortality and/or recurrence in female breast cancer patients. We excluded models that were developed for specific breast cancer subgroups or those based mainly on molecular factors. For each included study, we extracted detailed information about the study population, outcomes, predictors, methods used for development and/or validation, strengths and weaknesses. We also assessed the risk of bias within individual studies. • Model validation We validated PREDICT v2.0 and the NPI in newly diagnosed primary invasive breast cancer women in the Waikato District Health Board area between 01/01/2000 to 30/06/2014. We extracted relevant information from the Waikato Breast Cancer Registry. For PREDICT v2.0, we tested calibration for 10-year breast cancer-specific mortality, nonbreast cancer mortality, and total mortality by comparing the predicted and observed outcomes in the total study sample using ttests. For breast cancer-specific mortality, we also assessed calibration within strata of variables, tested discrimination using receiver operating characteristic (ROC) coupled with corresponding area under the curve (AUC), and assessed goodness-of-fit within each quintile of predicted risk using a calibration plot and a chi-square ( 2) test (5 degrees of freedom). We also conducted a subgroup analysis in Māori patients and a sensitivity analysis in patients diagnosed since 2006, when human epidermal growth factor receptor 2 (HER2) tests became a common practice in New Zealand. All analyses were conducted separately for oestrogen receptor (ER) negative and positive patients. For the NPI, we assessed discrimination using Kaplan-Meier survival curves and the log-rank test. The NPI was tested using the total study sample and the Māori patient subgroup. We also compared the mortality predicted by PREDICT v2.0 between the three NPI prognostic groups using box plot graphs and one-way ANOVA. Within each NPI prognostic group, we compared the predicted and observed mortality using ttests. Results • Systematic review From the 96 studies included in this review, we identified 58 models, which predicted mortality (n=28), recurrence (n=23), or both (n=7). The most frequently used predictors were nodal status (n=49), tumour size (n=42), tumour grade (n=29), age at diagnosis (n=24), and ER status (n=21). Models were developed between 1982 and 2016, mostly in Europe (n=25), followed by Asia (n=13), North America (n=12), Australia (n=1), but none in New Zealand. Models were validated in the same populations that were used for model development (n=43) and/or in independent populations (n=17), by comparing the predicted outcomes with the observed outcomes (n=55) and/or with the outcomes predicted by other models (n=32), or by individual prognostic factors (n=8). Cox proportional hazards regression was mainly used for model development (n=32), whereas calibration plots (n=27) and the absolute differences between the predicted and the observed outcomes (E-O) (n=30) were most commonly used for model calibration, and C-index/AUC (n=44) were most commonly used for model discrimination. The NPI, Adjuvant!Online, PREDICT v1.3, and CancerMath were used by independent researchers for comparisons with other models. In general, the models performed well in the original populations used for model development, but less accurately in some independent populations, particularly in patients with high risk, in young and elderly patients. An exception is the NPI, which retains its predicting ability in most of the externally validated cohorts. However, no model appeared to be superior to another in the studies that directly compared the models in independent settings by independent investigators. • Model validation In 398 ER negative patients included in the validation of PREDICT v2.0, E-O were 7% (P=0.3603) for total mortality, 14% (P=0.1638) for breast cancer-specific mortality, and -12% (P=0.4739) for non-breast cancer mortality. In 2,213 ER positive patients included, E-O were -7% (P=0.1734) for total mortality, -14% (P=0.0411) for breast cancer-specific mortality, and 1% (P=0.8710) for non-breast cancer mortality. In most of the pre-defined subgroups, E-O for breast cancer-specific mortality were not significantly different. AUC were 0.7649 for ER negative patients, and 0.9011 for ER positive patients. Goodness-of-fit tests showed 2=2.72 (P=0.7438) for ER negative patients and 2=10.26 (P=0.0683) for ER positive patients. There was a slight underestimation of breast cancer-specific mortality in the highest risk group of ER positive patients. The results in 1,666 patients included for the sensitivity analysis were similar to those in the main analysis. In the 423 Māori patients included, the model showed good calibration, discrimination, and goodness-of-fit for breast cancer-specific mortality, but it did not accurately predict non-breast cancer mortality. In 2,691 patients included in the validation of the NPI, 2 ranged from 225.44 to 233.95 (P=0.0000). In 442 Māori patients included, 2 ranged from 38.07 to 40.75 (P=0.0000). The mean 10-year breast cancer-specific mortality predicted by PREDICT v2.0 was significantly different between the three NPI prognostic groups, although the ranges of prediction overlapped. Within each NPI prognostic group, the range was large, especially in the poor prognostic groups. The differences between the observed mortality and the mortality predicted by PREDICT v2.0 in all three NPI prognostic groups were not statistically significant, for total mortality, breast cancer-specific mortality, and non-breast cancer mortality. Conclusions Many prognostic models have been developed for breast cancer, but only a few have been validated widely in different settings. Importantly, their performance was suboptimal in independent populations and in patients with high risk, in young and elderly patients. The NPI showed good discrimination in New Zealand patients and in Māori patients. PREDICT v2.0 accurately predicted 10-year breast cancer-specific mortality, non-breast cancer mortality, and total mortality in New Zealand patients, although it slightly underestimated breast cancer-specific mortality in the highest risk group of ER positive patients. In Māori patients, PREDICT v2.0 accurately predicted breast cancer-specific mortality but significantly underestimated non-breast cancer mortality.