Validating the ACS-NSQIP® Surgical Risk Calculator in microvascular breast reconstruction
Anne C. O'Neill, MBBCh PhD, Shaghayegh Bagher, MSc, Marina Barandun, MD, Stefan OP Hofer, MD PhD, Toni Zhong, MD MPH.
University Health Network University of Toronto, Toronto, ON, Canada.
The American College of Surgeons National Surgery Quality Improvement Program (ACS NSQIP®) Surgical Risk Calculator is an open access online tool that estimates the risk of adverse post-operative events. This study evaluates the predictive value of the ACS NSQIP® calculator in patients undergoing microvascular breast reconstruction.
Details of 759 microvascular breast reconstructions in 515 patients were entered into the online calculator. The predicted rates of post-operative complications were recorded and compared to observed complications identified on chart review. The calculator was validated using three statistical measures described in the original development of the ACS NSQIP® model.
The calculator predicted that complications would occur in 9.9% of breast reconstructions while the observed rate was 10.5%. Hosmer-Lemeshow test did not find any statistical difference between these rates (p=0.69) indicating that the calculator accurately measured what is was intended to measure. The area under the curve or c-statistic (measure of discrimination, was found to be low at 0.548, suggesting that the model has random performance in this patient population. The Brier score was found to be higher than that reported in the original ACS calculator model development (0.094 vs 0.069) demonstrating poor correlation between predicted probability and actual probability. These scores indicate that the calculator cannot effectively discriminate between patients who are at risk of complications and those who are not.
This study demonstrates that the ACS NSQIP® Universal risk calculator does not maintain its predictive value in microvascular breast reconstruction, illustrating the importance of validation of universal calculators in individual patient populations.
Back to 2015 Annual Meeting Posters