Supervised machine learning to predict non-home discharge following surgical treatment of pelvic fractures

INTRODUCTION

Over the past several decades, healthcare costs in the United States have progressively increased, and recent economic models estimate approximately 4.3 trillion dollars were spent on healthcare in 2021.¹ In response, bundled payments (BCPI) have been proposed to combat recent surges in healthcare expenditures and emphasize comprehensive, cost-effective healthcare management and services.² While such a payment model encompasses a multifarious variety of nuanced components, a fundamental element involves mitigating peri-operative cost and morbidity while simultaneously improving surgical outcomes. This initiative has made an impact on all fields of surgery, including orthopedics, in which trends aimed at attenuating healthcare costs include emphasis on outpatient surgery, standardized systems to reduce post-surgical complications, and utilization of less invasive procedures when applicable.²

Despite its utility, the implementation of bundled payments is convoluted and hindered by a variety of systemic limitations, including technological restraints. However, recent developments in artificial intelligence and machine learning (ML) represent a versatile technological conduit that providers can employ to ameliorate many of these systemic limitations by predicting surgical outcomes.^3,4 In addition to outcome prediction, the ML algorithm random forest (RF) can rank variables based on permutation feature importance (PFI), which stratifies pre-specified variables based on a quantitated, correlative power to outcomes of interest.⁵ PFI allows identification of patient characteristics, comorbidities, and variables associated with surgical complications within the RF algorithm structure.

ML can be applied toward evaluation of patients who undergo surgical fixation of pelvis fractures. Pelvis fractures are often consequences of traumatic events that necessitate surgical fixation followed by a period of rehabilitation. While the incidence of pelvic fractures is estimated to be only 3% of all adult fractures, proper medical and surgical management of patients with this injury is of paramount orthopedic importance, as associated spectrum of morbidity can include severe functional deficits, significantly diminished quality of life, and even death.⁶

While surgical fixation of pelvis fractures is amongst the most technically challenging operations performed by orthopedists, a fundamental element of mitigating peri-operative morbidity includes proper post-operative medical care and discharge planning. Pelvic fractures often result in limited mobility that increases chances of discharge to an inpatient nursing facility rather than direct discharge home. While discharge to a facility may benefit many patients, recent research demonstrates this type of discharge destination may be associated with increased risk of postoperative complications and cost.^7-9

A better understanding of the predictors for discharge destination following orthopedic pelvis surgery would permit surgeons to identify and address modifiable perioperative variables and patient characteristics associated with higher likelihood of discharge to non-home destinations. Additionally, earlier recognition may expedite the discharge process and decrease hospital length of stay. Therefore, elucidating the predictors for discharge destination after pelvic ring fixation may help attenuate post-operative morbidity, improve outcomes, and mitigate associated healthcare cost. Currently, the literature on predicting outcomes for pelvic fractures has relied on traditional statistical methods as a part of retrospective cohort studies, but ML has not yet been utilized for this purpose.^4,6

The objective of this study is to apply RF to expand upon the existing literature by validating and ranking known predictors of discharge to a facility as well as identifying the most important predictors of discharge to a facility following surgical pelvis stabilization.

METHODS

This study was brought before the institutional review board (IRB) of the study institution and determined to be IRB exempt. The study was a retrospective observational study, obtaining and analyzing patient characteristics from the American College of Surgeons national surgical quality improvement program (ACS-NSQIP), which includes over 250 variables from several hundred participating hospitals throughout the United States.

The patient sample was obtained using R studio (version 4.1.0) to filter the NSQIP database for all patients that underwent operative orthopedic intervention for pelvic ring fractures (CPT codes 27215, 27217, 27218, 27226, 27227, and 27228) from years 2008-2018. The patients were classified based on discharge destination such home discharge (“home”, “against medical advice”, and “facility which was home”) versus non-home discharge (“skilled care, not home”, “unskilled facility not home”, “separate acute care”, “rehab”). Age, body mass index (BMI), American Society of Anesthesiologists (ASA) classification, dependent functional status, smoking history, diabetes, history of congestive heart failure, dialysis, ascites, disseminated cancer, hypertension requiring medication, dyspnea, steroid use, renal failure, bleeding disorder, and recent weight loss were variables chosen for ML algorithm training.

RF algorithm was accessed using the Python package scikit learn.¹⁰ Prior to analysis, data was arbitrarily divided into training and testing datasets in a 4:1 ratio.¹¹ The algorithm was trained to predict outcomes for each variables using the predetermined testing dataset. The algorithm produced a list of the most impactful variables ranked in order of PFI. The Chi-square test was used to detect differences in categorical variables (home versus facility) and t-test for continuous variables.

The efficacy and performance of the algorithm in predicting the outcome of interest was compared to traditional metrics, including ASA, using the area under the receiver operator curve (AUC) to assess performance. An AUC of 0.7 is recognized as the cutoff for an ML algorithm with acceptable discrimination and separability.¹²

RESULTS

A total of 492 patients, mean age 48.5 years old, underwent orthopedic pelvic surgery between 2008 and 2018 in ACS-NSQIP (Table 1). Ultimately, 184 patients were discharged to a facility, while 308 patients were discharged home.

RF identified age, ASA classification, and preoperative hematocrit as top predictors for discharge to facility following pelvis trauma surgery (Figure 1 and Table 1). Patients being discharged home on average tended to be younger (43.9 years versus 56.0 years), less likely to have ASA scores 3 and above (23.7% of home discharge versus 58.2% of non-home discharge) and had higher preoperative hematocrit (37.7% versus 34.8%) compared to their counterparts with non-home discharge (p<0.05 for all). The accuracy of the algorithm was 67.4%, with an AUC of 0.74.

DISCUSSION

The recent emphasis of BCPI on cost-effective, outcome-driven healthcare has compelled providers, administrators, and industry to search for innovative solutions. This comprehensive approach to re-evaluating the contemporary standard of the American healthcare paradigm is broad, convoluted, and somewhat ethereal. However, significant progress has been achieved via paradoxically small changes to nuanced surgical domains through the incorporation of artificial intelligence. Advances to technology, including the refinement of ML, have revolutionized the ability to mine, filter, and analyze vast data sets in an unprecedented manner. Moreover, when compared to conventional statistical modeling, ML prioritizes accurate, reproducible predictive strength. In turn, the application of ML and neural networks to orthopedic surgery has demonstrated unprecedented utility for feature selection and outcome prediction, augmenting the strength of a variety of critical peri-operative domains, including pre-operative risk stratification and post-operative adverse event prevention. For example, Harris et al validated ML constructs that accurately predict renal and cardiac complications, along with mortality, after total hip and total knee arthroplasty.¹³ Another article incorporated ML to predict a patient’s propensity to achieve minimum increases in subjective knee evaluation scores after anterior cruciate ligament reconstruction.¹⁴ Additionally, another group utilized ML to accurately predict adverse post-operative outcomes after elective operative decompression for degenerative cervical myelopathy.¹⁵

Recent research has implicated discharge destination, specifically to a rehabilitation facility, as a risk factor for post-operative adverse events and increased peri-operative costs. Currently, studies on post-operative complications following pelvic fracture fixation are limited to specific adverse events. Moreover, these studies rely on traditional statistical methods like logistic regression, which provide interpretability instead of repetitive predictions. For example, one cohort study found that patients with pelvis fracture along with concomitant urinary tract infections (UTI) incurred a higher cumulative risk of both mortality and hospital readmission at 1-year post-operatively compared with the non-UTI control group.¹⁶ Another study by Malik et al, reported that patients with ASA grade >2, greater co-morbidity burden, and prolonged operative times were likely to experience adverse events and have a longer length of stay following orthopedic intervention for pelvic ring injuries.¹⁷ Therefore, application of ML to the identification of patient characteristics and peri-operative variables predictive of discharge to a facility following pelvic fixation may be associated with improved resource utilization, refined pre-operative risk stratification, and attenuated cost and post-operative adverse events.

Given the progressive recognition of ML’s ability to incorporate feature selection to predict post-operative outcomes within orthopedic surgery, we elected to determine whether RF could accurately predict discharge location following operative pelvic ring injuries. RF is among the most commonly used algorithms used to predict orthopedic surgery outcomes due to its superior ability to categorize heterogeneous data without limitations to linearly separable datasets, lending it superior discriminatory and predictive performance compared to other ML algorithms and traditional statistical analyses.^18,19 In a study by Jurgensmeier et al, RF outperformed other ML algorithms and logistic regression in predicting meniscus injury during ACL surgery as demonstrated by its low AUC.¹⁹ In general surgery, RF similarly identified variables that predicted severe sepsis and organ space infection after trauma laparotomy.³ Using patient characteristics and variables readily available in ACS-NSQIP, we were able to train and validate a random forest ML algorithm capable of accurately and efficiently predicting post-operative discharge location following orthopedic fixation of pelvic ring injuries.

In our study, patient age had the highest PFI, indicative of high predictive value within the RF algorithm. The mean age of patients discharged home was 43.9 years, while the mean age of non-home discharge was 56.0 years (p<0.05). Such findings are also seen in the orthopedic arthroplasty literature, where patients undergoing elective total hip arthroplasty were significantly more likely to be discharged to a non-home destination if they were older than 70 years of age.²⁰ Another study by Sokas et al showed higher frailty index and age were associated with higher chance of non-home discharge following various elective general and vascular surgeries.^20,21 One possible explanation discussed by those authors is the increased deconditioning and overall frailty associated with geriatric patients, which, in turn, requires additional assistance and extensive rehabilitation during the recovery process.

In our study, patients discharged home tended to be in overall better health compared to those who were discharged to a facility. For example, 76.3% of patients discharged home were ASA 1 or 2, compared to 41.8% of those discharged to a facility (p<0.05). Higher ASA scores reflect increased comorbidity burden, which can increase postoperative complications whose sequelae may require prolonged care. Even following other osseous injuries such as proximal humerus fractures, ASA scores were shown to be a strong predictor of non-home discharge.²² Other research regarding pelvis fractures such as the work by Malik et al found that patients with ASA grade >2 and greater co-morbidity burden were more likely to experience adverse events and have a longer length of stay after pelvis or acetabular surgery.¹⁷ Orthopedic surgeons who recognize patients with advanced age and higher ASA scores as potential candidates for nursing homes or rehabilitation facilities may initiate the appropriate placement protocols earlier in the hospitalization.

RF ranked preoperative hematocrit as the third most important variable to predict home discharge following pelvis surgery. Patients discharged to a facility had an average hematocrit of 34.8% versus 37.7% who were discharged home. This finding has been found among other surgeries as well. In patients undergoing lumbar decompression without fusion, a retrospective cohort study found hematocrit (<35% versus >35%, OR 1.76) to be a predictive factor of facility discharge.²³ Awareness of hematocrit as a predictor of adverse outcomes such discharge to facility should prompt close monitoring and earlier treatment of anemia in the perioperative period if deemed appropriate.

Limitations

Database studies such as ones based on the ACS-NSQIP database are retrospective in nature and are not able to establish causality between outcomes and studied variables. The variables included in the study may not be completely independent from one another, with confounding effects. Furthermore, the ACS-NSQIP database does not include patients who meet criteria for severe acute trauma. For this reason, the patients of the database tend to be older in age with more comorbidities compared to patients in dedicated trauma registries such as national trauma data bank.²⁴

CONCLUSION

RF identified age, ASA classification, and preoperative hematocrit as the strongest ranked predictors of discharge destination following orthopedic pelvis trauma surgery. Discharge to a rehabilitation facility has been shown to increase adverse outcomes following surgery and associated costs. By identifying predictors associated with facility discharge, orthopedic surgeons may better optimize patients preoperatively and streamline the discharge process for higher risk patients. ML has the capability to predict discharge to facility and associated risk factors, allowing providers to identify and correct ameliorable comorbidities, such as low hematocrit, and facilitate postoperative planning.

ACKNOWLEDGEMENTS

The authors would like to acknowledge the support of the Loma Linda University, orthopedics department and Elisabeth Clarke, Loma Linda University, orthopedics research coordinator.

Funding: No funding sources

Conflict of interest: None declared

Ethical approval: The study was approved by the Institutional Ethics Committee

REFERENCES

1. 1. Centers for Medicare & Medicaid Services. NHE fact sheet. 2015. Available at: https://www.cms.gov/data-research/statistics-trends-and-reports/national-health-expenditure-data/nhe-fact-sheet. Accessed on 09 December 2023.
2. 2. Kim JS, Iorio R. The 5 clinical pillars of value for total joint arthroplasty in a bundled payment paradigm. J Arthroplasty. 2017;32:1712-6.
3. 3. Gelbard RB, Hensman H, Schobel S, Khatri V, Tracy BM, Dente CJ, et al. Random forest modeling can predict infectious complications following trauma laparotomy. J Trauma Acute Care Surg. 2019;87(5):1125-32.
4. 4. Tseng P-Y, Chen Y-T, Wang C-H, Chiu K-M, Peng Y-S, Hsu S-P, et al. Prediction of the development of acute kidney injury following cardiac surgery by machine learning. Critical Care. 2020;24(1):478.
5. 5. Kaneko H. Cross-validated permutation feature importance considering correlation between features. Analytical Sci Advances. 2022;3(9-10):278-87.
6. 6. Hossain A, Islam S, Haque Qasem MF, Faisal Eskander SM, Hasan MT, Nahar M. Epidemiology of pelvic fractures in adult: Our experience at two tertiary care hospital in Dhaka, Bangladesh. J Clin Orthop Trauma. 2020;11(6):1162-7.
7. 7. Rosman M, Rachminov O, Segal O, Segal G. Prolonged patients’ In-Hospital Waiting Period after discharge eligibility is associated with increased risk of infection, morbidity and mortality: a retrospective cohort analysis. BMC Health Serv Res. 2015;15(1):246.
8. 8. Lavoie-Gagne O, Lu Y, MacLean I, Forlenza E, Forsythe B. Discharge Destination After Shoulder Arthroplasty: An Analysis of Discharge Outcomes, Placement Risk Factors, and Recent Trends. J Am Acad Orthop Surg. 2021;29(19):e969-78.
9. 9. Eastlack RK, Ledesma JB, Tran S, Khalsa A, Park P, Mummaneni PV, et al. Home Versus Rehabilitation: Factors that Influence Disposition After Minimally Invasive Surgery in Adult Spinal Deformity Surgery. World Neurosurg. 2018;118:e610-5.
10. 10. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion V. Scikit-learn: Machine Learning in Python. J Machine Learn Res. 2011;12:2825-30.
11. 11. Gholamy A, Kreinovich V, Kosheleva O. Why 70/30 or 80/20 Relation Between Training and Testing Sets: A Pedagogical Explanation. 2018;1209.
12. 12. David Hosmer SL, Rodney Sturdivant. Applied Logistic Regression. Third edition. John Wiley & Sons, Inc. 2013.
13. 13. Harris AHS, Kuo AC, Weng Y, Trickey AW, Bowe T, Giori NJ. Can Machine Learning Methods Produce Accurate and Easy-to-use Prediction Models of 30-day Complications and Mortality After Knee or Hip Arthroplasty? Clin Orthop Relat Res. 2019;477(2):452-60.
14. 14. Kunze KN, Polce EM, Ranawat AS, Randsborg PH, Williams RJ, Allen AA, et al. Application of Machine Learning Algorithms to Predict Clinically Meaningful Improvement After Arthroscopic Anterior Cruciate Ligament Reconstruction. Orthop J Sports Med. 2021;9(10):23259671211046575.
15. 15. Merali ZG, Witiw CD, Badhiwala JH, Wilson JR, Fehlings MG. Using a machine learning approach to predict outcome after surgery for degenerative cervical myelopathy. PLoS One. 2019;14(4):e0215133.
16. 16. Chen Y-C, Chuang C-H, Hsieh M-H, Yeh H-W, Yang S-F, Lin C-W, et al. Risk of Mortality and Readmission among Patients with Pelvic Fracture and Urinary Tract Infection: A Population-Based Cohort Study. Int J Env Res Public Health. 2021;18(9):4868.
17. 17. Malik AT, Quatman CE, Phieffer LS, Jain N, Khan SN, Ly TV. 30-day adverse events, length of stay and re-admissions following surgical management of pelvic/acetabular fractures. J Clin Orthop Trauma. 2019;10(5):890-5.
18. 18. Ogink PT, Groot OQ, Karhade AV, Bongers MER, Oner FC, Verlaan JJ, et al. Wide range of applications for machine-learning prediction models in orthopedic surgical outcome: a systematic review. Acta Orthop. 2021;92(5):526-31.
19. 19. Jurgensmeier K, Till SE, Lu Y, Arguello AM, Stuart MJ, Saris DBF, et al. Risk factors for secondary meniscus tears can be accurately predicted through machine learning, creating a resource for patient education and intervention. Knee Surg Sports Traumatol Arthrosc. 2023;31(2):518-29.
20. 20. Gordon AM, Malik AT, Khan SN. Risk Factors for Discharge to a Non-Home Destination and Reoperation Following Outpatient Total Hip Arthroplasty (THA) in Medicare-Eligible Patients. Geriatr Orthop Surg Rehab. 2021;12:2151459321991500.
21. 21. Sokas CM, Cowan J, Dalton MK, Coogan K, Bader A, Bernacki R, et al. Association between patient‐reported frailty and non‐home discharge among older adults undergoing surgery. J Am Geriatrics Soc. 2020;68(12):2909-13.
22. 22. Malik AT, Barlow JD, Jain N, Khan SN. Incidence, risk factors, and clinical impact of non-home discharge following surgical management of proximal humerus fractures. Shoulder Elbow. 2018;11(6):430-9.
23. 23. Murphy ME, Maloney PR, McCutcheon BA, Rinaldo L, Shepherd D, Kerezoudis P, et al. Predictors of Discharge to a Nonhome Facility in Patients Undergoing Lumbar Decompression Without Fusion for Degenerative Spine Disease. Neurosurgery. 2017;81(4):638-49.
24. 24. Samuel AM, Lukasiewicz AM, Webb ML, Bohl DD, Basques BA, Varthi AG, et al. Do we really know our patient population in database research? A comparison of the femoral shaft fracture patient populations in three commonly used national databases. Bone Joint J. 2016;98-b(3):425-32.