Developing and Validating a New Model to Predict In-Hospital Mortality in Patients with Acute Myocardial Infarction

By Selma Atay Islam, Mehmet Muzaffer Islam, Hande Akbal Kahraman, Muhammed Fatih Ciril, Aysegul Mizrak, Ismail Tayfur

Affiliations

doi: 10.29271/jcpsp.2023.12.1361

ABSTRACT
Objective: To derive and validate a regression model that can successfully and robustly predict in-hospital mortality of patients who underwent percutaneous coronary intervention (PCI) after admission to the Department of Emergency Medicine (ED) with acute myocardial infarction (AMI).
Study Design: Cohort study.
Place and Duration of the Study: ED of University of Health Sciences, Sancaktepe Training and Research Hospital, that worked as a PCI centre between January and March 2022.
Methodology: Patients older than 18 years of age, diagnosed with acute ST elevation myocardial infarction (STEMI) or non-STEMI (NSTEMI) in the ED, and consequently underwent PCI were included. Patients with missing information of the outcome were excluded. For the regression model, backward stepwise logistic regression was utilised. The non-random split-sample development and validation method was used for the internal and external validation of the model. Ejection fraction, diastolic blood pressure, haemoglobin A1c, and haemoglobin were selected as the predictors.
Results: A total of 279 patients were included in the analysis. The area under the curve (AUC) of the final model in the derivation cohort was 0.982 (95% CI = 0.956-1.0). The sensitivity was 92.3% (95% CI = 64-99.8) and the specificity was 96.2% (95% CI = 92.3-98.4). The AUC of the final model in the validation cohort was 0.956 (95% CI = 0.904-1.0). The sensitivity was 80% (95% CI = 28.3-99.5) and the specificity was 92.3% (95% CI = 84-97.1).
Conclusion: The suggested model generated results that can be utilised as a screening tool for predicting in-hospital mortality in patients diagnosed with STEMI or NSTEMI who are admitted to PCI in emergency medicine settings. Nonetheless, it is essential to validate the model in different populations.

Key Words: Percutaneous coronary intervention, Mortality, In-hospital mortality, Prediction model, Logistic regression.

INTRODUCTION

The recent advances in evidence-based guidelines, as well as new pharmacotherapy and intervention options, have led to a significant reduction in the mortality rate of patients with acute myocardial infarction (AMI).¹ The mortality rate for patients with ST-segment elevation myocardial infarction (STEMI) who underwent percutaneous coronary intervention (PCI) was reported to be 7%, while the same for those with non-ST-segment elevation myocardial infarction (NSTEMI) was 4.9% if they can undergo catheterisation within the recommended time frames.²

Identifying patients at high risk of mortality who undergo PCI can enable researchers and clinicians to develop the specific approach and strategies for this subgroup and intervene earlier.

Several scoring systems had been described in the literature to predict mortality after PCI. Since most of these systems focus on all patients who underwent PCI (elective and acute procedures together), the predictors defined in the model can be complex variables and hence, their utility in the emergency medicine settings becomes questionable. Moreover, many of these models are outdated due to the rapid developments in the management of myocardial infarction (MI) and fundamental innovations in the treatment modalities, which requires these models to be frequently updated or redefined.^3-5

The primary object of this study was to derive and validate a regression model that can successfully and robustly predict in-hospital mortality of patients who underwent PCI after admission to the ED with acute AMI. It was also aimed to develop an SPSS calculator to facilitate the conduction of external validation studies.

METHODOLOGY

This single-centred retrospective, cohort study was conducted following the local Review Board approval (No. E-46059653- 050.99-207116293, dated 16.01.2023). The study was designed, conducted, and reported according to the "Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD statement".⁶ The study was conducted in the Department of Emergency Medicine of the University of Health Sciences, Sancaktepe Training and Research Hospital, Turkey.

The hospital records between January and March 2022 were retrospectively examined. The inclusion criteria for the study was determined to be patients over 18 years of age, diagnosed with acute STEMI or NSTEMI in the ED, and had to undergo PCI. Patients with missing information on the outcome and those who underwent elective PCI were excluded from the study.

The diagnosis and treatment of NSTEMI were managed by the 2020 European Society of Cardiology (ESC) guideline.⁷ Pre-interventionally, NSTEMI patients were administered a loading dose of 300mg of aspirin to inhibit platelet aggregation and minimise the risk of thrombotic events during PCI. According to the guidelines, the routine-use of P₂Y₁₂ receptor inhibitors was avoided in the patients whose coronary anatomy was unknown. Peri-interventionally, unfractionated heparin was administered at a dose of 70IU/kg during the procedure.

The management of the patients with STEMI was performed according to the 2017 ESC guidelines.⁸ A loading dose of 300mg aspirin and 180mg ticagrelor were administered pre-interventionally, and 70-100IU/kg dose of unfractionated heparin was administered peri-interventionally.

Since the study hospital functioned as a PCI centre, all patients diagnosed with NSTEMI were referred for PCI within the first 24 hours of diagnosis. STEMI and high-risk NSTEMI patients were prioritised and catheterised within the first 2 hours. All patients underwent coronary echocardiography just before the procedure.

A total of 4 predictors were included for the regression model. Patients were consecutively included until there were 20 in-hospital mortality events in total.⁹ The significant variables between the mortality groups were determined through univariable analysis. Then, by utilising non-random split-sample development and validation described in the TRIPOD statement, the patients were split into two groups based on the order of their admission dates to the ED. Non-random splitting was used because this method is methodologically superior in evaluating model performance.¹⁰ The first group, consisting of 70% of the patients, was designated as the derivation cohort, while the remaining 30% formed the validation cohort. Multiple imputation was considered to handle the missing data, but there was no missing data in the dataset.

The derivation cohort was used to derive the regression model. When the final model was determined, internal validation was performed with 3-fold cross-validation, and the risk of over-fitting was analysed. Finally, the regression model was externally validated on the validation cohort.

SPSS 29 (IBM Corp. Released 2019, IBM SPSS Statistics for Windows, Version 29.0. Armonk, NY: IBM Corp.) was used for the statistical analysis. The normality test was conducted using Shapiro-Wilk test. The normally distributed continuous data were expressed as mean ± standard deviation and non-normally distributed continuous data were expressed as median (25-75% quartiles). Student's t-test was used for the intergroup comparisons of the continuous data that exhibited normal distribution. Mann-Whitney U test was utilised to assess the differences between the continuous data that had non-normal distribution. The categorical data were expressed as frequency (%). For the categorical variables, the Chi-square test was used to compare the groups. Fisher’s Exact test was utilised, if necessary. To determine the area under the curve (AUC), the receiver operating characteristics (ROC) test was used.

For the multivariable analysis, logistic regression was used and variables with a significant difference in the univariable analysis (p<0.05) were included in the model with backward step-wise method. The Hosmer-Lemeshow test was utilised to evaluate the goodness of fit, while the assumption of multicollinearity was assessed both by examining the correlation matrix and performing a tolerance test. Outliers were assessed with the Cook’s distance. The overall performance of the final model was evaluated using Nagelkerke R square, accuracy, and AUC.

For the internal validation, 3-fold cross-validation was used. The derivation cohort was randomly divided into three groups. For every fold, two groups were used as training, and the remaining group was used for testing, and the performances of the training and testing for every fold was compared.

After the internal validation, an optimal cut-off point was determined using the Youden’s index. The final model was then externally validated using the validation cohort, and the performance measures were calculated for the derivation and validation cohorts. Finally, a secondary cut-off point was determined where the sensitivity was 100% in the derivation cohort, and the performance measures were also calculated for this cut-off point. The level of statistical significance was determined as p<0.05.

RESULTS

After applying the inclusion and exclusion criteria, a total of 279 patients were included in the study. The mean age of the patients was calculated as 59 ± 12 years; 216 (77.4%) patients were male. Eighteen (6.5%) patients died during the hospitalisation. As a result of the univariable analysis, significant differences were found between the mortality groups in terms of age, coronary artery disease, systolic blood pressure, diastolic blood pressure, troponin, BUN, creatinine, haemoglobin, AST, CRP, haemoglobin A1c, and ejection fraction. The results of the univariable analysis and the descriptives of the patients are summarised in Table I.

Table I: Descriptive characteristics of the study population and the univariable analysis of the in-hospital mortality groups.

	All patients (n=279)	Survived patients (n=261)	Patients with in-hospital mortality (n=18)	p-value
Age	59 (±12)	58 (±11)	70 (±10)	<0.001*
Gender (male)	216 (77.4%)	203 (77.8%)	13 (72.2%)	0.586***
Diabetes mellitus	94 (33.7%)	86 (33%)	8 (44.4%)	0.318***
Hypertension	132 (47.3%)	123 (47.1%)	9 (50%)	0.813***
Coronary artery disease	123 (44.1%)	110 (42.1%)	13 (72.2%)	0.013***
Hyperlipidaemia	50 (17.9%)	44 (16.9%)	6 (33.3%)	0.080***
Chronic renal disease	35 (12.5%)	31 (12%)	4 (22.2%)	0.175***
Systolic blood pressure	152 (±43)	154 (±42)	121 (±46)	<0.001*
Diastolic blood pressure	88 (±25)	89 (±25)	71 (±24)	0.002*
Pulse rate (bpm)	80 (67 to 96)	82 (69 to 96)	70 (50 to 86)	0.052**
SpO2 (%)	98 (94 to 99)	98 (94 to 99)	96 (92 to 98)	0.339**
Troponin (ng/L)	48.9 (17.7 to 353.2)	46 (17.6 to 301)	393 (66.4 to 2102)	0.008**
BUN (mg/dl)	32 (25 to 41)	32 (25 to 40.5)	41 (31.5 to 52.7)	0.010**
Creatinine (mg/dl)	0.87 (0.75 to 1.02)	0.85 (0.75 to 0.98)	1.12 (0.97 to 1.3)	<0.001**
White blood cell (103/µl)	10.2 (8.3 to 13.3)	10.2 (8.3 to 13.1)	10.9 (8.6 to 15)	0.366**
Neutrophil (103/µl)	6.4 (4.9 to 9.6)	6.4 (4.9 to 9.5)	6.8 (5 to 10.7)	0.631**
Lymphocyte (103/µl)	2.5 (1.7 to 3.5)	2.5 (1.8 to 3.5)	2.4 (1.6 to 4.7)	0.860**
Haemoglobin (g/dl)	14.3 (12.8 to15.5)	14.4 (13 to 15.5)	12.5 (11.6 to 14.2)	0.004**
Platelet (103/µl)	247 (209 to 297)	248 (209 to 298)	230 (201 to 315)	0.793**
AST (U/L)	25.7 (18 to 45)	25 (18 to 43)	47 (25 to 121)	0.006**
CRP (mg/L)	5.7 (2.4 to 16.5)	5.3 (2.3 to 15.6)	20.9 (5.8 to 52.4)	0.003**
Haemoglobin A1c (%)	6.26 (5.62 to 7.6)	6.3 (5.7 to 7.7)	5.2 (2.6 to 6.9)	0.002**
Ejection fraction (%)	50 (45 to 60)	50 (45 to 60)	35 (15 to 40)	<0.001**
MI type (STEMI)	129 (46.2%)	117 (44.8%)	12 (66.7%)	0.072***
AST: Aspartate aminotransferase, BUN: Blood urea nitrogen, CRP: C-reactive protein, STEMI: ST elevation myocardial infarction. *Student-t, **Mann-Whitney U, ***Chi-Square.

Table II: Logistic regression results and the final model summary.

	B-Coefficient	Wald	p-value	OR	95% CI
Ejection fraction	-0.256	14.593	<0.001	0.774	0.679 to 0.883
Diastolic blood pressure	-0.074	6.588	0.010	0.929	0.878 to 0.983
Haemoglobin A1c	-1.016	4.484	0.034	0.362	0.141 to 0.927
Haemoglobin	-0.622	3.918	0.048	0.537	0.290 to 0.994
Constant	27.834	12.331	<0.001	NA	NA
Regression function: 27.834 + (Ejection Fraction * - 0.256) + (Haemoglobin A1C * - 1.016) + (Diastolic Blood Pressure * - 0.074) + (Haemoglobin * - 0.622).

Table III: Comparison of the performance of the final model in the derivation and validation cohorts.

	Derivation Cohort (Cut-off point = -1.7264)	Validation Cohort (Cut-off point = -1.7264)	Validation Cohort (Cut-off point = -4.3819)
AUC	982 (95%CI = 0.956 to 1.0)	0.956 (95%CI = 0.904 to 1.0)	NA
Sensitivity	92.3% (95%CI = 64 to 99.8)	80% (95%CI = 28.3 to 99.5)	100% (95%CI = 47.8 to 100)
Specificity	96.2% (95%CI = 92.3 to 98.4)	92.3% (95%CI = 84 to 97.1)	80.7% (95%CI = 70.3 to 88.8)
Positive predictive value	63.2% (95%CI = 44.9 to 78.3)	40% (95%CI = 21.6 to 61.8)	25% (95%CI = 17.5 to 34.4)
Negative predictive value	99.4 (95%CI = 96.4 to 99.9)	98.6% (95%CI = 92.6 to 99.7)	100%
Positive likelihood ratio	24.13 (95%CI = 11.48 to 50.74)	10.4 (95%CI = 4.29 to 25.2)	5.2 (95%CI = 3.3 to 8.2)
Negative likelihood ratio	0.08 (95%CI = 0.01 to 0.53)	0.22 (95%CI = 0.04 to 1.27)	0
Accuracy	95.9 (95%CI = 92.2 to 98.2)	91.6 (95%CI = 83.4 to 96.5)	81.9% (95%CI = 72 to 89.5)
AUC: Area under the curve.

After randomly splitting patients for the derivation of the prediction model, 196 patients were recruited into the derivation cohort and 83 patients in the validation cohort. The mean age of the derivation cohort was 58.2 + 11.5 years, while the mean age of the validation cohort was 60.2 + 11.8 years, (p=0.203). In the derivation cohort, 151 (77%) patients were male, and in the validation cohort, 65 (78.3%) patients were male, with no significant difference between the two groups (p=0.816). In-hospital mortality occurred in 13 (6.6%) patients in the derivation cohort and 5 (6%) patients in the validation cohort, with no significant difference between the groups (p=0.850).

The backward step-wise method was utilised to derive the logistic regression model, and the results of the ninth step were reported. The assumption of goodness of fit (p=0.952, Hosmer and Lemeshow test) and the assumption of multicollinearity (tolerance>0.1 for every variable) were met for the final model. No significant correlation was present between the variables. There was only one outlier (patient number 6) and the Cook’s distance was 1.406, but this patient was not excluded from the model. The final model classified 95.9% of all patients correctly in the derivation cohort. The Nagelkerke R square of the model was 0.760 and it was able to explain 76% of all the variance.

Ejection fraction, diastolic blood pressure, haemoglobin A1c and haemoglobin were the selected predictors of the final model. All of these variables were found to be independent predictors (p<0.001, p=0.010, p=0.034, and p=0.048, respectively). The most valuable predictor was ejection fraction (Wald statistic = 14.593). Results of the final regression model are summarised in Table II.

When the 3-fold cross-validation was performed, the mean AUC of training and testing were calculated as 0.982 (0.954 to 1.0) and 0.977 (0.884 to 1.0), respectively.

The AUC of the final model in the derivation cohort was 0.982 (95% CI = 0.956 to 1.0). The optimal cut-off point was -1.7264 where the sum of the sensitivity and the specificity was the highest. The sensitivity was 92.3% (95% CI = 64 to 99.8), specificity was 96.2% (95% CI = 92.3 to 98.4), and the accuracy was 95.9 (95% CI = 92.2 to 98.2). The AUC of the final model in the validation cohort was 0.956 (95% CI = 0.904 to 1.0). The sensitivity was 80% (95% CI = 28.3 to 99.5), specificity was 92.3% (95% CI = 84 to 97.1), and the accuracy was 91.6 (95% CI = 83.4 to 96.5). When a secondary cut-off point was determined (-4.3819) at 100% sensitivity in the derivation cohort, the specificity of this point in the validation cohort was calculated as 80.7% (95% CI = 70.3 to 88.8) and accuracy as 81.9% (95% CI = 72 to 89.5). The performance of the final model is summarised in Table III.

DISCUSSION

The studied model showed excellent discrimination in both internal and external validation with good calibration. Three-fold cross-validation results were interpreted with the preferred predictors as consistent and concluded that there was no serious risk of overfitting in this model. Based on the optimal threshold value, the model achieved a sensitivity of 80% and a specificity of 92% in the validation cohort. Positive and negative likelihood ratios were calculated as 10.4 and 0.22 which indicated that the model had high diagnostic power of identifying the patients with the high risk for mortality.¹¹ When the cut-off value was changed to achieve 100% sensitivity in the derivation cohort, the model showed similar performance in the validation cohort and had 100% sensitivity and 80.7% specificity. Hence, it would be reasonable to claim that this model can be used as a screening tool to predict in-hospital mortality of patients who undergo PCI after admission to the emergency medicine department with acute myocardial infarction.

Numerous studies on this subject exist in the literature, but most of them are either outdated or focus on almost a decade-old cohorts. One of the most recent studies on this topic was published in 2022 by Song et al.¹² The authors created a nomogram to predict mortality in PCI patients and included approximately 10,000 patients. The nomogram showed a relatively satisfactory performance and had an AUC value of 0.884 in predicting mortality. It should be noted, however, that the study's patient cohort was from 2013.¹² Similarly, in a study in which approximately 479,000 patients were included, and multiple models were created with various methods, the AdaBoost classifier model, a machine learning algorithm, showed the highest performance for in-hospital mortality, and the AUC value was found to be 0.927. However, although it was published in 2019, the patient records between 2004 and 2012 were included in this study.¹³ Given the changes in management of acute MI, the validity of these models’ current performances is uncertain.

Brener et al. conducted a study in 2019 with approximately 24,600 patients from several randomised controlled trials. The AUC value of the model was reported as 0.848 in the derivation cohort and 0.828 in the validation cohort. Although the model did not demonstrate excellent discrimination, it was concluded to be a robust model with consistent results. However, some predictors used for the model in this study (e.g. lesion of the left anterior descending artery, suboptimal flow in the artery) may not be suitable for use in the emergency medicine department since they can only be obtained during PCI.¹⁴

In a Japanese study conducted with the records of approximately 23,000 patients published by Niimi et al. in 2022, a model derived with XGBoost classifier achieved a 0.960 AUC for in-hospital mortality.¹⁵ A total of 9 predictors were used for the model, including presence of cardiogenic shock at initial assessment and the type of MI (STEMI or NSTEMI). Although the model had a high AUC, the authors did not elaborate on the model's performance measures such as sensitivity or specificity. This model appeared to be highly successful, and the model of current study was able to achieve similar results with fewer predictors as compared to this study.

Many modelling studies had been pointed out with serious shortcomings in the reporting of their results. Moreover, a significant number of these studies do not provide enough information for the reproducibility of the derived model. Although relatively rare, some authors had provided detailed specifications about the methodology of the model in their study, a calculator or a ready-to-use model was usually not provided. The authors believed that scientific inquiry necessitates the ability to reproduce the reported results, and this issue is even more critical in modelling studies. Therefore, this study provided an SPSS model in the supplementary material (xml file) and the regression function in the footnote of Table III and fellow researchers are encouraged to externally validate this model.

There are several limitations to the study. Although the patients are included consecutively and there was no missing data in the data set, data reliability is under question due to the retrospective design of the study. Moreover, the generalisability is limited because the study is unicentric. The mainstream practice of modelling studies with logistic regression in the literature is that the number of events per predictor variable (EPV) has to be 10 at the very least. However, the authors took the less popular view in this topic that this rule could be relaxed up to EPV=5.⁹ Although the model provided consistent results in both internal and external validations, this may have affected the power of the study in a negative way.

While including STEMI and NSTEMI patients together in the same model may seem like a confounding factor, the analysis found no significant difference in rates of in-hospital mortality between the two groups. Furthermore, no significant difference was observed in the frequency of STEMI and NSTEMI cases between the derivation and validation cohorts. The authors had come across some studies with similar inclusion criteria in the literature as well.^12,15 While this can not be entirely ruled out, its impact on the results is unlikely to be significant.

CONCLUSION

The studied model generated results that can be utilised as a screening tool for predicting in-hospital mortality in patients diagnosed with STEMI or NSTEMI admitted to PCI in emergency medicine settings. Nonetheless, it is essential to validate this model in different populations.

ETHICAL APPROVAL:
This study was conducted following the local Review Board approval number E-46059653-050.99-207116293 on 16^th January 2023.

PATIENTS’ CONSENT:
There was no informed consent due to the retrospective nature of the study.

COMPETING INTEREST:
The authors declared no competing interest.

AUTHORS’ CONTRIBUTIONS:
SAI: Conceptualisation, methodology, validation, writing original draft, and review, editing, visualisation, and project administration.
MMI: Conceptualisation, methodology, validation, formal analysis and investigation, writing original draft and review, editing, and visualisation.
HAK: Writing original draft, and review, editing, and visualisation.
MFC: Writing original draft, and review, and editing.
AM: Writing original draft.
IT: Supervision.
All authors approved the final version of the manuscript to be published.

REFERENCES

1. Greco A, Finocchiaro S, Angiolillo DJ, Capodanno D. Advances in the available pharmacotherapy for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation. Expert Opin Pharmacother 2023; 24(4):453-71. doi: 10.1080/14656566. 2023.2171788.
2. Bhatt DL, Lopes RD, Harrington RA. Diagnosis and treatment of acute coronary syndromes: A review. JAMA 2022; 327(7):662-75. doi: 10.1001/jama.2022.0358.
3. Singh M, Rihal CS, Lennon RJ, Spertus J, Rumsfeld JS, Jr Holmes DR. Bedside estimation of risk from percutaneous coronary intervention: The new Mayo Clinic risk scores. Mayo Clin Proc 2007;82(6):701-8. doi: 10.4065/82.6.701.
4. Hamburger JN, Walsh SJ, Khurana R, Ding L, Gao M, Humphries KH, et al. Percutaneous coronary intervention and 30-day mortality: The British Columbia PCI risk score. Catheter Cardiovasc Interv 2009; 74(3):377-85. doi: 10. 1002/ccd.22151.
5. Moscucci M, Kline-Rogers E, Share D, O'Donnell M, Maxwell-Eward A, Meengs WL, et al. Simple bedside additive tool for prediction of in-hospital mortality after percutaneous coronary interventions. Circulation 2001; 104(3):263-8. doi: 10.1161/01.cir.104.3.263.
6. Collins GS, Reitsma JB, Altman DG, Moons KG; TRIPOD Group. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD statement. The TRIPOD group. Circulation 2015; 131(2):211-9. doi: 10.1161/CIRCULATIONAHA.114.014 508.
7. Collet JP, Thiele H, Barbato E, Barthélémy O, Bauersachs J, Bhatt DL, et al. 2020 ESC Guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation. Eur Heart J 2021; 42(14):1289-367. doi: 10.1093/eurheartj/ehaa575.
8. Ibanez B, James S, Agewall S, Antunes MJ, Bucciarelli-Ducci C, Bueno H, et al. ESC Guidelines for the management of acute myocardial infarction in patients presenting with ST-segment elevation: The task force for the management of acute myocardial infarction in patients presenting with ST-segment elevation of the European Society of Cardiology (ESC). Eur Heart J 2018; 39(2):119-77. doi: 10.1093/eurheartj/ehx393.
9. Vittinghoff E, McCulloch CE. Relaxing the rule of ten events per variable in logistic and Cox regression. Am J Epidemiol 2007; 165(6):710-8. doi: 10.1093/aje/kwk052.
10. Moons KG, Kengne AP, Grobbee DE, Royston P, Vergouwe Y, Altman DG, et al. Risk prediction models: II. External validation, model updating, and impact assessment. Heart 2012; 98(9):691-8. doi: 10.1136/heartjnl-2011-301247.
11. Deeks JJ, Altman DG. Diagnostic tests 4: likelihood ratios. BMJ 2004; 329(7458):168-9. doi: 10.1136/bmj.329.7458. 168.
12. Song J, Liu Y, Wang W, Chen J, Yang J, Wen J, et al. A nomogram predicting 30-day mortality in patients undergoing percutaneous coronary intervention. Front Cardiovasc Med 2022; 9:897020. doi: 10.3389/fcvm.2022.897020.
13. Al'Aref SJ, Singh G, van Rosendael AR, Kolli KK, Ma X, Maliakal G, et al. Determinants of in-hospital mortality after percutaneous coronary intervention: A machine learning approach. J Am Heart Assoc 2019; 8(5):e011160. doi: 10.1161/JAHA.118.011160.
     
14. Brener SJ, Leon MB, Serruys PW, Smits PC, von Birgelen C, Mehran R, et al. Derivation and external validation of a novel risk score for prediction of 30-day mortality after percutaneous coronary intervention. Euro Intervention 2019; 15(6):e551-e557. doi: 10.4244/EIJ-D-19-00262.
15. Niimi N, Shiraishi Y, Sawano M, Ikemura N, Inohara T, Ueda I, et al. Machine learning models for prediction of adverse events after percutaneous coronary intervention. Sci Rep 2022; 12(1):6262. doi: 10.1038/s41598-022- 10346-1.