13rd Department of Cardiology, Faculty of Medical Sciences in Zabrze, Medical University of Silesia, Katowice, Poland

23rd Department of Cardiology, Silesian Center for Heart Disease, Zabrze, Poland

3Department of Science, Education and New Medical Technologies, Silesian Center for Heart Disease, Zabrze, Poland

4Department of Cardiology and Structural Heart Diseases, Medical University of Silesia, Katowice, Poland

5Department of Cardiology, Institute of Medical Sciences, University of Opole, Opole, Poland

62nd Department of Clinical Cardiology and Cardiovascular Interventions, Institute of Cardiology, Jagiellonian University Medical College, Kraków, Poland

7Department of Interventional Cardiology and Angiology, Institute of Cardiology, Warszawa, Poland

8Department of Preventive Medicine and Education, Medical University of Gdansk, Gdańsk, Poland

9Department of Cardiology, Poznan University of Medical Sciences, Poznań, Poland

10American Heart of Poland, Katowice, Poland

INTRODUCTION

Analyzing the material containing a large sample of STEMI patients from the Polish Registry of Acute Coronary Syndromes (PL-ACS) with complete follow-up, we decided to compare the most refined LR with the most advanced NN analysis to assess the accuracy of post-discharge mortality prediction.

METHODS

The PL-ACS Registry is the first national, multicenter, and prospective Polish medical registry, in which clinical and procedural data on ACS patients have been recorded since 2003 [8]. The registry is a joint initiative of the Silesian Center for Heart Disease in Zabrze and the Polish Ministry of Health. The data on more than 700 000 ACS from the whole country have been gathered. The complete follow-up containing all-cause mortality was obtained from the National Health Fund, the only Polish non-commercial medical insurer [9].

Material

The population of 175 895 patients with ACS was analyzed, of which anterior STEMI treated with pPCI in LAD was found in 21 420 individuals discharged alive between 2009 and 2015. Patients with previous ACS and thrombolysis and coronary angioplasty of arteries other than LAD were excluded, and 17 793 patients were included in the analysis.

Statistical analysis

Different activation (linear, logistic, tanh, exponential, sinus) and error (sum-of-squares, cross-entropy) functions were tested with the Broyden-Fletcher-Goldfarb-Shanno algorithm. General sensitivity analysis of variables included in the NN model was performed, and parameters with marginal influence on the outcome (mean <1.001) were removed from the model one at a time, starting from the least relevant.

Clinical potential of the analyzed models was evaluated with Area Under the Receiver Operating Characteristics Curve (AUROC), which is not sensitive to class imbalance. AUROC is the area under the curve plotted for all possible cut-off thresholds for sensitivity as a function of 1-specificity. It ranges between 0 and 1, where 0.5 is random guessing, and 1 is a perfect classifier. Values >0.7 are considered as a clinically useful classifier [10]. The NN with the highest AUROC was selected and compared with the AUROC of the LR model [10]. All analyses were performed with the Statsoft Statistica 13.1 software (TIBCO Software, Palo Alto, CA, USA) using the STATISTICA Automated Neural Networks (SANN) module.

Ethics

All procedures were undertaken according to Helsinki Declaration, and the protocol was reviewed and accepted by the local Ethical Committee. All patients have expressed their informed consent.

RESULTS

The analysis included 17 793 patients with anterior STEMI without previous ACS treated with pPCI of LAD and discharged alive. The learning group included 10 675 individuals, while the validation and test groups comprised 3 559 patients each. Baseline characteristics are presented in Table 1. Characteristics of treatment and events during hospitalization and treatment recommended at hospital discharge are shown in Table 2.

Six-month mortality since the onset of STEMI was 5.9%. The results of the LR analysis are presented in Table 3.

The best NN had 97 neurons in the hidden layer and used sinus activation and cross-entropy error functions. The prediction values for 6-month mortality after the onset of STEMI were good in both approaches. Better results were obtained in the NN approach regarding the statistical quality of the models— AUROC 0.8422 vs. 0.8137 for LR (P <0.0001). AUROC in the test groups were 0.8103 and 0.7939, respectively (P = 0.037) (Figure 2). However, the difference seems to be too small to implement the NN approach in clinical practice.

DISCUSSION

In our study, a better predictive value of NN in comparison with LR was shown in terms of 6-month mortality in patients with the first anterior STEMI treated with pPCI of LAD. Nevertheless, LR had some advantages, such as a more negligible difference between learning and test groups and a possibility of interpretation of particular parameters included in the model. Moreover, the NN model is more challenging to build and does not provide data on the particular risk factor included in the analysis. To our best knowledge, it is the first study that compared these two statistical methods in STEMI patients. Hitherto, analyses comparing LR with models based on machine learning (ML), but not with NN, were conducted, and no advantage of ML over LR was shown [11–13]. The difference was not documented even though a predictive value of LR models involving more than 20 variables decreased while in ML models, it was improved [13]. Thus, it may be assumed that the predictive value of modern statistical approaches (ML, NN) might be better in analyses involving a more significant number of parameters. Another potential advantage of NN is the possibility of including other than categorical or continuous variables in the models, i.e., diagnostic images or videos [14]. The NN was examined in patients with ACS to predict the outcomes based on electronic health records. The model included all the electronic health records variables and had a similar predictive value to classical statistical methods [15]. NN’s advantages over previously used methods were documented in analyzing ST-T changes on electrocardiograms and mortality prediction on echocardiography imaging [16, 17]. The NN was utilized in analysis of magnetic resonance or coronary angiography; both approaches are still under scientific research [18–20]. Therefore, it may be concluded that the increase of data sets and the use of more complex variables, such as digital electrocardiograms or diagnostic images, may reveal the superiority of NN over the LR approach [14].

One of the primary purposes of analyses performed in the PL-ACS registry is to identify the clinically significant predictors of unfavorable events and patients with high mortality risk [21–23]. The post-STEMI risk stratification is an important clinical issue highlighted in the “Gaps in the evidence and areas for future research” section of the European Society of Cardiology guidelines for the management of acute myocardial infarction in patients presenting with ST-segment elevation [24]. In the same document, the Global Registry of Acute Coronary Events (GRACE) risk score was recommended for risk assessment and adjustment without the class of recommendation [24]. Other risk scales mentioned in the guidelines were TIMI-STEMI, Zwolle, and PAMI-II. Although they might help distinguish between low- and high-risk individuals, they were developed for patients managed in an old-fashioned way. The TIMI risk scale was worked out for patients treated with thrombolysis. Among patients included in the Zwolle risk scale development, only about half had a stent implanted during coronary angioplasty, while in the PAMI-II cohort, only balloon angioplasty was performed without stent implantation [25–27].

Moreover, it may be assumed that patients with STEMI and NSTEMI differ in many aspects, including pathophysiology, baseline characteristics of therapeutic management, and survival [24, 28]. For that reason, it seems to be reasonable to develop separate risk scores for STEMI and NSTEMI. Besides, baseline characteristics and therapeutic methods in STEMI/NSTEMI patients have changed, so risk scales should be modified and updated in the most recent population.

In our study, patients who died in the 6-month follow-up after STEMI were older than survivors, had a higher heart rate and the Killip class, lower blood pressure and ejection fraction, more often had diabetes, heart failure, and chronic kidney disease. However, they less often received optimal treatment at discharge. When patients with worse clinical characteristics are treated with lower intensity, this relation has been described and is known as the treatment-risk paradox [29]. It is not proof of the cause-and-effect relation, and the direction of the potential relations is also not established. The worse clinical status could be a contraindication for the use of ACEI/ARB or β-blockers (cardiogenic shock or hypotension). On the other hand, the link between mortality and lower doses of statin or ASA use could not be excluded. Another possible explanation could be postponing therapy due to a planned transfer to another unit or hospital. The protective effect of ACE-I and β-blocker administration at discharge should be highlighted (multivariate LR analysis results are shown in Table 3).

Study strengths and limitations

Among the advantages, a large sample of patients, whole-country coverage, and the registry’s prospective design may be mentioned. Moreover, the data were collected directly from the hospitals, and complete follow-up was obtained.

The study also has some limitations. As no data on cardiovascular mortality was available, all-cause mortality was used. Secondly, there is no information on biochemical parameters, such as complete blood count or creatinine concentration. Moreover, no data on post-discharge treatment and environmental factors, such as compliance, were available. Thirdly, we could not gather information on whether the patient was discharged home or transferred to another hospital, which may have lowered the in-hospital mortality and increased mortality after discharge. However, these limitations affect both LR and NN approaches in the same way and probably are not relevant in comparing these two statistical methods. The most relevant limitation in terms of clinical implications is a restriction in inclusion criteria. The study included only STEMI patients treated with pPCI of LAD.

CONCLUSIONS

To conclude, the neural network may have a better predictive value than logistic regression in patients after the first STEMI treated with primary percutaneous coronary intervention in the left anterior descending artery. However, the statistical difference is slight and may not be relevant from the clinical point of view. Further studies on larger data sets containing different data types are required to verify NN’s clinical usefulness in the prediction of death risk after myocardial infarction.

Article information

Conflict of interest: None declared.

Open access: This article is available in open access under Creative Common Attribution-Non-Commercial-No Derivatives 4.0 International (CC BY-NC-ND 4.0) license, allowing to download articles and share them with others as long as they credit the authors and the publisher, but without permission to change them in any way or use them commercially. For commercial use, please contact the journal office at kardiologiapolska@ptkardio.pl.

How to cite: Niedziela JT, Cieśla D, Wojakowski W, et al. Is neural network better than logistic regression in death prediction in patients after ST-segment elevation myocardial infarction? Kardiol Pol. 2021; 79(12): 1353–1361, doi: 10.33963/KP.a2021.0142.

REFERENCES