Introduction
In a large group of patients with ischemic heart disease, the evaluation of myocardial viability is crucial for clinical and therapeutic decisions. In the last European Society of Cardiology guidelines for the revascularization of the heart muscle, the confirmation of myocardial viability has a class IIb level of recommendations for the qualification for coronary artery by-pass graft or percutaneous coronary intervention (PCI) [1]. In selected groups of patients like those with low ejection fraction < 30% and severe mitral regurgitation, the evaluation of myocardial viability has even an indication of level IIa [1]. In everyday clinical practice for the myocardial viability assessment, we use stress echo with dobutamine, single-photon emission computed tomography, cardiac magnetic resonance (CMR), and positron emission tomography. According to the cost-effectiveness, the most commonly used is the stress echo with dobutamine. Stress echo has some side effects, is observer-dependent, and the possibility of diagnosing myocardial viability with regular echocardiographic examination may be very appealing. Echocardiography can be extended to assess myocardial perfusion in rest and stress conditions by administration of contrast material which improves the quality of images and allows for the evaluation of tissue properties [2].
The future of medicine is to individualize the therapy for each patient, also regarding the coronary interventions or management of valvular diseases. Therefore, the role of myocardial viability and diagnostic procedures for its evaluation is crucial, and the use of artificial intelligence (AI) in many of these processes can be optimized and automated, reducing workload, time to diagnosis, treatment, and, not unimportantly, costs.
One of the most interesting and extensively studied branches of AI is Artificial Neural Networks (ANNs). Machine learning (ML) is a subfield of AI where the algorithms learn to perform a task based on expert engineered characteristics describing the data. ANNs are a family of ML where statistical learning models are used to estimate or approximate functions that can depend on many inputs and are predominantly unknown. ANNs are mathematical algorithms generated by computers. ANNs learn from standard data and capture the knowledge contained in the data. This technique can find optimal temporal features better than other deep/machine learning methods [3]. Ultrasonographic tissue characterization is a potential application for ANNs, which will allow us better and more cost-effective diagnosis [4–6]. Also, segmentation of the cardiovascular image is a developing field in which AI methods have shown substantial performance improvements [7, 8]. AI is quickly becoming present in various aspects of diagnosis and treatment of cardiovascular diseases and cardiac imaging is especially important regarding this development [9].
A recently published position paper by the European Association of Cardiovascular Imaging and the European Association of Nuclear Medicine highlights the use of ML and deep learning in everyday practice and the importance of the human ability to make final judgment and diagnosis [10].
The current study aimed to determine the meaning of the texture markers for the recovery and viability of the myocardial muscle after a heart infarct using echocardiographic images analyzed by the neural network.
Methods
Study group
Sixty-one consecutive patients admitted to the Cardiology Department of the Medical University of Lodz with acute coronary syndrome with ST-segment elevation, and coronary angiographic confirmation of occlusion of a single coronary artery were enrolled in the study (43 men, mean age 61 ± 9 years). Standard criteria for ST segment elevation myocardial infarction diagnosis were used, including symptoms of ischemia, ST-segment elevation on electrocardiogram, and significant troponin elevation, with at least one measurement exceeding the 99th percentile of the reference range [11]. All patients were successfully treated with PCI within 10 hours from the onset of symptoms.
Cardiac imaging
Echocardiographic images were obtained seven days (7 ± 1.3 days) after PCI using the Siemens Sequoia A512 and 4V1c transducer (4–1 MHz). Transthoracic echocardiography (TTE) images were acquired in all patients [12]. For the analysis, 533 segments of the heart echo images were used (native or contrast — obtained in resting TTE). Myocardial perfusion echocardiography was performed after iv Sonovue injection with dedicated software and recorded using contrast perfusion sequence (CPS, mechanical index < 0.16, 30 FPS). Analysis was performed for monochrome images (after conversion from originally recorded RGB data) or a red component as it contains the majority of image information among all RGB components. Within 7 to 10 days after PCI, each patient underwent a low-dose dobutamine stress echo (LDDSE). LDDSE included the administration of dobutamine every 3 minutes from 5–10–20 μg/kg/min with the storage of images on VIVID 7 (GE US). Each patient underwent a CMR study (Siemens Avanto 1.5T, DE) with gadolinium (Gadovist, Bayer DE) late enhancement imaging to define the % transmurality of necrosis. CMR was performed 14–21 days after PCI. All patients after 12 ± 1 month follow-up underwent echocardiographic examination with the evaluation of myocardial function recovery and viability (Fig. 1). Follow-up images have been obtained with VIVID 7 (GE – US). Two independent, experienced echocardiographers evaluated all images, and in case of doubts, a third expert opinion was crucial for the decision.
Image analysis
Custom analytic software (MaZda 5.20, Institute of Electronics, Lodz University of Technology) was used for extracting myocardial texture parameters from bitmaps where each bitmap represents a unique myocardial segment. The texture feature vectors were then used as input to artificial intelligence algorithms, which were used to distinguish between viable and non-viable myocardial segments and to predict myocardial viability. CMR was the gold standard for the viability assessment. A number of methods (multilayer perceptrons [MLP], non-linear discriminant analysis [NDA], support vector machine [SVM], Adaboost) were performed to identify the optimal analytic method correlating with CMR information regarding the scar transmurality. Based on the CMR visualization six viability classes were defined for evaluating the late gadolinium enhancement (Fig. 2). All applied methods represent supervised ML algorithms. Every myocardial segment (represented by a texture feature vector) was labeled by one of the classes defined in Figure 2 for further ML methods training and validation purposes. Myocardial function recovery assessment was performed for every patient after 12 ± 1 months and as a reference method, echocardiographic examination was used.
Images from end-systolic frames derived from two consecutive cardiac cycles in three apical echocardiographic views were used for the analysis. Images were evaluated from the four-, three- and two-chamber view of the heart with myocardium analyzed in a 17-segment model [13].
The image analysis included determining the regions of interest (ROIs) by the physician performing the examination. Non-overlapping ROIs were manually traced during end-systole in all images (all 17 segments were divided into two samples, and in each projection from two sequences, the segments have been marked). For each ROI, 283 texture features were calculated for defined regions of interest in each image, including 9 features from the histogram, 5 from the gradient matrix, 20 from the run-length matrix, 220 from the co-occurrence matrix, 5 from the autoregression model, and 44 from wavelet transform. Analysis for co-occurrence matrix evaluation was performed at a distance of 1 to 5 pixels and in four directions (horizontal, vertical, diagonal, and anti-diagonal) [14, 15]. Sample co-occurrence matrix for textures representing heart tissues with different necrosis levels are shown in Supplementary Figure S1.
Contrast-enhanced echocardiographic images were analyzed in the same manner after selecting the most informative red component for further evaluation.
To overcome the significant low counts in viability classes 1 and 5 (Fig. 2), it was decided to divide all the segments in the 17th segment model into two smaller ROIs (side-by-side, full thickness; Fig. 1).
Digital processing — artificial intelligence
For prediction (and for classification) texture features were selected using a two step procedure. First, among 283 texture parameters only the features that were characterized by the lowest variability were selected. For this purpose, normalized standard deviation ϵ was calculated (1):
where mf and σf are mean value and standard deviation of the texture feature f evaluated for all image ROIs, respectively. The evaluation was performed separately for each ROI location and for different viability grades. Finally, 30 features with the lowest ϵ were selected. In the second stage, a further reduction in the number of features was done by means of the Fisher criterion and the algorithms designed for minimizing the misclassification (POE) with an average correlation coefficient (ACC). Such POE + ACC method is based on reducing both the probability of misclassification (POE) and the average correlation coefficient (ACC) between the selected parameters. As a result, a selection process is achieved that ensures minor classification error, with a limited correlation coefficient between the selected features. Based on both approaches, 10 to 13 were selected from 30 previously chosen. Since experiments have shown that the classification and prediction results do not depend on the feature reduction method used (Fisher or POE + ACC), the results were obtained using the POE + ACC method because it generates a set of uncorrelated texture parameters. The two-stage feature selection method provides a target feature vector that is characterized by limited variability within the analyzed classes, contains uncorrelated features and minimizes the classification error.
To classify selected features (as well as for prediction of the recovery of myocardial function), neural networks — MLP, NDA, and SVM classifier with two kernels (polynomial and radial basis functions) were implemented [16, 17]. The results were correlated with information from the CMR concerning the degree of tissue necrosis to determine the optimal analysis method. Moreover, decision trees were used as “weak” classifiers in the AdaBoost algorithm implemented to predict parameter values that assess postinfarction myocardial viability [18]. Texture feature evaluation, feature selection and MLP and NDA classification was performed using MaZda 5.2 software while prediction of the recovery of myocardial function using MLP, SVM and AdaBoost algorithms was done with use of Weka 3.8 package (Waikato University, New Zealand) [19].
All results presented in Tables 1 and 2 were obtained using 5-fold cross-validation used for assessing classifiers quality. The final results of the classifiers’ metrics were calculated as the average value of the five obtained results of their testing.
|
Groups |
Two-layer perceptron |
Nonlinear discriminant analysis |
Support vector machine (polynomial kernel) |
|||
|
Red — RGB |
Monochrome |
Red — RGB |
Monochromatic |
Red — RGB |
Monochrome |
|
|
0 vs. (2,3,4,5) |
76.36% |
63.34% |
73.35% |
62.40% |
76.71% |
68.33% |
|
0 vs. (2,3) vs. (4,5) |
60.73% |
58.14% |
54.58% |
56.98% |
42.54% |
55.64% |
|
(0,2) vs. (3,4,5) |
74.44% |
63.37% |
71.56% |
61.37% |
74.31% |
64.46% |
|
Method of clasification |
Parameter of the analysis |
Sensitivity |
Specificity |
PPV |
NPV |
Best |
|
MLP |
R_LDDSE |
78.57% |
35.00% |
55.93% |
60.87% |
55.16% |
|
GREY_LDDSE |
92.86% |
24.39% |
55.71% |
76.92% |
55.25% |
|
|
BW_LDDSE |
26.32% |
100.00% |
100.00% |
70.21% |
67.47% |
|
|
R_FUP |
67.69% |
36.36% |
75.86% |
27.59% |
74.50% |
|
|
GREY_FUP |
96.92% |
13.04% |
75.90% |
60.00% |
74.43% |
|
|
rb_SVM |
R_LDDSE |
83.33% |
36.59% |
57.38% |
68.18% |
59.99% |
|
GREY_LDDSE |
86.05% |
57.50% |
68.52% |
79.31% |
59.17% |
|
|
poli_SVM |
R_LDDSE |
71.43% |
47.50% |
58.82% |
61.29% |
57.58% |
|
GREY_LDDSE |
76.74% |
65.00% |
70.21% |
72.22% |
58.56% |
|
|
AdaBoost |
R_LDDSE |
76.19% |
40.00% |
57.14% |
61.54% |
57.88% |
|
GREY_LDDSE |
64.29% |
62.50% |
64.29% |
62.50% |
57.96% |
|
|
BW_LDDSE |
30.00% |
93.75% |
75.00% |
68.18% |
66.50% |
|
|
R_FUP |
93.85% |
4.35% |
73.49% |
20.00% |
73.36% |
|
|
GREY_FUP |
83.33% |
36.36% |
79.71% |
42.11% |
74.43% |
Perceptron is one of the basic models of one-way networks used for classification. The purpose of each neuron in the network is to receive signals from other (not necessarily all) of the neurons, sum these signals, processing the summary signal using the concept [20–22].
In the present study, 10 to 13 input features corresponding to the number of elements of the input layer were used. The hidden layer contained four neurons. The output layer consisted of 2 to 3 neurons corresponding to the number of analyzed classes.
Statistical analysis
Normal distribution was checked with the Shapiro-Wilk test. The cross-table was used for the analysis of viability, evaluation of positive predictive value, and negative predictive value. All the analyses were performed with IBM SPSS Statistic 20 software.
Results
A group of selected stable parameters, including selected run-length matrix features, the percentage of non-zero elements of the gradient have been evaluated. For the analysis, the myocardium viability was divided into different groups according to the thickness of the viable muscle in CMR. Because of the irregular distribution of samples in each class, as shown in Figure 2, the analysis combined the different groups. In addition, a negligible number of samples of class 1 prevented their reliable classification. Therefore, this group has not been analyzed. In the primary analysis, the comparison between areas in class 2 and class 3 to 5 was performed (Table 3).
|
No. |
Groups |
Transmurality of the impaired myocardial function from cardiac magnetic resonance |
|
1 |
0 vs. 2 + 3 + 4 + 5 |
0% vs. 26–100% |
|
2 |
0 + 2 vs. 3 + 4 + 5 |
0% + (26–50%) vs. 51–100% |
|
3 |
0 vs. 2 + 3 vs. 4 + 5 |
0% vs. 26–75% vs. 76–99% |
From the clinical point of view, the most important for us was the analysis and comparison between healthy and diseased areas. The results obtained for the different types of classifiers for both monochrome and color images after the use of the contrast agent are shown in Table 1. The highest values of perception were obtained for the red component in differentiation necrosis versus no necrosis (76%) and for necrosis up to 50% and more than 50% (74%) (Table 1). These results represent the mean percentage of positive validation obtained after a 5-fold cross-validation (Table 1).
The neural network approach allowed correct classification regarding the absence of necrosis in 73% of segments and 76% of images representing different levels of transmurality (≤ 50% or > 50%) based on resting contrast imaging. These results were obtained for a red component of the CPS color scale as mean values of 5-fold network cross-validation. A similar feature selection and classification procedure was applied for native grayscale images yielding worse results (68% and 79% correct classification for segments and individual images, respectively). Advanced classification of segments into 3 classes: no necrosis (80% of regions), 1–50% necrosis (5%), > 50% necrosis by magnetic resonance imaging (15%) was 70% correct for CPS and 60% correct for native images.
For the analysis of the myocardial function recovery, both monochrome images and color images were analyzed. As a result of the analysis, it was an upheld theory from the previous phases of research that texture parameters of red color ultrasound images after contrast administration carry information capable of classifying regions in terms of heart viability and recovery of the myocardial function. In the transthoracic echocardiographic examination, there were no significant differences among the various techniques used to predict functional recovery. The best results have been obtained for images with contrast administration for the red and monochrome data (74% of proper classification). Interestingly in dobutamine stress echocardiography the results of proper prediction were only 67% for monochromatic images. Also, in LDDSE for function recovery more effective were images with contrast (sensitivity up to 97% for monochrome data in MLP), as in the diagnosis of lack of recovery monochromatic ones (specificity up to 100% for MLP) (Table 2). This seems that in the LDDSE examination, both evaluation methods should be performed because they are complementary.
Discussion
Viability assessment is extremely important in patient classification for PCI or coronary artery by-pass graft procedures. Various techniques may be used for the evaluation of viable myocardium. However, cost-effectiveness starts to play a very important role in everyday practice. According to a recent expert consensus from the European Association of Cardiovascular Imaging, even with the strain analysis, viability assessment is limited by a lack of specificity [23]. Support from AI is going to help doctors by saving time and increasing their effectiveness. A recently performed study using the neural network, based on 50.000 echocardiograms, confirmed the high correlation (r = 0.94, limits of agreement ± 14.4, sensitivity 0.93, specificity 087) with the measurements of ejection fraction done by a cardiologist with 20 years of experience [24]. The evaluation of the wall motion and the thickness of the myocardium with the texture analysis can be a good alternative for viability testing [25]. The present study results suggest that quantitative myocardial textural parameters provide valuable information on the evaluation of myocardial viability in the standard transthoracic examination. In an animal model, Milunksi et al. [26] have confirmed that independently of wall thickening, the backscatter analysis allows differentiating potentially stunned, viable myocardium from necrosis. The use of a contrast agent improves the detection of viable myocardium segments in echocardiographic examination in the analysis of texture entropy which was also confirmed by histological findings [27]. However, in the present study it was confirmed for the diagnosis in viability during rest echocardiography but not in LDDSE. One of the biggest studies evaluating the use of machine learning on more than 170 thousand patients whose echocardiograms and clinical data have been used to predict survival confirmed that artificial intelligence has superior accuracy (all AUC above 0.82) in standard clinical models [28]. It was confirmed in that study that echocardiographic data have been crucial for predicting survival because only 10 variables were needed to achieve 96% of the maximum prediction accuracy, with 6 of these variables being derived from echocardiography. Similar to the current results, Bae et al. [29] have confirmed that texture features other than the mean gray level can objectively distinguish nonperfused from perfused myocardium in myocardial contrast echocardiography images and may thus augment the diagnostic accuracy of current analysis techniques. This may be another indication to increase a class of recommendation for contrast use in future guidelines.
Many studies have confirmed that low dose or combined low and high-dose dobutamine infusion protocols performed by clinicians have clinically useful sensitivity (75 to 80%) and specificity (80 to 85%) for the identification of viable segments with functional recovery after revascularization [30, 31]. This confirms that contrast and monochrome images with neural network analysis may have similar values as in studies with regular evaluation. Moreover, a study by Omar et al. [32] used imaging-derived models of three-dimensional (3D) motion at rest and stress within random forests, support vector machines, and a deep learning approach consisting of a convolutional neural network. They found that the convolutional neural network provided the most sensitive model, with a sensitivity of 81.1% in a training dataset compared to expert operator interpretation [32]. Even when some augmentation techniques are used to increase the number of training samples, there is a need to collect hundreds of echocardiograms representing different degrees of necrosis.
The recent study, which aimed to use our study of ANN in evaluating myocardial perfusion in coronary angiography to predict the result of angiography or obstructive coronary artery disease, confirmed the added value of the use of ANN from 11% to 20% [33]. In CMR, where deep learning algorithms analyzed the 3d dataset, the correlation coefficient can be r > 0.95 in the manual and automatic evaluation of the left and right ventricular volumes [34]. A very high correlation r = 0.9 with p < 0.001 between automatic and manual analysis of segmented scar volumes quantification in hypertrophic cardiomyopathy has been obtained in the CMR examination [35]. However, CMR has some limitations like cost, patient claustrophobia, or history of pacemaker insertion. The 3D TTE automated analysis also provides a very good agreement in left ventricular end-diastolic and left ventricular end-systolic volumes with CMR (r = 0.84 to 0.95) [36]. In different cardiovascular imaging modalities, AI has been used to improve speed and quality of acquisition, reduce measurement time, and allow prompt diagnoses, improving patient care. It is only a matter of time before we will be using various algorithms for predicting patient survival and definitely, results from echo or CMR will be a very important part of them.
Limitations of this study
The acoustic properties of tissue were translated into echo texture by the ultrasound instrument. Therefore, the reproducibility of texture parameters depends on instrument settings and performance. One of them is also the algorithm of the saved image in bmp format that reduces the number of image gray levels to 256. It was the only format in which image files could be exported in the case of Siemens Sequoia scanner. The sample size was limited but still sufficient for obtaining significant results. Moreover, rigorous quality assessment of the echocardiographic assessment is essential before the implementation of these methods in clinical practice. Before making it global, it should be kept in mind that homogenization of clinical data recording and standardizing imaging protocols is crucial before data from different centers can be fed to ANNs input. Another important limitation is the single-center design of the study. Despite the use of 5-fold validation of the implemented machine learning models used, their actual assessment of generalization would be possible if image data from many medical centers were used, acquired with various ultrasound devices. Therefore, multicenter studies are needed for a more realistic efficiency evaluation of the applied texture analysis and machine learning methods for prediction of postinfarction myocardial viability in echo images.
Conclusions
Artificial intelligence using a neural network based on texture analysis of echocardiograms may provide valuable data on myocardial viability early after myocardial infarction without the need for stress testing. The present study confirmed the best detection of necrotic tissue in group 3 (51–75% of necrosis of the wall) and 4 (76–99% of necrosis of the wall). It was also confirmed that myocardial contrast enhancement allows a superior classification of necrotic tissue compared to native grayscale images. AI combined with texture analysis has enormous potential for creating advanced prognostic tools in cardiovascular disease, even using standard or contrast-enhanced echocardiographic imaging.
Conflict of interest: None declared.
Funding: None declared.
