Optimized target delineation procedure for the radiosurgery treatment of ventricular tachycardia: observer-independent accuracy

Abstract

Background: Part of the current stereotactic arrythmia radioablation (STAR) workflow is transfer of findings from the electroanatomic mapping (EAM) to computed tomography (CT).

Here, we analyzed inter- and intraobserver variation in a modified EAM-CT registration using automatic registration algorithms designed to yield higher robustness.

Materials and methods:This work is based on data of 10 patients who had previously undergone STAR.

Two observers participated in this study: (1) an electrophysiologist technician (cardiology) with substatial experience in EAM-CT merge, and (2) a clinical engineer (radiotherapy) with minimum experience with EAM-CT merge.

EAM-CT merge consists of 3 main steps: segmentation of left ventricle from CT (CT LV), registration of the CT LV and EAM, clinical target volume (CTV) delineation from EAM specific points.

Mean Hausdorff distance (MHD), Dice Similarity Coefficient (DSC) and absolute difference in Center of Gravity (CoG) were used to assess intra/interobserver variability.

Results:

Intraobserver variability: The mean DSC and MHD for 3 CT LVs altogether was 0.92 ± 0.01 and 1.49 ± 0.23 mm. The mean DSC and MHD for 3 CTVs altogether was 0.82 ± 0.06 and 0.71 ± 0.22 mm.

Interobserver variability: Segmented CT LVs showed great similarity (mean DSC of 0.91 ± 0.01, MHD of 1.86 ± 0.47 mm). The mean DSC comparing CTVs from both observers was 0.81 ± 0.11 and MHD was 0.87 ± 0.45 mm.

Conclusions: The high interobserver similarity of segmented LVs and delineated CTVs confirmed the robustness of the proposed method. Even an inexperienced user can perform a precise EAM-CT merge following workflow instructions.

Introduction

Scar-related ventricular tachycardia (VT) is a potentially lethal complication of structural heart disease, and catheter ablation of the arrhythmogenic substrate is essential in complex therapeutic management [1]. However, catheter ablation can sometimes be limited because of inaccessible substrates (inability to reach epicardial surface or deep location within the wall or septum) [2]. Stereotactic arrythmia radioablation (STAR) is an alternative treatment method for recurrent VTs after failed catheter ablation [3–5]. Compared with catheter ablation, STAR does not include substrate localization and must be performed by an electrophysiologist. Many factors can affect its overall accuracy, as has been well established by the treatment of malignant, benign, and functional diseases over many decades [6]. Important part of the current stereotactic arrythmia radioablation workflow is transfer of the electroanatomical study to computed tomography (CT) to be imported as secondary series to radiotherapy planning software and ensure maximum accuracy of target delineation

Several methods of target delineation have been proposed since the first VT patient was treated with STAR in 2012 [7]. The simplest approach involved manual contouring the clinical target volume (CTV) in the CT scan using exported images from an electroanatomic mapping (EAM) system and tagging the critical part of the arrhythmogenic substrate [8]. This method is clearly time-consuming and prone to error because of the subjective transfer of CTV from several three-dimensional (3D) visualizations to CT slices. Other techniques are based on merging maps with pre-procedural CT (secondary radiotherapy planning series) and following marking of the target [9–13]. In principle, two types of EAM-CT merge were proposed: manual alignment of EAM structures with segmented CT structures [9] or use of a semi-automatic algorithm [12]. These methods, which require a fast intravenous bolus of contrast agent for CT, are gradually being improved, including the extension of EAM to other structures such as the left main coronary artery, coronary sinus, thoracic aorta, and chambers. This comprehensive approach offers the possibility of greater use of automated algorithms with the potential to reduce inaccuracies.

Here we analyzed inter- and intraobserver variation in a modified EAM-CT registration using automatic registration algorithms designed to yield higher robustness than non-automated approaches in the hands of inexperienced users.

Materials and methods

Patient selection

For this analysis, we selected 10 of 39 patients who had previously undergone STAR for scar-related VT after failed catheter ablation. Their EAM data were used for CTV definition under the supervision of two electrophysiologists. The location of the targets within the left ventricle (LV) covered more basal than apical segments (Fig. 1).

Figure 1. Location of individual targets within the left ventricle using a 17-segment heart model (target located in segment 4 in four cases)

Electroanatomical data (EAM LV = left ventricle; EAM points = points with a specific tag for CTV delineation) were exported from CARTO software (CARTO® 3 system V6; Biosense Webster, Diamond Bar, CA, US) and converted to a format (VTK- The Visualization Toolkit and FCSV- Fiducials Comma-Separated Values) compatible with 3D Slicer software (open source, www.slicer.org) using an in-house program written in C#. All patients had fast bolus contrast-enhanced CT (exhale breathhold, 1mm slice thickness, reduced field of view resulting in voxel size < 1 x 1 x 1 mm, 400 mA) from the time of STAR pre-treatment simulation. Registration of EAM and CT was performed using 3D Slicer (versions 4.8.1 and 4.10) following a previously published strategy [13] with significant modifications. The procedure is briefly described in Figure 2 (see Supplementary File for details).

Figure 2. Visual presentation of the electroanatomic mapping–computed tomography (EAM-CT) merge procedure

Segmentation variability analysis

Two observers participated in this study. Observer 1 was an electrophysiologist technician from the cardiology department who was highly experienced in EAM-CT merge, CTV delineation, and anatomy of the heart, and the author of all modifications in the referenced procedure. Observer 2 was a clinical engineer from the oncology department, who was responsible for STAR treatment planning and minimally experienced with EAM-CT merge.

Both observers segmented LV from the CT image series for each case using a “grow from seeds” algorithm three times, with an interval of at least 8 h. Each segmentation was visually checked and manually adjusted if needed, in particular for features such as crossing sections with another structure and areas with metal artifacts from an implantable cardioverter defibrillator lead. The segmented LV CT was smoothed and the correction transformation applied to unify the CT coordinate system with EAM LV. Although CTVs cannot vary in volume (created from EAM data with a specific tag marking the STAR target area), the final position of the CTV within the LV could be affected by the precision of the CT LV segmentation and subsequent CT LV to EAM LV registration.

Variation was measured using the mean Hausdorff distance, Dice similarity coefficient (DSC), and absolute difference in center of gravity coordinates (CoG). The DSC represents the similarity of two 3D volumes in a range of 0-1 where 1 indicates a 100% match [14]. The Hausdorff mean distance is the mean distance between each point of one compared structure to the closest point in the other structure [15].

Intraobserver variability was considered as the first step in assessing the robustness of the proposed procedure. Observer 1, who had considerable experience in EAM-CT merge, performed the EAM-CT merge three times for each case, with an interval of at least 8 h. Intraobserver variability was evaluated using mean Hausdorff distance, DSC, and absolute difference in CoG of segmented CT LVs and CTVs individually for each case and in total. Segmented CT LVs were compared with the relevant EAM LVs.

Observer 2, who had minimal experience with EAM-CT merge, performed the EAM-CT merge once for each patient following instructions written by observer 1 (see Supplementary File). Interobserver variability was evaluated by mean Hausdorff distance, DSC, and difference in CoG. We compared the first CT LV and the first CTV from observer 1 with the CT LV and CTV from observer 2.

Statistical analysis

Statistical analysis was performed using STATISTICA 13 software (StatSoft, Tulsa, OK, USA). All quantitative data were expressed as mean and standard deviation. To compare segmented LVs between the two observers, we used t-tests. Repeated-measures ANOVA was used to compare the trios of segmented LV volumes from observer 1. All tests were performed at the 5% level of significance.

For observer 1, the similarity between CT LV and EAM LV was high (mean DSC of 0.84 ± 0.01), with a DSC of < 0.8 in only one case, in which CT was the poorest quality in the term of insufficient contrast agent within the LV (as a result of excesive inhomogenous endocardial hypertrabeculation and/or fibrotization). Contrast agent within LV was insufficient in this case, making segmentation more difficult and yielding a very irregular EAM LV. Table 1 shows the mean of absolute values of CoG differences, as averaging with negative values would have reduced the actual differences. When not using absolute values, we found that the mean differences for all cases were –0.23 mm, 0.08 mm, and 0.23 mm for X, Y, and Z coordinates, respectively, indicating no systematic error in any direction.

Table 1. Comparison of segmented left ventricles with left ventricles from electroanatomical mapping to evaluate intraobserver variability for observer 1

CT LV vs. EAM LV
					Center of gravity coordinates difference [mm]
Patient	DSC*	SD	Mean Hausdorf distance [mm]*	SD	X**	Y**	Z**
1	0.88	0.01	2.63	0.23	0.69	1.09	0.66
2	0.88	0.00	2.53	0.08	0.80	1.07	1.07
3	0.86	0.03	2.88	0.43	1.63	1.50	0.17
4	0.83	0.01	3.43	0.12	0.83	1.15	1.19
5	0.82	0.02	3.23	0.27	0.34	0.39	0.80
6	0.80	0.01	3.73	0.13	1.50	0.76	1.20
7	0.85	0.02	3.04	0.58	2.10	1.19	1.04
8	0.85	0.01	3.11	0.15	0.71	0.98	1.24
9	0.86	0.02	3.08	0.50	1.29	1.56	1.18
10	0.76	0.02	4.71	0.60	2.51	1.92	1.32
Mean	0.84	0.01	3.24	0.31	1.24	1.16	0.99

*mean value of Dice similarity coefficient (DSC) from three segmentations; **mean of the absolute value of coordinate differences from three segmentations; CT — computed tomography; LV — left ventricle; EAM — electroanatomic mapping; SD — standard deviation

Intraobserver variability

The mean DSC was 0.92 ± 0.01 for all cases (30 in total, three comparisons for each case), indicating a high similarity of all segmented LVs (Tab. 2). Repeated-measures ANOVA confirmed no significant difference among three segmented LVs for each case (p = 0.445). The mean Hausdorf distance was 1.49 ± 0.23 for all cases, and the mean absolute difference in CoG coordinates was < 1.5 mm, also indicating good intraobserver agreement. The mean differences in CoG coordinates (without absolute values) for all cases were 0.69 mm, –0.04 mm, and –0.05 mm for the X, Y, and Z coordinates, respectively.

Table 2. Comparison among three segmented left ventricles (LVs) for each case to evaluate intraobserver variability and reproducibility of LV segmentation over time

Intraobserver variability CT LV
					Center of gravity coordinates difference [mm]
Patient	DSC*	SD	Mean Hausdorf distance [mm]*	SD	X**	Y**	Z**
1	0.94	0.01	1.22	0.08	0.18	0.78	0.20
2	0.94	0.01	1.38	0.10	1.47	1.28	0.50
3	0.94	0.02	1.26	0.26	1.24	1.09	0.63
4	0.93	0.01	1.39	0.25	0.77	1.63	0.73
5	0.92	0.01	1.12	0.10	1.34	1.27	1.04
6	0.94	0.01	1.08	0.18	0.56	0.93	1.36
7	0.90	0.02	1.88	0.38	2.31	1.29	1.16
8	0.95	0.01	0.98	0.17	0.75	0.23	0.50
9	0.89	0.01	2.41	0.15	2.56	0.79	2.79
10	0.89	0.03	2.16	0.64	2.21	2.64	1.20
Mean	0.92	0.01	1.49	0.23	1.34	1.19	1.01

*mean value of Dice similarity coefficient from three segmentations; **mean of the absolute value of coordinate differences from three segmentations; CT — computed tomography; EAM — electroanatomic mapping; SD — standard deviation

Table 3 shows the similarity and comparison of CTV localization for all cases (mean value of three EAM-CT merges for each case). The mean differences in CoG coordinates (without absolute values) for all cases were 0.16 mm, –0.1 mm, and –0.06 mm for X, Y, and Z coordinates, respectively, again indicating no systematic error in any direction.

Table 3. Comparison of three delineated clinical target volumes (CTVs) for each case to evaluate intraobserver variability for observer 1

Intraobserver variability CTV
					Center of gravity coordinates difference [mm]
Patient	DSC*	SD	Mean Hausdorf distance (mm)*	SD	X**	Y**	Z**
1	0.60	0.19	1.19	0.59	1.10	1.04	1.69
2	0.88	0.03	0.82	0.24	1.27	1.87	0.40
3	0.84	0.03	0.37	0.07	0.80	0.59	0.25
4	0.91	0.02	0.51	0.07	0.21	0.78	0.57
5	0.90	0.02	0.73	0.13	0.81	0.98	0.21
6	0.93	0.02	0.50	0.08	0.13	0.51	0.42
7	0.89	0.02	0.50	0.09	0.32	0.87	0.81
8	0.87	0.05	0.77	0.28	0.60	1.22	1.07
9	0.71	0.10	1.23	0.45	0.42	0.93	1.77
10	0.71	0.09	0.51	0.15	0.72	0.75	0.58
Mean	0.82	0.06	0.71	0.22	0.64	0.95	0.78

*mean value of Dice similarity coefficient (DSC) from three segmentations; **mean of the absolute value of coordinate differences from three segmentations; CT — computed tomography; EAM — electroanatomic mapping; SD — standard deviation

DSC values for the comparison among CTVs was lower in all cases compared with DSC values among CT LVs. In contrast, the mean Hausdorf distance between CTVs was lower than for CT LVs. CTV was created from specific EAM points that were distributed in a surface area and did not vary much in the depth of the ventricle wall. In such cases where the object (CTV) is thin and planar (Figure 3), a small object translation can make a big difference in DSC, even if the mean Hausdorf distance remains low. Figure 3 shows an example of three CTVs with a mean DSC of 0.6 and mean Hausdorf distance of 1.19 mm.

Figure 3. Example of three clinical target volumes (CTVs) delineated at least 8 hours apart, with the worst Dice coefficient from all cases

Interobserver variability

Segmented CT LVs showed great similarity, with a mean DSC of 0.91 (range 0.88–0.94) for all cases. However, in the CTV comparison between observers, the mean DSC of all cases was 0.81 (range 0.66–0.94).

Interobserver differences in CT LV segmentation are reflected in different volumes of CT LVs, which can affect CoG and the corresponding CTV position after CT LV to EAM LV registration. CT LV volumes between both observers did not differ significantly (paired t-test p = 0.06), but difference in volumes correlated with DSC values for the compared CTVs (Pearson’s correlation coefficient R = 0.8, p = 0.005).

Discussion

The precise transfer of EAM data into the radiotherapy planning CT is an important part of STAR in patients with scar-related VT after failed catheter ablation. We developed a modification of EAM-CT registration that ensures low intra- and interobserver variability using automatic algorithms. High robustness in overcoming incompatibility between the EAM and DICOM formats carries the prospect of precise targeting without unnecessarily increasing radiation volumes.

To date, different methods have been designed to merge EAM data with CT [8, 9, 1–13]. The simplest method is “indirect” target delineation, based on the manual contouring of CTV from several 3D visualizations to CT slices [8]. However, this method requires good anatomical knowledge and could be very operator dependent and time consuming.

This work was based on a previously published procedure [13], and our current goal was to modify the approach to be faster and more robust and involve minimal manual steps. In particular, we wanted to use specific EAM points to create a CTV model using an algorithm and to replace manual registration of EAM LV to CT LV. We also simulated a scenario in which EAM data contained only the LV and no other structures. The availability of EAM data for at least three chambers can yield a higher EAM-CT merge accuracy [9].

Comparison of segmented left ventricles with left ventricles from electroanatomical mapping showed high similarity between CT LV and EAM LV with DSC values of 0.84 ± 0.01 for observer 1 and 0.85 ± 0.04 for observer 2 and only one case with a DSC < 0.8. This singular case involved the poorest quality of CT with insufficient contrast agent within the LV, rendering segmentation difficult and yielding very irregular EAM LV. Kafi et al.9 also published a method of EAM-CT merge using 3D Slicer software, but with more anatomical segments (cardiac cavities, ascending aorta, and pulmonary artery) and manual alignment of EAM data to CT. The target area was indicated on the surface of the EAM LV as 10 points, all manually connected. After projection of the target contour onto the CT images, observers drew a CTV throughout the LV wall thickness. For the EAM LV and CT LV, this group reported median DSC values of 0.86 (0.78–0.89) for observer 1 and 0.84 (0.82–0.88) for observer 2. These findings are comparable with our results.

To evaluate interobserver agreement we compared segmented CT LVs (mean Hausdorff distance of 1.86 mm, DSC 0.91) on which we moved the EAM LV using surface registration. Kafi et al. [9] evaluated interobserver agreement as median distance between EAM surfaces (2.6 mm) and DSC (0.86). We did not compare volumes of CTVs, which were exactly the same because we used only a set of EAM points with specific tags (marked during electroanatomical mapping). Comparison of CTV similarity between observers showed a mean DSC of 0.81 ± 0.11 and mean Hausdorff distance for all cases of 1.18 ± 1.11 (Tab. 4). We created CTV from marked points on the EAM map, so from this perspective, comparison of our CTVs is a subset of EAM comparisons. Kafi et al. [9] compared the target surface area of interest and volume of CTV. Finally, the median distance among the 10 points (area of interest) was 7 mm (4.3–10 mm) for both observers in their study.

Table 4. Comparison of interobserver variability between observers 1 and 2 in segmented left ventricles (LVs) and delineated clinical target volumes (CTVs)

Interobserver variability
Comparison of CT LV							Comparison of CTV
Patient	DSC	Mean Hausdorf distance [mm]	X*	Y*	Z*	Δ Volume CT LV [cm3]	DSC	Mean Hausdorf distance [mm]	X*	Y*	Z*
1	0.93	1.45	0.74	0.26	0.37	9.54	0.89	0.39	0.51	0.11	0.12
2	0.93	1.52	0.29	0.08	0.13	3.56	0.94	0.48	1.23	0.27	0.16
3	0.92	1.67	0.49	0.40	0.22	24.88	0.68	0.80	2.07	0.58	1.28
4	0.92	1.79	0.54	1.39	0.21	27.48	0.71	1.59	0.05	3.26	0.81
5	0.88	1.98	0.18	1.80	1.89	2.12	0.89	0.85	0.21	0.75	1.79
6	0.92	1.57	2.87	2.68	0.72	2.04	0.84	1.17	1.75	2.01	1.60
7	0.88	2.66	0.18	2.76	1.41	43.32	0.74	1.31	2.60	2.31	2.11
8	0.94	1.33	1.00	0.66	0.62	13.39	0.95	0.32	0.22	0.07	0.15
9	0.89	2.65	1.46	1.93	1.60	38.40	0.66	1.29	3.02	1.27	0.34
10	0.92	1.93	0.49	2.64	0.53	28.73	0.78	0.46	0.14	0.70	1.36
Mean	0.91	1.86	0.82	1.46	0.77		0.81	0.87	1.18	1.13	0.97

*Absolute value of coordinate difference; DSC — Dice similarity coefficient CT — computed tomography; SD — standard deviation

Our data from both observers for all cases altogether showed a mean Hausdorf distance of 3.2 mm (95th percentile, 7.95 mm) for the EAM map to the CT LV and a DSC of 0.84 between the EAM LV and CT LV. Hohmann et al. [12] published a similar method of EAM-CT merge, with a merge process based on manual alignment followed by automated registration using an iterative closest point algorithm (with the LV and thoracic aorta as anatomic landmarks) in 3D Slicer software. For one case, they reported a mean pointwise distance of 3.1 mm (95th percentile, 8.3 mm) for the EAM map to the CT LV endocardial surface and a DSC of 0.83 between the EAM map of the LV and corresponding CT LV.

Brett et al. [11] proposed a workflow for conversion of EAM to DICOM files that requires several manual steps, including placing fiducials to mark the target. This technique relies heavily on the identification of anatomic landmarks. Their method allows for higher precision than an indirect approach, but they did not report intra/interobserver variability or the associated reproducibility rate.

During the conducting of this work two other publications addressing the same issue were produced. Wang et al. [16] built a plug-in module on the 3D Slicer platform. Registration of EAM to cardiac MR/CT data starts with manually placing landmark pairs requiring certain anatomical knowledge. No reproducibility of target delineation is described here. Oh et al. [17] developed an in-house software to read EAM maps, registered them to a planning image set and converted them to DICOM files. A total of 7 patients underwent STAR treatment by defining the target with the proposed method. The authors reported DSC of 0.814 ± 0.053 between the EAM LV and CT LV from 7 cases. The target location on 3D space was inconsistent with no clear trend in terms of similarity but the authors compared the clinical target from EAM with the target defined by 17-segment model [18].

Figure 4. Examples of the best and worst interobserver agreement values for segmented LVs and delineated CTVs: LVs and CTVs from case 8 (DSC of 0.94 for LVs and 0.95 for CTVs) in transversal (A), coronal (B), and sagittal (C) planes (green = observer 1 , pink = observer 2); and LVs and CTVs from case 9 (DCS of 0.89 for LVs and 0.66 for CTVs) in (D) transversal, (E) coronal, and sagittal (F) planes (green = observer 1, pink = observer 2)

Our results demonstrate that we can achieve DSCs of 0.95 and 0.93 for intraobserver comparisons of multiple segmented CT LVs and created CTVs. Similarly, we achieved DSCs of 0.94 and 0.95 for interobserver comparison of segmented CT LV and created CTVs. The fact that observer 2 had only a basic knowledge of heart anatomy from radiotherapy planning underscores the robustness of the proposed procedure. Such results were achieved in cases where the chamber was well defined on contrast CT.

We did find DCS values of 0.6 and 0.66 in the intra- and interobserver comparison of CTVs (the worst cases). The AAPM guideline [19] recommends a DSC of 0.8–0.9 for testing of deformable image registration performance. DSC calculations depend on the volume of a structure so that very large or very small structures may have different expected DSC values for contour uncertainty. As shown in Figure 3, if EAM points for creation of CTV are distributed almost on the same plane, even a small shift will yield a significant difference in the DSC, whereas mean Hausdorff distance will remain low.

Our study is limited by the use of EAM from endocardial mapping, and we did not test our procedure using EAM from epicardial access. All EAM data were exported from the CARTO® 3 system. The weakest step of our procedure is the manual segmentation, which influences the final localization of the CTV. Although EAM inherently cannot generate a perfect model of the LV, as Table 1 shows, a high similarity between CT LV and EAM LV is possible. Our method is fast and can be completed within 10–15 minutes, with results that are equivalent to or better than those previously reported.

Worth noting is that good intra- or interobserver agreement does not guarantee correct localization of the CTV within the heart. The big advantage of our method is elimination of all manual shifts for any of the structures. We believe that manually moving the EAM LV or EAM points could be a significant source of potential error. Also worth noting is that the quality of the LV segmentation affects the position of CTV using our method. In addition, with this method, we delineate the CTV only according to the given EAM points. For STAR purposes, the CTV must be expanded throughout the LV wall thickness. This step is not a part of our method, and the final CTV is created in the radiotherapy planning system by a radiation oncologist.

Conclusion

The high interobserver similarity of segmented LVs and delineated CTVs confirmed the robustness of the proposed method. Even an inexperienced user can perform a precise EAM-CT merge following our workflow instructions. Accurate segmentation of LV is crucial for successful, precise CTV delineation.

Conflict of interest

Authors declare no conflict of interests.

Funding

This study was supported by Ministry of Health, Czech Republic—conceptual development of research organization RVO-FNOs/2022, by grant project AZV NU20-02-00244 from the Ministry of Health of the Czech Republic and by funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No 945119.