Vol 94, No 10 (2023)
Research paper
Published online: 2023-02-28

open access

Page views 604
Article views/downloads 642
Get Citation

Connect on Social Media

Connect on Social Media

Evaluating the risk of malignancy in adnexal masses: validation of O-RADS and comparison with ADNEX model, SA, and RMI

Rongling Wang1, Zongli Yang1
Pubmed: 36929789
Ginekol Pol 2023;94(10):799-806.

Abstract

Objectives: To evaluate the diagnostic value of Ovarian-adnexal Reporting and Data System (O-RADS), and to compare it with Assessment of Different NEoplasias in the adnexa (ADNEX) model, Subjective Assessment (SA), and Risk of Malignancy Index (RMI) in differentiating benign and malignant adnexal masses (AMs).

Material and methods: Ultrasound characteristics of 445 patients included in the study were retrospectively analyzed and evaluated using diagnostic models. The diagnostic performances of ultrasound diagnostic models were measured by assessing, receiver-operating characteristic curves, sensitivities, positive predictive values, positive likelihood ratios, specificities, negative predictive values, and negative likelihood ratios. Kappa values were used to evaluate inter-reviewer agreement (IRA).

Results: Of the 445 AMs, 265 were benign and 180 were malignant. The area under the curve (AUC) of O-RADS (0.941), ADNEX model (0.925), and SA (0.931) were higher than RMI (0.815) (all p < 0.05). The sensitivity of O-RADS (93.3%), ADNEX model (94.4%), and SA (96.1%) were higher than RMI (70.6%) (p > 0.05), and there was no statistical significance among them (p > 0.05). The specificity of O-RADS, ADNEX model, SA, and RMI was 90.2%, 90.6%, 90.2%, and 92.5%, respectively, with no statistical significance (p > 0.05). All four ultrasound diagnostic methods showed better IRA.

Conclusions: O-RADS, ADNEX model and SA have better diagnostic value in differentiating benign and malignant AMs than RMI.

ORIGINAL PAPER / Gynecology

Ginekologia Polska

2023, vol. 94, no. 10, 799–806

Copyright © 2023 PTGiP

ISSN 0017–0011, e-ISSN 2543–6767

DOI 10.5603/GP.a2023.0019

Evaluating the risk of malignancy in adnexal masses: validation of O-RADS and comparison with ADNEX model, SA, and RMI

Rongling WangZongli Yang
Department of Abdominal Ultrasound, the Affiliated Hospital of Qingdao University, Shandong Province, China
ABSTRACT
Objectives: To evaluate the diagnostic value of Ovarian-adnexal Reporting and Data System (O-RADS), and to compare it with Assessment of Different NEoplasias in the adnexa (ADNEX) model, subjective assessment (SA), and risk of malignancy index (RMI) in differentiating benign and malignant adnexal masses (AMs).
Material and methods: Ultrasound characteristics of 445 patients included in the study were retrospectively analyzed and evaluated using diagnostic models. The diagnostic performances of ultrasound diagnostic models were measured by assessing, receiver-operating characteristic curves, sensitivities, positive predictive values, positive likelihood ratios, specificities, negative predictive values, and negative likelihood ratios. Kappa values were used to evaluate inter-reviewer agreement (IRA).
Results: Of the 445 AMs, 265 were benign and 180 were malignant. The area under the curve (AUC) of O-RADS (0.941), ADNEX model (0.925), and SA (0.931) were higher than RMI (0.815) (all p < 0.05). The sensitivity of O-RADS (93.3%), ADNEX model (94.4%), and SA (96.1%) were higher than RMI (70.6%) (p > 0.05), and there was no statistical significance among them (p > 0.05). The specificity of O-RADS, ADNEX model, SA, and RMI was 90.2%, 90.6%, 90.2%, and 92.5%, respectively, with no statistical significance (p > 0.05). All four ultrasound diagnostic methods showed better IRA.
Conclusions: O-RADS, ADNEX model and SA have better diagnostic value in differentiating benign and malignant AMs than RMI.
Keywords: adnexal masses; O-RADS; ADNEX model; Subjective assessment; RMI
Ginekologia Polska 2023; 94, 10: 799806

Corresponding author:

Zongli Yang

Department of Abdominal Ultrasound, the Affiliated Hospital of Qingdao University, No. 16, Jiangsu Road, Qingdao, Shandong Province, China

e-mail: qingyichaosheng@126.com

Received: 3.06.2022 Accepted: 12.02.2023 Early publication date: 28.02.2023

This article is available in open access under Creative Common Attribution-Non-Commercial-No Derivatives 4.0 International (CC BY-NC-ND 4.0) license, allowing to download articles and share them with others as long as they credit the authors and the publisher, but without permission to change them in any way or use them commercially.

INTRODUCTION

Ovarian cancer is the most aggressive malignant tumor in gynecology, accounting for about 50% of all gynecological cancer deaths. Early identification of benign and malignant ovarian tumors can help improve patient survival [1]. Most adnexal masses (AMs) are found incidentally on physical examination. Almost adnexal masses are asymptomatic, and only a small proportion of AMs may present with symptoms of acute or intermittent pain [1]. Accurate pre-operative diagnosis of the benign and malignant AMs is critical to the prognosis of the patient because the best diagnostic process helps to choose the best treatment plan [2].

The most common imaging method used to find AMs is ultrasound. Subjective assessment (SA) by an experienced sonographer is generally known as the best method for pre-operative differentiation of benign and malignant AMs [3]. However, experienced ultrasound experts are not always available clinically. For less experienced sonographers (less than 5 years of experience in gynecological ultrasound diagnosis), it is important to use objective methods to diagnose AMs [4]. Ultrasound-based diagnostic models and scoring systems [5–8] can be used to predict the malignancy of AMs to help inexperienced sonographers in the diagnosis of AMs. A commonly used diagnostic model is the Risk of Malignancy Index (RMI) [9], which is a risk index calculated based on serum CA125, menopausal status, and ultrasound characteristics to identify benign and malignant AMs, and is currently recommended by many national for AMs properties. The International Ovarian Tumor Analysis Group (IOTA) has developed a multi-tumor prediction model, Assessment of Different NEoplasias in the adneXa (ADNEX) model [10], which is used to describe in detail the characteristics of AMs. ADNEX model can not only distinguish the probability of benign and malignant AMs, but also distinguish between borderline ovarian tumors, stage I ovarian cancer, stage II–IV ovarian cancer, and secondary metastatic ovarian cancers, which includes three clinical features and six ultrasound features [4, 11–13]. Ovarian-adnexal Reporting and Data System (O-RADS) [14] and management system published by the American College of Radiological is modeled after IOTA and aims to provide consistent interpretation, reduce or eliminate ambiguity in ultrasound diagnosis, and provide management recommendations for each risk category.

The purpose of our study was to validate the diagnostic efficacy of O-RADS and will compare it with the ADNEX model, RMI, and SA to differentiate benign from malignant AMs.

MATERIAL AND METHODS

Study design and setting

This is a retrospective study conducted in our hospital. Between January 2020 and December 2021, there were 445 patients with pathological findings of AMs diagnosed by ultrasound were included in the study. While data were being collected, the hospital’s ethics committee approved the study (ethics number: QYFY WZLL 26761). Inclusion criteria were as follows: 1) one or more masses in the adnexal area; 2) ovaries have not undergone surgery, radiotherapy, or chemotherapy; 3) High-quality ultrasound images stored in the database; 4) surgically managed patients and patients hospitalized. The exclusion criteria included O-RADS 0 and O-RADS 1, missing patient clinical data, pregnant women, and patients managed to employ expectant management/conservative methods and outpatients. For one or more adnexal masses, we selected the one with the most suspicious ultrasound features.

The GE Voluson E10, Mindray Reasona 8T were used for ultrasound examinations, respectively. Ultrasound images of adnexal masses were evaluated in all patients using O-RADS, ADNEX model, and RMI by two sonographers with more than five years of experience in gynecological ultrasonography. Ultrasound images of adnexal masses in all patients were subjectively assessed by two sonographers with more than 15 years of experience in gynecological ultrasonography. Focus on the following morphological characteristics for each examined AMs: maximum diameters, papillary projections, external contour, lesion category, wall thickness, the pattern and the score of color Doppler, and the presence of ascites or peritoneal implant. Collect general materials such as the patient’s age, serum tumor markers, and menopausal status. Postmenopausal refers to patients who have been amenorrhea for more than 1 year.

Prediction models

The O-RADS model arises from the International Ovarian Tumor Analysis (IOTA) score, which is based on a retrospective review of the evidence expected to be obtained from prospective Phase IOTA studies and other supporting studies [14]. O-RADS [14] were divided into six categories (O-RADS 0 to 5), covering a range from normal to highly malignant risk. O-RADS: 0) an incomplete evaluation; O-RADS 1) the physiologic category (normal premenopausal ovary); O-RADS 2) the almost certainly benign category ( < 1% risk of malignancy); O-RADS 3) lesions with low risk of malignancy (1% to < 10%); O-RADS 4) lesions with intermediate risk of malignancy (10% to < 50%); and O-RADS 5) lesions with a high risk of malignancy (≥ 50%).

The ADNEX model [10] includes nine variables: age (years), serum carbohydrate antigen 125 (CA125) level (U/mL), type of center (oncology center/other hospital), maximum diameter of the lesion (mm), maximal diameter of the largest solid part (mm), more than 10 locules (yes/no), number of papillary projections (0/1/2/3/≥ 3), acoustic shadow (yes/no), and ascites (yes/no), which allows counting the calculation of the malignant risk for AMs on the website (www.iotagroup.org/adnexmodel). AMs were considered malignant when the overall risk of malignancy was ≥ 10% [10, 15].

The RMI model determines the probability of malignancy risk by ultrasound characteristics (U), menopausal status (M), and serum CA125 level (U/mL). The ultrasound features include solid areas, multilocularity, bilaterality, intra-abdominal metastases, and ascites. When U = 0, ultrasound features are not included, when including ultrasound features, U = 1, and when there are two or more ultrasound features, U = 3. M = 1 for premenopausal women and M = 3 for postmenopausal women. RMI = U × M × CA125 (U/mL) a total score of ≥ 200 was used as a cut-off for malignancy.

Statistical analysis

We used SPSS version 26.0 or MedCalc version 19.0 for statistical analysis. We compared continuous variables using the independent samples t-test or the U-test. We compared categorical variables using the chi-square test. Calculate sensitivity (SE), positive predictive value (PPV), positive likelihood ratio (LR+), specificity (SP), negative predictive value (NPV), and negative likelihood ratio (LR−). We use the kappa index to evaluate the consistency between reviewers for the study. We used the applied receiver operating characteristic (ROC) curve to determine the optimal cutoff value, calculated the area under the curve (AUC), and compared the analysis validity. four diagnosis methods; p < 0.005 was considered significant.

RESULTS

Clinical and sonographic characteristics

A total of 445 patients were included in the study, including 265 benign tumors, 31 borderline tumors, and 149 malignant tumors. A summary of the patient enrollment process is shown in Figure 1. Teratomas are most common in benign tumors, and serous and clear cell carcinomas are most common in malignant tumors (Tab. 1). The clinical and sonographic characteristics of different diagnostic models are shown in Table 2. The age, serum CA125 and Human epididymal protein 4 (HE4) levels in AMs malignant tumor group were higher than those in the benign tumor group (p < 0.001). Irregular external walls, solid tissue, and ascites mainly existed in the malignant group, while acoustic shadow mainly existed in the benign group. In addition, there were significant differences in lesion diameter, lesion type, blood flow score, and the number of papillary locules between benign and malignant masses (p < 0.05).

Figure 1. Flow diagram showing the process of inclusion of patients with adnexal masses in the study; O-RADS — Ovarian-adnexal Reporting and Data System

Table 1. Ovarian-adnexal Reporting and Data System (O-RADS) and histopathological findings of 445 adnexal masses

Histological type

O-RADS

Total

[%]

2

3

4

5

Serous cystadenoma

10

34

2

0

46

10.34

Mucinous cystadenoma

9

46

0

0

55

12.36

Parovarian cyst

5

1

0

0

6

1.35

Endometrioma

23

19

4

3

49

11.01

Simple cyst

8

5

0

0

13

2.92

Mature teratoma

25

38

0

0

63

14.16

Fibroma

1

12

2

1

16

3.60

Hemorrhagic cyst

1

0

5

0

6

1.35

Theca cell tumors

0

5

2

1

8

1.80

Ovarian goiter

0

2

1

0

3

0.67

Brenner tumor

2

0

0

0

2

0.45

Borderline serous cystadenoma

0

1

13

3

17

3.82

Borderline mucinous cystadenoma

1

3

8

1

13

2.92

Borderline clear cell carcinoma

0

0

1

0

1

0.22

Serous carcinoma

0

2

17

75

94

21.12

Mucinous carcinoma

0

0

6

3

9

2.02

Clear cell carcinoma

0

1

12

5

18

4.04

Immature teratoma

0

0

1

1

2

0.45

Endometrioid carcinoma

0

0

5

1

6

1.35

Metastatic tumor

0

0

2

2

4

0.90

Granular cell tumor

0

1

5

0

6

1.35

Yolk Sac Tumor

0

0

0

2

2

0.45

Undifferentiated carcinoma

0

0

1

0

1

0.22

Carcinosarcoma

0

0

0

1

1

0.22

Small neuroendocrine carcinoma

0

0

1

0

1

0.22

Squamous cell carcinoma

0

0

0

1

1

0.22

Dysgerminoma

0

0

0

2

2

0.45

Total

85

170

88

102

445

100.00

Table 2. The clinical and sonographic characteristics for 445 adnexal masses

Pathological results

p

Benign (n = 265)

Malignant (n = 180)

Age, years (mean ± SD)

40.19 ± 15.96

52.34 ± 13.15

< 0.001

Menopausal status

Premenopausal

202

80

< 0.001

Postmenopausal

63

100

CA125 [U/mL]

19.36 (12.13, 35.15)

163.85 (34.83, 615.22)

< 0.001

HE4 [pmol/L]

43.2 (37.25, 51.57)

129.4 (55.56, 398.0)

< 0.001

Laterality of tumor

Bilateral

247

145

Unilateral

18

35

Lesion diameters [mm]

≤ 30

10

2

< 0.001

30 < D ≤ 50

54

14

50 < D < 100

111

68

D ≥ 100

90

96

Lesion category

Unilocular with no solid component

184

21

< 0.001

Unilocular with solid component

18

83

Multilocular cyst with no solid

35

8

Multilocular cyst with solid

6

14

Solid

22

54

Solid tissue

No

219

29

< 0.001

Yes

46

151

Maximum diameter of the lesion [mm]

95 ± 285

605 ± 492

< 0.001

Number of locules

0

22

55

< 0.001

1

204

103

2 10

34

19

> 10

5

3

Number of papillary projectionsa

0

199

94

< 0.001

1

25

23

2

16

19

3

11

8

≥ 4

14

36

Irregular cyst wall

No

249

29

< 0.001

Yes

16

151

Color scoreb

1

221

24

< 0.001

2

33

21

3

7

38

4

4

97

Acoustic shadow

No

204

176

< 0.001

Yes

61

4

Ascites

No

259

108

< 0.001

Yes

6

72

Metastases

No

264

117

< 0.001

Yes

1

63

a — papillary projection: height equal to or greater than 3 mm; b — color Doppler score: Score 1: no flow; Score 2: minimal flow; Score 3: moderate flow; Score 4: very strong flow; SD — standard deviation; CA125 — carbohydrate antigen 125; HE4 — human epididymis protein 4; Data are given as the median (interquartile range) or n (%)

Reliability analysis

The Kappa index of O-RADS, ADENX model, SA, and RMI was 0.865 (95% CI: 0.834–0.896), 0.851 (95% CI: 0.802–0.899), 0.877 (95% CI: 0.833 0.922), 0.847 (95% CI: 0.797–0.896).

Diagnostic performance of O-RADS, ADNEX model, SA, and RMI

Figure 2 and Table 3 show ROC curves of O-RADS, ADNEX model, SA, and RMI for the diagnosis of AMs in all study populations and premenopausal and postmenopausal women. The diagnostic performance of O-RADS, ADNEX model, and SA were all higher than RMI (p < 0.05), but there was no significant difference among them in pairwise comparison (p > 0.05).

Figure 2. Wg mnie powinno być inaczej: Receiver-operating characteristics curves for detection of malignant disease (including borderline ovarian tumors) for the Ovarian-adnexal Reporting and Data System (O-RADS), Assessment of Different NEoplasias in the adneXa (ADNEX model), subjective assessment (SA), Risk of Malignancy Index (RMI) in the whole population (n = 445); A. In premenopausal (n = 282); B. Postmenopausal (n = 163); C. Subgroups

Table 3. The receiver-operating characteristics (ROC) curve comparisons expressed as differences in area under the curve (AUC) and p values for the whole study population

d-AUC(P)

ADNEX

SA

RMI

O-RADS

0.015 (0.022–0.066) p = 0.09

0.009 (0.006–0.02) p = 0.262

0.126 (0.089–0.162)* p < 0.001

ADNEX

/

0.006 (0.005–0.028) p = 0.569

0.110 (0.069–0.150)* p < 0.001

SA

/

/

0.116 (0.076–0.157)* p < 0.001

The method in the left column is used as a reference standard for comparison; *The model in the left row outperforms the corresponding model in the column above; d-AUC, differences in area under the curve. Prediction models: O-RADS — Ovarian-adnexal Reporting and Data System; ADNEX model — Assessment of Different NEoplasias in the adneXa; SA — subjective assessment; RMI — Risk of Malignancy Index; values in parentheses are 95% CI

The diagnostic performance of O-RADS, ADNEX model, SA, and RMI for all study populations was shown in Table 4. When targeting the entire study population, the O-RADS, ADENX model, SA, and RMI had a sensitivity of distinguishing malignant tumors and benign was 93.3% (95% CI: 88.6–96.5), 94, 4% (95% CI: 90.7–97.3), 96.1% (95 %CI: 92.2–98.4), 70.56% (95%CI: 66.3–77.1), respectively, and the specificities of 90.2% (95%CI: 86.0–93.5), 90.6% (95%CI: 86.4–93.8), 90.2% (95% CI: 86.0–93.5), and 92.4% (95% CI: 88.6–95.3), respectively. The sensitivity of the O-RADS, ADNEX model, and SA were all higher than RMI (p < 0.05), while the difference in sensitivity between O-RADS, ADNEX, and SA was not significant (p > 0.05). There was no significant difference in specificity between O-RADS, ADNEX model, SA, and RMI (p > 0.05).

Table 4. Diagnostic performance indices for O-RADS, ADNEX model, SA, and RMI to differentiate benign from malignant adnexal masses in whole study population (n = 445) and in premenopausal (n = 282) and postmenopausal (n = 163) subgroups

SE (95%CI)

SP (95%CI)

PPV (95%CI)

NPV (95%CI)

LR+ (95%CI)

LP– (95%CI)

AUC (95%CI)

All patients

O-RADS

93.3 (88.6–96.5)

90.2 (86.0–93.5)

86.6 (81.7–90.3)

95.2 (92.0–97.2)

9.51 (6.6–13.7)

0.074 (0.04–0.1)

0.941 (0.915–0.961)

ADNEX

94.4 (90.0–97.3)

90.6 (86.4–93.8)

87.2 (82.4–90.8)

96.0 (92.9–97.8)

10.0 (6.9–14.6)

0.061 (0.03–0.1)

0.925 (0.897–0.948)

SA

96.1 (92.2–98.4)

90.2 (86.0–93.5)

86.9 (82.2–90.6)

97.2 (82.2–90.6)

9.8 (6.8–14.1)

0.043 (0.02–0.09)

0.931 (0.904–0.953)

RMI

70.6 (66.3–77.1)

92.5 (88.6–95.3)

86.4 (80.5–90.7)

82.2 (78.6–85.3)

9.35 (6.1–14.4)

0.32 (0.3–0.4)

0.815 (0.776–0.850)

Premenopausal

O-RADS

91.3 (82.8–96.4)

92.1 (87.5–95.4)

82.0 (73.9–88.0)

96.4 (92.9–98.2)

11.52 (7.2–18.5)

0.095 (0.05–0.2)

0.937 (0.902–0.962)

ADNEX

92.5 (84.4–97.2)

92.1 (87.5–95.4)

82.2 (74.2–88.1)

96.9 (93.5–98.5)

11.68 (7.3–18.8)

0.081 (0.04–0.2)

0.923 (0.885–0.951)

SA

97.5 (91.3–99.7)

92.6 (88.0–95.8)

83.9 (76.1–89.4)

98.9 (96.0–99.7)

13.13 (8.1–21.4)

0.027 (0.007–0.1)

0.950 (0.918–0.973)

RMI

57.5 (45.9–68.5)

91.58 (86.9–95.0)

73.0 (62.3–81.6)

84.5 (80.8–87.6)

6.83 (4.2–11.2)

0.46 (0.4–0.6)

0.745 (0.690–0.795)

Postmenopausal

O-RADS

95.0 (88.7–98.4)

84.1 (72.7–92.1)

90.5 (84.3–94.4)

91.4 (81.8–96.2)

5.99 (3.4–10.6)

0.059 (0.03–0.1)

0.930 (0.880–0.964)

SA

95.0 (88.7–98.4)

82.5 (70.9–90.9)

89.6 (83.4–93.7)

91.2 (81.5–96.1)

5.44 (3.2–9.3)

0.061 (0.03–0.1)

0.888 (0.829–0.932)

ADNEX

96.0 (90.1–98.9)

85.7 (74.6–93.3)

91.4 (85.3–95.1)

93.1 (83.7–97.3)

6.72 (3.7–12.3)

0.047 (0.02–0.1)

0.909 (0.853–0.948)

RMI

81.0 (71.9–88.2)

95.2 (86.7–99.0)

96.4 (89.9–98.8)

75.9 (67.7–82.6)

17.01 (5.6–51.5)

0.2 (0.1–0.3)

0.881 (0.821–0.927)

Values in parentheses are 95% CI. Prediction models: Ovarian-adnexal Reporting and Data System (O-RADS); (ADNEX) model — Assessment of Different NEoplasias in the adneXa; SA — subjective assessment; RMI — Risk of Malignancy Index; For O-RADS, cut-off value of > O-RADS 3 was used, for ADNEX models, cut-off value of 10% was used and for the RMI, cut-off value of 200 was used; SE — sensitivity; SP — specificity; PPV — positive predictive value; NPV — negative predictive value; LR+ — positive likelihood ratio; LP– — negative likelihood ratio; AUC — area under receiver-operating characteristic curve

Pre- and postmenopausal subgroups

The incidence of postmenopausal malignant mass was higher than that of premenopausal women (p < 0.05). In the premenopausal subgroup, SA had the highest diagnostic value in distinguishing between benign and malignant AMs, and in the postmenopausal subgroup, the ADNEX model had the highest diagnostic value. In premenopausal women, the difference between the AUC for SA and that for the O-RADS and ADNEX model was not significant (p = 0.271, p = 0.085, and p = 0.241, respectively). In the postmenopausal subgroup, the difference between the AUC for SA and O-RADS was significant (p = 0.011).

DISCUSSION

In this study, we compared O-RADS, ADNEX model SA, and RMI to identify benign and malignant AM. Verifying the diagnostic performance of O-RADS, ADNEX model, RMI, and SA in diagnosing benign and malignant adnexal masses can help improve the diagnostic accuracy of inexperienced sonographers in diagnosing benign and malignant adnexal masses, select appropriate treatment options for patients, and improve patient prognosis.

Our study has shown that the O-RADS, ADNEX model and SA has excellent diagnostic performance in distinguishing between a benign tumor and malignant tumor in AMs, with an AUC of 0.941 (95% CI: 0.915–0.961), 0.925 (95% CI: 0.897–0.948) and 0.931 (95% CI: 0.904–0.953), which was consistent with previous studies [16, 17]. In Basha et al. [18], the O-RADS of AUC was 0.98 (95% CI: 0.96–0.99), which was higher than in this study. However, in the study of Basha et al. [18], the study population was composed of hospital databases from three research institutions and ultrasound characteristics were evaluated by five experienced radiologists, while in this study, the study population was from only one research institution and evaluated by only two sonographers. We think that this is the reason why the AUC of this study is lower than that of Basha et al. [18]. Studies [17, 18] show that when the optimal cut-off is > O-RADS 3, O-RADS has the best diagnostic performance. Such as in the research of Cao L [17] et al., the sensitivity, and specificity of O-RADS were 98.7% (95% CI: 0.947–0.971) and 83.2% (95% CI: 0.802–0.858), respectively. In our study, the sensitivity and specificity of O-RADS were 93.3% (95% CI: 88.6–96.5) and 90.2% (95% CI: 86.6–93.5), respectively. The high sensitivity of O-RADS is because O-RADS uses standardized dictionaries to provide related descriptions and definitions for all normal ovaries and AMs. Using standardized terminology reduces the ambiguity of ultrasound reports and provides appropriate management policies for each lesion in O-RADS.

When compared with other diagnostic models, ROC showed that the AUC of ADNEX model and SA were 0.925 and 0.931 respectively, in this study, the sensitivity of ADNEX model and SA were 96.1% and 94.4%, respectively, which did not represent a significant difference of O-RADS, ADNEX model, and SA (p > 0.05). The results show that O-RADS and ADNEX models have similar diagnostic performance, we think that they may be derived from IOTA and have similar ultrasound terminology. SA by experienced ultrasound experts is considered the most sensitive method for evaluating AMs [3, 8]. However, experienced ultrasound experts are not always available clinically. In this study, the diagnostic performance of the O-RADS and ADNEX model was consistent with the SA (p > 0.05), suggesting that the O-RADS and ADNEX model have excellent diagnostic performance in distinguishing benign tumors from malignant tumors in AMs, and can help less experienced sonographers quickly and accurately determine the nature of AMs.

Although many countries advocate the use of RMI, the poorest performance was seen for RMI in our study. The sensitivity of RMI was 70.56% (95% CI: 66.3–77.1), lower than previous research [19–21]. Reduced sensitivity will result in the omission of some malignant tumors, which will greatly affect the prognosis of patients. We believe that the low sensitivity of RMI is due to the following reasons: firstly, the pathological types of ovarian tumors are complex and varied, and the serum CA125 specificity is not high. Some malignant tumors such as clear cell carcinoma, yolk sac tumor, mucinous carcinoma, etc. have no significant increase in serum CA125; on the contrary, some benign tumors such as endometrioma have a different increase. Secondly, the ultrasound features of RMI only include two-dimensional ultrasound, and do not include Color Doppler ultrasonography, and the ultrasound features of multilocular, solid components, and bilateral are also present in benign ovarian tumors. In our study, the incidence of clear cell carcinoma was second only to plasmacytoma. This may be caused by the different content and pathological types of samples collected. In addition, studies have shown that the incidence of clear cell carcinoma has significant ethnic and geographic differences, with a higher incidence in Asians than in Blacks and Whites, and Asia is also a region with a high incidence of clear cell carcinoma [22, 23].

There was no significant difference in the sensitivity of O-RADS, ADNEX model, and SA between premenopausal and postmenopausal women (all p > 0.05), while the specificity of O-RADS, ADNEX model, and SA in postmenopausal women was lower than that in premenopausal women (p < 0.05). Reduced specificity means an increase in false-positive cases, leading to overdiagnosis of adnexal masses. In this study, false-positive cases were mainly composed of cystadenomas and fibromas in postmenopausal women. Cystadenomas usually show malignant features of more than 10 cysts, and there are multiple papillary projections. Fibroids are solid tumors with abundant blood flow and are misjudged as malignant tumors. For the above-mentioned tumors, we believe that they can be further analyzed in combination with tumor markers [24] or contrast-enhanced ultrasound [25] to reduce the false-positive rate.

Consistency and reproducibility of ultrasound diagnostic models are important for differentiating benign and malignant AMs. The results indicated that the Kappa index values of O-RADS, ADNEX model, SA, and RMI were 0.865, 0.851, 0.877, and 0.847, respectively, indicating better consistency and reproducibility. The previous research [17, 26] conducted a consistency test on O-RADS and ADNEX model, which is consistent with this research.

The advantage of this study lies in the application of different ultrasound diagnostic models to consenting patients, making the evaluation results feasible and comparable.

However, there are limitations to our study: First, since this study only selected patients who underwent surgery, we may have overlooked patients treated conservatively will affect the accuracy of diagnosis; Secondly, this study analyzes static images. If the mass is too large to display all the features, it may cause information bias; Moreover, many malignant cases were included, which resulted in selection bias.

CONCLUSIONS

The sensitivity of O-RADS, ADNEX model, and SA in the diagnosis of adnexal malignant masses was similar and both are superior to RMI. O-RADS and ADENX models have good diagnostic performance and can be used as a substitute for SA to identify benign and malignant AMs. Sonographers with limited experience, have a good effect in differentiating benign and malignant AMs.

Article information and declarations
Acknowledgments

We thank all authors for their contributions to this study.

Conflict of interest

All authors declare no conflicts of interests.

REFERENCES

  1. American College of Obstetricians and Gynecologists’ Committee on Practice Bulletins—Gynecology. Practice Bulletin No. 174: Evaluation and Management of Adnexal Masses. Obstet Gynecol. 2016; 128(5): e210e226, doi: 10.1097/AOG.0000000000001768, indexed in Pubmed: 27776072.
  2. Wynants L, Timmerman D, Verbakel JY, et al. Clinical utility of risk models to refer patients with adnexal masses to specialized oncology care: multicenter external validation using decision curve analysis. Clin Cancer Res. 2017; 23(17): 50825090, doi: 10.1158/1078-0432.CCR-16-3248, indexed in Pubmed: 28512173.
  3. Meys EMJ, Kaijser J, Kruitwagen RF, et al. Subjective assessment versus ultrasound models to diagnose ovarian cancer: A systematic review and meta-analysis. Eur J Cancer. 2016; 58: 1729, doi: 10.1016/j.ejca.2016.01.007, indexed in Pubmed: 26922169.
  4. He P, Wang JJ, Duan W, et al. Estimating the risk of malignancy of adnexal masses: validation of the ADNEX model in the hands of nonexpert ultrasonographers in a gynaecological oncology centre in China. J Ovarian Res. 2021; 14(1): 169, doi: 10.1186/s13048-021-00922-w, indexed in Pubmed: 34857005.
  5. Terzic M, Aimagambetova G, Norton M, et al. Scoring systems for the evaluation of adnexal masses nature: current knowledge and clinical applications. J Obstet Gynaecol. 2021; 41(3): 340347, doi: 10.1080/01443615.2020.1732892, indexed in Pubmed: 32347750.
  6. Van Calster B, Valentin L, Froyman W. Validation of models to diagnose ovarian cancer in patients managed surgically or conservatively: multicentre cohort study. BMJ. 2020; 370: m2614, doi: 10.1136/bmj.m2614 , indexed in Pubmed: 32732303.
  7. Hiett AK, Sonek JD, Guy M, et al. Performance of IOTA Simple Rules, Simple Rules risk assessment, ADNEX model and O-RADS in differentiating between benign and malignant adnexal lesions in North American women. Ultrasound Obstet Gynecol. 2022; 59(5): 668676, doi: 10.1002/uog.24777, indexed in Pubmed: 34533862.
  8. Tavoraitė I, Kronlachner L, Opolskienė G, et al. Ultrasound assessment of adnexal pathology: standardized methods and different levels of experience. Medicina (Kaunas). 2021; 57(7), doi: 10.3390/medicina57070708, indexed in Pubmed: 34356989.
  9. Jacobs I, Oram D, Fairbanks J, et al. A risk of malignancy index incorporating CA 125, ultrasound and menopausal status for the accurate preoperative diagnosis of ovarian cancer. Br J Obstet Gynaecol. 1990; 97(10): 922–929, doi: 10.1111/j.1471-0528.1990.tb02448.x, indexed in Pubmed: 2223684.
  10. Van Calster B, Van Hoorde K, Valentin L, et al. Evaluating the risk of ovarian cancer before surgery using the ADNEX model to differentiate between benign, borderline, early and advanced stage invasive, and secondary metastatic tumours: prospective multicentre diagnostic study. BMJ. 2014; 349: g5920, doi: 10.1136/bmj.g5920, indexed in Pubmed: 25320247.
  11. Esquivel Villabona AL, Rodríguez JN, Ayala N, et al. Two-Step Strategy for Optimizing the Preoperative Classification of Adnexal Masses in a University Hospital, Using International Ovarian Tumor Analysis Models: Simple Rules and Assessment of Different NEoplasias in the adneXa Model. J Ultrasound Med. 2022; 41(2): 471482, doi: 10.1002/jum.15728, indexed in Pubmed: 33890698.
  12. Qian Le, Du Q, Jiang M, et al. Comparison of the diagnostic performances of ultrasound-based models for predicting malignancy in patients with adnexal masses. Front Oncol. 2021; 11: 673722, doi: 10.3389/fonc.2021.673722, indexed in Pubmed: 34141619.
  13. Viora E, Piovano E, Baima Poma C, et al. The ADNEX model to triage adnexal masses: An external validation study and comparison with the IOTA two-step strategy and subjective assessment by an experienced ultrasound operator. Eur J Obstet Gynecol Reprod Biol. 2020; 247: 207–211, doi: 10.1016/j.ejogrb.2020.02.022, indexed in Pubmed: 32146226.
  14. Andreotti RF, Timmerman D, Strachowski LM, et al. O-RADS US risk stratification and management system: a consensus guideline from the ACR ovarian-adnexal reporting and data system committee. Radiology. 2020; 294(1): 168185, doi: 10.1148/radiol.2019191150, indexed in Pubmed: 31687921.
  15. Van Calster B, Van Hoorde K, Froyman W, et al. Practical guidance for applying the ADNEX model from the IOTA group to discriminate between different subtypes of adnexal tumors. Facts Views Vis Obgyn. 2015; 7(1): 3241, indexed in Pubmed: 25897370.
  16. Lai HW, Lyu GR, Kang Z, et al. Comparison of O-RADS, GI-RADS, and ADNEX for diagnosis of adnexal masses: an external validation study conducted by junior sonologists. J Ultrasound Med. 2022; 41(6): 14971507, doi: 10.1002/jum.15834, indexed in Pubmed: 34549454.
  17. Cao L, Wei M, Liu Y, et al. Validation of American College of Radiology ovarian-adnexal reporting and data system ultrasound (O-RADS US): analysis on 1054 adnexal masses. Gynecol Oncol. 2021; 162(1): 107112, doi: 10.1016/j.ygyno.2021.04.031, indexed in Pubmed: 33966893.
  18. Basha MA, Metwally MI, Gamil SA, et al. Comparison of O-RADS, GI-RADS, and IOTA simple rules regarding malignancy rate, validity, and reliability for diagnosis of adnexal masses. Eur Radiol. 2021; 31(2): 674684, doi: 10.1007/s00330-020-07143-7, indexed in Pubmed: 32809166.
  19. Hada A, Han LP, Chen Y, et al. Comparison of the predictive performance of risk of malignancy indexes 1-4, HE4 and risk of malignancy algorithm in the triage of adnexal masses. J Ovarian Res. 2020; 13(1): 46, doi: 10.1186/s13048-020-00643-6, indexed in Pubmed: 32334618.
  20. Lycke M, Kristjansdottir B, Sundfeldt K. A multicenter clinical trial validating the performance of HE4, CA125, risk of ovarian malignancy algorithm and risk of malignancy index. Gynecol Oncol. 2018; 151(1): 159165, doi: 10.1016/j.ygyno.2018.08.025, indexed in Pubmed: 30149898.
  21. Chacón E, Dasí J, Caballero C, et al. Risk of ovarian malignancy algorithm versus risk malignancy index-i for preoperative assessment of adnexal masses: a systematic review and meta-analysis. Gynecol Obstet Invest. 2019; 84(6): 591598, doi: 10.1159/000501681, indexed in Pubmed: 31311023.
  22. Anglesio MS, Carey MS, Köbel M, et al. Vancouver Ovarian Clear Cell Symposium Speakers. Clear cell carcinoma of the ovary: a report from the first Ovarian Clear Cell Symposium, June 24th, 2010. Gynecol Oncol. 2011; 121(2): 407415, doi: 10.1016/j.ygyno.2011.01.005, indexed in Pubmed: 21276610.
  23. Machida H, Matsuo K, Yamagami W, et al. Trends and characteristics of epithelial ovarian cancer in Japan between 2002 and 2015: A JSGO-JSOG joint study. Gynecol Oncol. 2019; 153(3): 589596, doi: 10.1016/j.ygyno.2019.03.243, indexed in Pubmed: 30905436.
  24. Phinyo P, Patumanond J, Saenrungmuaeng P, et al. Diagnostic added-value of serum CA-125 on the IOTA simple rules and derivation of practical combined prediction models (IOTA SR x CA-125). Diagnostics (Basel). 2021; 11(2), doi: 10.3390/diagnostics11020173, indexed in Pubmed: 33530385.
  25. Xiang H, Huang R, Cheng J, et al. Value of three-dimensional contrast-enhanced ultrasound in the diagnosis of small adnexal masses. Ultrasound Med Biol. 2013; 39(5): 761768, doi: 10.1016/j.ultrasmedbio.2012.11.008, indexed in Pubmed: 23453372.
  26. Van Calster B. External validation of ADNEX model for diagnosing ovarian cancer: evaluating performance of differentiation between tumor subgroups. Ultrasound Obstet Gynecol. 2017; 50(3): 406407, doi: 10.1002/uog.17391, indexed in Pubmed: 28004459.