INTRODUCTION

Ovarian cancer is the most aggressive malignant tumor in gynecology, accounting for about 50% of all gynecological cancer deaths. Early identification of benign and malignant ovarian tumors can help improve patient survival [1]. Most adnexal masses (AMs) are found incidentally on physical examination. Almost adnexal masses are asymptomatic, and only a small proportion of AMs may present with symptoms of acute or intermittent pain [1]. Accurate pre-operative diagnosis of the benign and malignant AMs is critical to the prognosis of the patient because the best diagnostic process helps to choose the best treatment plan [2].

The most common imaging method used to find AMs is ultrasound. Subjective assessment (SA) by an experienced sonographer is generally known as the best method for pre-operative differentiation of benign and malignant AMs [3]. However, experienced ultrasound experts are not always available clinically. For less experienced sonographers (less than 5 years of experience in gynecological ultrasound diagnosis), it is important to use objective methods to diagnose AMs [4]. Ultrasound-based diagnostic models and scoring systems [5–8] can be used to predict the malignancy of AMs to help inexperienced sonographers in the diagnosis of AMs. A commonly used diagnostic model is the Risk of Malignancy Index (RMI) [9], which is a risk index calculated based on serum CA125, menopausal status, and ultrasound characteristics to identify benign and malignant AMs, and is currently recommended by many national for AMs properties. The International Ovarian Tumor Analysis Group (IOTA) has developed a multi-tumor prediction model, Assessment of Different NEoplasias in the adneXa (ADNEX) model [10], which is used to describe in detail the characteristics of AMs. ADNEX model can not only distinguish the probability of benign and malignant AMs, but also distinguish between borderline ovarian tumors, stage I ovarian cancer, stage II–IV ovarian cancer, and secondary metastatic ovarian cancers, which includes three clinical features and six ultrasound features [4, 11–13]. Ovarian-adnexal Reporting and Data System (O-RADS) [14] and management system published by the American College of Radiological is modeled after IOTA and aims to provide consistent interpretation, reduce or eliminate ambiguity in ultrasound diagnosis, and provide management recommendations for each risk category.

The purpose of our study was to validate the diagnostic efficacy of O-RADS and will compare it with the ADNEX model, RMI, and SA to differentiate benign from malignant AMs.

MATERIAL AND METHODS

Study design and setting

This is a retrospective study conducted in our hospital. Between January 2020 and December 2021, there were 445 patients with pathological findings of AMs diagnosed by ultrasound were included in the study. While data were being collected, the hospital’s ethics committee approved the study (ethics number: QYFY WZLL 26761). Inclusion criteria were as follows: 1) one or more masses in the adnexal area; 2) ovaries have not undergone surgery, radiotherapy, or chemotherapy; 3) High-quality ultrasound images stored in the database; 4) surgically managed patients and patients hospitalized. The exclusion criteria included O-RADS 0 and O-RADS 1, missing patient clinical data, pregnant women, and patients managed to employ expectant management/conservative methods and outpatients. For one or more adnexal masses, we selected the one with the most suspicious ultrasound features.

The GE Voluson E10, Mindray Reasona 8T were used for ultrasound examinations, respectively. Ultrasound images of adnexal masses were evaluated in all patients using O-RADS, ADNEX model, and RMI by two sonographers with more than five years of experience in gynecological ultrasonography. Ultrasound images of adnexal masses in all patients were subjectively assessed by two sonographers with more than 15 years of experience in gynecological ultrasonography. Focus on the following morphological characteristics for each examined AMs: maximum diameters, papillary projections, external contour, lesion category, wall thickness, the pattern and the score of color Doppler, and the presence of ascites or peritoneal implant. Collect general materials such as the patient’s age, serum tumor markers, and menopausal status. Postmenopausal refers to patients who have been amenorrhea for more than 1 year.

Prediction models

The O-RADS model arises from the International Ovarian Tumor Analysis (IOTA) score, which is based on a retrospective review of the evidence expected to be obtained from prospective Phase IOTA studies and other supporting studies [14]. O-RADS [14] were divided into six categories (O-RADS 0 to 5), covering a range from normal to highly malignant risk. O-RADS: 0) an incomplete evaluation; O-RADS 1) the physiologic category (normal premenopausal ovary); O-RADS 2) the almost certainly benign category ( < 1% risk of malignancy); O-RADS 3) lesions with low risk of malignancy (1% to < 10%); O-RADS 4) lesions with intermediate risk of malignancy (10% to < 50%); and O-RADS 5) lesions with a high risk of malignancy (≥ 50%).

The ADNEX model [10] includes nine variables: age (years), serum carbohydrate antigen 125 (CA125) level (U/mL), type of center (oncology center/other hospital), maximum diameter of the lesion (mm), maximal diameter of the largest solid part (mm), more than 10 locules (yes/no), number of papillary projections (0/1/2/3/≥ 3), acoustic shadow (yes/no), and ascites (yes/no), which allows counting the calculation of the malignant risk for AMs on the website (www.iotagroup.org/adnexmodel). AMs were considered malignant when the overall risk of malignancy was ≥ 10% [10, 15].

The RMI model determines the probability of malignancy risk by ultrasound characteristics (U), menopausal status (M), and serum CA125 level (U/mL). The ultrasound features include solid areas, multilocularity, bilaterality, intra-abdominal metastases, and ascites. When U = 0, ultrasound features are not included, when including ultrasound features, U = 1, and when there are two or more ultrasound features, U = 3. M = 1 for premenopausal women and M = 3 for postmenopausal women. RMI = U × M × CA125 (U/mL) a total score of ≥ 200 was used as a cut-off for malignancy.

Statistical analysis

We used SPSS version 26.0 or MedCalc version 19.0 for statistical analysis. We compared continuous variables using the independent samples t-test or the U-test. We compared categorical variables using the chi-square test. Calculate sensitivity (SE), positive predictive value (PPV), positive likelihood ratio (LR+), specificity (SP), negative predictive value (NPV), and negative likelihood ratio (LR−). We use the kappa index to evaluate the consistency between reviewers for the study. We used the applied receiver operating characteristic (ROC) curve to determine the optimal cutoff value, calculated the area under the curve (AUC), and compared the analysis validity. four diagnosis methods; p < 0.005 was considered significant.

RESULTS

Clinical and sonographic characteristics

A total of 445 patients were included in the study, including 265 benign tumors, 31 borderline tumors, and 149 malignant tumors. A summary of the patient enrollment process is shown in Figure 1. Teratomas are most common in benign tumors, and serous and clear cell carcinomas are most common in malignant tumors (Tab. 1). The clinical and sonographic characteristics of different diagnostic models are shown in Table 2. The age, serum CA125 and Human epididymal protein 4 (HE4) levels in AMs malignant tumor group were higher than those in the benign tumor group (p < 0.001). Irregular external walls, solid tissue, and ascites mainly existed in the malignant group, while acoustic shadow mainly existed in the benign group. In addition, there were significant differences in lesion diameter, lesion type, blood flow score, and the number of papillary locules between benign and malignant masses (p < 0.05).

Figure 1. Flow diagram showing the process of inclusion of patients with adnexal masses in the study; O-RADS — Ovarian-adnexal Reporting and Data System

Reliability analysis

The Kappa index of O-RADS, ADENX model, SA, and RMI was 0.865 (95% CI: 0.834–0.896), 0.851 (95% CI: 0.802–0.899), 0.877 (95% CI: 0.833 0.922), 0.847 (95% CI: 0.797–0.896).

Diagnostic performance of O-RADS, ADNEX model, SA, and RMI

Figure 2 and Table 3 show ROC curves of O-RADS, ADNEX model, SA, and RMI for the diagnosis of AMs in all study populations and premenopausal and postmenopausal women. The diagnostic performance of O-RADS, ADNEX model, and SA were all higher than RMI (p < 0.05), but there was no significant difference among them in pairwise comparison (p > 0.05).

Figure 2. Wg mnie powinno być inaczej: Receiver-operating characteristics curves for detection of malignant disease (including borderline ovarian tumors) for the Ovarian-adnexal Reporting and Data System (O-RADS), Assessment of Different NEoplasias in the adneXa (ADNEX model), subjective assessment (SA), Risk of Malignancy Index (RMI) in the whole population (n = 445); A. In premenopausal (n = 282); B. Postmenopausal (n = 163); C. Subgroups

The diagnostic performance of O-RADS, ADNEX model, SA, and RMI for all study populations was shown in Table 4. When targeting the entire study population, the O-RADS, ADENX model, SA, and RMI had a sensitivity of distinguishing malignant tumors and benign was 93.3% (95% CI: 88.6–96.5), 94, 4% (95% CI: 90.7–97.3), 96.1% (95 %CI: 92.2–98.4), 70.56% (95%CI: 66.3–77.1), respectively, and the specificities of 90.2% (95%CI: 86.0–93.5), 90.6% (95%CI: 86.4–93.8), 90.2% (95% CI: 86.0–93.5), and 92.4% (95% CI: 88.6–95.3), respectively. The sensitivity of the O-RADS, ADNEX model, and SA were all higher than RMI (p < 0.05), while the difference in sensitivity between O-RADS, ADNEX, and SA was not significant (p > 0.05). There was no significant difference in specificity between O-RADS, ADNEX model, SA, and RMI (p > 0.05).

Pre- and postmenopausal subgroups

The incidence of postmenopausal malignant mass was higher than that of premenopausal women (p < 0.05). In the premenopausal subgroup, SA had the highest diagnostic value in distinguishing between benign and malignant AMs, and in the postmenopausal subgroup, the ADNEX model had the highest diagnostic value. In premenopausal women, the difference between the AUC for SA and that for the O-RADS and ADNEX model was not significant (p = 0.271, p = 0.085, and p = 0.241, respectively). In the postmenopausal subgroup, the difference between the AUC for SA and O-RADS was significant (p = 0.011).

DISCUSSION

In this study, we compared O-RADS, ADNEX model SA, and RMI to identify benign and malignant AM. Verifying the diagnostic performance of O-RADS, ADNEX model, RMI, and SA in diagnosing benign and malignant adnexal masses can help improve the diagnostic accuracy of inexperienced sonographers in diagnosing benign and malignant adnexal masses, select appropriate treatment options for patients, and improve patient prognosis.

Our study has shown that the O-RADS, ADNEX model and SA has excellent diagnostic performance in distinguishing between a benign tumor and malignant tumor in AMs, with an AUC of 0.941 (95% CI: 0.915–0.961), 0.925 (95% CI: 0.897–0.948) and 0.931 (95% CI: 0.904–0.953), which was consistent with previous studies [16, 17]. In Basha et al. [18], the O-RADS of AUC was 0.98 (95% CI: 0.96–0.99), which was higher than in this study. However, in the study of Basha et al. [18], the study population was composed of hospital databases from three research institutions and ultrasound characteristics were evaluated by five experienced radiologists, while in this study, the study population was from only one research institution and evaluated by only two sonographers. We think that this is the reason why the AUC of this study is lower than that of Basha et al. [18]. Studies [17, 18] show that when the optimal cut-off is > O-RADS 3, O-RADS has the best diagnostic performance. Such as in the research of Cao L [17] et al., the sensitivity, and specificity of O-RADS were 98.7% (95% CI: 0.947–0.971) and 83.2% (95% CI: 0.802–0.858), respectively. In our study, the sensitivity and specificity of O-RADS were 93.3% (95% CI: 88.6–96.5) and 90.2% (95% CI: 86.6–93.5), respectively. The high sensitivity of O-RADS is because O-RADS uses standardized dictionaries to provide related descriptions and definitions for all normal ovaries and AMs. Using standardized terminology reduces the ambiguity of ultrasound reports and provides appropriate management policies for each lesion in O-RADS.

When compared with other diagnostic models, ROC showed that the AUC of ADNEX model and SA were 0.925 and 0.931 respectively, in this study, the sensitivity of ADNEX model and SA were 96.1% and 94.4%, respectively, which did not represent a significant difference of O-RADS, ADNEX model, and SA (p > 0.05). The results show that O-RADS and ADNEX models have similar diagnostic performance, we think that they may be derived from IOTA and have similar ultrasound terminology. SA by experienced ultrasound experts is considered the most sensitive method for evaluating AMs [3, 8]. However, experienced ultrasound experts are not always available clinically. In this study, the diagnostic performance of the O-RADS and ADNEX model was consistent with the SA (p > 0.05), suggesting that the O-RADS and ADNEX model have excellent diagnostic performance in distinguishing benign tumors from malignant tumors in AMs, and can help less experienced sonographers quickly and accurately determine the nature of AMs.

Although many countries advocate the use of RMI, the poorest performance was seen for RMI in our study. The sensitivity of RMI was 70.56% (95% CI: 66.3–77.1), lower than previous research [19–21]. Reduced sensitivity will result in the omission of some malignant tumors, which will greatly affect the prognosis of patients. We believe that the low sensitivity of RMI is due to the following reasons: firstly, the pathological types of ovarian tumors are complex and varied, and the serum CA125 specificity is not high. Some malignant tumors such as clear cell carcinoma, yolk sac tumor, mucinous carcinoma, etc. have no significant increase in serum CA125; on the contrary, some benign tumors such as endometrioma have a different increase. Secondly, the ultrasound features of RMI only include two-dimensional ultrasound, and do not include Color Doppler ultrasonography, and the ultrasound features of multilocular, solid components, and bilateral are also present in benign ovarian tumors. In our study, the incidence of clear cell carcinoma was second only to plasmacytoma. This may be caused by the different content and pathological types of samples collected. In addition, studies have shown that the incidence of clear cell carcinoma has significant ethnic and geographic differences, with a higher incidence in Asians than in Blacks and Whites, and Asia is also a region with a high incidence of clear cell carcinoma [22, 23].

There was no significant difference in the sensitivity of O-RADS, ADNEX model, and SA between premenopausal and postmenopausal women (all p > 0.05), while the specificity of O-RADS, ADNEX model, and SA in postmenopausal women was lower than that in premenopausal women (p < 0.05). Reduced specificity means an increase in false-positive cases, leading to overdiagnosis of adnexal masses. In this study, false-positive cases were mainly composed of cystadenomas and fibromas in postmenopausal women. Cystadenomas usually show malignant features of more than 10 cysts, and there are multiple papillary projections. Fibroids are solid tumors with abundant blood flow and are misjudged as malignant tumors. For the above-mentioned tumors, we believe that they can be further analyzed in combination with tumor markers [24] or contrast-enhanced ultrasound [25] to reduce the false-positive rate.

Consistency and reproducibility of ultrasound diagnostic models are important for differentiating benign and malignant AMs. The results indicated that the Kappa index values of O-RADS, ADNEX model, SA, and RMI were 0.865, 0.851, 0.877, and 0.847, respectively, indicating better consistency and reproducibility. The previous research [17, 26] conducted a consistency test on O-RADS and ADNEX model, which is consistent with this research.

The advantage of this study lies in the application of different ultrasound diagnostic models to consenting patients, making the evaluation results feasible and comparable.

However, there are limitations to our study: First, since this study only selected patients who underwent surgery, we may have overlooked patients treated conservatively will affect the accuracy of diagnosis; Secondly, this study analyzes static images. If the mass is too large to display all the features, it may cause information bias; Moreover, many malignant cases were included, which resulted in selection bias.

CONCLUSIONS

The sensitivity of O-RADS, ADNEX model, and SA in the diagnosis of adnexal malignant masses was similar and both are superior to RMI. O-RADS and ADENX models have good diagnostic performance and can be used as a substitute for SA to identify benign and malignant AMs. Sonographers with limited experience, have a good effect in differentiating benign and malignant AMs.

Article information and declarations

Acknowledgments

We thank all authors for their contributions to this study.

Conflict of interest

All authors declare no conflicts of interests.