ORIGINAL PAPER / GYNECOLOGY

Cumulative summation test for learning curve (LC-CUSUM) in outpatient hysteroscopy

Tadeusz Issat1, 2, Jaroslaw Beta3, Marcin Januszewski1, Artur J. Jakimiuk1, 2

1Department of Obstetrics, Women’s Diseases and Oncogynecology, Central Clinical Hospital of Ministry of Interior, Warsaw, Poland
2Center for Reproductive Health, Institute of Mother and Child, Warsaw, Poland
3Women’s Health Directorate; Medway Maritime Hospital, Gillingham, United Kingdom

Corresponding author:

Artur J. Jakimiuk

Center for Reproductive Health

Institute of Mother and Child

Kasprzaka St. 17a, Warsaw, Poland

e-mail: jakimiuk@yahoo.com

ABSTRACT

Objectives: Outpatient hysteroscopy has become an integral part of postgraduate training in gynecology. It is an operator-dependent procedure, however there are no recommendations regarding total number of performed procedures to reach proficiency.

Material and methods: This study aimed to assess the learning curve (LC) using cumulative summation test for learning curve (LC-CUSUM).

Results: A success rate of 97% a failure rate ≥ 10% were established to denote an adequate and an inadequate performance. A third-year trainee needed 56 procedures to reach the predefined level of performance.

Conclusions: As the length of the LC for outpatient hysteroscopy seems highly variable, it is reasonable to provide tailored monitoring while training.

Key words: LC-CUSUM, CUSUM, learning curve, hysteroscopy, outpatient hysteroscopy

Ginekologia Polska 2017; 88, 1: 9–12

INTRODUCTION

Hysteroscopy is one of the basic diagnostic tools in modern gynecology and, as such, has become an essential part of postgraduate and subspecialty training curricula. Learning a new procedure is supervised by experienced surgeons. Regardless, it is difficult to evaluate the level of competence of a trainee, and even then the decision may be subject to bias. Time and number of the procedures depend on the trainee, their tutor, and the department profile, as well as the complexity of the learnt procedure. Numerous authors have emphasized the discrepancies regarding the number of the surgical procedures one need to perform to be deemed competent [1]. Cumulative sum control charts (CUSUM) are widely applied for industrial quality control, and have been found effective for measuring and monitoring healthcare outcomes [2–5]. A new statistical tool, the cumulative summation test for learning curve (LC-CUSUM), was developed to address that issue and to assist in assessment of the learning curve (LC) and competency of a given trainee [6–9]. Both, LC-CUSUM and CUSUM approaches have similar mathematical concept, simple and easy-to-follow graphical forms, and are designed to detect even small changes in performance [10]. LC-CUSUM enables quantitative evaluation of learning processes of individual trainees and it provides a graphical monitoring of the learning curve, hence might be useful for supervisors responsible for the certification. Instead of completing an arbitrary number of procedures, trainees can follow an individualized learning plan, which can be continued until certain level of competency has been reached. The aim of the study was to assess feasibility of the LC-CUSUM in monitoring the learning curve for outpatient hysteroscopy and to evaluate the number of the procedures required to achieve competency.

MATERIAL AND METHODS

Design

Local Ethics Committee approved of the study (73/2012). A total of 100 women underwent outpatient hysteroscopy at the Department of Obstetrics, Women’s Diseases and Oncogynecology, Central Clinical Hospital of Ministry of Interior, Warsaw, between January 1 and June 30, 2015. Written informed consent was obtained. The patients (aged > 18) were referred for hysteroscopy due to the diagnosis of an abnormal endometrium on ultrasound, endometrial polyps, and uterine bleeding. Pelvic ultrasound examination was performed to confirm the initial diagnosis. Women with endometrial polyps of > 30 mm in diameter were excluded and referred for operative hysteroscopy under anesthesia. Potentially pregnant patients, as well as women with lower genital tract infections, gestational trophoblastic disease, and endocervical polyps visualized on a speculum examination, asthma, acute porphyria, hepatitis, renal failure and lactation were also excluded.

Procedure

All procedures were performed by an ob/gyn trainee (post-graduate year 3), only theoretically introduced to the procedures of hysteroscopy or laparoscopy. The 3.2 mm Versascope hysteroscope (Gynecare division of Ethicon, Inc., Somerville, New Jersey, US) was used with normal saline solution as a distention medium. When appropriate, the polyps or fibroids were cut with a Versapoint (Gynecare division of Ethicon, Inc., Somerville, New Jersey, US), and their fragments were extracted with the 5F forceps. For the simple biopsy of the endometrium only the 5F forceps were used. Other devices included a 300W Xenon lamp and a video camera. Distention fluid pressure was generated using an automated flow-meter pump set for 120 mm H2O of intrauterine pressure (BL-Hyst 100093-V2, MISS Medizintechnik GmbH, Germany).

Acceptable and unacceptable failure rates

Failure was defined as ‘inability to complete the procedure due to supervisor taking over, a technical difficulty, or pain intolerance’. Based on our experience (2106 procedures with the failure rate of 3.3%) and the available literature (2–5% failure rate) [11, 12], an acceptable failure rate was set at 3% (p0 = 0.03) and unacceptable failure rate at 10% (p1 = 0.10). As reported previously, the level of acceptable type I error was at α = 0.05 (probability of claiming that the trainee is competent while they are not) and type II error at β = 0.20 (probability of rejecting the fact that the trainee has reached their competency). Based on previously published data, our sample weight X = 0 (success) was calculated at 0.0080043, sample weight X = 1 (failure) at –1.38629, the Average Run Length ARL0 = 40 and the decision interval h = 2.5 [5–9]

Statistical analysis

Patient characteristics were compared using the Chi square test and the Fisher exact test for categorical variables and the U-Mann Whitney for continuous variables. Statistical software package SPSS 17.0 (SPSS Inc. Chicago, US) and Microsoft Excel 2010 software (Microsoft Excel 2010, UK) were used for data analysis.

RESULTS

There were no differences in patient characteristics between success and failure groups (patient characteristics are presented in Table 1). As expected, failure was associated with extended time of the procedure and bigger volume of distension medium. In the study, a trainee needed 56 procedures to reach the predefined proficiency (Figure 1). The model used in the study would confirm surgical competency with 30 procedures, if all performed successfully.

Table 1. Patient characteristics and differences between the two subgroups
Patient characteristics	Success N = 49	Failure N = 7
Age (years); median (IQR)	48.0 (42.0–62.0)	42.0 (38.0–58.0)
Weight (kg); median (IQR)	68.0 (61.0–84.0)	66.0 (65.0–71.0)
Height (cm); median (IQR)	165.0 (160.0–170.0)	160.0 (157.0–168.0)
BMI (kg/m2); median (IQR)	24.8 (22.5–30.5)	25.7 (24.9–29.4)
Parity
Nulliparous; n(%)	15 (30.6)	2 (28.6)
Menopause
Postmenopausal; n(%)	20 (40.8)	3 (42.9)
Indication
Operative; n(%)	29 (59.2)	4 (57.1)
Diagnostic; n(%)	20 (40.8)	3 (42.9)
Procedure
Time (s); median(IQR)	259.0 (202.0–459.0)	350.0 (295.0–671.0)*
Time > 600 s (10 min); n(%)	6 (12.2)	3 (42.9)*
Distension medium (mL); median (IQR)	600.0 (400.0–937.5)	800.0 (750.0–1650.0)*
Distension medium > 1500 mL; n(%)	8 (16.3)	3 (42.9)*
* p < 0.05

 

Figure1. LC-CUSUM plot of obstetrics and gynaecology trainee learning outpatient hysteroscopy

DISCUSSION

The LC-CUSUM was developed to evaluate whether a process has reached a predefined level of performance [9]. The LC-CUSUM is designed to reject the null hypothesis, i.e. that performance is inadequate when the score reaches certain values (decision interval or h). When the score is equal or superior to decision interval, null hypothesis is rejected, resulting in performance being deemed adequate. LC-CUSUM graph plots a score based on the completed procedures on the y-axis against the successive procedures on the x-axis. Performance of a trainee is considered inadequate as long as the score remains below a limit h. Once the score reaches h, a trainee is considered to have reached competency. Importantly, the LC-CUSUM, unlike regular CUSUM charts, has got a holding barrier at zero, which cannot be crossed. The LC-CUSUM score remains at zero, even if a trainee accumulates failures. With such approach, the trainees will not have to unnecessarily compensate for accumulated failures and with appropriate adjustments in their technique may be able to improve quickly and achieve their competency. In this study, the acceptable and unacceptable failure rates were based on our previous experience and the available data [11, 12]. In the proposed model, proficiency is reached when there is an adequate sequence of unsuccessful and successful hysteroscopies confirmed with LC-CUSUM. It has shown an advantage over performing an arbitrary number of the procedures, as the number required to learn outpatient hysteroscopy was 56 rather than the assumed 30. Following the learning curve, our trainee has been signed off and continued to perform the procedures reaching an overall success rate of 97.2% after 200 performed hysteroscopies. The initial settings of LC-CUSUM require background data and should be set accordingly. However, adjustments are simple, making the model easily reproducible, user-friendly, and adaptable for any unit or department. Most importantly, implementing CUSUM as an objective and bias-free assessment tool might help trainees to improve and adjust their technique, before they are ready to perform without supervision.

CONCLUSIONS

As the length of the learning curve for outpatient hysteroscopy seems highly variable, it seems prudent to provide tailored monitoring while training. LC-CUSUM appears to be suitable for monitoring the process of acquiring surgical competency.

REFERENCES

1. 1. Dagash H, Chowdhury M, Pierro A. When can I be proficient in laparoscopic surgery? A systematic review of the evidence. J. Pediatr. Surg. 2003; 38(5): 720–724, doi: 10.1016/jpsu.2003.50192, indexed in Pubmed: 12720179.
2. 2. Steiner SH, Cook RJ, Farewell VT. Monitoring paired binary surgical outcomes using cumulative sum charts. Stat Med. 1999; 18(1): 69–86, doi: 10.1002/(sici)1097-0258(19990115)18:1<69::aid-sim966>3.0.co;2-l, indexed in Pubmed: 9990693.
3. 3. Nizard RS, Porcher R, Ravaud P, et al. Use of the Cusum technique for evaluation of a CT-based navigation system for total knee replacement. Clin. Orthop. Relat. Res. 2004(425): 180–188, doi: 10.1097/01.blo.0000136902.01368.69, indexed in Pubmed: 15292806.
4. 4. Boulkedid R, Sibony O, Bossu-Salvador C, et al. Monitoring healthcare quality in an obstetrics and gynaecology department using a CUSUM chart. BJOG. 2010; 117(10): 1225–1235, doi: 10.1111/j.1471-0528.2010.02632.x, indexed in Pubmed: 20560945.
5. 5. Balsyte D, Schäffer L, Burkhardt T, et al. Continuous independent quality control for fetal ultrasound biometry provided by the cumulative summation technique. Ultrasound Obstet Gynecol. 2010; 35(4): 449–455, doi: 10.1002/uog.7545, indexed in Pubmed: 20052663.
6. 6. Biau DJ, Porcher R, Salomon LJ. CUSUM: a tool for ongoing assessment of performance. Ultrasound Obstet Gynecol. 2008; 31(3): 252–255, doi: 10.1002/uog.5270, indexed in Pubmed: 18307195.
7. 7. Biau DJ, Williams SM, Schlup MM, et al. Quantitative and individualized assessment of the learning curve using LC-CUSUM. Br J Surg. 2008; 95(7): 925–929, doi: 10.1002/bjs.6056, indexed in Pubmed: 18498126.
8. 8. Dessolle L, Fréour T, Barrière P, et al. How soon can I be proficient in embryo transfer? Lessons from the cumulative summation test for learning curve (LC-CUSUM). Hum. Reprod. 2010; 25(2): 380–386, doi: 10.1093/humrep/dep391, indexed in Pubmed: 19923166.
9. 9. Campbell RD, Hecker KG, Biau DJ, et al. Student attainment of proficiency in a clinical skill: the assessment of individual learning curves. PLoS ONE. 2014; 9(2): e88526, doi: 10.1371/journal.pone.0088526, indexed in Pubmed: 24586337.
10. 10. Montgomery DC. Introduction to Statistical Quality Control. 6th edn. Chichester: John Wiley & Sons 2008.
11. 11. Issat T, Beta J, Nowicka MA, et al. Accuracy and diagnostic value of outpatient hysteroscopy for malign and benign disease. Eur. J. Gynaecol. Oncol. 2014; 35(1): 52–55, doi: 10.1016/s0002-9378(97)70376-4, indexed in Pubmed: 24654462.
12. 12. De Angelis C, Santoro G, Re ME, et al. Office hysteroscopy and compliance: mini-hysteroscopy versus traditional hysteroscopy in a randomized trial. Hum. Reprod. 2003; 18(11): 2441–2445, doi: 10.1093/humrep/deg463, indexed in Pubmed: 14585898.