INTRODUCTION

The intra-uterine device (IUD) is the first long-acting contraception method used by women in the world [1]. The World Health Organization supports increasing the availability and acceptability of the use of long-acting reversible contraceptives (IUD and implants), inserted if possible, by midwives, general practitioners, and gynecologists [2]. The insertion of an IUD is a procedure that is not without risks, expulsion, malpositioning, and uterine perforation [3]. Uterine perforation is rare, but it may cause serious problems, including bleeding, bowel or bladder perforation, and fistula formation. Many risk factors were suggested, including inexperience of the inserter and inappropriate technique during the IUD insertion [3]. There exists a correlation between lack of training and occurrence of uterine perforation: the onset of perforation is linked to the clinical inexperience of the practitioner and nearly 2/3 of perforations occur during the first 25 IUD insertions of a practitioner [4–6]. Being trained in IUD insertion and feeling self-confident in this procedure are the two most widely identified factors conducive to supporting IUD as a means of contraception during a consultation [7].

The American nonprofit organization, Jhpiego, listed in its training program the different steps of learning how to perform an IUD insertion [8]. Nevertheless, only a few studies have reported on the benefits of simulation-based training (SBT) in terms of performance during IUD insertion training using a validated checklist. Most of them used only self-assessment of satisfaction of trained learners [9, 10]. Nippita used a specific checklist for IUD insertion from the training resource package of family planning and demonstrated the benefits of repetition of simulations regardless of the model used [11]. However, the checklist was not psychometrically tested for validity and reliability evidence. Psychometric analysis ensures that the checklist scores accurately reflect the construct it is intended to measure, i.e., the ability to perform the procedure [12]. Therefore, the primary objective of this study was to provide validity and reliability evidence for the Jhpiego IUD insertion performance assessment checklist enriched by other items for objective evaluation. Secondary objectives were: 1 — to determine the predictive cut-off score for successful IUD insertion in simulation setting; 2 — to analyze the satisfaction of trainees.

MATERIAL AND METHODS

Study design

This prospective, multi-site, non-randomized study took place in different hospitals of the Poitou-Charentes region (1.8 106 inhabitants, France) (Niort, La Rochelle, Rochefort) and the ABS Lab (Anatomy Biomechanics Simulation Laboratory) of the Faculty of Medicine of Poitiers, for a duration of two years from September 3, 2018 to September 3, 2020.

The ABS Lab has an agreement from the Regional Health Agency of Poitou-Charentes for research on healthy volunteers. IRB approval was obtained from the Research Board of the Faculty of Medicine under the number # 2018-04-DO.

All participants signed an informed consent form. Results were kept anonymous.

Creation of the instrument

The checklist was created following INSPIRE guidelines for developing checklists for simulation-based education and simulation-based assessments [12].

Contents

Based on the analysis of comprehensive literature review conducted using the Ovid MEDLINE and PubMed databases, many evaluation tools for IUD exist [13]. Our choice was directed using a checklist for a two-handed technique of insertion of IUD, currently used in Europe. Among the available checklists, the Jhpiego checklist was chosen because it was concise and complete compared to others [8]. It was enriched by some items from the IUD Levonorgestrel (Mirena®) leaflet to implement missing items for “good practice of IUD insertion” (Bayer HealthCare France, Mirena® 52 mg (20 micrograms/24 hours), intrauterine device) [14]. This Mirena® IUD was chosen because its placement procedure was similar to those of the majority of IUDs. It was more convenient for the simulation model used, i.e., with a wide cervix, as Mirena® IUD is wider than cupper IUDs.

The Jhpiego checklist is divided into two sections: preparation of the equipment for IUD insertion, and IUD insertion technique per se. This checklist contains several items that were considered inappropriate for teaching in the French setting (i.e., ask the woman to wash herself, to empty her bladder, remove the speculum after IUD insertion, etc.). Furthermore, a reformulation of some items was necessary for clarification, with less ambiguity.

A Delphi method was carried out to reach a consensus to determine the required steps of the IUD insertion procedure [15]. Our aim was to recruit 10 experts. Inclusion criteria were to be general practitioners (with competence in gynecology) and gynecologists, inserting at least one IUD per week for at least five years. Recruited experts were then contacted by e-mail giving information about the study and sending them the first questionnaire if they consented to participate. Anonymity of answers was respected, each expert receiving a personal e-mail with no knowledge of the other experts’ answers.

Among the 18 French experts chosen, 10 agreed to participate in the study. There were 5 gynecologists, and 5 general practitioners for whom IUD insertion is a frequent procedure in their medical practice. The checklist was then sent to these experts, asking them to rank the importance of each item to successful performance of the task and to avoid the immediate risk of perforation during IUD insertion by using a 7-point Likert scale according to INSPIRE guidelines with free comments [12]. The mean score for each item was calculated. For the next round, Items with a mean rating of 1 to 3 (i.e., not important) were removed from the checklist. Some questions were modified with more precise/accurate definitions according to some experts’ remarks. This process aimed to continue until all experts agreed on the checklist items (i.e., all items have an average score ≥ 4), and no additional revisions are needed, based on the experts’ comments [12]. Two rounds were carried out to obtain a consensus.

Within the initial checklist, 10 items were deleted (check for the absence of bleeding, insertion of the speculum, removal of the speculum, removal of the Pozzi forceps…), three were rephrased (hand washing became hand disinfection) and modified through the Delphi process. The final checklist assigned the same weight to each item, but three items specifically denoted success of insertion: intrauterine position of the IUD, not protruding into the cervix, and correct deployment of the IUD in the uterine cavity (no twisting or pushing against the wall). Because the technical procedures for inserting an IUD are relatively simple, it was more appropriate to evaluate participants by a yes/no choice for each item rather than using a multiple-choice scale. The checklist is presented on Table S1.

Response process

Pilot testing of the checklist was carried out in a simulation setting similar to the one in which we planned to apply it. The response process was tested during an assessment of 10 participants (2 gynecologists, 7 general practitioners, and 1 midwife). Two independent observers who were certified for Gynecology and Obstetrics performed assessment. Sources of errors in transcription of the score and discordance among observers were identified (more than 10% discordance on item score). If necessary, they led to modifications of the checklist (rephrasing aimed at a more precise description of items) to end up with the definitive version. The final version of the IUD insertion performance assessment checklist included 27 items divided into two sections: seven items dealt with the search for contraindications of IUD in the history of the patient, and 20 items assessed the technical aspect of IUD use (equipment set-up, respect of hygiene, technical aspects of insertion) (Tab. S1).

Psychometric testing

Participants and simulation setting

Comparison of scores at different training times was conducted with two other populations of participants: 1/the first (sample A) included experienced practitioners who inserted at least one IUD per month and medical residents who had no practice of IUD insertion. This sample was used to analyze internal consistency and inter-observer reproducibility; 2/the second (sample B) included medical residents from the 1st to 3rd year. These participants were novices, had inserted between 1 and 10 IUDs during their residency, and in a very irregular manner depending on their rotation. Sample B was used to assess evolution of performance scores over time. Each participant was given three attempts at IUD insertion during one hour under direct supervision, called simulation 1 (S1), simulation 2 (S2) and simulation 3 (S3). Each attempt was interspersed with a debriefing and the instructions remained the same.

For all participants, a video was displayed to show the insertion of a cupper IUD and a Levonorgestrel IUD (Mirena® or Jaydess®) on a mannequin prior to the simulation. Moreover, a briefing was given to the participants explaining the anatomy of the mannequin and the equipment available to perform the procedure. Each participant was then given time for hands-on procedures to perform vaginal touch and insert a speculum in the mannequin under supervision.

During simulation, the participants were asked to insert an IUD on the model without supervision. They had their performance assessed using the final version of the checklist. Success rate was assessed during three attempts at IUD insertion with direct supervision. Since the procedures for inserting an IUD are simple, there seemed little point in repeating the insertion more than three times in a single session; this was confirmed during the period when the feasibility of the protocol was tested. The intra-uterine position of the IUD was videotaped using a fibroscopic camera inserted in the wall of the uterus of the mannequin (Fig. S1). The three insertion attempts were filmed with an external camera and timed with a stopwatch. The time between IUD placement in the insertion tube and display of the IUD in the uterine cavity had to be inferior to five minutes for the IUD insertion to be considered as a success (the insertion leaflet recommends not to leave the IUD more than five minutes inside the insertion tube) [14]. The three simulations were successively performed on the same day, each of them followed by a good-judgment debriefing [16].

Figure S1. Modified gynecologic mannequin Zoe (Gaumard®) to observe the placement of the IUD inside the uterine cavity using a fibroscopic camera; A. Gynecologic mannequin Zoe (Gaumard®); B, C. Small cut of the uterine wall and insertion of a fibroscopic camera; D. Visualization of the uterine cavity; E. equipment required for IUD insertion; F. IUD insertion in the cervical canal; G. Remove of the pusher and then the insertion tube in two steps

The primary outcome was the performance assessment score established on the checklist. Secondary outcomes were success rate (IUD not in the uterine cervix, correct display of IUD in uterine cavity), and insertion time less than five minutes. These criteria were assessed after each of the three tries. Other outcomes were concordance between assessment by direct observation and on video.

All the participants assessed the benefit of SBT in IUD insertion by filling out the satisfaction questionnaire at the end of the sessions.

Model

The model used for simulation sessions was the gynecologic mannequin Zoe (Gaumard®). This task-trainer is made up of a female abdomen and pelvis, with an anteverted uterus (8 cm height). This model was modified in our laboratory, the objective being to observe the placement of the IUD inside the uterine cavity by a small cut of the uterine wall and insertion of a fibroscopic camera. It made it possible to evaluate the correct performance of hysterometry and to check the position of the IUD (Suppl. Fig. 1). This choice was made because the model was not suitable for ultrasound monitoring of IUD insertion.

The simulation environment was standardized; all participants from the various hospitals came to the simulation lab for testing. The IUD inserted in the model was the Mirena® IUD.

Observers

The same two trained observers assessed performance for all simulations. All simulations were videotaped. Videos were anonymized and reviewed by the two observers in random order.

Statistics

Analysis was performed with Statview version 4.5 (SAS Institute Inc., Cary, NC). The number of required participants was determined to analyze internal consistency and inter-observer reproducibility in sample A, and to analyze performance in sample B. Based on a previous simulation study with similar design that aimed at developing a performance checklist [17]. The number of required simulations was at least 22 for sample A and 26 for sample B. Kolmogorov-Smirnov test was used to check normal distribution for assessed measures. Continuous variables were described as mean (SD) or median and 1st and 3rd quartile (Q1; Q3). Categorical variables were described as numbers and percentage (%). Internal consistency of the checklist was analyzed by the Cronbach alpha coefficient. Inter-observer reproducibility was analyzed by the intra-class coefficient (ICC), comparison of means, and linear regression analysis (Spearman coefficient and R2). F-test was used to compare variance of scores of observers 1 and 2. Performance of participants was assessed at three different times (S1, S2, and S3). The evolution of performance scores was analyzed, using an ANOVA for repeated measures. In case of statistically significant results, the Scheffe Post hoc test was pre-specified to explore differences between multiple means while controlling the experiment-wise error rate. Performance scores were compared at different times using a Student t-test. Success rate was expressed as percentage. A p value < 0.05 was considered significant. The data collected were used to plot receiver operating characteristic (ROC) curves for the score values. The area under the curve (AUC) as well as the sensitivity, specificity and likelihood ratios were determined for different score thresholds.

RESULTS

General findings

The final version of the checklist included 27 items distributed in four sections: patient history, pre-insertion procedure steps, preparation of IUD, and insertion. All in all, 57 participants carried out 111 simulations (Tab. 1). In sample A, 30 participants carried out 30 simulations. This sample included 19 experienced practitioners (i.e., at least 1 IUD per month): 5 gynecologists, 6 midwives, and 8 general practitioners. This sample also included 11 medical students who had no practice of IUD insertion. Median age was 36 (25; 39) and sex ratio was 9 (30%) males and 21 (70%) females. In sample B, 27 participants carried out 81 simulations. This sample included 27 medical residents from 1st year (n = 9), 2nd year (n = 12), and 3rd year (n = 6). These participants were novices (i.e., fewer than 10 IUDs during their residency). Median age was 26 (25; 27) and sex ratio was 10 (37%) males and 21 (63%) females.

Validity analysis

Validity analysis was carried out on sample A. Twenty-seven residents performed an IUD insertion in simulation and were assessed by the checklist. Internal consistency of the checklist gave a Cronbach alpha coefficient of 0.79 (Tab. 1).

The checklist is discriminative, showing a significant increase in performance scores when the simulations were repeated (F = 77.6, p < 0.0001). The Scheffe post-hoc test showed a difference between S1 and S2 (p < 0.0001). All the subsection scores increased between S1 and S2 (p < 0.0001). All results are given in Table 2. Success rate was 19% for S1, 46% for S2 and 53.8% for S3. The success rate significantly increased between S1 and S2 (p = 0.04).

Receiver operating characteristic (ROC) curve [AUC: 0.792 (95% CI: 0.71–0.89); p < 0.0001] revealed the best score cutoff predictive of 100% sensitivity, i.e., true positive rate or success rate on this model during IUD insertion (Fig. 1). The point maximizing the positive likelihood ratio is the point of 100% sensitivity and the best ratio between sensitivity and specificity, corresponding to a score of 22/27. This finding reflected a very strong correlation between ‘success’ and ‘process’ (score on the checklist) of IUD insertion for a score ≥ 22/27.

Figure 1. Receiver operating characteristic (ROC) curve for the performance score at S3 (n = 27); AUC — area under the curve

Reliability analysis

Reliability analysis was performed on sample A and sample B (Tab. 1). There was no difference between the mean scores of observers 1 and 2 (p = 0.47) or between the mean scores of direct observation and observation of video, carried out by observer 1 (p = 0.99). Comparison of variances of means between the two observers did not find any significant difference (p = 0.94).

There was a very strong correlation between the scores of the two observers: ICC = 0.99 for sample A and B, which represented particularly high inter-observer reproducibility (Tab. 1). In linear regression there was a strong correlation with Spearman coefficient of 0.99 and R2 = 0.90 for sample A (Fig. 2). Linear regression showed a very strong correlation for sample B with R2 = 0.99.

Figure 2. Sample A, correlation between mean scores of observer 1 and observer 2 (n = 30)

The participants considered the mannequin realistic. The vaginal cavity was often considered too smooth with a cervix that was too easy to find. Although not questioned, the presence of the intrauterine camera was greatly appreciated. Participants were very satisfied with their participation in this simulation session (Fig. 3).

Figure 3. Answers to the satisfaction questionnaire (n = 57)

DISCUSSION

Main results

The existing Jhpiego checklist for evaluation of an IUD insertion performance was redesigned and assessed. It was used as an assessment instrument during SBT on a task-trainer. The final version of the checklist included 27 items divided into two sections with a total score of 27. Psychometric testing showed excellent reliability and good internal consistency. To our knowledge, there currently exists no other adjustable tool with psychometric tests showing validity and reliability designed to assess clinical performance during IUD insertion. Furthermore, performance score was highly correlated to success rate of the procedure. Performance assessment with this instrument was easy to apply and well-accepted.

Instrument development and psychometric properties (validity & reliability)

Creating a checklist with strong initial evidence of validity ensures that the steps for performing the procedure are valid, accurate, and comprehensive [12]. The use of this valid and reliable checklist allowed an objective evaluation of the performance in simulation before considering performance of the procedure in a clinical situation. A major improvement of performance scores was found between S1 and S2 in the present study. It could be explained by the benefit of debriefing between each try. This specific reflection time allowed participants to express their feelings, to analyze and correct their performance gaps, and it represented an essential step of the simulation sessions [18], providing safe clinical care and new knowledge [19]. Otherwise, improvement in performance scores could be partially explained by rapidly cycled deliberate practice between simulation assessments [20]: the participant repeats a task until reaching mastery level; along with feedback to close performance gaps, this constitutes an active learning technique [20]. Nevertheless, a memory effect, especially in the past-history section of the performance score, may have explained this improvement.

Use of the instrument

Although some studies comparing simulation to classical teaching methods have not found it beneficial [20], other studies have, and implementation of simulation in curriculum has increased residents’ performances [21]. As a result, the French National Gynecologist and Obstetricians College recommended the completion of traditional teaching by SBT to improve performance of procedures [22]. The present study determined a cut-off score guaranteeing 100% success of insertion of the IUD in a simulation setting. Although the situation in simulation may differ from the clinical situation, from an ethical point of view, the determination of such a score makes it possible to have a minimum pedagogical objective to reach based on metric analysis before considering the real clinical situation.

Intra-uterine device insertion simulation-based education could also be implemented in initial education of different health care providers. It could furthermore be used for reassessment of professionals’ competence in continuous medical education. The low cost of the task-trainer and the minimal equipment required to perform the procedure render this tool usable for teaching in low-income countries, even if the anatomical landmarks could be improved. The checklist could be enriched as practices evolve regarding the placement of IUDs, the evolution of these devices, and the modification of recommendations regarding screening for sexually transmitted infections.

Limitations

The present study is not without limitations. The first limitation involved the task-trainer (as revealed on the questionnaire). The anatomical structures of the mannequin presented technical limitations: the cervix was relatively near the entry of the vagina, favoring IUD placement, which is not the case in clinical practice. The hard plastic of the model offered strong resistance to its manipulation and made it necessary to pull back the insertion tube to replace the IUD inside the cervix, subsequently overestimating this event and decreasing in parallel the success rate. The second limitation involved complication of IUD insertion. While perforation is often assumed to occur at insertion, the phenomenon of late migration of the IUD out of an intra-uterine location is well-recognized as potentially leading to unintended pregnancy. Future research should look for factors that may predict late migration of IUDs from the uterus. For this purpose, the use of the present checklist in clinical situations could be considered. Finally, limitations concerning generalization of the use of the checklist exist. The checklist was tested with a cupper IUD, excluding the multiload and LNG52 cupper IUD and other models of IUD that were not available in France. Moreover, the checklist is related to the “two handed” inserter used in Europe, as opposed to the “single-handed” inserter in use in the USA and Canada. It would be interesting to test the checklist in these application conditions to improve its applicability to the later settings.

CONCLUSIONS

This designed and tested, valid and reproducible checklist for IUD insertion provides an objective assessment of the procedure during SBT. According to the results, it is suggested to use this assessment checklist during simulation sessions, with the aim of obtaining a score ≥ 22/27 — a performance level guaranteeing success in simulation setting — prior to clinical practice. Future studies should focus on assessment of the impact of simulation practice on real-life IUD insertions.

Article information and declarations

Data availability statement

All data analyzed during this study are included in the manuscript and supplemental files. Materials described in the manuscript, including all relevant raw data, are freely available to any scientist wishing to use them for non-commercial purposes, without breaching participant confidentiality. For more details, please contact the corresponding author.

Ethics statement

Research has been performed in accordance with the Declaration of Helsinki. IRB agreement was obtained from the Research Board of the Faculty of Medicine under the number # 2018-04-DO. Participants consented to participate, and “informed consent” was obtained from all subjects. Participants consented to the publication of de-identified data and individual details. The manuscript does not contain individual images or videos.

Author contributions

Contributions of authors: SM supervised the study, provided educational support for the simulation sessions, reviewed and completed the text; ACB and MA carried out the simulation sessions, collected and analyzed the data, analyzed the literature, and wrote the text; CB modified the Zoe mannequin (GaumardR) to obtain an intra-uterine assessment of IUD placement; JPR reviewed the text and made suggestions for the discussion; DAG: was responsible of the methodology; DO had the idea for this research and designed the study. DAG and DO drafted the manuscript, and all authors contributed substantially to its revision. All authors read and approved the final manuscript.

Funding

None declared. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Acknowledgments

The authors would like to thank the experts who participated in the nominal group and all the medical students who participated to the simulation sessions. The authors would like to thank Jeffrey Arsham, an American scientific translator, for having reviewed the English language text.

Conflict of interest

None of the authors have any financial conflicts of interest in connection with the work to disclose.