Visual inspection of transvaginal ultrasound videos to characterize uterine peristalsis: an inter-observer agreement study

Purpose: Contractions in non-pregnant uterine can be assessed by visual inspection of transvaginal ultrasound (TVUS). Many authors have used this method to extract features like contraction frequency and direction. However, visual inspection is a subjective method and the outcome is dependent on the sonographers and video analysts. In this study, we wanted to see which uterine feature is reproducible enough, in terms of inter-observer agreement, to serve as a reliable control for future research. Methods: Six observers assessed 80 TVUS videos, and rated video quality, contraction frequency, direction and timing. One observer assessed operating time. A Fleiss’ kappa (κ) or an intra-class correlation (ICC) was calculated to determine the inter-observer agreement of all features. Results: The inter-observer agreement in frequency was substantial (ICC = 0.68). Conversely, there was just slight to fair agreement in contraction timing and direction and in video quality: ICC = 0.26, κ = 0.17 and κ = 0.16, respectively. Overall, agreement among technical engineers was better than between medical professionals. The level of agreement was correlated with video quality, phase of the menstrual cycle and individual patient (all χ² with p < 0.00). The time to analyze one video ranged between 6 and 20 min. Conclusions: This study shows that visual inspection of TVUS videos is a fairly reproducible method to assess contraction frequency. However, the operating time is too extensive to implement this method in daily practice. Automated methods could offer a solution for this problem in the future.