Endometriosis Laparoscopy Annotation: Building Surgical Video Datasets for AI
Every laparoscopic endometriosis case looks a little different once you’re actually inside the pelvis. Two patients can come in with nearly identical imaging and leave the OR with completely different findings: one with a few scattered peritoneal spots, the other with dense adhesions fusing the rectum to the back of the uterus. That’s part of why endometriosis is so hard to standardize as a diagnosis, and honestly, it’s the same reason it’s such a pain to turn into a dataset.
If you work in medical AI, you’ve probably already run into this. There’s plenty of interest in AI for gynecologic surgery right now, but not nearly enough laparoscopic video that’s actually been labeled well enough to train on. Diagnostic delays for endometriosis still commonly stretch past seven years, and a good chunk of that delay comes down to how inconsistently the disease gets documented in the first place, whether that’s in imaging, operative notes, or raw footage sitting unlabeled on a hospital server somewhere.
This article is about that specific problem: what it actually takes to turn hours of raw laparoscopic footage into something structured and AI-ready. We’ve written before about how AI is changing endometriosis detection and treatment at a broader level, and about the annotation side of endometriosis data across modalities. This one stays focused on the surgical video piece specifically, because video annotation comes with its own set of problems that ultrasound and MRI simply don’t have.
What “endometriosis laparoscopy annotation” actually means
Raw laparoscopic video isn’t training data on its own. It’s just a video. Somewhere between the recording and the model, someone has to sit down and go through that footage frame by frame, deciding whether it shows disease, what kind, where exactly, how confident they are, and whether any of that even matters for the task the model is being built for.
Depending on what the AI is meant to do, that labeling can look pretty different:
- Classification: does this frame or clip contain visible endometriosis, yes or no
- Detection: where in the frame the lesion is, marked with a bounding box
- Segmentation: a precise, pixel-level outline of the lesion or anatomical structure
- Temporal annotation: marking surgical phases, instrument use, or specific events across the timeline of the video
These aren’t interchangeable, and it’s worth being upfront about that early on. A model built for lesion screening doesn’t need pixel-perfect segmentation masks. A model meant to support surgical planning or estimate disease extent usually does. Getting this wrong at the start, over-annotating when something simpler would have worked, or under-annotating when precision was actually needed, ends up costing a lot of clinical review time down the line.
Why endometriosis is one of the hardest things to label on video
Compare this to something like laparoscopic cholecystectomy, which has fairly standardized anatomy and a fairly predictable sequence of steps. It’s not a coincidence that most public surgical video datasets are built around procedures like that one. Endometriosis just doesn’t offer that kind of consistency.
A few reasons it’s genuinely difficult:
The disease doesn’t look the same twice. Superficial peritoneal lesions can be small, pigmented spots that are easy to miss on a quick pass through the footage. Deep infiltrating endometriosis, on the other hand, often distorts the anatomy so much that the “normal” landmarks an annotator would usually rely on simply aren’t there anymore.
Image quality shifts within a single case. Blood, smoke from energy devices, glare off wet tissue, and instruments blocking the view are all just part of surgery, and all of them make frame-by-frame labeling harder. A good annotation protocol defines upfront how to handle an ambiguous or obstructed frame instead of leaving that call to whoever happens to be labeling it that day.
Specialists don’t always agree. Even experienced gynecologic surgeons can look at the same area and disagree on whether it’s disease, scar tissue, or a normal variant. It’s worth being honest about this rather than glossing over it. Inter-observer disagreement in endometriosis is well documented in the surgical literature, and a dataset that pretends otherwise, forcing every frame into a strict positive or negative label, tends to produce a model that’s confidently wrong on exactly the cases that were ambiguous to begin with.
Confirmed disease depends on more than what’s visible. In plenty of cases, what looked like endometriosis during surgery only gets confirmed, or ruled out, by pathology afterward. ESHRE’s 2022 guideline actually moved away from treating laparoscopy alone as the diagnostic gold standard for this exact reason. That has a real, practical consequence for annotation: video-only labels, without a link back to pathology results, are a weaker source of ground truth than people sometimes assume.
Building a taxonomy before anyone starts labeling
None of this is a reason to avoid building the dataset. It’s a reason to be deliberate about the taxonomy before a single frame gets touched. A reasonable starting structure looks something like this, though the final version really needs to be worked out together with both the clinical and ML sides of the project:
The confidence row probably matters more than it looks like at first glance. Ambiguous findings shouldn’t get forced into a clean positive or negative bucket just to keep the dataset tidy, because that’s exactly how bias creeps in early. A good taxonomy leaves room for “uncertainty,” with a clear path to send those frames to a senior reviewer instead of just guessing.
Not every frame needs to be labeled
A full laparoscopic procedure can run well over an hour at a high frame rate, and annotating every single frame is rarely necessary, and almost never a good use of a clinician’s time. Most projects land somewhere more targeted: selected keyframes, clips around clinically meaningful events, or continuous annotation limited to the segments a model genuinely needs to learn from. The right approach depends on the task itself. A classification model can often work fine from sparser, well-chosen frames, while a temporal or phase-recognition model needs continuous coverage across the relevant part of the case.
Who should actually be doing the labeling
This is where a lot of annotation projects quietly go wrong: treating surgical video labeling like a generic image-tagging job and handing it to annotators with no clinical background at all. Endometriosis doesn’t really tolerate that shortcut. Telling a hemorrhagic cyst apart from an endometrioma, or catching a subtle deep infiltrating lesion near the bowel wall, takes someone who’s actually spent time looking at pelvic anatomy.
A workable setup usually pairs trained medical annotators, often supervised medical students or clinically trained staff, for the first pass, with a gynecologist or surgeon reviewing anything flagged or uncertain. Disagreements go to a senior clinical reviewer rather than getting resolved by majority vote among non-specialists. It’s slower than pure crowd-labeling, no question, but for a disease this visually inconsistent, that’s really the point of doing it this way.
Splitting the dataset without leaking information
One mistake that’s easy to make and genuinely painful to fix later is splitting frames randomly across training, validation, and test sets. Adjacent frames from the same video are often nearly identical, so a random split can end up putting frames from the same patient into both the training set and the test set, meaning the model is effectively being evaluated on something it’s already seen. Splits need to happen at the patient or procedure level, not the frame level, and the finished dataset should be checked for class imbalance and underrepresented lesion types before anyone calls it done.
Privacy and de-identification for surgical video specifically
Laparoscopic video comes with its own de-identification headaches beyond what applies to still images. Overlays showing patient names or dates, audio picked up in the OR, visible monitor screens, and any external camera angle that happens to catch a face all need to be reviewed and stripped before the footage ever reaches an annotator. It’s a similar problem to what came up in a laparoscopic hernia video collection project we ran for a US healthcare AI company, where full anonymization across dozens of procedures from multiple hospital sites had to be built into the workflow from day one instead of tacked on at the end.
What a workable QA process looks like end to end
Roughly, in this order:
- Define the intended use of the model and lock the label taxonomy
- Write annotation guidelines with direct clinical input
- Run a pilot batch and measure inter-annotator agreement
- Refine the guidelines based on where disagreement showed up
- Move to production annotation
- Escalate uncertain cases for senior clinical review
- Audit a representative sample of the finished dataset
- Document annotation provenance and known limitations before delivery
That last step gets skipped more than it should. A dataset that comes with honest documentation of what wasn’t labeled, where confidence ran low, and what the protocol simply didn’t cover is far more useful to a downstream ML team than one that shows up with no caveats at all.
A quick technical note for teams building this from scratch
If you’re trying to figure out how much annotated video you actually need, or benchmarking against what’s already out there, it’s worth looking at recent work on gynecologic surgical video datasets specifically, rather than assuming general-surgery benchmarks (mostly built around cholecystectomy) will transfer cleanly. A 2025 gynecologic laparoscopy dataset paper, GynSurg, walks through how a large, multitask annotated dataset was put together specifically for this surgical domain, and it’s a decent reference point for scale and annotation structure if you’re scoping something similar.
The takeaway
Endometriosis laparoscopic video annotation isn’t harder than other kinds of surgical annotation because of the tools involved. It’s harder because the disease itself resists tidy labels. Lesions look different from patient to patient, image quality inside the pelvis is all over the place, and even experienced surgeons don’t always agree on what they’re looking at. Building a usable dataset means designing for that uncertainty from the start: a taxonomy with room for “uncertain,” clinical review built into the workflow instead of bolted on afterward, patient-level data splits, and documentation that’s honest about where the dataset falls short. Get that foundation right and the annotation actually holds up once a model is trained on it.
Frequently Asked Questions
How is endometriosis annotated in laparoscopic video? Depending on the AI task, annotators use frame-level classification, bounding boxes, polygon outlines, pixel-level segmentation masks, or temporal labels marking surgical phases and events. Most projects end up combining several of these depending on what the model actually needs to learn.
Should every frame in a laparoscopic video be labeled? Usually not. Most projects rely on carefully selected keyframes or clips around clinically relevant events rather than exhaustive frame-by-frame labeling. Full continuous annotation is generally reserved for temporal or phase-recognition tasks that actually need it.
Who should annotate endometriosis surgical video? Trained medical annotators can handle the first pass, but gynecologists or surgeons need to review anything flagged, uncertain, or complex. Given how visually inconsistent endometriosis is, skipping clinical oversight tends to produce lower-quality, inconsistent labels.
How should uncertain or ambiguous lesions be handled? They shouldn’t be forced into a definitive positive or negative label. A solid protocol includes an uncertainty category, a clear path to escalate ambiguous frames to a senior reviewer, and documentation of how often that actually happens across the dataset.
How is data leakage prevented in surgical video datasets? By splitting at the patient or procedure level instead of the frame level. Adjacent frames from the same video are often nearly identical, so a random frame-level split can place near-duplicate footage in both training and test sets, which inflates how good the model looks on paper.
Can ultrasound, MRI, and laparoscopy annotations be combined for one AI project? Yes, but each modality needs its own annotation protocol, and the records need to be linked at the patient level rather than assumed to represent identical findings. Not every lesion visible on video shows up the same way on imaging, and the reverse is true too.
