Medical data annotation is the work of adding expert labels to clinical data (scans, slides, reports) so an AI model can learn from it. Get it right and you have a dataset you can trust. Get it wrong and the model learns your mistakes at scale. This guide covers what the work involves, how to run a project, how to keep quality high and how to handle patient data under EU rules.
We’ll keep it practical. Grab a coffee!
What medical data annotation actually means
In imaging, annotation associates labels with images to create labelled training data. Those labels can describe what an image shows, outline a structure, mark a region of interest or pin down a landmark.
Pathology works the same way, with its own twist. Annotation there can range from outlining regions in a whole-slide image to assigning a single category to the slide, and the type you choose depends on what the algorithm is meant to do.
So “annotation” is not one task. It is a family of tasks, and the first decision is always: what question should the model answer?
Which tasks and data types come up most
Three pathology tasks show the range well, as described in the same pathology annotation review:
- Semantic segmentation: pixel-level outlining of areas such as tumours.
- Object detection: finding and counting objects in the image.
- Instance segmentation: detecting and separating each individual object.
The data itself is just as varied. Radiology work talks about formats such as DICOM and NIfTI, while pathology uses whole-slide image formats, and research projects may also use TIFF, BMP and PNG. Add clinical text such as discharge summaries and reports, and you also need text labelling: entities, diagnoses, medications, negations.
Why does that matter for you? Because each format and task needs its own tooling, its own guidelines and its own kind of expert.
How a project runs, step by step
A well-run project follows a clear order. The expert recommendation published in Intelligent Medicine describes it as identifying the machine-learning tasks and data types, choosing annotation methods such as software tools, defining hierarchical label relationships and a structured storage format, then assigning the annotation to trained clinicians for review and revision.
Here is how we’d turn that into a checklist:
- Define the task. One sentence: “The model should flag X in Y.” If you can’t write it, stop here.
- List the data types. Images, text, audio, video? Which formats?
- Write the label hierarchy. Parent and child classes, with definitions and edge cases.
- Pick the tool and storage format. Labels and images must stay linked and versioned.
- Pilot on a small batch. Fix the guidelines before you scale.
- Annotate, review, revise. Never skip the review round.
- Freeze and document the dataset. Future you (and the auditor) will say thanks.
If your project involves human subjects or data derived from them, check whether an ethics committee needs to sign off. Universities run their own, for example the ethics committee at Comenius University’s Faculty of Mathematics, Physics and Informatics, which assesses the ethical acceptability of projects involving human subjects or data obtained from them.
Who should do the labelling?
Short answer: people who understand the data. In digital pathology, expert pathologists assess cases using diagnostic criteria and can help establish the reference standard for an AI model. The same logic holds for radiology, cardiology or clinical text.
A practical split often works well:
- Clinicians define the guidelines and handle ambiguous or high-stakes cases.
- Trained annotators do the volume work under those guidelines.
- A reviewer or arbiter settles disagreements.
Language matters too. If your reports are in Slovak, Czech, German or Hungarian, annotators must read clinical language natively, including abbreviations and local phrasing. A mislabelled “negative finding” is not a typo, it is a data defect.
Keeping labels consistent
Two annotators will rarely agree on everything. Your job is to measure that and shrink it. The research points to a few habits that work:
- Use a standard reference and written guidelines. Following a standard protocol or guideline and using software to track and version data supports consistency.
- Recruit expert clinicians and have evaluators review cases independently.
- Resolve disagreements openly, by voting or through a separate arbiter, rather than quietly picking one label.
One more thing: schedule a quality check before delivery, not after the model underperforms. We’ve written up how to do this in our guide to the audit of annotation quality.
A quick example
Imagine two radiologists outlining a lung nodule. One includes the surrounding haze, the other doesn’t. Both are “right” by their habits. Without a written rule on boundaries, your model learns a blurry average. With a rule and an arbiter for borderline cases, it learns a clear target.
Protecting patient data during annotation
Health data is among the most sensitive data you can handle, so privacy design belongs at the start of the project, not the end.
De-identify first. Medical images used to develop and validate AI tools are typically de-identified, and depending on the project pseudonymisation may be used instead of anonymisation. The difference matters legally: pseudonymised data is still personal data. Our playbook on anonymising personal data explains where that line sits.
Add technical and organisational safeguards. EDPB guidance on health-data research lists pseudonymisation, encryption, confidentiality agreements, restricted access and access logs as measures, and notes that a data protection impact assessment is required when processing is likely to pose a high risk. The same guidance says that for scientific research, GDPR exceptions and restrictions must be applied only to the extent necessary.
Be transparent with patients. Slovakia’s data protection authority stresses that transparency is a practical prerequisite for patients to exercise their rights, and that providers must clearly explain the legal basis and purpose of processing. If consent is your basis, the wording has to be understood by the person signing, which is why informed consent translation deserves real care in multilingual studies.
Mind the AI Act. The European Economic and Social Committee notes that the EU AI Act classifies healthcare AI as high-risk, requiring explainable and auditable models, while GDPR protects health data. For annotation, that means documented datasets: who labelled what, under which guideline, with which review. Good records today save a painful reconstruction later.
Our perspective
We see the same pattern again and again: teams spend months on model architecture and days on the labelling guide. Then they wonder why results wobble. Our view is simple. Annotation is where a healthcare AI project is won or lost, so it deserves the same rigour as the model.
glocco® has worked since 2014 on the language side of regulated industries, and medical is one of the sectors our clients come from. That is why we treat annotation as a language and quality discipline, not just a labelling queue. Clear guidelines, the right people on the right data, a review step and a paper trail. Boring? Maybe. Effective? Absolutely.
One more opinion, free of charge: decide your privacy approach before you send a single file. It is far cheaper to design de-identification into the workflow than to retrofit it.
— glocco®
Where glocco® fits in
If you are building a medical AI product or research dataset in the EU, our data annotation service covers annotation, data collection and data validation across 76 languages. We work with companies in regulated sectors, so documentation and review are part of the job, not an add-on.
Planning a budget? Our guide to data annotation pricing models helps you structure a procurement brief. Prices depend on the project, so tell us about your data and we’ll talk scope.
FAQ
Who can annotate medical data?
People with relevant clinical expertise, supported by trained annotators working to written guidelines. In pathology, for instance, expert pathologists help establish the reference standard. Complex or ambiguous cases should go to a clinician.
How do you check that annotations are accurate?
Use a standard reference and clear guidelines, have several experts review cases independently, and settle disagreements by voting or with an arbiter. Version your data so every change is traceable. A formal annotation quality audit adds an independent check before delivery.
Is anonymised data still covered by GDPR?
Truly anonymised data is outside GDPR, but pseudonymised data is still personal data. Because projects often use pseudonymisation rather than full anonymisation, as the radiology overview notes, plan for GDPR duties unless a data protection specialist confirms otherwise.
Does the AI Act affect annotation work?
Indirectly, yes. Healthcare AI is treated as high-risk, with explainability and auditability expected, so the datasets behind a model need clear documentation. Ask your legal or compliance team how it applies to your specific product.
Sources
- Image annotation and curation in radiology: an overview
- Annotating for Artificial Intelligence Applications in Digital Pathology
- Expert recommendation on collection, storage, annotation
- Ethics committee, Comenius University Faculty of Mathematics, Physics and Informatics
- EDPB Guidelines 3/2020 on health data processing for scientific research
- Slovak Office for Personal Data Protection: transparency enforcement action
- EESC opinion, Official Journal C/2026/24
