EU PII Anonimization Multilingual Detector v2

A multilingual PII detector for the 24 official EU languages, built to work on the text real systems see: web pages and raw scrapes, mailing lists, invoices, PDF tables and OCR output. It is the successor of bardsai/eu-pii-anonimization-multilang (v1) and keeps the same label set.

• Model: ModernBERT token classifier, fine-tuned from jhu-clsp/mmBERT-base, 307M parameters, up to 8192-token context.
• Weights: a uniform weight average ("model soup") of three fine-tuning runs from the same base.
• Labels: 35 entity types in BIO format (listed below).

Results

All numbers below use strict span scoring. A gold span counts as found only if every letter and digit in it is predicted with the right type, so missing part of a word is a miss. A predicted span counts as correct only if it lies inside a gold span of the same type and does not cut a word. Types are grouped into person, email, phone, address, ID, secret and URL.

Test set (real text, hand-checked gold)	Spans	v1	v2
Mailing lists, Impressum/contact pages, raw scrapes, PDF text	332	0.891	0.982
↳ of which a PDF registry table		0.873	0.988
Fresh web pages (contact pages from 10 sites)	75	0.921	0.968
FineWeb-2 chunks (multilingual, held out)	48	0.835	0.948
All real text	455	0.885	0.973
Synthetic OCR'd business documents		0.876	0.988

On all 455 real spans v2 has strict precision 0.966 and recall 0.980. The real test sets are small, so differences under about 0.01 are noise.

Public benchmarks (span F1, 7 coarse types; Nemotron and OpenPII train splits are part of both models' training data, Gretel is out of distribution):

Benchmark	v1	v2
gretelai finance PII, multilingual (OOD)	0.615	0.692
nvidia/Nemotron-PII	0.882	0.933
ai4privacy OpenPII	0.928	0.997

Quick start

The scores above use the word-level decoder in pii_detector.py, shipped in this repo. Use it rather than a plain per-token argmax, which emits sub-word fragments.

from huggingface_hub import hf_hub_download
import importlib.util, sys

repo = "bardsai/eu-pii-anonimization-multilang-v2"
path = hf_hub_download(repo, "pii_detector.py")
spec = importlib.util.spec_from_file_location("pii_detector", path)
pii_detector = importlib.util.module_from_spec(spec); spec.loader.exec_module(pii_detector)

det = pii_detector.PIIDetector(repo)          # GPU if available, else CPU
text = "Kontakt: Jan Kowalski, tel. 600 123 456, jan.kowalski@example.pl, ul. Długa 5, 00-238 Warszawa"
for span in det.predict(text):
    print(span)
# {'start': 9, 'end': 21, 'label': 'PERSON_NAME', 'text': 'Jan Kowalski'} ...

print(det.anonymize(text))
# Kontakt: [PERSON_NAME], tel. [PHONE_NUMBER], [EMAIL_ADDRESS], [POSTAL_ADDRESS]

det.anonymize(text, labels={"PERSON_NAME", "EMAIL_ADDRESS"}) redacts only the listed types. Requires transformers (a version with ModernBERT support) and torch.

ONNX / CPU

ONNX exports are in onnx/. The decoder can run them with onnxruntime (the tokenizer still comes from transformers):

det = pii_detector.PIIDetector(repo, onnx="onnx/model_quantized.onnx")   # or "onnx/model.onnx"

Weights	Size	Strict F1, all real text (455 spans)
PyTorch / onnx/model.onnx (FP32)	1.2 GB	0.973
onnx/model_quantized.onnx (dynamic INT8)	0.3 GB	0.968

The FP32 export gives the same predictions as PyTorch. The INT8 file is 4x smaller and loses about half a point.

How the decoder works

1. The text is split into 512-token windows with a 128-token overlap.
2. Each word gets one label from the mean of its sub-token probabilities, with B- and I- probability pooled per type. Words are split on whitespace and on , ; | " < > ( ) [ ] { } * ^, and edge punctuation is trimmed.
3. Neighbouring words of the same type are merged unless the next word looks like the start of a new entity.
4. Format checks drop implausible spans: a link must look like a URL, a phone number needs at least 5 digits, and ID types need at least 2 digits.
5. Every regex-matched email address is returned as EMAIL_ADDRESS.

Labels

PERSON_NAME, PERSON_ALIAS, PERSON_IDENTIFIER, PERSON_ATTRIBUTE, PERSON_ROLE_OR_TITLE, DATE_OF_BIRTH, EMAIL_ADDRESS, PHONE_NUMBER, CONTACT_HANDLE, POSTAL_ADDRESS, LOCATION, GEO_LOCATION, IP_ADDRESS, DEVICE_IDENTIFIER, IDENTIFYING_LINK, ACCOUNT_IDENTIFIER, AUTH_SECRET, BANK_ACCOUNT_IDENTIFIER, PAYMENT_CARD, PAYMENT_CARD_SECURITY, FINANCIAL_AMOUNT, DOCUMENT_IDENTIFIER, DOCUMENT_REFERENCE, VEHICLE_IDENTIFIER, ORGANIZATION_NAME, ORGANIZATION_IDENTIFIER, PROPER_NAME, HEALTH_DATA, BIOMETRIC_DATA, ETHNIC_ORIGIN, POLITICAL_OPINION, RELIGION_OR_BELIEF, SEXUAL_ORIENTATION, TRADE_UNION_MEMBERSHIP, CRIMINAL_OFFENCE_DATA.

Company registration and VAT numbers (NIP, REGON, KvK, USt-IdNr, SIREN and similar) are ORGANIZATION_IDENTIFIER, not personal IDs. The evaluation above scores person, contact, address, ID, secret and link types; GDPR special-category labels are kept from v1 but are not covered by these test sets.

What changed from v1

• Base model: mmBERT-base (ModernBERT, 8k context) instead of XLM-R base.
• Real web text with teacher labels: about 37k chunks of FineWeb-2 / FineWeb text in 24 languages, labelled by Qwen3-30B-A3B-Instruct together with an earlier version of this model. Spans where the teachers disagree are masked out of the loss instead of being taught as "not PII", and every personal value is replaced with a format-preserving surrogate.
• Partial-annotation loss: source datasets that do not annotate a type do not teach the model that this type is "O".
• Synthetic documents: generated invoices, forms, contact blocks, staff and registry tables, Impressum pages and OCR-degraded renders, with country-specific identifier formats, local phone formats and street abbreviations (ul., Avda., C/, Str., P.O. boxes), plus documents written by an open-weight LLM.
• Word-level decoding with format checks (see above).

Training data

All training data is openly licensed or generated:

• ai4privacy/pii-masking-openpii-1m (CC-BY-4.0)
• nvidia/Nemotron-PII (CC-BY-4.0)
• piimb/privy
• HuggingFaceFW/fineweb-2 and FineWeb (ODC-By) text, labelled by Qwen3-30B-A3B-Instruct (Apache-2.0), with surrogate values
• Synthetic documents from bards.ai generators and an Apache-2.0 open-weight LLM

The domains used in the real-text test sets were excluded from the training text.

Limitations

• Remaining error patterns on real text: multi-line addresses with an extra campus or building line, labels glued to values ("T 09 331 01 01"), names glued to the next word, institute names pulled into an address, and venue or organisation names tagged as locations.
• Performance varies by language, domain and input quality. Heavy OCR noise and unusual layouts lower recall.
• This is a detection model, not a compliance program. Use it to support a redaction workflow with review where the stakes require it.

Files

• model.safetensors, config.json: weights and label mapping
• tokenizer.json, tokenizer_config.json: tokenizer
• onnx/model.onnx: FP32 ONNX export (inputs input_ids, attention_mask; output logits)
• onnx/model_quantized.onnx: dynamic INT8 quantization of the ONNX export, for CPU
• pii_detector.py: the decoder used for all reported scores (PyTorch or ONNX)

About bards.ai

We build product ML for teams shipping AI to real users. bards.ai