AstrLink Guard

AstrLink Guard is a small model that finds personal information and credentials in Chinese, English and mixed Chinese and English text. It also works on source code, configuration files, logs, and the API requests that AI coding assistants such as Claude Code, Codex and Cursor send to language models. It runs locally on CPU with ONNX Runtime and returns each detected span with its label and score. Your application decides what to mask.

AstrLink uses this model for privacy detection.

This repository holds version 0.1.2 in two formats: int8/ (dynamic INT8, recommended) and fp32/ (FP32, the higher-accuracy reference).

中文说明

What's new in 0.1.2

Behavior change: private_date now covers only dates about a specific person. The model still flags birth dates, dates on ID cards, passports and other documents, medical and other appointments, and personal travel dates. It no longer flags dates in logs, code and configuration, file listings, HTTP and git headers, record timestamps (created_at, updated_at, …), release and build dates, or system prompts. If your integration expects every date to be masked, stay on 0.1.1 (revision="v0.1.1") or add your own date rule.

• The model flags far fewer non-personal dates. Of 7,743 non-personal dates in coding-assistant requests, INT8 now flags 21, down from 7,435, and FP32 flags 22, down from 7,439. A separate set of 2,815 non-personal dates comes from logs, file listings, git output, headers, record timestamps, release notes, code, system prompts and work events. On that set, INT8 flags 106, down from 2,786, and FP32 flags 112, down from 2,802.
• The trade-off is that the model now misses some personal dates. Of 1,760 personal dates, 0.1.2 finds 81.5% with INT8 and 82.7% with FP32. 0.1.1, which flagged nearly every date, found 98.6% with INT8 and 98.8% with FP32. The model finds birth dates and travel dates most often and misses dates on documents and medical appointments most often (see Dates).
• The model misses fewer credentials entirely. Of 2,158 credentials in coding-assistant requests, INT8 leaves 51 with no part masked, down from 228, and FP32 leaves 45, down from 159. For the 91 AWS access key IDs (AKIA…), the count falls from 8 to 1 with INT8 and from 5 to 1 with FP32.
• There are fewer false positives in coding-assistant requests. Of 187 requests with no personal data or credentials, 20 still get a flagged span with INT8, down from 24, and 24 with FP32, down from 31 (dates not counted).
• The model detects the other nine types more accurately in coding-assistant requests. Macro F1 over the nine types other than private_date rises from 87.91 to 90.58 with INT8 and from 89.42 to 90.84 with FP32. With INT8, common_secret F1 rises from 85.97 to 93.54 and account F1 from 70.36 to 77.27.
• These are the same as in 0.1.1: architecture, tokenizer, the 10 labels, input format, decoding rules, recommended minimum span score, file names and repository layout. Existing integrations can switch models with the same code. Of the changes in what gets flagged, private_date is the one that may need attention.

Model details

Version	0.1.2 (previous release: 0.1.1)
Architecture	BertForTokenClassification, 12 layers, hidden size 384
Parameters	36.9M
Base model	microsoft/Multilingual-MiniLM-L12-H384
Tokenizer	XLM-R SentencePiece Unigram, 40,003 tokens
Languages	Chinese, English
Window	512 tokens (128-token overlap for long inputs)
Formats	ONNX dynamic INT8 (recommended) and ONNX FP32, with weights in an external data file
License	Apache-2.0

Formats

Folder	Format	Model size	General text F1	Code F1	Coding-assistant macro F1 (9 types)
int8/ (recommended)	Dynamic INT8	36 MiB	97.46	93.54	90.58
fp32/	FP32	141 MiB	97.35	93.83	90.84

All scores use the recommended minimum span score of 0.8.

INT8 is the recommended format. It is about a quarter of the size of FP32 and usually faster on CPU.

FP32 is the higher-accuracy reference. Use fp32/ when accuracy matters most, especially for code and credentials. Compared with FP32, INT8 scores 0.29 F1 points lower on code and 0.26 macro F1 points lower on coding-assistant requests. In coding-assistant requests it leaves 51 of 2,158 credentials with no part masked, against 45 for FP32, and it finds 81.5% of personal dates, against 82.7%. In return, INT8 flags fewer harmless values in coding-assistant requests: 20 of 187 entity-free requests get a false positive, against 24 for FP32. On general text the two formats are on par, with INT8 0.11 F1 points higher.

Files

Path	Description
int8/model_int8.onnx	INT8 ONNX graph
int8/model_int8.onnx_data	INT8 weights (external data, must stay next to model_int8.onnx)
fp32/model.onnx	FP32 ONNX graph
fp32/model.onnx_data	FP32 weights (external data, must stay next to model.onnx)
config.json	Model config, label map and decoding rules (one in each folder)
tokenizer.json, tokenizer_config.json	Tokenizer (one in each folder)
labels.json, label_mapping.json	BIO tags and entity type mapping (one in each folder)
astrlink-model.json	AstrLink import descriptor (one in each folder)
CHECKSUMS.json	sha256 and size of the model files in the folder
licenses/	License texts, notices and data sources
SHA256SUMS	sha256 of every file in this repository

int8/ and fp32/ are each a complete model folder. The config.json at the repository root is the same file as in int8/ and fp32/.

File	Size (bytes)	sha256
int8/model_int8.onnx	330,781	0a070d3113615e33ba3b84d6bd486f663a377c04c4b056e7381732179f51832a
int8/model_int8.onnx_data	37,124,608	0ba34216ceead01e0ae2dfd8e46d52b67e7aee80b04f722d9b758613a91c0067
fp32/model.onnx	219,740	994d61395afadffd0b81aa94c95f9f7b6ba02e79771b6109a72eeb9b541c156d
fp32/model.onnx_data	147,443,712	2298b89ea747a212820abb81260027b2bc6383fa6615277e24546e535031b734
tokenizer.json (both folders)	1,502,881	0cf3b235ef125658015a6c8026a5fe6f28088bf825101a990ec18a10200feef4
config.json (root and both folders)	1,804	2c20003590d642e273c03c4c7c82d9849fa75f64bf29a71ec0a0c9a20ba4a150

Entity types

The model uses BIO tagging with 21 tags.

Label	Description
private_person	Person name
phone	Phone number (mobile, or landline with area code)
email	Email address
private_address	Physical address (home, shipping, office)
private_date	Date, time of day or timestamp about a specific person: birth date, dates on ID cards, passports and other documents, appointments, personal travel
account	Personal account or service number: user name, QQ / WeChat ID, member or employee number, non-bank stored-value or campus card number
payment_card	Bank or credit card number
ip_address	IP address (without port)
url	URL
common_secret	Password, PIN, API key, access token, cookie key=value, private key, public key

The model leaves these values unlabeled:

• order, ticket, tracking and transaction numbers
• amounts, counts, durations and version numbers
• hashes, checksums and request IDs
• key fingerprints and signatures
• OAuth client IDs
• short internal numeric IDs
• partially masked values, for example with only the last 4 characters visible
• placeholders
• dates and times that are not about a specific person, such as log timestamps, dates in code and configuration, file listings, HTTP and git headers, record timestamps, release dates and the current date in a system prompt

Quick start

pip install onnxruntime tokenizers numpy huggingface_hub

import json

import numpy as np
import onnxruntime as ort
from huggingface_hub import snapshot_download
from tokenizers import Tokenizer

d = snapshot_download("QuantumNous/astrlink-guard", allow_patterns=["config.json", "int8/*"]) + "/int8"
onnx_file = "model_int8.onnx"  # for the FP32 variant: allow_patterns=["config.json", "fp32/*"], "/fp32", "model.onnx"

tok = Tokenizer.from_file(f"{d}/tokenizer.json")
id2label = json.load(open(f"{d}/labels.json"))["id2label"]
sess = ort.InferenceSession(f"{d}/{onnx_file}", providers=["CPUExecutionProvider"])


def detect(text):
    enc = tok.encode(text)  # one window: at most 512 tokens including <s> and </s>
    ids = np.array([enc.ids], dtype=np.int64)
    feed = {"input_ids": ids, "attention_mask": np.ones_like(ids), "token_type_ids": np.zeros_like(ids)}
    logits = sess.run(["logits"], feed)[0][0]
    probs = np.exp(logits - logits.max(-1, keepdims=True))
    probs /= probs.sum(-1, keepdims=True)
    spans, cur = [], None
    for label_id, p, (start, end) in zip(probs.argmax(-1), probs.max(-1), enc.offsets):
        if start == end:  # special tokens
            continue
        tag, _, kind = id2label[str(label_id)].partition("-")
        if tag == "O":
            cur = None
        elif tag == "B" or cur is None or cur["label"] != kind:
            cur = {"label": kind, "start": start, "end": end, "score": float(p)}
            spans.append(cur)
        else:
            cur["end"] = end
            cur["score"] = min(cur["score"], float(p))
    for s in spans:
        s["text"] = text[s["start"]:s["end"]]
    return spans


print(detect("请联系张伟，电话 13812345678，邮箱 zhangwei@example.com，收货地址：北京市海淀区中关村大街 27 号。"))

To pin this release, pass revision="v0.1.2" to snapshot_download.

Example output with INT8 (all values are made up):

请联系张伟，电话 13812345678，邮箱 zhangwei@example.com，收货地址：北京市海淀区中关村大街 27 号。
  private_person   张伟                               0.9953
  phone            13812345678                        0.9952
  email            zhangwei@example.com               0.9966
  private_address  北京市海淀区中关村大街 27 号       0.9943

Contact Jane Doe at jane.doe@example.org or +1 415-555-0132. Server 10.0.3.17, api_key = "q7Vt2LmX9pRk4sWz8bNc3yHd6fJg1aE5"
  private_person   Jane Doe                           0.9952
  email            jane.doe@example.org               0.9965
  phone            +1 415-555-0132                    0.9946
  ip_address       10.0.3.17                          0.9896
  common_secret    q7Vt2LmX9pRk4sWz8bNc3yHd6fJg1aE5   0.9963

订单号 202409301234567890，金额 128.00 元，已于 2026-09-30 14:05 发货。
  (no spans)

王芳的生日是 1992 年 7 月 8 日，护照有效期至 2031-05-01。
  private_person   王芳                               0.9940
  private_date     1992 年 7 月 8 日                  0.9843
  private_date     2031-05-01                         0.9618

[2026-09-30 14:05:12] INFO request served in 35 ms
  (no spans)

FP32 returns the same spans for these examples, with scores within 0.01 of INT8. The shipping time in the third example and the log timestamp in the last one are not about a person, so 0.1.2 leaves them unflagged (0.1.1 flagged the shipping time as private_date). The model flags the birth date and the passport expiry date in the fourth example, which are about a person.

Offsets are Unicode character positions in the input string.

Long inputs

The example handles a single 512-token window. For longer text, run 512-token windows with a 128-token overlap and merge the spans.

Decoding and scores

config.json records the decoding rules: BIO decoding constrained by character offsets (bio-offset-consistency-v1) and a cleanup step that corrects where IP address and URL spans start and end (existing-ip-and-dynamic-url-v1). The plain argmax decoding in the example may give slightly different spans.

Every span has a score. Drop spans that score below the recommended minimum of 0.8. This release is calibrated for that setting. A higher minimum gives fewer false positives at some cost in recall, and a lower one adds false positives.

Using with AstrLink

The following are AstrLink settings and behavior. Code that runs the model directly can choose its own.

• AstrLink drops spans that score below 0.8 by default.
• AstrLink's model catalog recommends the INT8 variant.
• astrlink-model.json in each folder is AstrLink's import descriptor.
• AstrLink accepts up to 131,072 tokens in a single request and rejects a longer request in full, with an explicit error. This cap belongs to AstrLink. With the windowing described in Long inputs, the model can process text of any length.
• When this repository gets a new tagged release, AstrLink offers it as an update.

Evaluation

All results come from in-house benchmarks. The evaluation drops spans that score below the recommended minimum of 0.8, as AstrLink does. P, R and F1 are entity-level exact-match percentages, and the 95% confidence intervals come from a document-level bootstrap.

The benchmarks follow the 0.1.2 labeling, where only dates about a specific person are entities. The same benchmarks and scoring apply to 0.1.1, so its numbers here are lower than in the 0.1.1 README, mostly because it flags the other dates. Rows marked "9 types" leave out private_date and compare the other nine types.

General text (2,200 documents: 840 English, 840 Chinese, 520 mixed)

	Entities	FP32 P	FP32 R	FP32 F1	INT8 F1
All	3,681	97.75	96.96	97.35 [96.83, 97.82]	97.46
English	1,442	97.23	94.87	96.03	96.21
Chinese	1,437	97.43	97.49	97.46	97.59
Mixed	802	99.26	99.75	99.50	99.44

Entity type	Entities	FP32 F1	INT8 F1
url	360	100.00	100.00
email	407	99.26	99.26
ip_address	315	100.00	100.00
payment_card	319	99.37	99.37
phone	446	98.76	98.99
private_person	507	97.52	98.12
common_secret	352	99.14	98.86
private_address	365	97.55	97.81
account	514	92.95	92.92
private_date	96	75.11	74.65

Macro F1 over the 9 types other than private_date is 98.29 with FP32 and 98.37 with INT8. Documents with no entities that get a false positive: 0 of 755 with FP32, 1 of 755 with INT8.

Code, configs and logs (709 documents)

	Entities	FP32 P	FP32 R	FP32 F1	INT8 F1
All	1,473	95.31	92.40	93.83 [91.91, 95.42]	93.54
English	755	95.78	90.20	92.91	92.25

FP32 F1 by type: url 98.54, common_secret 91.71, account 94.21, ip_address 100.00, private_person 94.74. Documents with no entities that get a false positive: 3 of 261 (1.1%) with FP32, 5 of 261 (1.9%) with INT8.

Coding-assistant requests (1,200 requests, 22,667 text segments)

The set holds fictional API requests from coding assistants, in Anthropic Messages, OpenAI Responses and Chat Completions formats. The evaluation scores each string in the request JSON as a separate text segment, the same way AstrLink splits a request. There are 187 requests with no entity at all. Such a request counts as a false positive if any of its segments gets a span. That count leaves out dates, which are covered in Dates. The set has only 7 personal dates, so macro F1 is over the other nine types.

	FP32	INT8
Macro F1, 9 types	90.84	90.58
Macro F1, 9 types, weakest language (English)	89.62	89.34
account F1	78.11	77.27
common_secret F1	93.35	93.54
Entity-free requests with a false positive	24 / 187 (12.8%)	20 / 187 (10.7%)
Credentials with no part masked	45 / 2,158	51 / 2,158
AWS access key IDs with no part masked	1 / 91	1 / 91
Non-personal dates flagged	22 / 7,743	21 / 7,743

A credential counts as having no part masked when no kept span of any type overlaps it. Its full value would then be sent in clear text.

Dates

The personal dates are 1,760 dates about a specific person, taken from coding-assistant requests, general text and a separate personal-date set (1,657 dates in Chinese, English and mixed text). The non-personal dates are 2,815 dates from the same kinds of text that are not about a person. A personal date counts as found, and a non-personal date as flagged, when a kept span overlaps it. The personal-date category rows cover the separate personal-date set.

	0.1.1 INT8	0.1.2 INT8	0.1.1 FP32	0.1.2 FP32
Personal dates found (of 1,760)	98.6%	81.5%	98.8%	82.7%
Birth dates (396)	99.7%	92.4%	99.7%	93.2%
Personal travel (436)	99.3%	86.7%	99.3%	88.1%
Dates on ID cards, passports and other documents (414)	97.3%	74.9%	97.6%	76.8%
Medical and other appointments (411)	99.5%	70.8%	99.5%	71.5%
Non-personal dates flagged (of 2,815)	2,786	106	2,802	112
Log lines (360)	359	0	360	0
git output (372)	372	0	372	0
File listings (326)	320	1	325	1
HTTP and email headers (218)	218	10	218	12
Record timestamps (658)	653	19	658	26
Release and build dates, certificate and token validity (182)	177	0	178	0
Dates in code, configuration and queries (219)	208	23	212	24
Current date in a system prompt (304)	303	12	303	9
Work events: deploy windows, incidents, deadlines, meetings (176)	176	41	176	40
Non-personal dates flagged in coding-assistant requests (of 7,743)	7,435	21	7,439	22

With INT8, the model finds 74.8% of personal dates in tool output (file contents, query results, API responses) and 92.4% of personal dates that the user writes.

Keys inside URLs (300 documents, 225 keys, 248 URLs)

These documents put keys and tokens in URL query strings, paths, fragments and connection strings, in Chinese, English and mixed text.

	FP32	INT8
Keys detected as common_secret	222 / 225 (98.7%)	222 / 225 (98.7%)
Keys swallowed into a URL span	3	3
URLs with exact start and end	241 / 248	241 / 248

Compared with 0.1.1

Both versions use the same benchmarks and scoring, at minimum score 0.8, and each pair of columns compares the same format.

	0.1.1 INT8	0.1.2 INT8	0.1.1 FP32	0.1.2 FP32
Non-personal dates flagged in coding-assistant requests (of 7,743)	7,435	21	7,439	22
Non-personal dates flagged, separate set (of 2,815)	2,786	106	2,802	112
Personal dates found (of 1,760)	98.6%	81.5%	98.8%	82.7%
Credentials with no part masked (of 2,158)	228	51	159	45
AWS access key IDs with no part masked (of 91)	8	1	5	1
Entity-free requests with a false positive, dates not counted (of 187)	24	20	31	24
Coding-assistant macro F1, 9 types	87.91	90.58	89.42	90.84
General text F1	90.49	97.46	90.79	97.35
General text macro F1, 9 types	97.89	98.37	98.13	98.29
Code F1	90.09	93.54	91.19	93.83
Keys inside URLs detected (of 225)	97.8%	98.7%	98.2%	98.7%
URLs with exact start and end	240 / 248	241 / 248	240 / 248	241 / 248

Some results got worse. In coding-assistant requests, private_address F1 falls from 88.06 to 86.42 with INT8 (FP32: 89.05 to 87.10). Keys swallowed into a URL span go from 2 to 3. With INT8, entity-free code documents with a false positive go from 4 to 5 of 261.

Limitations

• The model can miss entities and flag harmless values. Treat it as one layer of privacy protection.
• The model misses personal dates more often than 0.1.1 did. With INT8, 0.1.2 finds 81.5% of personal dates (82.7% with FP32), against 98.6% for 0.1.1. It misses dates on documents, medical appointments and dates in tool output most often. If every date has to be masked, stay on 0.1.1 or add a rule-based date filter.
• The model still flags some non-personal dates. Most of them are work events such as deploy windows, deadlines and meetings (41 of 176 with INT8), dates in code and configuration (23 of 219) and record timestamps (19 of 658).
• INT8 gives up some accuracy for its smaller size. In coding-assistant requests it leaves more credentials with no part masked than FP32 (51 against 45 of 2,158) and finds fewer personal dates (81.5% against 82.7%). Use fp32/ where a missed key costs more than a larger model.
• Code and coding-assistant requests are harder than prose. With INT8, 1.9% of entity-free code documents and 10.7% of entity-free coding-assistant requests still get at least one false positive (FP32: 1.1% and 12.8%).
• account is the weakest entity type in coding-assistant requests, and common_secret is the weakest in code.
• The model is most likely to miss passwords, along with keys that have no field name or vendor prefix nearby, such as dash-separated keys.
• The model may label card numbers from issuers it did not see in training, such as transit, membership or badge numbers, as payment_card when they should be account.
• The model may label configuration values whose key name ends in key as common_secret, even when they are not secrets.
• Government ID numbers, such as national ID card numbers, have no dedicated entity type.
• The evaluation covers Chinese and English only.

Training data

Training used these public datasets and repositories:

• NVIDIA Nemotron-PII (CC BY 4.0)
• Wismut/nym-pii-multilingual-data (MIT)
• Samsung CredData (Apache-2.0) and the open-source repositories it references
• Other open-source code and documentation, including django/django (BSD-3-Clause), dromara/hutool (MulanPSL-2.0) and google/eng-practices (CC BY 3.0)

Training also used fictional synthetic examples written for this project, including code, configuration files, logs and coding-assistant requests. The full list with revisions and license texts is in licenses/. This repository contains no training data.

License

The model is released under the Apache License 2.0. The base model, microsoft/Multilingual-MiniLM-L12-H384, is MIT-licensed.

Attribution:

• Contains data derived from NVIDIA Nemotron-PII, licensed under CC BY 4.0. The data was modified.
• Contains data derived from google/eng-practices, licensed under CC BY 3.0. The data was modified.
• Contains data derived from Django (BSD-3-Clause). This model is not endorsed by the Django project or its contributors.

Full license texts and notices for the base model, datasets and source repositories are in licenses/. SHA256SUMS lists the sha256 of every file in this repository.

---

中文说明

AstrLink Guard 是一个小模型，用来找出文本里的个人信息和凭证，中文、英文和中英混排都支持。源代码、配置文件、日志，以及 AI 编程工具（如 Claude Code、Codex、Cursor）发给大模型的请求（下文简称“编程工具请求”），它也能处理。模型用 ONNX Runtime 在本机 CPU 上运行，对每个检出的片段给出位置、类别和分数，要遮盖哪些由调用方决定。AstrLink 的隐私检测用的就是这个模型。

本仓库是 0.1.2 版，有两种格式：int8/（动态 INT8，推荐）和 fp32/（FP32，精度更高的参照版本）。

0.1.2 更新内容

行为变化：private_date 现在只标属于具体某个人的日期。 出生日期、身份证和护照等证件上的日期、就诊及其他预约、个人行程日期照常标出。日志、代码、配置、文件列表、HTTP 和 git 头部信息、系统提示词里的日期，以及记录时间戳（created_at、updated_at 等）、发布和构建日期，现在都不再标出。如果你的程序需要遮盖所有日期，请继续用 0.1.1（revision="v0.1.1"），或者自己加一条日期规则。

• 被标出的非个人日期少了很多。编程工具请求里共有 7,743 个非个人日期，INT8 标出的从 7,435 个降到 21 个，FP32 从 7,439 个降到 22 个。另有一组 2,815 个非个人日期，取自日志、文件列表、git 输出、请求头、记录时间戳、发布说明、代码、系统提示词和工作事件，INT8 标出的从 2,786 个降到 106 个，FP32 从 2,802 个降到 112 个。
• 代价是一部分个人日期会漏掉。1,760 个个人日期里，0.1.2 用 INT8 能找出 81.5%，用 FP32 能找出 82.7%。0.1.1 几乎把所有日期都标出来，这两个比例分别是 98.6% 和 98.8%。出生日期和行程日期找出的比例最高，证件上的日期和就诊预约漏得最多（见上文 Dates 一节）。
• 一处都没遮住的凭证少了。编程工具请求里共有 2,158 个凭证，这样整个漏掉的，INT8 从 228 个降到 51 个，FP32 从 159 个降到 45 个。91 个 AWS 访问密钥 ID（AKIA…）里，整个漏掉的 INT8 从 8 个降到 1 个，FP32 从 5 个降到 1 个。
• 编程工具请求的误报少了。187 个不含个人信息和凭证的请求里，仍被标出片段的（日期不计），INT8 从 24 个降到 20 个，FP32 从 31 个降到 24 个。
• 其余九类更准了。编程工具请求上，除 private_date 以外九类的 macro F1，INT8 从 87.91 升到 90.58，FP32 从 89.42 升到 90.84。
• 模型结构、分词器、10 类标签、输入格式、解码规则、建议最低分数、文件名和仓库目录结构都和 0.1.1 一样。已有的集成可以直接换模型，代码照旧。标注上需要留意的变化是 private_date 的范围。

两种格式

目录	格式	模型大小	通用文本 F1	代码 F1	编程工具请求 macro F1（九类）
int8/（推荐）	动态 INT8	36 MiB	97.46	93.54	90.58
fp32/	FP32	141 MiB	97.35	93.83	90.84

表中数字都按建议最低分数 0.8 计算。

推荐用 int8/。它的体积约为 FP32 的四分之一，在 CPU 上通常也更快。

fp32/ 是精度更高的参照版本。最看重精度时用它，代码和凭证场景尤其如此。和 FP32 相比，INT8 的代码 F1 低 0.29 点，编程工具请求九类 macro F1 低 0.26 点。编程工具请求里整个漏掉的凭证，INT8 是 51 个，FP32 是 45 个（共 2,158 个）。个人日期 INT8 找出 81.5%，FP32 找出 82.7%。不过 INT8 在编程工具请求上误报更少：187 个不含实体的请求里，INT8 有 20 个出现误报，FP32 有 24 个。通用文本上两者基本持平，INT8 还高 0.11 点。

int8/ 和 fp32/ 都是完整的模型目录。根目录的 config.json 和这两个目录里的 config.json 是同一个文件。

实体类别

模型识别人名、电话、邮箱、实际地址（住址、收货地址、办公地址）、属于具体个人的日期和时间、个人账号、银行卡号、IP 地址、URL 和凭证（密码、PIN、API key、token、Cookie、私钥、公钥），用 BIO 标注，共 21 个标签。

下面这些按普通文本处理：订单号、快递单号、交易号、金额、版本号、哈希、请求 ID、OAuth client_id、部分脱敏的值、占位符，以及不属于具体个人的日期和时间，例如日志时间、代码和配置里的日期、文件列表、HTTP 和 git 头部信息、记录时间戳、发布日期、系统提示词里的当前日期。

用法

代码见上文 Quick start。示例下载并加载 int8/。要用 FP32，就下载 fp32/、加载 model.onnx。想固定在这一版，给 snapshot_download 传 revision="v0.1.2"。

示例一次只处理一个 512 token 的窗口。更长的文本按每个窗口 512 token、相邻窗口重叠 128 token 切开，分别检测后合并结果。

每个片段都有分数。建议丢掉分数低于 0.8 的片段，本版就是按这个最低分数校准的。调高最低分数，误报会少一些，漏检会多一些。调低则误报增多。

在 AstrLink 中使用

下面几项都是 AstrLink 的设置和做法。在自己的代码里直接调用模型时，可以另行设定。

• AstrLink 默认丢掉分数低于 0.8 的片段。
• AstrLink 的模型列表推荐 INT8 版本。
• int8/ 和 fp32/ 里的 astrlink-model.json 是 AstrLink 的导入描述文件。
• AstrLink 单次请求最多处理 131,072 token，更长的请求会被整条拒绝，并明确报错。这个上限是 AstrLink 定的。模型本身配合上面的窗口切分，可以处理任意长度的文本。
• 本仓库发布带 tag 的新版本后，AstrLink 会把它列为可用更新。

效果

下表都按建议最低分数 0.8 计算。评测集按 0.1.2 的方式标注，只有属于具体个人的日期才算实体。

评测集	INT8	FP32
通用文本 F1（2,200 篇，中 / 英 / 混排）	97.46	97.35
代码、配置、日志 F1（709 篇）	93.54	93.83
编程工具请求九类 macro F1（1,200 个请求）	90.58	90.84
无实体请求出现误报（日期不计，共 187 个）	20	24
整个漏掉的凭证（共 2,158 个）	51	45
整个漏掉的 AWS 访问密钥 ID（共 91 个）	1	1
个人日期找出比例（共 1,760 个）	81.5%	82.7%
被标出的非个人日期（共 2,815 个）	106	112
URL 里的密钥检出率（共 225 个）	98.7%	98.7%

在单独的个人日期集上，INT8 按类别的检出率是：出生日期 92.4%，个人行程 86.7%，证件日期 74.9%，就诊和其他预约 70.8%。

同样评测方式下与 0.1.1 的对比，见上文英文部分的 Compared with 0.1.1 一节。也有几项变差了。编程工具请求里 private_address 的 F1，INT8 从 88.06 降到 86.42，FP32 从 89.05 降到 87.10。被并进 URL 片段的密钥从 2 个变成 3 个。261 个不含实体的代码文档里，INT8 出现误报的从 4 个变成 5 个。

局限

• 模型会漏检，也会把无害的值标出来。请把它当作隐私保护中的一层。
• 个人日期比 0.1.1 更容易漏。0.1.2 用 INT8 能找出 81.5% 的个人日期（FP32 为 82.7%），0.1.1 是 98.6%。证件日期、就诊预约和工具输出里的日期最容易漏。需要遮盖所有日期的场景，请继续用 0.1.1，或者另加基于规则的日期过滤。
• 仍有少量非个人日期会被标出，主要是部署窗口、截止日期、会议等工作事件（INT8 41 / 176），代码和配置里的日期（23 / 219），以及部分记录时间戳（19 / 658）。
• INT8 用一点精度换来更小的体积。编程工具请求里，INT8 整个漏掉的凭证比 FP32 多（51 个对 45 个，共 2,158 个），个人日期也找得少一些（81.5% 对 82.7%）。宁可模型大一些也不想漏掉密钥时，请用 fp32/。
• 代码和编程工具请求比普通文本难。用 INT8 时，不含实体的代码文档约有 1.9%、不含实体的编程工具请求约有 10.7% 仍会出现误报（FP32 为 1.1% 和 12.8%）。
• 编程工具请求里效果最差的类别是账号（account），代码里是凭证（common_secret）。
• 最容易漏的是密码，以及附近没有字段名或厂商前缀的密钥（例如用连字符分段的密钥）。
• 训练时没见过的发卡方的卡号（交通卡、会员卡、工牌等）可能被判成银行卡。
• 键名以 key 结尾的配置值，即使不是凭证，也可能被判成凭证。
• 身份证等政府证件号没有专门的类别。

许可

模型以 Apache-2.0 许可发布，底座模型 MiniLM 是 MIT 许可。训练用到的公开数据包括 NVIDIA Nemotron-PII（CC BY 4.0）、google/eng-practices（CC BY 3.0）和 Django（BSD-3-Clause）等，另有为本项目编写的虚构合成样本。署名见上文 License 一节，许可原文和数据来源在 licenses/ 目录里。