MAABench

MAABench is a benchmark for machine-text attribution: given a document written by an LLM, predict which model family wrote it. There are five families: gpt, claude, gemini, mistral and llama.

Besides an in-distribution split, it tests three kinds of distribution shift: unseen domains, newer model versions, and both at once.

Files

file	domains	model versions	rows
maabench_id.jsonl	academic, comment, news, wiki	training versions	169,735
maabench_domain.jsonl	patent, stackexchange	training versions	7,470
maabench_version.jsonl	academic, comment, news, wiki	3 newer versions per family	29,700
maabench_joint.jsonl	patent, stackexchange	3 newer versions per family	21,615

maabench_id.jsonl holds 29,952 train, 1,998 val and 1,997 test rows per family. Every family has the same number of rows in every file.

Usage

from datasets import load_dataset

# one split
id_data = load_dataset("ming9999/maabench", split="id")
train = id_data.filter(lambda r: r["split"] == "train")
test = id_data.filter(lambda r: r["split"] == "test")

# all four files
maabench = load_dataset("ming9999/maabench")
version_shift = maabench["version"]
newest = version_shift.filter(lambda r: r["role"] == "test-3")

The label is source. Train on the train rows of the id split, tune on val, and report accuracy on the test rows of id and on the domain, version and joint splits.

Statistics

Rows per domain (each count is split evenly over the five families):

file	academic	comment	news	wiki	patent	stackexchange	total
id (train)	37,485	37,500	37,420	37,355	–	–	149,760
id (val)	2,495	2,500	2,500	2,495	–	–	9,990
id (test)	2,500	2,500	2,495	2,490	–	–	9,985
domain	–	–	–	–	3,725	3,745	7,470
version	7,485	7,500	7,455	7,260	–	–	29,700
joint	–	–	–	–	11,100	10,515	21,615

In version and joint, each of test-1, test-2 and test-3 has a third of the rows (9,900 and 7,205 rows respectively).

Seed documents:

file	unique seeds	rows per seed
id	33,947 (29,952 train / 1,998 val / 1,997 test)	5 (one per family)
domain	1,494	5
version	1,980	15 (5 families x 3 versions)
joint	1,441	15

Train, val and test in id share no seeds. Every version seed is an id test seed, so version rows can be compared with the id test rows written from the same seed by the training version. 1,439 of the 1,441 joint seeds are also domain seeds.

Text length in whitespace-separated words:

file	min	median	mean	max
id	81	191	184.8	242
domain	82	189	184.2	242
version	81	186	183.7	242
joint	82	188	185.6	242

Model versions

family	training version	newer versions (test-1 / test-2 / test-3)
gpt	gpt-5	gpt-5.2 / gpt-5.4 / gpt-5.6-terra
claude	claude-opus-4.6	claude-opus-4.7 / claude-opus-4.8 / claude-opus-5
gemini	gemini-2.5-flash	gemini-3-flash-preview / gemini-3.5-flash / gemini-3.6-flash
mistral	Mistral-Small-24B-Instruct-2501	Mistral-Small-3.1 (2503) / Mistral-Small-3.2 (2506) / mistral-small-2603
llama	Meta-Llama-3-70B-Instruct	Llama-3.1-70B-Instruct / Llama-3.3-70B-Instruct / Llama-4-Scout-17B

Fields

field	meaning
id	row id: <seed_id>:<source>:<role>
text	the generated document
source	label: gpt, claude, gemini, mistral or llama
model	the exact model that generated the text
role	train for the training version, test-1/2/3 for the newer versions
domain	academic, comment, news, wiki, patent or stackexchange
split	train, val or test in maabench_id; test elsewhere
seed_id	the human-written document the prompt was built from (domain + hash of its text)

Human-written text

Only machine-written rows are included. Each prompt was built from a human-written document from a public corpus, and those documents are not redistributed here. The code release can rebuild them from the original corpora and checks every rebuilt text against its seed_id hash.

Intended use and limitations

MAABench is meant for research on attributing machine-written text to the model family that wrote it, and on how attribution holds up under domain and model-version shift.

• Not for decisions about people. Attribution accuracy on this benchmark does not show that a detector is reliable on real-world text. Do not use models trained or tuned on MAABench as the sole basis for judging whether a person used an LLM, e.g. in academic-integrity, hiring or content-moderation decisions.
• Closed label set. Every text comes from one of five families. Text from other families, human-written text, and human-edited or mixed text are not covered.
• One training model per family. Each family is represented in training by a single model, so results reflect that model's style as much as the family's.
• Fixed format. All texts are English and roughly 80-240 words long, generated from prompts seeded by a human-written document. Shorter, longer, multilingual or freely prompted text may behave differently.
• Snapshot in time. API models change behind the same name; the model field records the identifier used at generation time.
• Generated content. Texts were not fact-checked and may contain invented names, numbers and claims. Do not treat them as factual.
• Model terms. The texts are outputs of commercial and open-weight models. Users are responsible for complying with the respective providers' terms when using them.