Wikidata-Search-Traces is an open corpus of 10,235 reasoning trajectories over Wikidata, developed by Pleias
as part of AI-BRIDGES, with funding from Wikimedia Switzerland and in close collaboration with Wikimedia Deutschland.
In each trajectory, an agent answers a question by writing Python in a REPL: it searches entities, follows relations, reads qualifiers and references,
keeps intermediate results in variables, and returns an answer checked exactly against the graph.
Wikidata describes more than 120 million entities in over 300 languages, and AI systems have no good way to explore it.
SPARQL takes expertise and pasting graph data into a context window degrades as the question grows.
Recursive Language Models (RLMs; Zhang, Kraska & Khattab, 2026) fit this task better: the model explores the graph in code and calls itself on the parts that matter.
Training on trajectories from a larger model turns a small model into an effective RLM, but no such trajectories existed for a knowledge graph. This corpus provides them.
| Category | In short | Question types | Example | Traces |
|---|---|---|---|---|
| Single-entity | Simple lookup around one node | A property's value (entity, year or quantity), a qualifier, a reference, the preferred value, the label in French, German, Chinese, Arabic or Russian, every value of a property, how many entities point at it | The Japanese submarine I-53 was launched during World War II, but do you know exactly when that happened? → 1942 | 9,563 |
| Multi-hop | Long-horizon search through the graph | The target never appears by name; a chain of facts about unnamed entities identifies it. The question asks for the target, a year, a quantity, how many entities point at it, the list of those entities, or the one with the highest or lowest value | A cave in Lebanon is located within a district belonging to a specific governorate that shares a border with the South Governorate. Which cave is this? → Afqa Grotto | 672 |
A single-entity trace takes 1 turn at the median; a multi-hop trace takes 10, up to 27. Every answer is an entity (QID), a year, a quantity, a number, a string or a list of entities, so you can score a model exactly, with no judge. The graph is a frozen Wikidata dump of February 2026 (48 M entities, 262 M edges).
Each row holds the question, the answer format, the expected answer, the model and the conversation in chat format.
We evaluated five systems on 100 questions built with the pipeline: 50 single-entity and 50 multi-hop.
null when the value has no unit.| System | Model | Setup |
|---|---|---|
| Qwen3.8-27B + RLM | Qwen3.8-27B (FP8, served with vLLM on one H100) | RLM harness |
| gpt-6-luna + RLM | gpt-6-luna | RLM harness |
| glm-5.3-flash agent | glm-5.3-flash | tool-calling agent |
| gpt-6-luna agent | gpt-6-luna | tool-calling agent |
| gemini-3.1-flash-lite agent | gemini-3.1-flash-lite | tool-calling agent |
| System | Single-entity | Multi-hop | Overall | Cost / question | Tokens / question (median) | Seconds / question (median) |
|---|---|---|---|---|---|---|
| Qwen3.8-27B + RLM | 46/50 | 28/50 | 74/100 | $0.04 ¹ | 35,754 | 25 ² |
| gpt-6-luna + RLM | 42/50 | 19/50 | 61/100 | $0.013 | 24,942 | 18 |
| glm-5.3-flash agent | 37/50 | 13/50 | 50/100 | $0.067 | 30,478 | 33 |
| gpt-6-luna agent | 40/50 | 9/50 | 49/100 | $0.005 | 24,924 | 16 |
| gemini-3.1-flash-lite agent | 31/50 | 10/50 | 41/100 | $0.19 | 460,426 | 104 |
For the API systems, cost is input tokens × input price + output tokens × output price, with prices from LiteLLM's model price table. For every model, reasoning tokens are counted as output tokens, in both the token counts and the costs.
¹ Estimated cost of batched serving on an H100 in Google Colab, about 40 compute units in total. ² Processing one question at a time.
Facts from Wikidata (CC0 1.0). Question style from NVIDIA Nemotron-Personas-USA (CC BY 4.0); no persona text appears in a question.
Train a small model to search a knowledge graph This is what the corpus is for. Each trace is a full worked example: the question, the Python the agent wrote, what the graph returned, and how it got to the answer. Fine-tuning on trajectories from a larger model is the established route to making a small model an effective RLM, and these are the first such trajectories for a knowledge graph. Our next step is a ~12B model trained on them; yours can be too.
Benchmark your own system, exactly The 100 evaluation questions have reference answers that are QIDs, years, quantities, counts or lists, so scoring is deterministic string and identifier matching. No LLM judge, no rubric, no drift between runs. Results are comparable with the table above and with anyone else's.
Study how agents fail on long-horizon search The traces keep the dead ends, the repeated searches and the runs that never reached the target, not only the clean paths. That makes this a dataset about search behaviour, not just about answers: which conditions get resolved first, where a chain breaks, how an agent recovers.
Build question sets for your own domain The construction method generates certified questions: every one provably has a single answer in the frozen graph, and every clue is necessary to identify it. The same approach extends to other entity types, languages and answer formats.
Ask Wikidata hard questions without SPARQL The harness that produced these traces is open. An open-weight model on a single GPU answers 74 of 100 questions at about $0.04 each, with every answer traceable to the statements it read, so an institution can run the whole thing in-house, with no commercial API and no data leaving its own infrastructure. The traces were generated with gemini-3.1-flash-lite: we provide them for demonstration purposes of our harness design.
We want your questions. If you work in a gallery, library, archive, museum, government, educational / research institution, specifically the Humanities, tell us what you'd ask Wikidata if the technical barrier disappeared. You can add your questinos to this doc or contact us via email here: contact@ai-bridges.org.
This corpus was built by Pleias as part of the AI-BRIDGES project. AI-BRIDGES is a research project, directed by Dr. Shani Evenstein Sigalov at the Digital Humanities Research Hub, School of Advanced Study, University of London. The project is focuses on creating bridges between institutional data, Linked Open Data platforms (such as Wikidata & Wikibase), and AI systems.
AI-BRIDGES is supported by the European Commission through a Marie Skłodowska-Curie Postdoctoral Research Fellowship (Grant ID 101203096). This corpus was released as part of the first phase of the AI-BRIDGES technical development, which was funded by Wikimedia Switzerland (WMCH), in close collaboration with Wikimedia Deutschland (WMDE). Compute through grant n°AD011014736R2 on the Jean Zay cluster.