XingChen-AGI/Xing4.0-29B-Intent-oss

Xing4.0-29B-Intent-oss is the companion dataset for Xing4.0-29B-Intent, a model for schema-guided intent recognition and structured information extraction. It is curated and released by the Xingchen General Artificial Intelligence Laboratory of China Telecom Artificial Intelligence Technology Co., Ltd.

For complete schema definitions, conversion rules, and training/inference example assembly instructions, see the dataset documentation on GitHub: English | 简体中文.

Overview

The dataset combines two sources:

• General: 61 public information extraction and spoken language understanding datasets reconstructed into a unified schema-guided format.
• XingChen: China Telecom's in-house intent and slot annotations for smart-device assistant scenarios.

It supports supervised fine-tuning, schema-guided semantic understanding evaluation, and research on generalization across tasks, domains, and languages. Reproducing a specific evaluation result also requires matching data versions, evaluation scripts, and inference settings.

Data Distribution

Statistic	Value
Total samples	2,753,000
Reconstructed public datasets	61
Domains	19
Languages	52, including Chinese and English
Task types	NER, RE, EE, and intent recognition / slot filling

Task	Samples	Share	Representative datasets
Named entity recognition (NER)	823,000	29.9%	MSRA, OntoNotes, CoNLL2003, CMeEE, WeiboNER
Relation extraction (RE)	905,000	32.9%	DuIE, NYT, TACRED, BioRED, SemEval
Event extraction (EE)	56,000	2.0%	ACE05, RAMS, FewFC, MAVEN, DuEE
Intent recognition / slot filling (SLU)	969,000	35.2%	Snips, ATIS, MASSIVE
Total	2,753,000	100%	—

Data Format

Dataset files are JSON arrays, with one sample per element. Each sample includes its input conversation, a schema reference, and a serialized reference output.

Fields	Description
id, turn	Sample and conversation-turn identifiers
task, source, domain, language	Task and provenance metadata
schema_path, schema	Schema file path or an embedded schema array; a non-empty embedded array takes precedence
prompt_path, prompt	Optional prompt file or embedded prompt template
dialogs	Input turns containing role and content
output	Reference answer stored as a JSON string

Illustrative sample:

{
  "id": "idx",
  "turn": "idx-0",
  "task": "Intent Recognition",
  "source": "XingChen",
  "domain": "Phone",
  "language": "Chinese",
  "schema_path": "./schema_description/schema.json",
  "schema": "",
  "prompt_path": "",
  "prompt": "",
  "dialogs": [
    {"role": "user", "content": "明天的紫外线强度"}
  ],
  "output": "[{\"category\": \"查询天气\", \"args\": [{\"时间\": \"明天\", \"天气指数\": \"紫外线指数\"}]}]"
}

The input schema defines candidate categories and their argument names and descriptions. The parsed output uses a common structure across all tasks:

[
  {
    "category": "查询天气",
    "args": [{"时间": "明天", "天气指数": "紫外线指数"}]
  }
]

Here, category identifies the intent, entity type, relation type, or event type. Each object in args contains one set of extracted arguments. Multiple categories use separate top-level objects; multiple instances of a category use separate objects in args. Use [] when there are no arguments to fill.

The output field must be parsed with json.loads(output) to obtain this structure. Schema and prompt paths are resolved relative to the corresponding data root, such as General/test/ or XingChen/, rather than the individual JSON file's directory.

Download and Use

Browse and download the published files from the Files and versions tab. Keep dataset files together with their associated schema and prompt files, preserving their relative paths.

The GitHub repository provides build_examples.py to assemble records into messages and output pairs. Follow the setup and usage examples to arrange the downloaded data and run the script from the model repository root.

• Training: convert the assembled records to the format required by your training framework, for example by appending output as an assistant reply.
• Inference: use messages as the request's message field and add parameters such as model. Never include reference answers in inference requests.
• Evaluation: preserve task-specific single-label or multi-label constraints, construct each sample independently, and compare parsed output structures rather than raw JSON strings.

License

This dataset is released under the Apache License 2.0. The original license terms of the public source datasets also apply; check their respective terms before commercial use.