Edit the requested text or table. Preserve the rest of the document.
VDEBench evaluates instruction-guided image editing on dense Chinese and English document pages. Models must make precise local changes while preserving surrounding text, table structure, typography, and page layout. The benchmark contains 3,506 examples, covering text replacement, addition, deletion, and table content and structure editing.
Read the paper · Get the code · Explore the data
Overview illustration from the VDE-Bench GitHub repository.
| Examples | Text edits | Table edits | Languages | Task types |
|---|---|---|---|---|
| 3,506 | 1,578 | 1,928 | English · Chinese | 7 |
Each example pairs an input image, an editing instruction, and a reference target image with editing-region annotations and source provenance. Available masks describe the allowed editing region and the pixels that actually changed.
The following input–target pairs come from this dataset. They show the editing tasks and reference targets, rather than predictions from a model under evaluation. Click an image to inspect it at full resolution.
Instruction: 将标题“国资背景基金情况”替换为“国资背景基金分析”
Replace the title “国资背景基金情况” with “国资背景基金分析”.
Example 48234 · text configuration · text replacement.
Task: Replace several table headers, cell entries, and the conclusion according to one instruction.
Example 46007 · table configuration · text modification.
修改表格内的文字内容：将表头第二列的“描述听到的声音的强弱变化”改为“观察到的现象”；将第一列的“实验”改为“测试”；将“轻轻拨动钢尺”改为“缓慢移动滑块”，“振动幅度小 声音弱（小）”改为“移动距离短 速度慢（低）”；将“用力拨动钢尺”改为“快速移动滑块”，“振动幅度大 声音强（大）”改为“移动距离长 速度快（高）”；将“我的发现”改为“结论”；并将最后一行关于音量的描述替换为“速度是由物体移动的距离和时间决定的，移动距离越长，速度就越快；移动距离越短，速度就越慢。”
pip install -U datasets pillow
Load the benchmark with Hugging Face Datasets. Streaming lets you inspect examples without first downloading the entire collection.
from datasets import load_dataset
ds = load_dataset(
"hongzhuyi/VDEBench",
name="default",
split="test",
streaming=True,
)
example = next(iter(ds))
print(example["id"], example["instruction"])
example["image_input"].save("input.png")
example["image_output"].save("reference.png")
Choose name="text" for text edits or name="table" for table edits. Remove streaming=True to download and load the complete configuration. The repository is public; authentication is optional.
Images and available masks are embedded in Parquet, so this interface needs no custom loading script or archive extraction.
| Region | Editing task | Examples |
|---|---|---|
| Text | Replacement | 575 |
| Text | Addition | 504 |
| Text | Deletion | 499 |
| Table | Text modification | 524 |
| Table | Text addition | 472 |
| Table | Text deletion | 472 |
| Table | Structure editing | 460 |
| Total | 3,506 |
All configurations expose the frozen benchmark through a single test split. The default configuration contains all 3,506 examples; text and table are its two subsets, not additional examples. When constructing new partitions, group examples by source document to prevent source overlap.
The VDE-Bench repository provides data-generation and evaluation tools. The evaluation protocol combines local edit-region assessment with global full-page assessment to examine both the requested change and its surrounding document context.
See the repository's installation and usage instructions for environment setup, OCR backends, and evaluation commands.
| Fields | Contents |
|---|---|
id, instruction | Persistent example ID and editing instruction |
region, edit_type, language, data_source | Editing task, language, and document category |
image_input, image_output | Input and reference images, decoded as PIL images |
allowed_mask, actual_difference_mask | Optional editing-region and changed-pixel masks; missing masks are null |
label_input_json, label_output_json | JSON-encoded annotation structures |
record_json, provenance_json, annotation_json | Compatibility records and provenance sidecars |
source_dataset, source_group | Source identity and grouping |
image_input_path, image_output_path | Relative image paths in the compatibility archives |
Annotation rectangles use percentages of the recorded annotation canvas, which can differ from the image's native dimensions. The allowed editing region and actual changed pixels have separate annotations. See DATA_DICTIONARY.md for field definitions and coordinate conventions.
The root-level text_info_all.json and table_info_all.json retain the VDE-compatible records. Input, target, and mask files are stored in compatibility/assets-*.tar.
from pathlib import Path
import tarfile
from huggingface_hub import snapshot_download
root = Path(snapshot_download("hongzhuyi/VDEBench", repo_type="dataset"))
for archive in sorted((root / "compatibility").glob("assets-*.tar")):
with tarfile.open(archive) as tar:
tar.extractall(root, filter="data")
Use a Python version supporting tarfile extraction filters (Python 3.12+ recommended). load_dataset.py supports local JSON filtering. asset_manifest.json describes unpacked assets; files_sha256.json records repository file hashes except its own.
Source datasets include OmniDocBench, DocGenome, and DocLayNet, alongside the recorded source papers.
The data follows source-specific terms. See SOURCE_LICENSES.md and provenance_and_validation.json for attribution and reuse conditions.
If you use VDEBench in your research, please cite the paper:
@article{yi2026vdebench,
title = {VDE Bench: Evaluating The Capability of Image Editing Models to Modify Visual Documents},
author = {Yi, Hongzhu and Yang, Yujia and Wang, Yuanxiang and Li, Tong and Guan, Zhenyu and Zong, Tianyu and Chen, Jiahuan and Bao, Chenxi and Yang, Tiankun and Jin, Haopeng and Yuan, Yixuan and Wang, Xinming and Yu, Tao and Gao, Ruilin and Tao, Ruiwen and Liang, Haijin and Ma, Jin and Luo, Jinwen and Yeshani and Zuo, Xinyu and Xu, Jungang},
journal = {arXiv preprint arXiv:2602.00122},
year = {2026},
url = {https://arxiv.org/abs/2602.00122}
}