Dataset
Welcome to the official page for our paper, which introduces CT-RATE—a pioneering dataset in 3D medical imaging that uniquely pairs textual data with image data focused on chest CT volumes. Here, you will find the CT-RATE dataset, comprising chest CT volumes paired with corresponding radiology text reports, multi-abnormality labels, and metadata, all freely accessible to researchers.
A major challenge in computational research in 3D medical imaging is the lack of comprehensive datasets. Addressing this issue, we present CT-RATE, the first 3D medical imaging dataset that pairs images with textual reports. CT-RATE consists of 25,692 non-contrast chest CT volumes, expanded to 50,188 through various reconstructions, from 21,304 unique patients, along with corresponding radiology text reports, multi-abnormality labels, and metadata. We divided the cohort into two groups: 20,000 patients were allocated to the training set and 1,304 to the validation set. Our folders are structured as split_patientID_scanID_reconstructionID. For instance, "valid_53_a_1" indicates that this is a CT volume from the validation set, scan "a" from patient 53, and reconstruction 1 of scan "a". This naming convention applies to all files.
Leveraging CT-RATE, we developed CT-CLIP, a CT-focused contrastive language-image pre-training framework. As a versatile, self-supervised model, CT-CLIP is designed for broad application and does not require task-specific training. Remarkably, CT-CLIP outperforms state-of-the-art, fully supervised methods in multi-abnormality detection across all key metrics, thus eliminating the need for manual annotation. We also demonstrate its utility in case retrieval, whether using imagery or textual queries, thereby advancing knowledge dissemination. Our complete codebase is openly available on our official GitHub repository.
Leveraging the VQA dataset derived from CT-RATE and pretrained 3D vision encoder from CT-CLIP, we developed CT-CHAT, a multimodal AI assistant designed to enhance the interpretation and diagnostic capabilities of 3D chest CT imaging. Building on the strong foundation of CT-CLIP, it integrates both visual and language processing to handle diverse tasks like visual question answering, report generation, and multiple-choice questions. Trained on over 2.7 million question-answer pairs from CT-RATE, it leverages 3D spatial information, making it superior to 2D-based models. CT-CHAT not only improves radiologist workflows by reducing interpretation time but also delivers highly accurate and clinically relevant responses, pushing the boundaries of 3D medical imaging tasks. Our complete codebase is openly available on our official GitHub repository.
When using this dataset, please consider citing the following related papers:
1. @article{hamamci2026generalist,
title={Generalist foundation models from a multimodal dataset for 3D computed tomography},
author={Hamamci, Ibrahim Ethem and Er, Sezgin and Wang, Chenyu and Almas, Furkan and Simsek, Ayse Gulnihan and Esirgun, Sevval Nil and Dogan, Irem and Durugol, Omer Faruk and Hou, Benjamin and Shit, Suprosanna and others},
journal={Nature Biomedical Engineering},
pages={1--19},
year={2026},
publisher={Nature Publishing Group UK London}
}
2. @inproceedings{hamamci2024generatect,
title={Generatect: Text-conditional generation of 3d chest ct volumes},
author={Hamamci, Ibrahim Ethem and Er, Sezgin and Sekuboyina, Anjany and Simsar, Enis and Tezcan, Alperen and Simsek, Ayse Gulnihan and Esirgun, Sevval Nil and Almas, Furkan and Do{\u{g}}an, Irem and Dasdelen, Muhammed Furkan and others},
booktitle={European Conference on Computer Vision},
pages={126--143},
year={2024},
organization={Springer}
}
3. @inproceedings{hamamci2024ct2rep,
title={Ct2rep: Automated radiology report generation for 3d medical imaging},
author={Hamamci, Ibrahim Ethem and Er, Sezgin and Menze, Bjoern},
booktitle={International Conference on Medical Image Computing and Computer-Assisted Intervention},
pages={476--486},
year={2024},
organization={Springer}
}
4. @inproceedings{hamamci2025bettertokensbetter3d,
title={Better Tokens for Better 3D: Advancing Vision-Language Modeling in 3D Medical Imaging},
author={Hamamci, Ibrahim Ethem and Er, Sezgin and Shit, Suprosanna and Reynaud, Hadrien and Yang, Dong and Guo, Pengfei and Edgar, Marc and Xu, Daguang and Kainz, Bernhard and Menze, Bjoern},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems}
}
5. @inproceedings{hamamci2025crg,
title={CRG Score: A Distribution-Aware Clinical Metric for Radiology Report Generation},
author={Hamamci, Ibrahim Ethem and Er, Sezgin and Shit, Suprosanna and Reynaud, Hadrien and Kainz, Bernhard and Menze, Bjoern},
booktitle={Medical Imaging with Deep Learning-Short Papers}
}
For those who require ethical approval to apply for grants with this dataset, it can be accessed here.
We are committed to fostering innovation and collaboration in the research community. To this end, all elements of the CT-RATE dataset are released under a Creative Commons Attribution (CC-BY-NC-SA) license. This licensing framework ensures that our contributions can be freely used for non-commercial research purposes, while also encouraging contributions and modifications, provided that the original work is properly cited and any derivative works are shared under similar terms.
For commercial inquiries related to CT-RATE, please contact: contact@forithmus.com.
To help you navigate and utilize the CT-RATE dataset effectively, we provide the following supplementary documents:
📂 Dataset Structure Explanation
A detailed guide explaining the organization and contents of each folder and file within the dataset is available here.
🛠️ Data Correction Notes
Corrections such as intensity normalization and spacing adjustments — already available in the metadata CSV — have now been implemented directly in the provided NIfTI files. Full details can be found here.
We recommend reviewing these documents before using the dataset for your research.
We extend our gratitude to the Helmut Horten Foundation for their support, which made this work possible. We also sincerely thank Istanbul Medipol University Mega Hospital for their support and for providing the data. High-performance computing resources were provided in part by the Erlangen National High Performance Computing Center (NHR@FAU) at Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), under the NHR projects b143dc and b180dc. NHR is funded by federal and Bavarian state authorities, and NHR@FAU hardware is partially funded by the German Research Foundation (DFG) – 440719683.