Dataset
The French-shiKomori Parallel Dataset v1.0 is a French–Shingazidja parallel corpus developed by Komor-IA as part of its efforts to build open language resources and artificial intelligence technologies for the Comorian languages.
This first version contains 1,424 aligned French–Shingazidja sentence pairs, with French sentences paired with their corresponding translations in Shingazidja.
The dataset is the first release of a broader French-shiKomori dataset series. Future versions may progressively incorporate additional varieties of Shikomori in order to provide broader linguistic coverage of the Comorian language continuum.
The resource is designed primarily for research and development in machine translation, computational linguistics, language education, speech technologies, and other natural language processing applications involving low-resource African languages.
fr)Note: Shingazidja is the variety represented in this first version. The broader French-shiKomori project is intended to evolve toward the inclusion of additional Shikomori varieties.
Version 1.0 contains:
The thematic categories include:
| French | Shingazidja |
|---|---|
| Bonjour comment vas-tu aujourd'hui ? | ɓariza lewo, ngawe mnono ? |
| Où est la mosquée la plus proche ? | Omsihiri wakariɓu nguo nɗahu ? |
| Je bois du thé chaque matin. | Mi hunwa cai haina trasi. |
| Parle fort. | Rongoa hakuu. |
| Le médecin vaccine les nourrissons chaque semaine. | Etwaɓiɓu ye ucandja owana watiti haina mfumo. |
The data were collected through the Komor-IA language data contribution campaign, which involves contributions from Comorian volunteers.
The objective of the campaign is to progressively build high-quality language resources for the development and evaluation of AI systems adapted to Comorian languages.
Contributors provide language data that can subsequently be reviewed and incorporated into Komor-IA datasets.
The dataset has been developed with attention to linguistic quality and alignment between the French and Shingazidja sentences.
Translations are reviewed with the involvement of native speakers and Komor-IA linguistic contributors.
Particular attention is given to the representation of sounds that are important for Shingazidja phonology and orthography, including implosive consonants such as ɓ and ɗ.
Despite these validation efforts, the dataset may contain errors, linguistic variation, orthographic variation, or sentences requiring further review.
Users are encouraged to report potential issues through the repository's discussion or contribution mechanisms.
Shingazidja is a low-resource language for which standardized computational resources remain limited.
This dataset preserves linguistic forms used during the collection and validation process rather than attempting to impose a single computational normalization on all data.
The representation of phonological distinctions is particularly important for future speech and language technologies. The dataset therefore retains characters such as:
ɓɗUsers working with this resource should ensure that their processing pipelines correctly support Unicode and these characters.
The dataset may be used for:
Researchers may also use the corpus as a starting point for the development of larger multilingual or dialect-aware datasets.
This dataset should not be considered a complete representation of Shingazidja or of the broader Shikomori language.
The dataset is relatively small and should not be assumed to provide comprehensive coverage of:
The dataset should therefore be used with appropriate caution when developing systems intended for broad real-world deployment.
Several limitations should be considered:
Future releases are expected to expand the number of sentence pairs, thematic coverage, linguistic validation, and dialectal coverage.
Version 1.0 is currently released as a single parallel corpus.
No official train/validation/test split is provided at this stage.
Users creating machine learning benchmarks are encouraged to define their own splits carefully and document their methodology to ensure reproducibility.
The dataset is distributed as a CSV file containing aligned French and Shingazidja sentences.
The current file contains the following columns:
id
categorie
français
traduction
dialecte
The dialecte field identifies the Shikomori variety represented by the translation and is intended to support the progressive expansion of the dataset to additional Shikomori varieties in future releases.
Example:
001 | daily_life | Bonjour comment vas-tu aujourd'hui ? | ɓariza lewo, ngawe mnono ? | Shingazidja
002 | administration | Où est la mosquée la plus proche ? | Omsihiri wakariɓu nguo nɗahu ? | Shingazidja
This dataset is publicly visible on Hugging Face but uses gated access.
Users must request access before downloading the underlying dataset files.
Access requests are reviewed manually by Komor-IA.
This mechanism allows the project team to monitor access to the resource while maintaining a public research-oriented dataset page.
This dataset is released under the:
Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) license.
Under this license, users may share and adapt the dataset subject to the applicable license conditions, including:
Users should consult the full license terms before redistributing or incorporating the dataset into a project.
Users of this dataset are encouraged to:
Community feedback is welcome.
Users can report:
Contributions should follow the repository's contribution mechanisms.
The dataset is part of the broader Komor-IA language data initiative, whose objective is to create resources supporting the development of AI systems for Comorian languages.
The long-term objective is to progressively expand the resource beyond Shingazidja and include additional Shikomori varieties.
Future releases may therefore include:
Komor-IA
Komor-IA is a Comorian AI initiative working on artificial intelligence technologies adapted to Comorian realities, with a particular focus on language technologies and the development of resources for Shikomori.
Website: https://komor-ia.com
We thank the Comorian volunteers and contributors who participate in the Komor-IA data collection initiative.
Their contributions help build the language resources necessary for the development of future AI technologies for Comorian languages.
We also acknowledge the contribution of native speakers and linguistic reviewers involved in the validation and improvement of the data.
If you use this dataset in research, publications, benchmarks, or other public work, please cite the dataset as follows:
@dataset{komoria_french_shikomori_parallel_v1,
author = {Komor-IA},
title = {French-shiKomori Parallel Dataset v1.0},
year = {2026},
publisher = {Hugging Face},
version = {1.0},
url = {https://huggingface.co/datasets/Komor-IA/French-ShiKomori-Parallel-Dataset}
}
Version: 1.0 Dataset: French-shiKomori Parallel Dataset Current variety: Shingazidja Parallel pairs: 1,424 License: CC BY-NC 4.0 Maintainer: Komor-IA