Two small streaming detectors for the Mandarin wake word 小光 (xiǎo guāng). Both also fire on 小光小光. Inference code, usage and full results are at github.com/xiaonengmiao/xiaoguang-kws.
| Folder | Model | Size |
|---|---|---|
openwakeword/ | xiaoguang_both.onnx: a conv-attention classifier on openWakeWord's speech embeddings. It reads 16 embeddings, shape (1, 16, 96), and outputs a score. | 88 KB |
sherpa-onnx-kws-zipformer-wenetspeech-3.3M-xiaoguang-ft/ | A streaming zipformer transducer for sherpa-onnx: k2-fsa's wenetspeech 3.3M keyword spotter with its encoder fine-tuned on 小光 and its near-misses. It includes fp32 and int8 encoder, decoder and joiner, plus tokens.txt and keywords.txt. | 13.5 MB (5.4 MB int8) |
The openWakeWord classifier needs openWakeWord's front end: melspectrogram.onnx and embedding_model.onnx from
the openWakeWord v0.5.1 release. The GitHub package
downloads them automatically. The classifier also works with openwakeword.Model(wakeword_models=["xiaoguang_both.onnx"], inference_framework="onnx").
import sherpa_onnx
d = "sherpa-onnx-kws-zipformer-wenetspeech-3.3M-xiaoguang-ft/"
spotter = sherpa_onnx.KeywordSpotter(
tokens=d + "tokens.txt",
encoder=d + "encoder-epoch-10-avg-2-chunk-16-left-64.onnx",
decoder=d + "decoder-epoch-10-avg-2-chunk-16-left-64.onnx",
joiner=d + "joiner-epoch-10-avg-2-chunk-16-left-64.onnx",
keywords_file=d + "keywords.txt",
keywords_threshold=0.25,
keywords_score=0.2, # keep the boost low; at 1.0 this model reports near-misses such as 小关 as 小光
)
keywords.txt holds the wake word as pinyin units: x iǎo g uāng @小光.
All test audio is synthetic and comes from voices not used in training. The tables count how often a detector fired on each spoken phrase.
Held-out voices (VoxCPM2 clones of 20 Emilia speakers) and false activations (2 h of AISHELL-1 test speech):
| Spoken phrase | openWakeWord (0.65) | sherpa-onnx (0.25, boost 0.2) |
|---|---|---|
| 小光 | 95.5 % | 98.7 % |
| 小光小光 | 98.7 % | 99.8 % |
| 小关 | 7.0 % | 2.2 % |
| 小黄 | 16.8 % | 1.4 % |
| 消光, 小广, 小逛 | 80 % | 22 % |
| False activations per hour | 0.5 | 0 |
edge-tts (11 voices × 9 rate and pitch variants):
| Spoken phrase | openWakeWord | sherpa-onnx |
|---|---|---|
| 小光 | 96.0 % | 87.9 % |
| 小关 | 8.1 % | 0 % |
| 消光, 小广, 小逛 | 92 % | 11 % |
All of sherpa-onnx's edge-tts misses come from two dialect-accented voices. The GitHub README has the full table.
The weights are released under CC BY-NC 4.0 for non-commercial use, because the training data includes CC BY-NC material (the Emilia reference voices and ESC-50). The sherpa-onnx model is derived from k2-fsa's Apache-2.0 sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01. The training speech imitates the voices of real Emilia speakers, so do not use the models for impersonation.