💻 Code on GitHub · 📊 Full results and analysis · ✅ Check our numbers
Cue v1 is turn detection for voice agents that combines context and audio:
Cue v1 = the fine-tuned Gemma 4 E4B's logits (the words and context) × √Smart Turn v3.2 (the voice)
Combining the two gives better and faster results than either one alone. On LiveKit's eot-bench, with one model and one English prompt across 14 languages:
4th in English · 1st in Portuguese · about 13 decisions a second on one T4
Cue v1 works best with Shortwire; we'll share more details soon.
Code, every benchmark prediction and a one-command check of our numbers: github.com/PYPE-AI-MAIN/cue-v1
Thought Hold on a real call: the caller pauses mid-sentence while giving their details, the end-of-turn decision answers MID_THOUGHT (0.68), and the agent holds instead of cutting in. Recorded in Shortwire with the base Gemma 4 E4B asking this same question, before the fine-tune; Cue v1's Gemma part is that decision, trained. The phone number is masked.
This repository holds Cue v1's Gemma part: Gemma 4 E4B, fine-tuned to answer two typed questions: when the caller pauses, are they done, and if not, why not? and when the caller speaks over the agent, should the agent stop? Each answer is read from the logits, as probabilities over named states instead of a single score, so the agent knows not only whether to act but how long to wait and whether it was really interrupted:
| Answer | What it means | What the agent does | |
|---|---|---|---|
| End of turn (the caller pauses) | DONE | A complete question, answer, request or statement | Reply now |
| LIKELY_DONE | Probably finished, but it trails off | A short grace, then reply | |
| MID_THOUGHT | The sentence is left hanging: a trailing "and", "so", "my number is", half a number | Wait: more is coming | |
| HOLDING | Telling a story or explaining something: the pause is thinking | Wait longer, don't cut in | |
| Interruptions (the caller speaks while the agent talks) | STOP | A real interruption: a question, a correction, an objection, "wait", "hello?", or a full answer to the agent's question | Stop talking and listen |
| CONTINUE | Only listening: "hmm", "okay", "haan", "achha", laughter, noise, a fragment | Keep talking |
The Gemma part brings the context: what the agent asked, what the caller has said so far, whether a sentence, a list or a phone number has been left hanging. It does not hear the voice. Smart Turn, an audio turn model, hears the voice (a falling tone, a breath) but not the meaning. Together they beat either one alone:
| eot-bench, English | Latency @5% | Latency @10% | Cut-offs @300 ms | Cut-offs @600 ms |
|---|---|---|---|---|
| Smart Turn v3.2 only (audio) | 1051 ms | 739 ms | 35.2% | 14.8% |
| Gemma part only (words and context) | 750 ms | 552 ms | 34.0% | 7.8% |
| Cue v1 (both) | 719 ms | 532 ms | 26.8% | 7.5% |
Across all 14 eot-bench languages, adding the voice to the Gemma part makes it faster in 12 of 14 languages at
both the 5% and 10% budgets. The combination is one line, p_end × √p_voice (see
Using the probabilities). Smart Turn v3.2 is free (BSD-2), from Pipecat, and is not
included here: download it from pipecat-ai/smart-turn-v3.
In the results below, Cue v1 is always the combination; Gemma part only is the fine-tune on its own.
Cue v1 never writes an answer. At every pause:
Ask one question. The decision prompt lists the options as letters, followed by what was said so far.
Let the model compute one token. max_tokens: 1, with the top 20 log-probabilities returned
(logprobs: true). llama.cpp returns log-probabilities, the normalised form of the logits; for choosing between
options they are the same thing.
Read the probability of each letter and renormalise them so they sum to 1:
Agent asked: What's the best number to reach you on?
Caller said: Sure, it's nine eight four five,
MID_THOUGHT 0.999 DONE 0.001 LIKELY_DONE 0.000 HOLDING 0.000
Turn the probabilities into an action, as described in Using the probabilities:
Why logits instead of an answer in words or JSON:
Serve the GGUF with llama.cpp:
llama-server -m cue-v1-q8_0.gguf -c 8192 -np 4 -ngl 999 --jinja --port 8090
Ask it about a pause. The answer is the first token's probability mass on each option letter; nothing is generated beyond one token.
import json, math, httpx
END_OF_TURN = json.load(open("prompts/end_of_turn_phone.json"))
INTERRUPTION = json.load(open("prompts/barge_in.json"))
LETTERS = "ABCDEFGH"
LABELS = {"agent_question": "Agent asked", "caller_earlier": "Caller earlier", "caller_text": "Caller said",
"agent_said": "Agent is saying"}
def block(decision, inputs):
return "\n".join(f"{LABELS.get(k, k)}: {inputs.get(k) or '(nothing)'}" for k in decision["inputs"])
def system(decision):
ids = [o["id"] for o in decision["options"]]
lines = [decision["instruction"], "", "Options:"]
lines += [f"{LETTERS[i]}: {o['describe']}" for i, o in enumerate(decision["options"])]
lines += ["", "Answer with exactly one letter."]
if decision["examples"]:
lines += ["", "Examples:"]
for ex in decision["examples"]:
lines += ["", block(decision, ex["inputs"]), f"Answer: {LETTERS[ids.index(ex['answer'])]}"]
return "\n".join(lines)
def ask(decision, inputs):
r = httpx.post("http://localhost:8090/v1/chat/completions", json={
"messages": [{"role": "system", "content": system(decision)},
{"role": "user", "content": block(decision, inputs) + "\nAnswer:"}],
"max_tokens": 1, "temperature": 0, "logprobs": True, "top_logprobs": 20,
"reasoning_effort": "none", "chat_template_kwargs": {"enable_thinking": False}}).json()
raw = {}
for tp in r["choices"][0]["logprobs"]["content"][0]["top_logprobs"]:
t = tp["token"].strip()
if len(t) == 1 and t in LETTERS[:len(decision["options"])]:
opt = decision["options"][LETTERS.index(t)]["id"]
raw[opt] = raw.get(opt, 0.0) + math.exp(tp["logprob"])
total = sum(raw.values()) or 1.0
return {k: v / total for k, v in raw.items()}
probs = ask(END_OF_TURN, {"agent_question": "What's the best number to reach you on?",
"caller_earlier": "", "caller_text": "Sure, it's nine eight four five,",
"transcript_end": "no sentence end"})
print(probs) # {'MID_THOUGHT': 0.999, 'DONE': 0.001, 'LIKELY_DONE': 0.0002, 'HOLDING': 0.0001}: half a number
Ask again whenever the transcript changes, and re-check the score as the silence grows. Until the first words of the turn are transcribed, treat the end-of-turn score as 0.5 (or the wait as 1.0 s).
The words decide most; an audio turn model breaks ties with what the words can't carry, such as a falling tone. We use Smart Turn v3.2 (BSD-2), rescored as the pause goes on.
p_end = probs.get("DONE", 0) + probs.get("LIKELY_DONE", 0)
score = p_end * p_voice ** 0.5 # p_voice: the audio model's end-of-turn probability for this moment
if score >= THETA:
reply()
THETA sets the trade between replying fast and cutting callers off. Choose it on your own calls for the false cut-off rate you can accept. This is the setup behind the eot-bench numbers below: at a 5% false cut-off rate, Cue v1 replies after 719 ms on average in English.
Here the answer sets how long to wait. Give each option a silence budget and blend by probability:
WAITS = {"DONE": 0.5, "LIKELY_DONE": 1.0, "MID_THOUGHT": 1.75, "HOLDING": 2.25} # seconds of silence
wait = sum(p * WAITS[o] for o, p in probs.items())
if silence_so_far >= wait:
take_the_turn()
A confident DONE ends the turn after half a second; a likely HOLDING waits over two seconds, so someone thinking mid-story isn't cut off. Here use the Gemma part on its own, without Smart Turn: in our two-person tests, adding Smart Turn's score made it slightly worse.
When the caller says something while the agent is talking, ask the interruption question with what the agent is saying, the question it last asked, and the caller's words so far:
probs = ask(INTERRUPTION, {"agent_said": "...so the appointment is on Tuesday at", "agent_question": "",
"caller_text": "wait, Tuesday doesn't work"})
# {'STOP': 0.999, 'CONTINUE': 0.001}. The same question with "hmm" gives {'CONTINUE': 1.0, 'STOP': 0.0}
if probs.get("STOP", 0) >= 0.5:
stop_speaking() # a real interruption: listen
# else: a backchannel ("hmm", "okay"), keep talking
Ask again as more of the caller's words arrive: a fragment can read like a backchannel until the rest of it comes in. The threshold trades stopping on noise against talking over a caller; 0.5 is a sensible start.
LiveKit's eot-bench, all 400 English turns, scored with the harness's own metrics. Latency at a 5% and 10% false cut-off rate, and false cut-offs at a 300 ms and 600 ms latency budget.
| Model | Latency @5% | Latency @10% | Cut-offs @300 ms | Cut-offs @600 ms |
|---|---|---|---|---|
| LiveKit turn-detector v1 (hosted) | 543 ms | 295 ms | 9.9% | 4.5% |
| Baton v1 | 577 ms | 350 ms | 12.3% | 4.8% |
| Soniox stt-rt-preview | 647 ms | 512 ms | – | 5.5% |
| Cue v1 | 719 ms | 532 ms | 26.8% | 7.5% |
| Cue v1, Gemma part only | 750 ms | 552 ms | 34.0% | 7.8% |
| Deepgram flux-general-multi | 1151 ms | 548 ms | 12.9% | 9.9% |
| ultraVAD | 899 ms | 663 ms | 27.7% | 11.9% |
| LiveKit turn-detector v1-mini (local) | 1070 ms | 698 ms | 27.8% | 12.1% |
| Smart Turn v3.2 | 1051 ms | 739 ms | 35.2% | 14.8% |
Cue v1 is 4th on all four views (AUC 0.961), behind LiveKit v1, Baton v1 and Soniox, and ahead of Deepgram Flux, ultraVAD, LiveKit's local model and Smart Turn v3.2 on its own. The Gemma part on its own (AUC 0.920) is also 4th at the 5% budget and on cut-offs at 600 ms.
Honest notes on this table:
Cue v1 and its Gemma part on all 14 eot-bench languages, with one model and the same English prompt, English examples included: no translation, no per-language examples, no per-language tuning. The fine-tune itself only ever saw English. The one language-aware step is automatic. Where most caller turns end without punctuation in the transcript (Arabic, Hindi, Indonesian, Japanese, Korean, Chinese), the "how the transcript ends" input is set to neutral, because a missing full stop means nothing there.
Each cell is Gemma part only → Cue v1, with Cue v1's rank on the board in brackets.
| Language | Latency @5% (ms) | Latency @10% (ms) | Cut-offs @300 ms | Cut-offs @600 ms |
|---|---|---|---|---|
| English | 750 → 719 (4th of 13) | 552 → 532 (4th) | 34.0% → 26.8% (4th) | 7.8% → 7.5% (4th) |
| Portuguese | 800 → 681 (1st of 11) | 583 → 498 (3rd) | 27.4% → 27.7% (3rd) | 9.4% → 6.5% (2nd) |
| Spanish | 884 → 792 (2nd of 11) | 690 → 603 (3rd) | 40.2% → 29.9% (4th) | 13.5% → 10.1% (3rd) |
| French | 738 → 647 (3rd of 11) | 524 → 449 (3rd) | 23.7% → 24.4% (4th) | 8.1% → 5.2% (2nd) |
| German | 802 → 786 (3rd of 11) | 548 → 559 (4th) | 22.7% → 22.7% (3rd) | 8.5% → 9.2% (4th) |
| Italian | 924 → 912 (3rd of 11) | 620 → 595 (3rd) | 34.6% → 31.6% (3rd) | 10.4% → 9.9% (3rd) |
| Dutch | 730 → 770 (3rd of 10) | 590 → 554 (4th) | 23.7% → 24.4% (3rd) | 9.6% → 9.4% (4th) |
| Japanese | 567 → 557 (3rd of 10) | 374 → 406 (3rd) | 13.1% → 15.1% (3rd) | 3.9% → 4.6% (3rd) |
| Korean | 593 → 608 (2nd of 9) | 462 → 406 (3rd) | 19.6% → 21.2% (3rd) | 4.7% → 5.5% (2nd) |
| Chinese | 862 → 838 (2nd of 9) | 645 → 637 (3rd) | 40.1% → 35.5% (3rd) | 11.8% → 11.7% (3rd) |
| Indonesian | 881 → 871 (2nd of 9) | 673 → 598 (2nd) | 41.1% → 34.4% (2nd) | 13.4% → 9.9% (2nd) |
| Hindi | 628 → 607 (4th of 10) | 513 → 494 (3rd) | 37.5% → 30.8% (4th) | 6.5% → 5.3% (4th) |
| Turkish | 1015 → 1012 (3rd of 9) | 743 → 692 (3rd) | 41.3% → 33.9% (2nd) | 14.9% → 11.6% (3rd) |
| Arabic | 829 → 815 (6th of 9) |
Ranks are among the models the benchmark reports for that language. Not every model covers every language.
Adding the voice helps in most languages: Cue v1 is faster than its Gemma part alone in 12 of 14 languages at both budgets, and has fewer cut-offs at 600 ms in 11 of 14. In Dutch and Korean, the Gemma part alone is slightly faster at the 5% budget.
In Portuguese, Cue v1 is 1st at a 5% false cut-off rate, ahead of LiveKit v1 and Soniox (both 702 ms).
Fine-tuning on English made the other languages better too. Against Cue v1 built on the base Gemma (same prompt, same Smart Turn), the fine-tune is faster at the 5% budget in 11 of 14 languages and at the 10% budget in 11 of 14. The biggest moves: Hindi from 7th to 4th, Portuguese from 4th to 1st, French 698 to 647 ms, Korean 485 to 406 ms at 10%. It is slower in Dutch (688 to 770 ms at 5%), and at the 10% budget in English (502 to 532 ms) and Chinese (606 to 637 ms).
The whole board, with Cue v1 in the outlined column. The small rank under each Cue value counts every model eot-bench reports for that language; the outlined cell in each row is the best. Grey cells are languages a model does not support.
Across the 14 languages, counting the 5% and 10% budgets:
| Against | Cue v1 is faster in |
|---|---|
| OpenAI GPT Realtime 2 (semantic VAD) | 28 of 28 |
| VAP | 28 of 28 |
| Gradium | 28 of 28 |
| Smart Turn v3.2 | 26 of 28 |
| ultraVAD | 25 of 28 |
| LiveKit turn-detector v1-mini (local) | 25 of 28 |
| Deepgram Flux | 12 of 18 |
| Soniox | 12 of 28 |
| LiveKit turn-detector v1 (hosted) | 1 of 28: still the one to beat |
Arabic is the weak spot (6th at 5%).
The eot-bench latencies are not hardware speed. They measure when Cue v1 decides that the caller is done, counted from the moment the caller actually stopped. They come out the same on any hardware, as long as each decision is ready in time: the benchmark assumes words arrive 0.5 s after they're spoken, covering the ASR and the decision. On slow hardware (a CPU-only machine, a busy laptop), a late decision makes the real-world wait longer than these numbers.
Turn detection runs on Gemma logits: one token, no generation. Every decision is one forward pass that reads the option letters' probabilities, with the long prompt shared and cached across calls. Measured on one NVIDIA T4 (llama.cpp, q8_0, this repo's phone prompt, about 70 new tokens per question):
| Questions in flight | Decisions per second | Time per decision (median / 95th percentile) |
|---|---|---|
| 1 | 12.5 | 79 / 96 ms |
| 2 | 13.6 | 145 / 173 ms |
| 4 | 10.5 | 364 / 419 ms |
| 8 | 12.2 | 649 / 718 ms |
One T4 serves about 13 decisions a second; beyond two at once, questions queue. On eot-bench, Cue asked about six questions per caller turn. At an assumed six caller turns a minute, that is roughly 20 concurrent calls per T4 with every decision under 200 ms, or about $0.0005 per call-minute for turn detection on a T4 at around $0.58 an hour (an estimate: measure your own call pattern). Smart Turn v3.2, the audio part, is a small ONNX model that runs on the CPU. A faster GPU serves more.
The fine-tune never saw a real recording or a benchmark transcript.
On held-out simulated conversations (10% of the conversations, never trained on):
| Base Gemma 4 E4B | Cue v1's Gemma part | |
|---|---|---|
| Exact answer, every held-out question | 38.3% | 83.2% |
| End of turn: done vs not done | 58.2% | 86.9% |
| Interruptions: STOP vs CONTINUE (90 questions) | 93.3% | 98.9% |
The base model almost never answers HOLDING. That is the main thing the fine-tune teaches.
prompts/*.json. Changing a description or adding examples in another language changes behaviour without
retraining.| File | What |
|---|---|
cue-v1-q8_0.gguf | Cue v1's Gemma part: the fine-tune merged and quantised to q8_0, for llama.cpp |
adapter/ | The LoRA adapter (safetensors) for google/gemma-4-E4B-it |
prompts/end_of_turn_phone.json | End of turn, caller and agent on a phone call |
prompts/end_of_turn_dialogue.json | End of turn, two people talking |
prompts/barge_in.json | Interruptions: STOP or CONTINUE while the agent is speaking |
| Code, benchmark predictions, verification | github.com/PYPE-AI-MAIN/cue-v1: the cue client, every eot-bench prediction, verify.py, RESULTS.md |
assets/ | The English table, the 14-language heatmaps and the script that draws them; the Thought Hold demo clip |
Turn-taking inside voice agents and other spoken-dialogue systems that already have an ASR: deciding when to reply, when to keep listening, and whether something the caller says over the agent should stop it.
Cue is the turn engine in Shortwire, Pype's open source voice AI stack. Shortwire splits a call in two:
Every decision the cascade makes is auditable: each one is logged with its answer and probability, and can be replayed after the call. In Shortwire, MID_THOUGHT and HOLDING are what we call Thought Hold, and the record of every decision is the Decision Trace.
Knows when to wait. Knows when to stop.
Cue v1 works best with Shortwire. We'll share more details soon.
Apache 2.0, the same as the base model, Gemma 4 E4B.
@misc{pype2026cue,
title = {Cue v1: turn detection from context and audio, with a fine-tuned decision model},
author = {Pype},
year = {2026},
url = {https://huggingface.co/Pype/cue-v1}
}
| 685 → 640 (5th) |
| 51.5% → 39.9% (4th) |
| 14.8% → 12.4% (4th) |