Local Whisper: hallucinated tails on trailing silence ("Продолжение следует…", "Thank you for watching") — local whisper-server gets no anti-hallucination thresholds

[xlurie]

Split out of #1158 at @gabrielste1n's request:

#3 (the "Спасибо" tails) is genuinely separate — that's classic Whisper trailing-silence hallucination,
as @ekhodzitsky said. Please do open a separate issue for it; it needs its own fix and I don't want it
lost when this one closes.

This is only about hallucinated tail content on trailing silence. The mid-word segment splits from
#1158 are a different bug, already fixed by #1369 (and reported upstream as ggml-org/whisper.cpp#3968).

Summary

On the local Whisper path, dictations sometimes end with a phrase that was never spoken — subtitle/outro
boilerplate from Whisper's training data. In a 1,469-dictation sample from the bundled local engine it hit
2.25% of dictations overall, and 47% of dictations whose recording ran 30–32 s (62% at exactly
30 s, 0% at 29 s — the jump lands on whisper's 30-second decode window). The bundled whisper-server is
launched without any of whisper.cpp's decoder-side anti-hallucination thresholds, and the transcription
request does not set their per-request equivalents either. The Dictation Cleanup LLM removed the tail in
only 1 of the 29 large-v3 cases, so the phrase reaches the user's cursor.

Environment

• Windows 11 Pro x64 (26200), RTX 5070 Ti, Ryzen 7 5800X3D
• Code claims below are checked against 1.8.1 (current install). The measurements come from this
  machine's dictation history over 2026-05-23 → 06-13, recorded on an earlier 1.7.x release — see the
  Windows installer omits ggml-silero-v5.1.2.bin — VAD silently disabled #1057 caveat further down, it matters for how you read the VAD numbers.
• Local Whisper (bundled whisper-server-win32-x64.exe; binaries pinned to release tag 0.0.8 = whisper.cpp v1.9.1 in 1.8.x)
• Models: ggml-large-v3.bin (primary) and ggml-large-v3-turbo.bin
• Language setting: auto (the user dictates mixed Russian + English technical terms, so pinning ru is not an option)
• Dictation Cleanup enabled (self-hosted Gemma 3 12B)

What happens

The transcript ends with boilerplate the user never said:

Raw Whisper output (tail)	Translation
…Есть вопросы еще? Продолжение следует...	"…Any more questions? To be continued..."
…предложи улучшенную версию. Продолжение следует.	"…suggest an improved version. To be continued."
…сфокусирован на разных кусочках. Субтитры создавал DimaTorzok	"…focused on different pieces. Subtitles by DimaTorzok"

Продолжение следует ("to be continued") and Субтитры создавал <name> ("subtitles by ") are the
Russian equivalents of "Thank you for watching" / "Subtitles by …" — YouTube outro text that is heavily
represented in Whisper's training data. Catalogue of known Russian variants:
waveletdeboshir's gist,
openai/whisper#2372.

Frequency (measured, not estimated)

Measured over the local transcription history in transcriptions.db (raw_text = pre-cleanup Whisper
output). A dictation counts as a hit when a known boilerplate phrase appears in the last 70 characters of
raw_text.

By engine / model (bundled local engine, 2026-05-23 → 2026-06-13):

Engine	Dictations	Tail hits	Rate	Removed by Cleanup LLM
bundled local, large-v3	1380	29	2.10%	1 / 29
bundled local, large-v3-turbo	89	4	4.49%	0 / 4

(The turbo sample is small — treat the turbo-is-worse direction as consistent with
openai/whisper#2363, not as proof.)

By recording length (bundled local engine, both models, n = 1469):

Recording length	Dictations	Tail hits	Rate
0–15 s	518	1	0.19%
15–29 s	401	0	0.00%
29–32 s	63	18	28.57%
32–60 s	295	5	1.69%
60–90 s	110	5	4.55%
90 s+	82	4	4.88%

Resolved to 1-second bins, the edge is sharp enough to name the mechanism:

Recording length	Dictations	Tail hits	Rate
28 s	23	0	0%
29 s	25	0	0%
30 s	24	15	62.5%
31 s	14	3	21.4%
32 s	25	0	0%
33 s	24	1	4.2%

Nothing at 29 s, 18 of the sample's 33 total hits in the 30–31 s bins, nothing again at 32 s. That is
whisper's fixed 30-second decode window: a recording that just crosses 30 s produces a second window
holding a second or two of speech and ~29 s of padding, decoded with the previous window's text as prompt
context — the textbook trailing-silence hallucination setup. Median recording length in the same sample is
21.7 s, so the band is not simply over-represented; the 30 s boundary is a property of the decoder, not of
how the user speaks.

Why it happens

Classic Whisper trailing-silence hallucination: the decoder is handed a window with little or no speech and
fills it with the most likely training-data continuation, which for YouTube-derived data is outro text.
Two things in the app make it reachable:

1. The local server gets no decoder-side thresholds — neither at spawn nor per request.
buildWhisperServerArgs in src/helpers/whisperServer.js (read out of the shipping 1.8.1 app.asar)
passes exactly:

--model … --host 127.0.0.1 --port … [--threads N] --language auto --no-timestamps
[--vad --vad-model … --vad-threshold … --vad-min-speech-duration-ms … --vad-min-silence-duration-ms …
 --vad-max-speech-duration-s … --vad-speech-pad-ms … --vad-samples-overlap …]

None of whisper.cpp's anti-hallucination knobs are passed, so they stay at their permissive defaults
(values from whisper-server --help, 0.0.8):

Flag	whisper.cpp default	Effect
-et, --entropy-thold	2.40	decoder-fail threshold; too permissive for large-v3
-lpt, --logprob-thold	-1.00	low-confidence segments survive
-sns, --suppress-nst	false	non-speech tokens not suppressed
-nth, --no-speech-thold	0.60	never tuned by the app

All four can be set either at launch (CLI flags above) or per request: whisper.cpp's server reads
entropy_thold, logprob_thold, no_speech_thold and suppress_nst as multipart form fields in
get_req_parameters (examples/server/server.cpp), alongside the language, prompt and
response_format fields the app already sends. So this is fixable in one line at either end — and the
per-request form also covers users pointing the app at their own whisper-server instance, which the
spawn arguments never touch.

2. The VAD defaults lean toward more silence than whisper.cpp's own. src/constants/whisperVad.json
DEFAULTS vs the binary's defaults (see the caveat below — VAD was not actually running in the sample, so
this one is reasoning, not measurement):

Setting	OpenWhispr default	whisper.cpp default
speechPadMs	100	30
samplesOverlap	0.5	0.10
minSilenceDurationMs	200	100
maxSpeechDurationS	30	unbounded

More padding and more overlap means more silence reaching the decoder per segment.

Important caveat, and a status update on #1057: on the release this sample was recorded on, the install
had no VAD at all — the Windows installer did not place whisper-vad/ggml-silero-v5.1.2.bin, and
whisper.js silently continues without VAD in that case ("VAD requested but ggml-silero model not found"). That is #1057, still open. So the numbers above measure the no-VAD path, and the VAD default
values in the previous table were not in play for them — they are listed because they will govern the
behaviour now that the file ships. Good news for #1057: 1.8.1 does ship the file (verified in
resources/bin/whisper-vad/), so that installer bug looks resolved and #1057 can probably be closed. The
silent fallback to "no VAD" on a missing model file is still worth a visible warning, since the user cannot
tell the difference from the UI.

What we measured as effective

Same machine, same model family, same user, same dictation style. We moved the STT step to a self-hosted
whisper-server (identical whisper.cpp build, ggml-large-v3.bin) reached through the app's custom STT
endpoint, and launched it with:

--language auto --vad --vad-model ggml-silero-v5.1.2.bin
--vad-threshold 0.5 --vad-min-silence-duration-ms 200
--vad-max-speech-duration-s 15      # app default 30
--vad-samples-overlap 0.1           # app default 0.5
--vad-speech-pad-ms 30              # app default 100
--entropy-thold 2.8                 # whisper.cpp default 2.4
--logprob-thold -1.25               # whisper.cpp default -1.0
--no-timestamps                     # same fix as #1369

Result over 4,814 dictations through that server (2026-06-13 → 2026-08-03), same detector:

Path	Dictations	Tail hits	Rate
bundled local engine (defaults, VAD inactive)	1469	33	2.25%
self-hosted whisper-server, flags above	4814	3	0.06%

Honest caveats on that comparison: it is not a single-variable A/B. The flags, the VAD state, and the
explicit large-v3 model file all changed at once, and the failure is rare enough that the residual 3
hits mean "much rarer", not "eliminated". The direction is nevertheless large and stable over two months
of daily use. We originally adopted these thresholds to kill runaway repetition loops (-et 2.8 is a
commonly suggested value for large-v3 in whisper.cpp discussions), and the tail rate dropped with them.

A second confound worth stating plainly: the bundled-engine sample ran with VAD inactive (#1057), while
the self-hosted server had VAD on. Part of the delta is therefore "VAD on vs VAD off", not the decoder
thresholds alone.

We have not measured -sns / -nth — they are listed below as untested suggestions, not as results.

Cleanup does not save the user

The Dictation Cleanup LLM (Gemma 3 12B, stock prompt) removed the tail in 1 of the 29 large-v3 hits
on the bundled path, and 0 of 4 on turbo — effectively never; the phrase is grammatical, plausible, and
short, so a "clean up this transcript" prompt keeps it. After we added one explicit rule to our own cleanup
prompt ("drop trailing subtitle/outro boilerplate"), 2 of the 3 residual hits on the self-hosted path were
caught by cleanup and never reached the cursor. That makes a cleanup-prompt rule a cheap safety net, but it
is a second line of defense, not the fix.

Proposed fixes, ranked

1. Set the decoder-side thresholds. entropy_thold 2.8 / logprob_thold -1.25 is what we run in
   production; both are cheap and neither costs latency. Two equivalent places:

   • as form fields on the existing /inference multipart request, next to language /
     response_format — this also covers a user-supplied remote whisper-server, and needs no restart;
   • or as --entropy-thold / --logprob-thold in buildWhisperServerArgs, if defaults-at-spawn fit
     the codebase better.

   Optionally suppress_nst and a tuned no_speech_thold — untested by us, listed for completeness.

2. Align the VAD defaults with whisper.cpp's (speechPadMs 100 → 30, samplesOverlap 0.5 → 0.1):
   less silence padded into each segment. Note this is a forward-looking suggestion — our measurements
   predate VAD actually running (Windows installer omits ggml-silero-v5.1.2.bin — VAD silently disabled #1057), so we have no before/after for these two values specifically.
   maxSpeechDurationS 30 → 15 is worth considering for a different reason: with VAD on it caps segment
   length, and 15 s keeps segments clear of whisper's 30 s window edge. (The 30 s spike in our histogram
   comes from whisper's own window, not from this setting, which was inactive at the time.)

3. Trim trailing silence before decode, or drop a final window that VAD reports as speechless. This
   targets the 30–32 s cluster directly and is the same direction Whisper hallucinations ("Thank you for watching!") during silence — needs smarter VAD for cloud backends #462 proposes for the cloud path — one
   implementation could serve both.

4. Surface a warning when VAD is silently disabled (missing model file) instead of only logging it —
   the remaining half of Windows installer omits ggml-silero-v5.1.2.bin — VAD silently disabled #1057, now that the file itself ships.

5. Cleanup-stage rule as a backstop: a line in the default cleanup prompt instructing removal of
   trailing subtitle/outro boilerplate. Cheapest possible change, catches whatever the decoder still emits.

Option 1 is a two-line change wherever you prefer it (request builder or arg builder), and is the one our
data actually supports.

Relation to existing issues

• Whisper hallucinations ("Thank you for watching!") during silence — needs smarter VAD for cloud backends #462 (open) — same phenomenon, scoped to cloud backends (Groq) and pre-transcription VAD gating;
  PR feat: add VAD-based silence compression for efficient transcription #504 there failed validation. Whisper hallucinations ("Thank you for watching!") during silence — needs smarter VAD for cloud backends #462's own body sets the local path aside ("With local Whisper, users
  can tune no_speech_threshold and related parameters… Cloud APIs like Groq don't expose those controls"),
  which is exactly the lever this issue is about — hence a separate report. If maintainers prefer, fold it
  in as the local-engine half.
• [RU] Regression: words split mid-word, broken sentence punctuation, and hallucinated tails in local Whisper #1158 / fix(whisper): stop local transcription splitting words mid-word #1369 — mid-word segment splits, fixed; unrelated mechanism (token timestamps forcing a
  60-char wrap). Reported upstream as server: default token_timestamps forces ~60-char mid-word line wrapping in all response formats (regression in v1.8.4) ggml-org/whisper.cpp#3968.
• Windows installer omits ggml-silero-v5.1.2.bin — VAD silently disabled #1057 (open) — Windows installer omitted ggml-silero-v5.1.2.bin, so VAD was silently off. That is
  the state our measurements were taken in. 1.8.1 ships the file, so the installer half looks fixed; the
  "silent fallback, no user-visible warning" half is item 4 above.
• Voice Activity Detection (VAD) #185 — VAD support, closed.

Happy to run further A/Bs on this hardware if a candidate patch appears — we have a 6,300-dictation
Russian/English corpus and the detector script that produced the tables above.