#2268

[BUG] Post-processing: an empty LLM response counts as success, so the dictation is silently dropped instead of falling back to the raw transcript

Openilyachaseopened 18d ago

Before You Submit

I searched existing issues. Related, but not the same: #1261 (transcript treated as instructions; closed via #1310). That one is about why a model might answer badly; this one is about what the app does after a model returns an empty string.

Bug Description

When the post-processing model returns an empty transcription (or whitespace only), Handy treats it as a successful post-processing result. The empty string becomes final_text, the paste step is skipped because final_text.is_empty(), and the user's dictation is lost from the focused app: nothing is pasted, the overlay just hides. The original text only survives in the history entry (where post_processed_text is empty).

Every other post-processing failure (HTTP error, timeout, provider error) falls back to the original transcription ("Falling back to original transcription."). An empty answer from a non-empty input is also a failure, but is not handled like one. Note that the Apple Intelligence branch already does this correctly: it treats an empty result as None.

Steps to Reproduce

  1. Settings → Post Process: provider OpenRouter, model google/gemma-4-31b-it, default or any prompt, structured outputs enabled (the default).
  2. Use the post-processing shortcut and dictate a short, imperative-sounding phrase, e.g. "продиктовал, проверь" ("dictated, check it").
  3. Result: nothing is pasted.

Log:

[DEBUG] Starting LLM post-processing with provider 'openrouter' (model: google/gemma-4-31b-it)
[DEBUG] Using structured outputs for provider 'openrouter'
[DEBUG] Chat completion response received with status 200 OK ...
[DEBUG] Structured output post-processing succeeded for provider 'openrouter'. Output length: 0 chars
[DEBUG] coordinator: processing finished (Processing took 1125ms)

(No "Text pasted successfully" line follows.) History entry: transcription_text = "Продиктовал, проверь.", post_processed_text = "".

The empty string comes from the model, not from the app, and which models do this varies. But the app has no safety net, so any model that sometimes returns {"transcription": ""} makes dictations disappear.

Expected Behavior

An empty or whitespace-only post-processing result for a non-blank input is treated as a failed post-processing attempt, and the original transcription is pasted (same as for HTTP errors).

Where in the code (v0.9.8, src-tauri/src/actions.rs)

  • Blank input is already guarded: is_blank_transcription (L93, used at L122), so the model is never called for empty input. An empty output is therefore always a model failure, never "correct".
  • Structured branch returns the parsed value without checking: L281 (return Some(result)).
  • Legacy branch also returns whatever it got: L324–L330.
  • Paste is skipped for an empty final_text: L741.
  • Precedent: the Apple Intelligence branch already does if result.trim().is_empty() { None } (L217–L218).

Proposed fix (small, no new settings)

In both the structured and the legacy branches, treat result.trim().is_empty() like the Apple Intelligence branch does: log it and return None, so process_transcription_output keeps the original text. I'm happy to send a PR for exactly this if you'd like.

System Information

App Version: code identical in v0.9.8 and current main for this path (I observed it on a build of a downstream fork of 0.9.7, which has the same actions.rs logic here; I then re-checked the v0.9.8 sources).

Operating System: Ubuntu 26.04 LTS (GNOME, Wayland)

CPU/GPU are not relevant (the failure is in the post-processing network path).

Notes

I used an AI assistant (Claude Code) to help investigate this and draft the report; I reproduced the behaviour and verified the code references myself.

Activity

0

No activity yet.