Follow-up to discussion #2229, as suggested by @cjpais.
Summary
Use the OS's active keyboard layout / input source as a signal for the output language:
- Active layout is English → output English (enable Whisper's Translate to English if the speech is in another language)
- Active layout is another language (e.g. Russian) → transcribe in that language, no translation
Use case
I dictate in Russian but write in both Russian and English depending on context (chats/docs vs. code, commits, GitHub). I already switch the keyboard layout when I switch context, so the layout reliably tells which language the text should be in. Today I have to open settings and flip "Translate to English" every time.
Prior work
- #559 "Add Follow OS Input Language" by @vladstudio. It implemented cross-platform layout detection (
input_source.rs) and mapped layouts to Whisper language codes. It was closed when its branch was deleted, not rejected. - This request extends it: besides picking the transcription language, an English layout would also switch on the translate task. Whisper's translate only targets English, so this needs no new models or LLM post-processing.
Proposed behavior
A "Follow keyboard layout" option, with the layout checked when recording starts:
| Layout | Speech | Result |
|---|---|---|
| Russian | Russian | Russian text |
| English | Russian | English text (translate) |
| English | English | English text |
Platform detection: Windows GetKeyboardLayout (foreground thread), macOS TISCopyCurrentKeyboardInputSource, Linux could be best-effort or unsupported initially.
Related: #2076, #1534, #2094
Activity
4
cjpais15d ago@binhna please just do the most basic thing (follow), if you add another option to it (translate to english) it will be rejected as a PR. Translate to english should be followed if the option is set (aka do nothing)
binhna15d agoPR is up: #2283. kept it to just following the layout like you said, translate to english works the same as before. tested on mac, windows still needs someone to try it
alexweiqi12d ago+1, same use case: I dictate in the language selected in Handy's settings, but some of the time I need the output in English instead.
An alternative that might be simpler than following the keyboard layout: a separate shortcut binding for "Transcribe + Translate to English" (Whisper's built-in translate), next to the existing "Transcribe" and "Transcribe with Post-Processing" bindings. Then e.g. Option+D gives text in the selected language and Option+Shift+D gives English, with no need to open settings and flip the toggle each time.
The post-processing shortcut with a "translate" prompt works as a workaround, but it needs a cloud LLM, while Whisper's translate is fully local.
hey, I'd like to pick this up. plan is to reuse the layout detection from #559 (thanks @vladstudio), add a "Follow keyboard layout" option, and turn on translate to English when the layout is English. will start with mac and windows, linux best effort. I'll link the PR here once it's ready