Skip to content

Whisper and code-switching

Why Whisper garbles
mixed Chinese and English.

Say "帮我 review 一下这个 pull request" to a Whisper-based dictation app and you may get the English words spelled out in Chinese characters, dropped, or the whole sentence turned into English. This page explains why, what you can try, and which models handle it properly.

Written by the TypeAloud team. TypeAloud is a free, open-source dictation app for Mac; wherever this page mentions it, we are describing our own product.

One language for the whole recording

Unless you set a language, standard Whisper listens to the first 30 seconds, decides which language you are speaking, and transcribes the rest in that language. A sentence that switches languages halfway tends to get pushed towards the one it chose, which shows up as dropped words, English spelled out in Chinese characters, or a translation. How bad it gets depends on the model size and the app's settings.

Its vocabulary is shared across languages and most of its training audio is in a single language, so switching mid-sentence is exactly the case it saw least. Research on Whisper and code-switching describes errors clustering at the switch points.

Workarounds if you stay with Whisper

Set the language to Chinese rather than letting it detect, so short English words do not flip the whole window into English.

If your app has a prompt or vocabulary setting (Whisper calls it the initial prompt), write a sentence in Simplified Chinese there that includes the English terms you use, such as product names. That nudges Whisper to keep them in Latin letters and to write Simplified rather than Traditional characters.

For developers: fine-tuning on mixed Chinese–English data helps most. Published work using small adapters on top of Whisper reports clear gains on mixed speech, but that is a research project, not a setting.

Use a model trained for it

Newer speech models trained heavily on Chinese handle switching natively. Qwen3-ASR can detect the language from the audio itself, and TypeAloud uses that detection, so you never choose a language first. TypeAloud runs Qwen3-ASR on your Mac as a dictation app, so a mixed sentence comes out as you said it. More on mixed Chinese–English dictation.

Details checked on 26 Oct 2026 against arXiv 2312.08856 (Whisper code-switching).

Common questions

Does Whisper support Chinese?
Yes. It transcribes Mandarin well when you speak only Chinese. The problem is switching languages within a sentence.
Why does Whisper output Traditional Chinese?
Its training data mixes Simplified and Traditional text. An initial prompt written in Simplified Chinese usually steers it to Simplified.
Which dictation apps use Whisper?
Many local Mac apps, including Superwhisper, VoiceInk and MacWhisper, offer Whisper models.
Is there a free app that handles mixed speech?
TypeAloud is free and open source, and uses Qwen3-ASR locally on your Mac.

About TypeAloud

TypeAloud: free dictation for Mac, transcribed on your Mac.

Hold Ctrl+Shift, speak, release, and the text appears at your cursor. No word limit, no account, open source.

Download for Mac