Guide

Why auto-captions get Cantonese wrong (and how to fix them)

You film a vlog in 廣東話, hit auto-captions, and what comes back reads like a news anchor: 不是 where you said 唔係, 他 where you said 佢, and every particle that made it sound like you — gone. This isn't your microphone's fault. It's a structural problem in how caption tools handle Cantonese.

Spoken Cantonese isn't written Chinese

Chinese has a split most languages don't: the way Cantonese is spoken (口語) and the way Chinese is conventionally written (書面語) are different registers with different vocabulary and grammar. Spoken Cantonese written down faithfully uses characters like 嘅、咗、喺、唔、佢、啲 — characters a Mandarin-trained system barely sees in its training data, because most Chinese text on the internet is written-register.

So when a caption tool meets Cantonese speech, one of two things happens. Either it was never trained on Cantonese and force-fits the audio into Mandarin — producing text that's simply wrong — or it was trained mostly on written Chinese and "translates" your speech into 書面語 as it transcribes. The second failure is sneakier: the captions are technically understandable, but they say 我們現在去哪裡 when you said 我哋而家去邊度. Your voice is casual; your captions wear a suit.

What breaks, specifically

• The particles vanish. Final particles — 啦、囉、㗎、咋、呢、嘛 — carry tone, attitude, and half the humour of Cantonese speech. Auto-captions drop or mangle them, and the personality goes with them.
• Code-switching confuses the model. Hong Kong speech mixes English into Cantonese sentences constantly (今日個 schedule 好 full). Recognizers locked to one language per pass mangle the switches, transliterate the English, or drop it.
• Homophones with six-plus tones. Cantonese distinguishes more tones than Mandarin, and tone recognition under music, room noise, or fast speech is fragile — one wrong tone, a different character, a different meaning.
• Colloquial vocabulary gets normalized. 飲咗 becomes 喝了, 好正 becomes 很好 — each substitution technically fine, together a different voice entirely.
• Names and places. Local names, shop names, and romanized words (MTR exits, dish names) come out as phonetic soup.

Why it matters if you're a creator

Captions are your voice in silent autoplay. For a Hong Kong audience, 書面語 captions under 口語 speech read as stiff at best and inauthentic at worst — the register mismatch is exactly what your audience is fluent in noticing. And if you caption bilingually, a wrong Cantonese transcript poisons the English line too: the translation inherits every error, so the mistakes ship twice.

How to get Cantonese captions right today

• Use a recognizer that actually models Cantonese. Not "Chinese" — Cantonese (yue / zh-HK). Apple's on-device speech recognition is one of the few mainstream engines with real Cantonese support; many web caption tools quietly route everything through Mandarin models. Test with one sentence full of particles and see what survives.
• Record for the recognizer. A close mic, less background music at record time (add it in the edit), and natural but unhurried delivery measurably improve tone recognition.
• Fix the transcript before you translate. Correct the Cantonese first — restore particles, fix homophones — and only then produce the English line. Never let a tool translate an unreviewed transcript.
• Keep the 口語. Resist the urge to "clean up" into written Chinese. The particles are the personality; your audience reads them fine because they're how everyone actually types anyway.
• Style the pair for reading order. Cantonese line dominant, English line secondary and visually distinct — different size and weight, so each audience finds its line instantly.

Weighing up the tools themselves? See every way to add bilingual subtitles on a Mac, compared.

Published 4 August 2026 · sordino