Why does AI cost more in Vietnamese?
Saying the same thing in Vietnamese takes 1.54× the tokens it takes in English. Since AI providers bill by the token, that is 54% added to every prompt you send — not for better answers, just for the encoding.
The measurement
12 of 25 by token costThe newer tokenizer closed 67% of the gap. That is real progress, but it is not the same as parity, and older models still bill at the old rate.
What that costs a real product
$4,075per year, wasted on encoding
Based on 500K calls a month at 500 English tokens each, priced at $2.50 per million input tokens. Input side only — output length is a property of the answer, not of your language.
Measured across five registers
One sentence would prove nothing, so the same five texts are measured in every language. The penalty ranges from 1.24× to 1.77× depending on what you are writing.
| Text type | Tokens | vs English |
|---|---|---|
| Business email | 53 | 1.77× |
| Support reply | 58 | 1.57× |
| Product description | 60 | 1.76× |
| Instructions | 42 | 1.24× |
| Casual message | 54 | 1.42× |
Why this happens
Byte-pair encoding builds its vocabulary from training text that is overwhelmingly English. Common English word-pieces earn a token of their own; the suffix chains and inflections of other languages do not, so words shatter into more pieces than they should.
Frequently asked questions
Why does Vietnamese use more tokens?
Tokenizers learn their vocabulary from training text that is overwhelmingly English, so English word-pieces get their own tokens while other languages' word forms are split into smaller fragments. The effect is mechanical, not a judgement about the language.
Can I do anything about it?
Three things help. Write system prompts and instructions in English while keeping user-facing content in Vietnamese — the model understands the instruction either way. Prefer newer models, whose tokenizers handle non-English text noticeably better. And trim boilerplate, since the penalty applies to every token you send, including the wasted ones.
Is this measured or estimated?
Measured. The real tokenizers are run over five parallel texts in each language and the counts are exact. The only soft input is the translation itself, which is why the corpus is published in full and the spread across all five texts is shown rather than a single flattering average.
Languages that cost about the same
Method: the real BPE tokenizers — cl100k_base for the GPT-4 generation and o200k_base for GPT-4o and later — are run over a parallel corpus of five texts per language. The token counts are exact, not estimated. The translations are ours and are kept in full in the repository, because a wordier rendering would cost more tokens and you should be able to check ours.