Why does AI cost more in Russian?
Saying the same thing in Russian takes 1.49× the tokens it takes in English. Since AI providers bill by the token, that is 49% added to every prompt you send — not for better answers, just for the encoding.
The measurement
15 of 25 by token costThe newer tokenizer closed 68% of the gap. That is real progress, but it is not the same as parity, and older models still bill at the old rate.
What that costs a real product
$3,685per year, wasted on encoding
Based on 500K calls a month at 500 English tokens each, priced at $2.50 per million input tokens. Input side only — output length is a property of the answer, not of your language.
Measured across five registers
One sentence would prove nothing, so the same five texts are measured in every language. The penalty ranges from 1.21× to 1.94× depending on what you are writing.
| Text type | Tokens | vs English |
|---|---|---|
| Business email | 49 | 1.63× |
| Support reply | 48 | 1.30× |
| Product description | 66 | 1.94× |
| Instructions | 49 | 1.44× |
| Casual message | 46 | 1.21× |
Why this happens
Cyrillic characters cost two bytes each in UTF-8 before merging even begins, and the tokenizer's vocabulary contains far fewer Cyrillic word-pieces than Latin ones. Both effects push the count up.
Frequently asked questions
Why does Russian use more tokens?
Tokenizers learn their vocabulary from training text that is overwhelmingly English, so English word-pieces get their own tokens while other languages' word forms are split into smaller fragments. The effect is mechanical, not a judgement about the language.
Can I do anything about it?
Three things help. Write system prompts and instructions in English while keeping user-facing content in Russian — the model understands the instruction either way. Prefer newer models, whose tokenizers handle non-English text noticeably better. And trim boilerplate, since the penalty applies to every token you send, including the wasted ones.
Is this measured or estimated?
Measured. The real tokenizers are run over five parallel texts in each language and the counts are exact. The only soft input is the translation itself, which is why the corpus is published in full and the spread across all five texts is shown rather than a single flattering average.
Languages that cost about the same
Method: the real BPE tokenizers — cl100k_base for the GPT-4 generation and o200k_base for GPT-4o and later — are run over a parallel corpus of five texts per language. The token counts are exact, not estimated. The translations are ours and are kept in full in the repository, because a wordier rendering would cost more tokens and you should be able to check ours.