Русский

Why does AI cost more in Russian?

Saying the same thing in Russian takes 1.49× the tokens it takes in English. Since AI providers bill by the token, that is 49% added to every prompt you send — not for better answers, just for the encoding.

The measurement

15 of 25 by token cost

1.49×more tokens than English for the same meaning — 49% on top of every prompt you send.

GPT-4 / GPT-3.5

2.55×

2.17 chars/token

GPT-4o / GPT-5

1.49×

3.75 chars/token

The newer tokenizer closed 68% of the gap. That is real progress, but it is not the same as parity, and older models still bill at the old rate.

What that costs a real product

$3,685per year, wasted on encoding

Based on 500K calls a month at 500 English tokens each, priced at $2.50 per million input tokens. Input side only — output length is a property of the answer, not of your language.

Measured across five registers

One sentence would prove nothing, so the same five texts are measured in every language. The penalty ranges from 1.21× to 1.94× depending on what you are writing.

Text type	Tokens	vs English
Business email	49	1.63×
Support reply	48	1.30×
Product description	66	1.94×
Instructions	49	1.44×
Casual message	46	1.21×

Why this happens

Cyrillic characters cost two bytes each in UTF-8 before merging even begins, and the tokenizer's vocabulary contains far fewer Cyrillic word-pieces than Latin ones. Both effects push the count up.

Same Russian text, different tokenizers

We ran the same Russian texts through each provider's own tokenizer. Prices are quoted per token, so a tokenizer that needs more tokens makes the same work cost more — even at an identical list price.

Tokenizer	Tokens	vs English	vs OpenAI
OpenAI o200k_baseUsed by GPT-6.1 Sol, GPT-6 Sol, GPT-6 Luna	258	1.49×	baseline
DeepSeek V3 – V4.1Used by DeepSeek V4.1 Flash, DeepSeek V4 Pro	302	1.71×	+17%

Counted October 2, 2026 with each tokenizer's published files or the provider's own token-counting API. Not measured yet: Claude 4.7 and later, Claude 4 – 4.6, Grok 4, Gemini 3 — their model pages use an estimate. How we measure

Frequently asked questions

Why does Russian use more tokens?

Tokenizers learn their vocabulary from training text that is overwhelmingly English, so English word-pieces get their own tokens while other languages' word forms are split into smaller fragments. The effect is mechanical, not a judgement about the language.

Can I do anything about it?

Three things help. Write system prompts and instructions in English while keeping user-facing content in Russian — the model understands the instruction either way. Prefer newer models, whose tokenizers handle non-English text noticeably better. And trim boilerplate, since the penalty applies to every token you send, including the wasted ones.

Is this measured or estimated?

Measured. The real tokenizers are run over five parallel texts in each language and the counts are exact. The only soft input is the translation itself, which is why the corpus is published in full and the spread across all five texts is shown rather than a single flattering average.

Italian1.49×Korean1.50×Arabic1.45×Vietnamese1.54×Hebrew1.58×German1.38×

Method: the real BPE tokenizers — cl100k_base for the GPT-4 generation and o200k_base for GPT-4o and later — are run over a parallel corpus of five texts per language. The token counts are exact, not estimated. The translations are ours and are published in full in the study's data download, because a wordier rendering would cost more tokens and you should be able to check ours.