The bites an AI actually eats
An AI does not read the way you do. It does not see letters, and it does not really see words either. Before your text reaches the model, it gets chopped into pieces called tokens. A short common word is usually one token. A long or rare word gets broken into fragments.
Why not just use whole words? Because language is endless. New slang appears every month, people make typos, languages glue words together. A model that only knew complete words would need an infinite dictionary, and it would freeze the first time someone wrote “kuya” instead of “kutya” (that is Hungarian for “dog”, with the letters swapped, the kind of typo we all make).
Why not single letters then? Because a letter alone means nothing. Building meaning up from individual letters would cost enormous computing power, for no benefit.
So the token is the compromise in the middle. Frequent words stay in one piece. Rare or complicated words get split into meaningful chunks, roots and endings, building blocks. This is why an AI can handle a word it has never seen before. It does not know the word, but it knows the parts.
The meter runs on words
Classic cloud computing is rented like a car. You pay for time. You book a server with this much memory and that much processing power, and the bill ticks as long as it runs, whether you use it or not.
AI services broke with that. You do not rent the machine. You pay per token, for every piece of text the model reads and every piece it writes. You are not paying for hardware time. You are paying for the thinking itself, by the gram.
There is a good reason for this. These models run on racks of extremely expensive specialized hardware. If you had to rent a whole cluster by the hour just to ask a few questions a day, it would be absurd, and the machines would sit idle most of the time. Instead, the providers run the hardware as one giant shared pool, and everyone pays only for the work actually done. This is also the quietly beautiful part: a hobby developer with a few cents can use the same supercomputer as a large corporation.
One detail in the price list tells you a lot about how these systems work. Reading and writing are not priced the same. The text you send in costs a few dollars per million tokens. The text the model generates costs three to five times more. Reading is cheap. Writing, inventing the answer piece by piece, is the expensive part.
And one more thing is measured in tokens: the AI’s working memory, the context window. A conversation has a token budget, and when you exceed it, the beginning quietly falls out. The AI does not forget in minutes. It forgets in tokens.
My language pays a tax
Now the part that is personal for me.
I am Hungarian. Hungarian is a language that glues its grammar onto the word itself. Where English says “with our dogs”, three little words, Hungarian says “kutyáinkkal”. One word. Compact, dense, efficient. For a human.
For a tokenizer, it is the opposite. These systems were tuned mostly on English text, so common English words are single, cheap tokens. My dense Hungarian words are foreign objects, and the machine shreds them. I ran the test myself with the tokenizer used by one of the big models. “With our dogs”: three tokens. “Kutyáinkkal”: five fragments, k, uty, á, ink, kal. The machine does not see my word. It sees confetti.
I also compared full sentences with the same meaning, and in my tests the Hungarian version cost roughly sixty to seventy-five percent more tokens than the English one. The consequences are completely mechanical. The same conversation in Hungarian costs more money. It fills the AI’s memory faster, so the model forgets the start of the conversation sooner. And it even appears on the screen more slowly, because the answer is built token by token.
Some developers call this the token tax. English is the AI’s mother tongue. The rest of us pay a small customs fee at the border, in every single sentence.
The new frugality
Older programmers remember a time when memory was counted in kilobytes and every CPU cycle mattered. That discipline mostly faded as hardware got cheap. Token pricing brought it back in a new costume: suddenly, every word costs something again.
The developer community has started joking about where this leads. Teams playing “token poker” in meetings, estimating features in millions of tokens instead of work days. “Pair prompting”, where two engineers polish a single prompt together before daring to press enter, because a sloppy one burns the team budget. Year-end bonuses for whoever spent the least of their token allowance. Jokes, for now. I would not bet they all stay jokes.
The Caveman plugin is the same instinct, except real and installable. It tells the model to drop the filler: the greetings, the hedging, the “great question” warm-ups, the polite essay before the actual answer. What remains is bare fragments and exact technical content. The project claims around sixty-five percent fewer output tokens, and my own experience is in that neighborhood.
The strange discovery is how little I lost. The code is the same code. The facts are the same facts. It turns out that a large share of an AI’s eloquence is decoration, and you can cut nearly all of it without losing meaning. I stopped missing the politeness within a day. My AI grunts at me, and the grunts contain everything I need.
Do machines need words at all?
Watch what happens when I code with AI today. I describe the task in human language. The model translates that into its internal mathematics, then writes code, which is itself a language full of English words. Then it explains back to me, in human language, what it did. Language everywhere, the whole chain long.
All of that translation exists for exactly one reason: a human is in the loop. Me.
But AI systems increasingly hand work to each other. One model writes the code, another reviews it, a third tests it. And when machine talks to machine, two computers exchanging polite English sentences is theatre. Expensive theatre, billed per token. Researchers are already exploring whether models could pass their internal mathematical state to each other directly, with no words involved. Less like a conversation, more like two minds touching foreheads and transferring the whole thought at once.
It would be radically more efficient. And there are two reasons we are not there. The practical one: text is the only format that every model from every company understands. The deeper one: if machines drop human language between themselves, we lose the ability to read along. Today, when something goes wrong, a human can open the log and read what the AI did, in sentences. A stream of raw numbers between two models is silence to us.
The cheapest possible communication would also be the first one we cannot check.
Closing thought
The token is a strange unit. Not a word, not a letter, an awkward fragment in between. And yet your money, the AI’s memory, its speed, even how well it speaks your native language, all of it is measured in this one unit.
But the token has an underrated property. As long as AI is built from tokens, its work stays in a form a human can still read. My caveman fragments are still language. I can audit every line. The day machines optimize the words away entirely, communication becomes nearly free, and perfectly opaque, at the same time.
So I sit deliberately in the middle. I want fewer tokens. I do not want zero.
How much of your AI’s output do you actually read today? And would you give up the ability to read along at all, if everything became twice as fast and half as expensive?