AI tokens and the cost of agentic coding

Source: Why AI Tokens are so Expensive - Computerphile, Computerphile, 25:22, uploaded 2026-07-02, playlist index 21.

Computerphile’s Mike starts with a change that has become visible to people who use coding agents. Anthropic places tighter caps on some premium accounts, and GitHub Copilot has moved from a request-based monthly price towards a credit system based on AI tokens. The change feels abrupt when a flat subscription has allowed an agent to work for hours. Mike’s explanation begins with the small unit behind the bill and ends with the repeated context that makes an apparently modest task expensive.

The model’s alphabet

A token is a word or part of a word. That definition is enough for the first question and leaves out the interesting part. A person tends to separate “the cat sat on the mat” into words, using spaces to find the boundaries. A language model treats spaces as part of the input. Full stops, curly brackets, code symbols, and Unicode characters can occupy tokens as well. Chinese characters may take one or two tokens because the tokenizer divides them differently from familiar English words.

The exact division depends on the tokenizer chosen by the model provider. The system builds its vocabulary around frequency, so a common word such as “the” makes a good token on its own. Mike gives 100,000 as a rough vocabulary size. That vocabulary includes common British English, other languages, programming syntax, and special characters. A few thousand entries do most of the work, while the larger vocabulary remains available for less common text.

The tokenizer is a string parser that sits before the language model. It turns each token into a numerical representation called an embedding. The embedding places the token in a high-dimensional space. “The” carries little meaning on its own, whilst “cat” and “at” occupy different positions. Similar words may end up in nearby regions because the training process gives them related uses. The tokenizer’s vocabulary is chosen during system development, while the embeddings are learned for the particular model.

Mike and the interviewer use a small sequence of rhymes to make the distinction concrete: “Mike on a bike”, “a bike on a mic”, and “Mike at a strike”. The phrases contain different tokens, though related words can sit near one another in the model’s learned space. A code-only vocabulary could exclude much of the language a model needs for other work. Providers therefore tend to start with a model trained on broad material, then refine it with domain-specific examples. The same system can write JavaScript or an English poem.

One prediction at a time

The token definition matters because a large language model produces output autoregressively. It reads the input as a list, makes a set of decisions, and emits one next token. It then uses the expanded list to choose the following token. The process works for the next word in “the cat sat on the mat” and for the next part of an unfinished if statement.

Mike describes the cost through a comparison with ordinary conversation. When a person answers a question, they usually retain enough of the exchange to respond without rereading the whole transcript. The model receives the context again as part of each forward pass. A short web chat may contain a 500-token question, around 1,000 tokens of hidden reasoning, and a 500-token answer. The visible answer understates the work because the reasoning also occupies the model’s context.

The simple version of the calculation looks severe. If the initial system prompt and user question total 1,100 tokens, then a 10,000-token thought process and a 1,000-token response create roughly 11,000 output tokens across the exchange. The model does not make one pass over the complete text and then stop. It produces a token, adds it to the context, and runs again. The context grows at each step, which increases the amount of computation and electricity involved.

Mike qualifies this picture with key-value caching, usually called KV caching. The model can retain intermediate relationships that it has already calculated, so it does not need to recompute every relationship between every old token on every step. Caching makes long contexts much cheaper than the naive calculation suggests. It also creates a systems problem. A provider must decide how long to keep a cache on a GPU. If someone leaves a long question to make coffee, the provider may evict the cache to serve other users. When the person returns, the system has to pre-fill the context again.

The cache changes the size of the bill without removing the basic pressure. A follow-up question has to include the original question, the model’s thoughts, its answer, and the new question. An exchange that began with 1,100 input tokens can reach nearly 11,000 on the next turn, then 20,000 and 30,000 as the conversation continues. A 50,000-token file adds that much context to every later iteration. This is where coding agents become expensive.

The extra work inside a coding agent

A chatbot answers through text. A coding agent can read files and make tool calls. Instead of returning a sentence to the user, it can ask the surrounding system to read a file, inspect particular lines, delete a file, or apply a change. The system performs the action, perhaps after asking permission, and sends the result back into the model. The agent then continues from the enlarged context.

Mike gives approximate provider prices of $2 to$3 for a million input tokens and about $15 for a million output tokens. These are his rough figures in the video, not a current price comparison or a source-backed market table. A tiny completion can therefore be cheap. Coding agents generate hidden thoughts, tool calls, file contents, patches, and final explanations, while each round carries the earlier context into the next round.

He works through a deliberately simple bug fix. The system prompt for a coding agent contains about 4,000 tokens, although the user never sees it. A short bug report adds 200 tokens, which creates an initial input of 4,200. The agent then thinks for around 2,000 output tokens and asks to read one file with a tool call of about 100 tokens. The system returns a file containing perhaps 5,000 tokens. The next model input therefore reaches about 11,300 tokens.

The agent thinks again for another 2,000 tokens, asks for a second file, and receives around 4,000 more. Mike’s running total reaches about 17,400 input tokens. A further thought, a large code patch, and a short success message push the final prompt towards 20,950 tokens. These figures are estimates on a sheet of paper, and Mike warns that he may have made an arithmetic mistake. Their purpose is to show the repeated context. A request that reads two files and applies one patch can consume roughly 55,000 to 60,000 tokens across its rounds, alongside several thousand output tokens.

The expense comes from the work around the answer. The final response may say that the bug is fixed, yet the model has already read the system instructions several times, reconsidered the user request, absorbed the files, and generated intermediate reasoning. The visible sentence gives little sense of that path.

A Copilot starfield

Mike then shows a real session in GitHub Copilot’s command-line interface. He asks it to build a Windows 3.11-style starfield screen saver, then requests changes such as adding colour and varying the speed of the stars. The agent introduces a bug and fixes it after Mike asks it to do so. The screen saver is a small, playful example that lets him watch the session grow.

After about five or six prompts, the session has moved from one file to three. Mike reads a display of roughly two million input tokens and 47,000 output tokens. He says that GitHub Copilot, with a Sonnet model behind it, wrote the program without his having to do the coding. He also says that he has not read the generated code and would not put it on somebody else’s computer. The session demonstrates token growth. It does not establish that the resulting program is correct, safe, or maintainable.

The example also explains why a coding agent feels different from a short completion. A person can ask a model to finish half of a for loop with a small amount of context. The model reads a limited piece of code and returns a narrow completion. An agent that searches a project, inspects several files, retries a failed change, and keeps its own reasoning in the conversation creates a much larger repeated input.

Token use as an incentive

Mike says that some companies measure their move towards AI by counting how many tokens their developers use. The interviewer compares that with judging a driver’s quality by how quickly the tyres wear out. Mike adds a broom that sweeps backwards and forwards through a pulley system, recalling a Simpsons gag in which a bird performs the work. The comparisons make a narrow point about measurement: token volume records activity and cost, while the useful result may remain unknown.

An incentive based on token use can also change behaviour. People may write longer prompts, choose more expensive models, allow a reasoning loop to run longer, or give the agent more files to inspect. A system that rewards consumption creates pressure to consume. Mike expects that model providers will have to make the economics visible because ordinary companies cannot pay indefinitely for work that produces no immediate return.

He remains doubtful that the generated code is good enough to justify that expense across all companies. A programmer still needs enough knowledge to judge what the agent has changed. Mike’s own starfield example makes the limit plain because he has not read the code. He sees a more plausible near-term use in short questions, quick fixes, and code completion. Agentic systems may find a place in software work, though the cost remains high whilst the industry is still subsidising much of the use.

The Copilot pricing change makes that subsidy visible. A flat monthly fee lets a user treat hours of agent activity as part of the subscription. Token credits expose the files and hidden reasoning that the earlier price kept out of view. The next question is whether a company can connect that expenditure to a real product return, such as features that produce revenue or a product that did not exist six months earlier. Mike leaves that question open.

Limits of the account

The video explains the shape of the cost through estimates, a hand-built example, and one Copilot session. It does not provide provider billing records, model specifications, a full account of KV-cache implementation, or technical sources for the quoted prices. The Anthropic caps, GitHub pricing change, token counts, and Copilot session totals remain claims or observations reported in the video.

The coding example also has a clear evidence limit. Mike has not inspected the generated starfield code, so it cannot support a judgement about software quality. The example shows how tool calls and repeated context can expand a session’s token count. It does not show that the agent solved the underlying problem well.

The description says that Computerphile tested three possible titles for the upload, including “Extreme Token Use of Agentic AI” and “What is a token and why does it cost so much?”. The final title puts the price question first. The source’s answer is more specific: the expensive part is the repeated context and tool use that sit around each visible response, with caching reducing the worst case without changing the underlying economics.