The tokens before your prompt starts

Paste a message into a chat product and price just that text, and the number you get is real but incomplete. The request that actually reaches the model carries more than what the user typed.

What rides along#

A typical call to a model API includes, ahead of the user's message:

• A system prompt — instructions, persona, formatting rules, house style — usually a few hundred to a few thousand words.
• Tool or function definitions, when the product does anything beyond replying with text: a JSON Schema object per tool, naming every parameter, its type, and a description of when to use it.
• Often, the running history of the conversation so far, resent in full because most APIs are stateless between calls.

None of that is visible in the chat window. All of it is counted.

Why the schema part is worse than it looks#

Tool definitions are JSON, and JSON is one of the more expensive things to tokenize: braces, quotes, colons, and repeated key names all cost tokens, and none of them carry meaning a compression-based tokenizer can exploit efficiently. A tool schema that reads as compact to a person — a dozen parameters across three tools — routinely runs to several hundred or low thousands of tokens once counted rather than eyeballed.

That cost does not depend on which tool actually gets called, or whether any tool gets called at all. The full set of definitions is typically sent on every request in the session, because the model has to see all its options each time it decides whether to use one.

A worked session#

Say a coding assistant carries a 3,000-token block of system prompt and tool schemas — not an unusual size for an agent with a handful of tools, each with several parameters and a description. Over a 50-turn session, that block is sent 50 times:

3,000 × 50 = 150,000 tokens of pure overhead, before a single line of the user's actual request is counted.

What that costs depends entirely on the model:

• GPT-4o mini at $0.15 per million input: 150,000 ÷ 1,000,000 × $0.15 = $0.0225.
• Claude Opus 5 at $5 per million input: 150,000 ÷ 1,000,000 × $5 = $0.75.
• GPT-5.6 Sol at $5 per million input: 150,000 ÷ 1,000,000 × $5 = $0.75.

Same schema, same session length, a 33x difference in what the overhead alone costs — before weighing anything the user actually said. On the two $5-per-million models, three quarters of a dollar was spent re-sending instructions the model had already seen forty-nine times.

Why this doesn't show up in casual testing#

A single test call looks cheap because a single system prompt and one tool block is cheap in isolation — a few thousand tokens is a rounding error next to a per-million rate. The cost is a multiplication problem, not a per-call problem: it is the schema size times the number of calls in a session times the number of sessions in a day. None of those three numbers is visible from pricing a single pasted prompt, which is exactly the kind of blind spot a one-off test cannot catch.

What to do with this#

If you're pricing an agentic or multi-turn product rather than a single prompt, count the system prompt and tool schemas once, then multiply by the number of calls a real session makes — not the number of messages a user types, which undercounts by ignoring every resend. A smaller, more selectively-loaded tool set is a cost lever independent of the model you pick: fewer or shorter tool definitions shrink every single call in the session, not just one of them.