Ask the industry how to make AI agents better and for two years the answer was the same: add more. A bigger system prompt. More tools. More guardrails, more instructions, more gating logic deciding which tool the model is allowed to see this turn. Every agent framework sold more steering as the path to reliability.
The teams shipping the best agents right now are doing the opposite. They are deleting scaffolding. And the side effect nobody prices in: once the steering lives in the harness and the tools, you stop needing a frontier model to hold the whole thing together. That is the real lever of LLM cost optimization — not cheaper tokens, architecture.
I spent the last two weeks running my coding work on four open-weight models instead of Claude. This post is the argument, and the bill.
The Industry Is Quietly Deleting Its Scaffolding
Anthropic said it first, back when nobody wanted to hear it. Their Building Effective Agents guidance from late 2024 states that the most successful agent implementations use simple, composable patterns — not complex frameworks. The industry read it and built frameworks anyway. I did too. I built my own agentic runtime in 3 days and killed it 11 days later, so I am not pointing fingers here.
Two years later the evidence caught up. Databricks benchmarked coding agents against their multi-million-line codebase — real pull requests, graded by the test suites their own developers wrote. The headline finding was not about models. Running the same model through different harnesses changed cost per task by more than 2x while quality stayed flat.
Same model. Same tasks. Double the cost. The difference was the scaffolding around the model, not the model.
The agent harness affects cost more than model choice. Databricks benchmarked coding agents on their multi-million-line codebase in 2026 and found the same model run through different harnesses differed more than 2x in cost per task at equal quality. The efficient harness sent about 3x less context per turn.
Better Harness, Less Steering
The harness that won on efficiency in the Databricks benchmark is pi — an open-source coding agent that sends about 3x less context per turn than the alternatives. It keeps a tight working set instead of dragging the whole session history through every call. Less context per turn means less steering: the model sees what matters now, not everything that ever happened.
That is the harness I run. Not because I predicted the benchmark — because I had already learned the build-vs-buy lesson the expensive way and picked an open-source harness with real community behind it. Subagent definitions, MCP servers, skills — the extension points are maintained by people who use them daily. When Kimi K3 weights dropped, pi supported it before I finished reading the announcement.
A good harness replaces a category of prompt engineering. I do not write instructions about context management, session compaction, or which files to keep in view. The harness does it structurally. Every instruction I delete is tokens I stop paying for on every single call — and one less thing a mid-tier model can get confused by.
Better Tools, Fewer Tools
The same inversion works at the tool level. My AI receptionist started with 12 narrow tools — collect_patient_name, collect_phone, fetch_slots, one tool per step, plus a system prompt explaining the choreography. The prompt grew to roughly 20,000 tokens of tool-ordering rules. The agent needed a top-tier model just to follow its own manual.
I rewrote it as 5 deep tools. One tool per domain, all parameters optional, and each tool's return includes a next_step field telling the model what is valid to do next. The state machine moved out of the prompt and into the tool returns. This is John Ousterhout's deep modules principle applied to agent tools: simple interface, complexity pulled down into the implementation.
Consolidating narrow agent tools into fewer deep tools cuts cost and improves quality at the same time. As of Q1 2026, rewriting my production receptionist agent from 12 tools to 5 dropped the system prompt from ~20,000 to ~1,200 tokens, cut median latency from 9.4s to 1.7s, reduced cost 5x — and the eval score went up, from 87.8% to 95.1%.
Read that last part again, because it is the whole thesis in one number: the version with less steering scored higher. 95.1% against 87.8%. The 20,000-token manual was not helping the model. It was in the way.
Loose tools follow the same logic. A tool that rejects a call because a parameter is missing forces the model to reason about error recovery. A tool that accepts partial input, fills the gaps server-side, and returns next_step: "pick_schemas" turns the error into a state transition. The model never sees a failure it has to think about.
The Dividend: Open-Weight Models Do the Work
Here is why this matters for your budget and not just for engineering taste. Steering is exactly what cheap models are bad at. Take away the 20K-token manual, the 20-tool menu, the context sprawl — and the gap between a frontier model and an open-weight model shrinks to the part that matters: can it write the code.
Databricks measured that gap. GLM-5.2, an open-weight model, landed statistically tied with Claude Opus 4.8 on quality in their benchmark — at $1.28 per task against Opus's $1.94. Their developers confirmed it as a daily driver, not a benchmark toy. Kimi K3's weights landed on Hugging Face on July 27 — a 2.8T-parameter model with a 1M-token context window, open for anyone to serve.
Open-weight models are production-ready for coding agents as of mid-2026. In Databricks' internal benchmark on real pull requests, GLM-5.2 statistically tied Claude Opus 4.8 on quality at $1.28 per task versus $1.94, and Databricks developers validated it as a daily-driver model.
My fleet runs four of them, each in the role it earned in an A/B tournament I ran across 100+ task runs in April:
| Role | Model | Why |
|---|---|---|
| Orchestrator | Kimi K3 | Strongest open model on agentic work; holds the main loop |
| Planner + worker | GLM-5.2 | Tied Opus on quality in Databricks' benchmark, at a third of Kimi's token price |
| Reviewer + scout | Qwen 3.6 Plus | Won the review discipline in my tournament; 1M context, cheap enough to read everything |
| Mechanical tasks | DeepSeek V4 Flash | Renames, transcription, file moves — near-free |
The routing rule underneath: weight quality by the cost of being wrong. A bad plan wastes N executions, so planning gets a quality-first model. A bad execution wastes one retry, so execution gets the balanced one. Review runs quality-first too — a missed bug costs more than the review.
Worker and reviewer agents should come from different model families because correlated biases produce correlated failures. A reviewer from the same family tends to accept the same wrong assumptions the worker made. In my fleet, GLM-5.2 writes and Qwen 3.6 Plus reviews — the family split is structural, not a preference.
LLM Cost Optimization in Practice: The $424 Bill
Theory ends here. From July 27 to August 9 the fleet made 11,291 API calls and processed 977 million tokens. The full OpenRouter bill:
| Model | Calls | Cost | Share |
|---|---|---|---|
| Kimi K3 (orchestrator) | 4,919 | $341.69 | 81% |
| GLM-5.2 (planner + worker) | 2,920 | $49.88 | 12% |
| Qwen 3.6 Plus (reviewer + scout) | 2,494 | $29.77 | 7% |
| DeepSeek V4 Flash (mechanical) | 915 | $2.13 | <1% |
| Total, 14 days | 11,291 | $424.46 |
That money bought shipped work, not benchmarks: a pay-per-report Stripe checkout on my live warranty SaaS (€2.99 single, €5.99 bundle), a production-down hotfix on the same product, a new Google Sheets MCP server with verified compare-and-swap writes and unit tests, and roughly 80 commits of home-lab infrastructure — Garmin training analysis tools, self-healing systemd timers for a media server. Merged PRs, running in production.
Two weeks of full-time coding-agent work on open-weight models cost me $424.46 as of August 2026: 11,291 API calls, 977 million tokens, blended $0.43 per million. Kimi K3 as orchestrator consumed 81% of the bill; the execution and review models combined cost $52.
Two things in that bill surprised me. First, the workers are nearly free. All the models that actually write and review code — GLM-5.2, Qwen, DeepSeek — cost $52 combined. The orchestrator ate $342. Gartner's March 2026 analysis puts agentic workloads at 5 to 30 times more tokens per task than chatbot use, and my log shows where those tokens live: not in the work, in the coordination. The main loop re-reads its context on every one of 4,919 calls. If you budget an agent deployment by looking at worker-model token prices, you are budgeting the cheap 19% of the bill.
Second, prompt caching is not an optimization. It is the business model. Of my 977 million tokens, 818 million were cache hits. Kimi K3 charges $3.00 per million input tokens, but $0.30 on cache hits — and my OpenRouter export shows $1,687.52 in cache discounts against the $424.46 I paid.
Prompt caching cut my agent fleet bill roughly 80% as of August 2026. Over two weeks, 818 million of 977 million tokens were cache hits, and OpenRouter's export shows $1,687.52 in cache discounts against a paid total of $424.46. The same traffic uncached would have billed around $2,100.
For scale: my peak day cost $103. One hard day of unsubsidized agent work costs half a Claude Max monthly subscription — the subscription I already calculated was heavily subsidized before the limits started tightening. Idle days cost under a dollar. And in the entire two weeks, exactly one call in my log went to a frontier Anthropic model. It cost $0.89.
Where Less Is More Breaks Down
The honest part. My workflow is younger than Claude Code's polish. I still adjust prompts, fix rough edges in my setup, rebuild habits the old tooling handled for me. That adjustment time is a real cost and it does not show on the OpenRouter bill.
The 81% orchestrator share is partly my fault too. I have not tuned the context flow — Kimi drags more history through the main loop than it needs, and I knew it while watching the bill grow. The stack does not demand that cost. My laziness does. Which is its own version of the thesis: the remaining fat in my bill is exactly where I still steer too much.
And frontier models still win somewhere. The hardest synthesis work — ambiguous requirements, subtle architecture calls — is where I would still reach for one. That is what the $0.89 call was. The escape hatch stays open; it just stopped being the default. My $1-a-day scheduled agent sits at the other end of the same spectrum: the less it has to think, the less the thinking has to cost.
The steering did not disappear. It moved — out of the system prompt and into the harness and the tools, where you pay for it once, at design time, instead of on every token of every call. That is the direction the whole industry is heading, one deleted instruction at a time. The teams that notice can run production coding agents for $424 a fortnight. The teams that do not will keep renting a frontier model to read their 20,000-token manual.
I help businesses figure out where automation actually pays — including what it should cost to run. If you are evaluating AI agents and want real numbers instead of vendor decks, get in touch.