The Most Expensive Shell Script I Ever Ran
I was three months into an AI agent system for an enterprise data warehouse automation platform — I had been building it for five years, automating code generation across 12+ target technologies — when I ran a codebase audit. The system had dozens of agents — each one reading a markdown playbook, calling an LLM, and producing output. It worked. Agents generated code, ran imports, checked for drift. The architecture was called "agentic." The team was proud of it.
Then I counted the files.
9 out of 10 files in the codebase never talked to an LLM. They read CSVs. They called APIs. They transformed data from one format to another. They moved files between directories. Regular Python doing regular work — wrapped in an architecture that routed every execution through a language model.
Each agent run took 30 to 60 seconds. The same work, extracted into a script, finishes in milliseconds. The LLM was not reasoning. It was not deciding. It was reading instructions from a markdown file and following them. The most expensive shell script I ever ran.
The system worked. I do not mean "sort of." I mean it produced correct output, handled edge cases, and delivered value to the business. The architecture was expensive and slow — but it was not wrong.
That distinction matters. Because the expensive prototype was also the smartest design decision the team made.
In a production enterprise AI agent system audit, 9 out of 10 files never communicated with an LLM. They read CSVs, called APIs, and transformed data — regular Python wrapped in an architecture that routed every execution through a language model. Each agent run took 30–60 seconds; the same work as a script finishes in milliseconds.
Why That Was the Right Way to Start
Here is the part most "AI agents are overhyped" articles skip: the prototype worked because it was expensive.
When you build an AI agent system from scratch, you do not know what the system needs to do. Not fully. You know the inputs and the desired outputs. You do not know the twelve intermediate steps, the three edge cases that surface only in production data, or the one transformation that looks simple until you hit null values in a field that was supposed to be required.
An LLM-driven prototype discovers those things for you. You write a markdown playbook — "take this metadata, produce this output." The LLM figures out the steps. Some steps work on the first try. Some fail. You adjust the playbook. The LLM tries again with different reasoning. Over weeks, the playbook converges on a process that works.
LLM agents are the best prototyping tool for workflow discovery — they let you build a working system before you fully understand the process. The LLM explores the problem space faster than any architecture diagram. The mistake is not using agents to prototype; it is treating the prototype as the production architecture.
This is the opposite of the "AI agents are a scam" take. They are not a scam. They are a discovery tool. A good one. The problem starts when discovery ends and nobody changes the architecture.
The Three Taxes You Pay After Discovery
Once the prototype works, you start paying three taxes. They compound every day you do not address them.
The latency tax. Every agent run routes through an LLM. The model reads the playbook, reasons about the steps, and executes them one by one. A process that needs 200 milliseconds as a Python script takes 30 to 60 seconds through the LLM. For a single run, that is fine. For a pipeline processing hundreds of items, it is the difference between "runs during a coffee break" and "runs overnight."
The non-determinism tax. Same input, same playbook, different execution path. The LLM might call functions in a different order, handle an edge case differently, or produce slightly different output formatting. You cannot write reliable tests against non-deterministic execution. You cannot guarantee the same result twice. For prototyping, this is acceptable — you are exploring. For production, where downstream systems depend on consistent output, it is a liability. Gartner predicts that over 40% of agentic AI projects will be scaled back or abandoned by 2027 — non-determinism and cost are the top reasons.
The economics tax. LLM agent runs take 30–60 seconds per step; the same work as a deterministic script finishes in milliseconds. As of Q2 2026, with AI providers moving to usage-based billing, every unnecessary LLM call becomes a real line item. At 85% per-agent accuracy across ten steps, end-to-end success drops to roughly 20%.
These three taxes are not failures. They are signals. When you notice them, your discovery phase is done. The prototype taught you what the system needs to do. Now it is time to build the production version.
The pattern I use comes from software engineering, not AI. It is called the strangler fig.
The Strangler Fig for AI Systems
The strangler fig is a tropical tree that grows around a host tree. It does not kill the host immediately. It wraps around it, replacing the host's root system with its own over years, until the host is gone and the fig stands alone. The host tree provides the scaffolding while the new tree grows.
Martin Fowler named the software pattern after it in 2004. The idea: when migrating a legacy system, do not rewrite it from scratch. Build the new system alongside it. Route one piece of traffic at a time from the old to the new. The old system keeps running — supporting everything the new system does not handle yet — until the new system handles everything.
The strangler fig for AI: audit every step the LLM performs, classify each as reasoning or execution, then extract execution steps into deterministic code one at a time. The LLM prototype keeps running for unextracted steps. Each extraction is independently testable and deployable. No big-bang rewrite — the deterministic pipeline grows around the prototype.
The key insight: you do not need to design the deterministic system from scratch. The prototype already mapped the workflow. You know what every step does because you watched the LLM do it for months. Extraction is a translation exercise, not a design exercise.
In the system I worked on, the first extraction was the import pipeline. A chain of steps that reads metadata from a database, transforms it into a canonical format, and loads it into a graph database. Zero reasoning needed. The LLM had been doing this work by reading a playbook and calling Python functions — the same functions that now run directly, without the LLM in the middle. The pipeline went from 60 seconds to under a second. Five years of domain knowledge made the extraction straightforward — I knew exactly what every step did because I had built the system it was processing. From non-deterministic to fully testable. From dollars per run to fractions of a cent.
The prototype stayed running for everything else. No rewrite. No migration risk. One extracted path at a time.
A graduated production system has three layers: deterministic pipelines handling the 90% (data ingestion, transformation, validation — fast, testable, cheap), LLM decision nodes handling the 10% (conflict resolution, semantic enrichment, fuzzy classification), and verification gates between them enforcing contracts in both directions.
What the Production System Looks Like
The end state is not "no AI." It is "AI where AI matters."
A production system that graduated from an LLM prototype has three layers.
Deterministic pipelines handle the 90%. Data ingestion, transformation, validation, file I/O, API calls, report generation. Fast, testable, cheap. They run on a schedule or on a trigger. Same output every time. No tokens consumed.
LLM decision nodes handle the 10%. The steps where the system genuinely needs reasoning. Conflict resolution when two data sources disagree. Semantic enrichment where the input is ambiguous. Classification tasks where the categories are fuzzy. These steps are expensive and non-deterministic by nature — but they are doing work that justifies the cost.
Verification gates sit between the two. When a deterministic pipeline produces output that feeds into an LLM node, a gate checks that the output meets the contract. When an LLM node produces output that feeds into a deterministic pipeline, another gate checks structural validity before the pipeline consumes it. Same principle from the autonomous agent experiment — structure and approval gates beat unconstrained execution.
The result: a system that uses less AI and works better. Faster, cheaper, testable, deterministic where it can be, intelligent where it needs to be. The AI reception system we deploy to clinics follows the same shape — the AI handles the conversation with patients, but the booking logic, scheduling checks, and notification system underneath are all deterministic code. The AI earns its cost at the interface layer. Everything else is a pipeline.
If you are building AI agents today, build the expensive prototype. Let the LLM figure out the workflow. Watch what it does. When the process stabilizes, start the strangler fig. Extract one deterministic path at a time. Keep the LLM where it earns its cost. Strangle the rest.
The companies that figure this out will have systems that are small enough to own — including the AI parts. The companies that stay in the prototype phase will keep paying the three taxes and wondering why their "AI system" feels fragile.
The prototype is not the product. It is the map. Build the product from the map.
Mind Momentum builds AI automation systems — from LLM prototypes to production pipelines. If you are evaluating an agent architecture and wondering what should stay AI and what should graduate to deterministic code, get in touch.