The AI Workshop Will Not Ship
AI adoption becomes real only when it survives workflows, systems, tests, deployments, and ownership.
AI Regression Is a Software Problem
9 min read
The Workflow Does Not Care About Your AI Lore
22.08.2026, By Stephan Schwab
A workflow works for weeks. Then one morning the AI stops downloading invoice attachments and starts taking screenshots of the mail instead. The room immediately reaches for mystical explanations. Maybe the model changed. Maybe the vendor tweaked something. Maybe AI is just moody. Convenient story. Usually wrong. When a business workflow flips from useful to absurd, you are not looking at a personality problem. You are looking at a software system that lacks clear contracts, reliable checks, and someone accountable for how the whole thing behaves after the demo. Prompt iterations, better instructions, and skill libraries can improve local behavior. They do not replace executable expectations. Large language models are non-deterministic by design. That is exactly why you do not put them in charge of a workflow step that must behave the same way every time. Deterministic behavior belongs in normal software: code, rules, validations, and tools with bounded behavior. If the workflow matters, somebody has to define what “correct” means, detect when reality drifts, and decide whether the fix belongs in prompts, tools, integration code, test coverage, permissions, or fallback behavior. That is not a vibes discussion. That is software delivery. The companies now asking whether AI has good days and bad days are often the same companies insisting they are just modernizing workflows or doing product work. Fine. Call it whatever keeps the budget meeting calm. The moment an AI agent touches customer communication, files, approvals, or money, the business is running software. Software responsibility arrived whether the org chart noticed or not. When the workflow regresses, can your team tell whether the failure sits in the prompt, tool, data, or integration?
The funniest part of many AI regressions is not the bug.
It is the vocabulary around the bug.
An agent that handled invoice mail correctly last week now behaves like a confused intern with a screen-capture addiction, and grown adults start asking whether the model is having one of its emotional phases. That language is understandable. It is also a small disaster. The moment the conversation turns mystical, the real ownership question leaves the room.
An AI workflow did not wake up in a mood.
Something changed in a system.
The AI workshop will not ship, because the workshop is not the system and the model is not the control plane.
A Regressing Workflow Is Already a Software System
If a failure changes money flow, customer communication, approvals, or records, you are not watching an experiment. You are watching software in production.
This is the mental trap for many product people, operations leads, founders, and workflow modernizers. They describe the work in business terms: automate invoice handling, reduce manual effort, connect tools, help the team move faster, roll out AI support. All reasonable language.
Then the workflow becomes important.
Now an inbox matters. Attachments matter. File types matter. Email layout matters. Permissions matter. Vendor APIs matter. Retry behavior matters. The distinction between downloading a document and taking a screenshot of it suddenly matters a lot, because one can feed the next step in a controlled process and the other is a clown mask pretending to be a document pipeline.
That is software.
Not because the company sells software.
Because the company now depends on software behavior to make a business process work.
This is exactly why seemingly “non-technical” AI efforts keep wandering into technical pain. The organization thinks it bought cleverness. What it really bought was a new operational dependency with moving parts, edge cases, integration assumptions, and failure modes.
And that dependency needs to be split properly.
Use the LLM where ambiguity is real: summarizing, drafting, classifying messy input, proposing options, helping a human judge a fuzzy situation.
Do not use the LLM as the control system for a step that must execute predictably.
If the job is “download the attachment, store the file, preserve metadata, and pass the correct object to the next step,” that is ordinary software. It should be handled by deterministic tools and code. Handing that responsibility to a probabilistic model is not innovation. It is category confusion.
“Script First” Is Not the Same as Specification
More prompting can improve local output. It cannot prove that the workflow still does the right thing tomorrow.
Teams often respond to regressions the same way they respond to a disappointing first draft from a human assistant: give more feedback, restate the instructions, add another constraint, expand the skill, tighten the wording, remind the agent what it should have done.
Sometimes that helps. Good. Use it.
But do not confuse “we improved the prompt” with “we regained control of the workflow.”
Control comes from explicit expectations that survive memory, staff turnover, vendor changes, and the next excited round of optimization. If the intended behavior is “download the attachment, store it in the right location, preserve metadata, and hand the file to the next step,” then that behavior needs more than a beautifully phrased instruction. It needs checks.
Can the system tell the difference between an attachment download and an image capture?
Can it detect that the wrong artifact type was produced?
Can it surface the failure before the next step quietly accepts garbage?
Can somebody replay the run and see where behavior diverged?
Can the workflow fall back safely instead of continuing with decorative nonsense?
If the answer to those questions is no, the regression is not surprising. The only surprising part is that the workflow behaved for as long as it did.
This is the part non-developers keep getting wrong.
They see an LLM perform something impressive once and assume the model is a general replacement for software. It is not. It is a probabilistic component that can be useful inside a system designed by people who understand where probability is acceptable and where it is not.
Normal software did not become obsolete because a model can narrate its own guesses with confidence.
Agentic coding is not “make me X”. The same mistake shows up there too: people confuse a useful probabilistic collaborator with a deterministic execution engine and then act surprised when the workflow drifts.
Mood Talk Usually Hides an Ownership Gap
Calling the model unpredictable is often a polite way of admitting nobody owns the full path from instruction to operational outcome.
Yes, providers change models. Yes, APIs shift. Yes, external dependencies occasionally regress. None of that rescues the buying organization from responsibility.
If your business process depends on a vendor model, then vendor drift is part of your system design. Pretending otherwise is managerial superstition.
The question is not whether the provider changed something. The question is what your organization built around that possibility.
Did anyone define the operational contract?
Did anyone decide what must never change silently?
Did anyone put monitoring around the critical outputs?
Did anyone make room for rollback, fallback, manual override, or approval gates?
Did anyone connect product intent, workflow knowledge, and software judgment in the same room?
That last part matters more than most teams want to admit. Many AI efforts are led by people who can see the business value, guided by people who can talk confidently about tools, and implemented by someone somewhere in the stack who is expected to make it work. What is missing is the cross-cutting senior who can translate all of that into a durable system instead of a streak of lucky runs.
Lucky runs are not reliability.
They are borrowed time.
The Wrong Question Sounds Smart for About Five Minutes
The wrong question is:
“Did the model get worse?”
That question is not useless. It is just too small.
It keeps attention on the most fashionable component in the stack while ignoring everything else that had to go right for the workflow to behave.
The better questions are uglier and therefore more useful:
- What exact output was expected?
- Where was that expectation defined?
- Which step could have detected the wrong artifact immediately?
- What changed in the surrounding system?
- Which failures are acceptable and which are business incidents?
- Who owns the fix when the cause crosses prompt, tool, workflow, and integration boundaries?
Those are software questions disguised as workflow questions.
That is the point. Much of modern AI adoption is software work in business clothing. The clothes confuse buyers. The blast radius does not.
And one of the software questions is brutally simple:
Should this step be handled by an LLM at all?
For deterministic control steps, the answer is often no.
Use the model to interpret. Use software to execute.
Product Strategy Still Needs Development Discipline
The sharper the product or workflow thinking becomes, the more dangerous it is to skip the development discipline underneath.
There is a reason this pattern now shows up in product circles instead of only in developer circles. Product and operations people are increasingly close to the real workflow problems. They see friction earlier. They see waste earlier. They can spot a valuable automation target before a technical team even gets invited.
That is good news.
The bad news is that workflow proximity can create false confidence. Because the pain is visible, the system starts to feel understandable. Because AI can produce an impressive demo quickly, the implementation starts to feel light. Because the people involved are smart and thoughtful, the missing discipline starts to feel like a detail that can be filled in later.
Later is where regressions breed.
A useful product strategist can identify where automation helps. A useful developer can make a system survive contact with reality. The market keeps pretending those are separable jobs once the AI tooling gets good enough.
It is nonsense.
The more critical the workflow becomes, the more the business needs both perspectives tied together by someone who can own the seam.
That seam is where most AI projects quietly fail. Product people can see the friction. Developers can see the failure modes. Only non-developers tend to believe the model itself replaces the need for ordinary software design. It does not. It just makes that misunderstanding more expensive, faster.
What Serious Teams Do Instead
They stop asking whether the model is temperamental and start designing for drift.
They define critical outputs in concrete terms.
They add small, brutal checks around the steps that matter.
They move deterministic behavior out of the model and into ordinary software wherever they can.
They log enough to replay failures without folklore.
They decide which behavior belongs in prompts and which belongs in code or tooling.
They treat vendor change as a normal operational risk, not an insult from the universe.
They give somebody authority to say, “No, this workflow is not stable enough yet to be trusted with money, customers, or compliance.”
And they stop filing AI delivery problems under motivational categories like adoption, enablement, or experimentation once the workflow has entered production reality.
That shift can feel annoyingly sober after the excitement of the first working runs.
Good.
Sobriety is usually what separates an impressive demo from a system the finance team can survive.
Talk It Through
Tell me what is happening. I listen, ask a few practical questions, and reflect back what I see: where the risk may sit, what may be blocking delivery, and what looks worth checking next. No pitch, no obligation. Confidential and direct.
Talk it through. Practical reflection, no pitch.
Start a Conversation
🎭 This Week's Episodes
Telenovelas show what we can't say in client meetings. The drama is heightened, but the patterns are real.
The Table That Tells You Everything
Don Ricardo Sánchez invites Stefan and Sophie for dinner at his family's estate. The food is extraordinary, the conversation is careful, and Sophie...
Process Theater and the Grind
Preston Hale arrives just in time to do what he does best: call fear professionalism and bless documents that cannot actually guide the work. By Th...
Technical Consultancy
Embedded Delivery Partner
Embedded into your team as an active contributor, reducing delivery friction and helping important work move cleanly.
Technical Advisor
Peer-style technical assessments before far-reaching decisions; reduce architectural and product risk early.
Product & Delivery
Ship working software to real users earlier. Measure impact and adapt based on evidence rather than assumption.
Custom Software Development
High quality, maintainable software. Short-term augmentation that leaves lasting capability in your own team.
Recent Articles
- AI Regression Is a Software Problem (22.08.2026)
- Your Website Must Be Open for Business (20.08.2026)
- The AI Workshop Will Not Ship (18.08.2026)
- The System Only One Person Understands (13.08.2026)
- Before You Hire an AI Executive, Stop Treating AI as Magic (08.08.2026)