From One-Off to Repeatable Workflows

A thing that worked once in a chat is not a thing that works. You coaxed a good result out of a back-and-forth, the answer was right, and the instinct is to keep that conversation as the recipe. But the conversation is the least repeatable artifact you produced. Rerun it tomorrow with a slightly different input and the model takes a different turn three messages in, and the careful sequence that landed yesterday now drifts somewhere you did not intend.

The move that makes it repeatable is to stop running the work as a conversation and start running it as a workflow. Anthropic draws the line cleanly: a workflow is a system where the model and its tools are orchestrated through code paths a human wrote, and an agent is a system where the model directs its own control flow — picks the next step, decides when it is done. The difference is who owns the sequence. In a workflow you own it. You decided there are three steps, in this order, with this check between step two and step three, and the model only fills in the cognitive blank at each node, and the unreliable part stays fenced inside a structure you control.

That framing also tells you why the conversation drifted. A multi-turn chat sits at the agent end of that line whether it feels like it or not. The path through the work is being improvised as you go, one message at a time. There is a measured cost to that improvisation. When Microsoft and Salesforce ran the same tasks two ways in their sharded-prompt study, once as a single assembled prompt and once with the same information spread across conversational turns, average performance dropped 39%, and on the hardest reasoning models the fall was steep: o3 went from 98.1 to 64.1. Their summary is the part to keep: "when LLMs take a wrong turn in a conversation, they get lost and do not recover." The drift is not bad luck. It is the structural property of running work as a chat, and assembling the same information into one deterministic call is what recovers most of the lost ground.

So you wrap the probabilistic calls in deterministic structure, and the cleanest version of that structure is a fixed, schema-validated container the model fills rather than a free-form exchange. A well-run internal command library tends to converge on this shape. Each runnable command is a file with a fixed anatomy, capture the current state, plan, execute, verify, summarize, and the model does the thinking inside each section while the section it belongs to, the order, and the validation are all decided in advance. Written that way, the command stops being documentation about a workflow and becomes the workflow, in a form an executor with no memory of the original chat can run correctly, including you next month.

There is a ladder of structure, and the operator skill is reading which rung a task actually needs. It runs roughly: a single prompt, then a prompt with one tool, then a fixed chain of prompts, then routing an input to the right specialized prompt, then orchestrator-workers where a coordinating call decomposes the task and farms pieces out, and finally the autonomous agent that owns its own loop. The rungs are ordered by how much control you hand the model, and the first three keep control flow in your code while the last two start giving it away. The discipline is to climb only when the rung you are on demonstrably fails, with evidence, not because the higher rung sounds more capable. Chip Huyen's observation from the field is that most teams over-shoot by two rungs — they build an orchestrator-workers system for a task a three-step chain would have handled, and then spend the life of the project debugging flexibility they never needed.

The fastest filter for the whole decision is a question you can answer before writing any code: can you draw the flowchart? If you can enumerate the steps and the branches in advance, the task shape is known, and you should build that flowchart as a workflow rather than hire a model to rediscover it on every run. You reach for an agent only when you genuinely cannot draw the flowchart ahead of time — when the steps depend on what earlier steps turn up in a way you cannot enumerate. Most production work that feels like it wants an agent is a flowchart you have not drawn yet.

The headline failure is over-engineering: building the autonomous agent when a deterministic workflow would have been more reliable, cheaper, and far easier to debug. Handing the model control flow buys flexibility you pay for in every dimension that matters in production, and you keep paying it long after the demo. If the task shape was enumerable, the agent was a mistake even when it works, because a workflow would have worked more often.

A separate trap runs the other direction: freezing the shape too early. The instinct after reading this is to take the chat that worked once and immediately harden it into a rigid pipeline. But the first version that worked was probably one example out of a class you do not understand yet. Freeze it now and you have abstracted over a pattern that has not stabilized, baking in assumptions the second and third real inputs will break. Let the shape prove itself across a handful of genuinely different runs before you commit it to code. Premature abstraction is as expensive as no abstraction.

Two quieter breaks round it out. Naive retries turn into a correctness hazard the moment a step has side effects, because rerunning a model call resamples it and you can get a different plan on the retry, so retrying after a partial write risks a double-write or an inconsistent one. Separating the model decision from the side effect and making that action idempotent is the fix, and the automation lesson covers it in full. The gate problem cuts both ways too. Put a human checkpoint on every step and you have rebuilt a slow manual process with extra friction, but omit the gate before an irreversible action and you ship errors at machine speed. Gate on irreversibility and blast radius, the email that sends, the card that charges, the table that drops, rather than on whether the model happened to sound unsure, because a confident model is not a correct one and a long chain at even 95% per-step reliability quietly decays below usable. A last trap worth naming is the heavy orchestration framework that hides the actual prompts and responses behind its own abstractions, so when the workflow misbehaves you are debugging the framework instead of the call. Build the bare version against the raw API once before you reach for a framework, so you know what it is hiding.