AI
Agent or workflow? How to scope agentic AI
|
5 minutes

For the latest insights and top stories, prefer RUBICON on Google.
Most AI conversations I have this year start one of two ways: "We want an AI agent" or "We need AI."
Before we talk about models or frameworks, I ask something plainer. Does this task need an agent, or a workflow with one or two AI steps in it?
That answer sets the cost per case, how you test the system, and how far you can trust it on day one. Few teams ask it first.
What an agent does

An agent is a language model running in a loop. It reads a goal, picks a tool (an API, a database query, a document search), runs it, reads the result and decides what to do next. It stops when the goal is met, when it hits a step limit, or when it needs a human to approve something.
The model is only one part. The rest is engineering: the tools it can call, the documents it can search, what it remembers between cases, and the rules that stop it from doing damage.
A workflow runs steps you fixed in advance, in order. An LLM can still sit inside a step, to classify an email or summarize a contract. It just doesn't choose what happens next.
The three-question test

Before anyone draws an architecture diagram, I ask three questions about the task.
1. Are the steps the same every time? If a person follows the same checklist for every case, build the checklist as a workflow. An agent only adds cost and unpredictability.
2. Does one lookup or one generation solve it? "Summarize this ticket" or "extract these fields from the invoice" is a single call. You don't need a loop.
3. Can a human check the result before it has an effect? Agents make mistakes that look plausible. If a wrong action is expensive and nobody reviews it first, the task isn't ready for an agent, whatever the demo shows.
Flip the answers and you have a job for an agent: the path changes from case to case, the task needs several tools over several turns, and a person reviews what it proposes.
Why it matters to the business
Workflows cost less to run. Every loop is another paid model call, and each call carries more context than the last. A hard case gets expensive fast.
They're also easier to test and explain. You check each step on its own, and when an auditor or a data protection officer asks why the system did something, "step 3 applied rule X" is a good answer. An agent's path changes from case to case. You can only test its outcomes across a set of real cases, and your explanation is a trace of the model's reasoning.
Most good designs are a mix

I rarely recommend all agent or all workflow. The design I suggest most often is a workflow with an agent in the one step that needs judgment.
Sometimes no step needs an agent, and one step shouldn't use a model at all. A proof of concept for a multi-team enterprise taught us the second part. Its leaders wanted to ask plain questions about people, roles and projects instead of digging through meeting notes before every check-in. Our first version let an LLM build the knowledge graph from the documents. It invented roles and created duplicate people. So we took the model out of that step. People, roles and projects came from a curated source, and the model did two narrow jobs: it turned a question into a graph query, and it wrote the answer with citations. The path was fixed. The model never decided what happened next. (Read the case.)
Take customer invoice disputes (an illustrative case):
A fixed step pulls the invoice, the order and the delivery records.
An LLM step classifies the dispute: price, quantity, damage or missing delivery.
An agent investigates. It picks the records that matter, checks the contract terms, compares quantities and looks for an existing credit note. The path depends on the dispute, so the loop pays off here.
A fixed step drafts the resolution in the finance system.
A person approves it before the customer sees anything.
My rule for any step that sets an amount of money, such as a price, a credit limit or a payment: fixed rules decide, and the model gathers facts, asks follow-up questions and explains the result. It never produces the number.
How we decide it in discovery
On our agentic AI projects, we make this call in the first step, use case discovery and scoping, before we build anything. We map the process with the people who run it and mark where judgment happens. Only those steps are candidates for an agent.
In that session I ask the client six questions:
Which process do you want to change, and how many cases does it handle each month?
Where do people lose the most time today, and where do they make mistakes?
Which decisions must stay with a person?
Which systems hold the data, and do they have APIs?
What does a correct result look like, and who can judge it?
What does your AI and data policy allow: which models, which regions, which data?
Question 5 stops more projects than the other five. If nobody in the room can say what a correct answer looks like, nobody can test the agent. The client's domain expert has to answer it. We can't do it for them.
Build the test set before the agent
That's why the most useful output of discovery is a set of real cases, each with the answer the domain expert expects.
When you change a prompt, swap a model or add a tool, you run the set again and see whether results got better or worse. "The agent seems good" becomes a number.
Two rules carry most of the weight here. Decide what correct means before you build, and make the agent ask before it acts.
What I'd tell a CTO planning an agent
Map the process first. Find the one step where the path changes from case to case and put the agent there. Keep the rest as a workflow you can test and explain.
Then go back to questions 4 to 6 above. If you know which systems the agent needs and they have APIs, someone can define a correct answer, and your AI policy allows the models and regions you have in mind, you're ready to prototype. If not, start with use case discovery and close those gaps before you pay for a prototype.
FAQ's
What is the difference between an AI agent and a workflow?
A workflow runs steps you fixed in advance, in order. An LLM can still sit inside a step, for example to classify an email or summarize a contract, but it doesn’t choose what happens next. An agent is a language model running in a loop: it reads a goal, picks a tool, runs it, reads the result and decides what to do next.
When does a task need an AI agent?
When the path changes from case to case, the task needs several tools over several turns, and a person reviews what the agent proposes. If the steps are the same every time, or one lookup or one generation solves it, build a workflow instead.
Why are workflows cheaper and easier to test than agents?
Every agent loop is another paid model call, and each call carries more context than the last. You can check a workflow step by step and explain exactly which rule each step applied. An agent’s path changes from case to case, so you can only test its outcomes across a set of real cases.
Can you combine a workflow and an agent?
Yes, and most good designs do. The design we suggest most often is a workflow with an agent in the one step that needs judgment. For any step that sets an amount of money, fixed rules decide, and the model only gathers facts and explains the result.
What do you need before prototyping an AI agent?
You need to know which systems the agent needs and whether they have APIs. Someone has to define what a correct result looks like, and your AI and data policy has to allow the models and regions you have in mind. The most useful output of discovery is a set of real cases, each with the answer your domain expert expects.
Explore more stories



