[00:00]
Artificial intelligence is said to transform IT. Yesterday I asked AI how many AI
[00:06]
agents are being created every day? The answer was 11,000 per day based on news releases and other
[00:13]
public sources. At that pace, we'll have over a million new AI agents deployed this
[00:20]
year. While there's no way to know exactly how many AI agents are being created each day, chances
[00:26]
are that you will be asked to work on a project to develop AI agents, or to begin working with an
[00:32]
orchestration platform to work with orchestration of complex workflows in your environment using AI
[00:39]
agents. There's some good news. Ah, orchestration of workflows with agents are in
[00:46]
many ways an extension of some of the established frameworks and tools that most developers have
[00:52]
worked with before. Today we'll take a look at how agent orchestration is fitting into the
[00:59]
existing IT ecosystem. Okay, now the big new new kid on the
[01:05]
block, uh, are these GPT models um that are that are allowing us to have these
[01:12]
large language models. And what we're what we're seeing with the LLM is that we're bringing
[01:19]
in a, a strong language um faculty that is allowing us to open
[01:25]
up the kind of logic that we're dealing with ah when we're automating a business task with
[01:31]
technology. Right. So we know that the LLMs are trained on massive text datasets. They understand
[01:37]
human language. Um. And and so this is a really nice um component
[01:43]
to to pull into our design framework when we're when we're building software. So, some of the
[01:49]
really, um, you know, some of the really common software that we're, that we're dealing with is
[01:55]
broadly assistance. Okay. And,
[02:03]
agents. And the simple thing to remember about assistants and agents um are
[02:10]
that they're actually very similar. Um. But assistants are going to, to really be driven in a, in a prompt
[02:16]
response framework. Right? So, you know, we ask a question. It's called a prompt. And then we get, you
[02:23]
know, basically we get an answer uh, a response. Okay. Now, with agents. Um, agents
[02:30]
don't really need to be prompted in this way. Um, so with agents, what we're really doing is
[02:36]
we're talking about defining goals. And then what we want out are
[02:43]
outcomes. Okay. And so again, assistance, we're sort of asking
[02:50]
questions and looking for answers. Agents, we're defining goals and we're looking for outcomes. Now, the
[02:56]
the really big um, the really big difference here is down to the definition of the word agency. An
[03:03]
agency means that we are giving the software agency to actually take action at its
[03:09]
discretion within the boundaries that we set. Whereas an assistant, an assistant is just gonna
[03:15]
sit there until it's prompted. Right? And then it's gonna be at the ready to answer us when
[03:20]
we prompt it. Okay? So, remember that when we're developing assistants and when we're developing
[03:26]
agents, um, there's a lot of news but new buzzwords there's sort of small language models, large language
[03:32]
models ,constrain language models. Um, you know, with agents, there are a whole bunch of different
[03:37]
design decisions that that we make. Um, but at the end of the day, it's it's very important to remember
[03:43]
that this is software. Right? And so experienced software engineers should come into this space ah with
[03:49]
some confidence that you can bring all of your best practices, ah, and past project experience. Um.
[03:56]
And and all of those those same things are gonna serve you very well. So, most of the developers, um, that
[04:02]
that I interact with have have have mentioned that ,once they really jump in and get started
[04:07]
working with assistants and working in these agentic frameworks, that they're able to have,
[04:12]
they're able to make progress quite quickly um, and have a little bit of fun, as well, with these new
[04:16]
with these new technologies. So, I was having a conversation ah recently, maybe a little bit of a
[04:22]
debate on with a friend of mine, and we were talking about agents and orchestration layers.
[04:29]
And my friend said, hey, why? What is different about an orchestration layer? Is it really just
[04:34]
robotic process automation with LLMs added into it? And ah, you know, I thought that was a that was an
[04:41]
interesting take. Um. So interesting that, you know, I think, when we're talking about people who are
[04:47]
building things and and talking about things, we often like an example. So, let's let's imagine that we
[04:53]
have a business process here with three steps. And and, obviously, the three steps here are gonna be
[04:58]
supported by technology. And, let's talk about
[05:06]
what it would look like to bring an orchestration layer
[05:14]
into this process. And so, the the first thing we want to remember when we're talking about
[05:20]
orchestration. So we have we have a flow like this. And, when we when we talk about orchestration and
[05:27]
agents, we just said a moment ago that we're bringing, we're bringing, um, we're bringing LLMs into
[05:33]
place. What I'm gonna do is I'm gonna create a little triangle agent here. This is an agent. And
[05:39]
we said that, you know, agents have LLM capabilities. And, we we stressed that that we're
[05:46]
talking about goals and outcomes. Okay? So,
[05:53]
let's say let's take an example of a process where, you know, the thing that we want out is a
[05:59]
customer quote. So an actual quote that we can send to a customer.
[06:06]
And the thing that we're putting in is a goal. Okay? And the goal is to create
[06:13]
a quote a create a quote. So we want to create a commercial quote
[06:20]
that's good enough so that our sales team can send that quote out to a customer and not get us
[06:26]
in trouble with all the finance people and the product people. So, the the the way to think about
[06:31]
orchestration versus robotic process
[06:38]
automation.
[06:44]
Okay? So, you know, ultimately, they're similar in
[06:51]
that we wanna understand our process. So, in this case, it's kind of left to right. We wanna
[06:55]
understand our process. And we want to boil this down into a really, really tight job story so that
[07:01]
we understand, if we want to create a quote, what has to happen so that we get a quote on the other
[07:06]
side that's conforming to all of our, ah, to all of our desires. So, in the old days, robotic process
[07:12]
automation would typically have to interact with, let's say, the first step here is maybe we
[07:19]
have ah an application like a CRM. And this is where we kinda understand, you know, where we are in
[07:25]
step in a sales process. And this is where we know when we need to when it's time to create a quote.
[07:32]
Okay. Now, the API would would probably have a difficult time um, discerning
[07:38]
when to when it's time to make a quote, unless we had a really strong like programmatic step built
[07:45]
in here where we click, you know, make a quote or something. It would have to be really obvious. Okay?
[07:51]
Because we're gonna through the API, we're gonna be able to access very specific actions,
[07:56]
like almost like calling a so a software process. And, we're gonna be able to access like very, very
[08:02]
specific structured data. So, usually like data tables, you know, or keyword pairs or things that
[08:09]
are very structured in the RPA framework. Okay? So, after we after we go in here, let's assume we can
[08:15]
go in here and we can detect when we need to create a quote. And let's say that we can also
[08:20]
retrieve valuable information, like the customer name and the customer address and some of the
[08:24]
other, you know, particulars that we're gonna need along the way here to get over here ah, where we wanna
[08:29]
get to to our quote. So, then let's say the middle process is something like, um, you know,
[08:35]
product or product SKUs. Um, and, and maybe and maybe
[08:42]
product catalog. Okay. So here maybe we have a database of ah of
[08:49]
of approved product SKUs and that maybe we have a separate database that's kind of in this
[08:54]
container that's going to give us detailed product catalog and more, more descriptions um of
[09:00]
those SKUs. And then lastly, you know, over here, this might be some kind of like ah sort of financial
[09:07]
application or fric financial container that knows how to look at what we pull together and sort of, um, you
[09:13]
know, correctly put prices on things. Um, and then maybe there's also sort of a legal container ah,
[09:20]
or legal information in this container where we kind of know our, our Ts and Cs that we wanna
[09:26]
attach ah based on the based on the combination of SKUs that we put together. Okay. So, so this is kind
[09:32]
of a very simple cartoon of, of a process where we're trying to, you know, when we wanna create a
[09:37]
quote, we wanna leverage, you know, a CRM application, um, a data source that has product SKUs
[09:43]
and product catalog. then we wanna get into some kind of financial application that can price
[09:48]
SKUs and and and also apply legal T's and C's based on those SKUs. Okay? Now, in the in the RPA
[09:55]
world, you know, what we're gonna need is we're gonna need APIs um, and and kind of really, well,
[10:02]
well-defined data tables into each of these
[10:08]
resources. But we're probably gonna run into some real problems when we when we try to
[10:15]
do this use case, um, because we're gonna really need to make sure that all of our, our
[10:20]
applications are configured in ways that are very, very highly structured and that really provide us
[10:26]
explicit triggers and explicit structure um around this problem. So, probably not
[10:32]
impossible. But if any of you have tried to use RPA to do something like creating quotes for your
[10:36]
sales team uh, drop it in the comments and tell us how it went. I'm sure there there I'm sure there's some
[10:41]
interesting challenges that you've run into. Now, when we go to the orchestration, we suddenly have
[10:46]
agents. Okay? And the beautiful thing about agents is that we can have many of them, okay, that are
[10:52]
going to be working together. So, with the agents, we're going to we're gonna create, ah, you know, in
[10:59]
reality, let me make sure I have my agent colors still the same here. Um. In reality, we're gonna
[11:03]
make, you know, we're probably gonna make an army of of little agents. Right? So, agents, you you know do
[11:10]
really well when we keep them kind of narrowly defined and and and keep their job stories really, really
[11:16]
tight. So, because we're gonna give these things agency, and and that means we don't want them
[11:22]
to get off the rails. We don't want them. We don't wanna hand these things LLMs and give them a big
[11:26]
scope and, you know, and have these things doing ah a lot of things that are not useful. So, we're gonna
[11:31]
create ,we're probably gonna have kind of a master agent for this process that the
[11:35]
orchestration layer is gonna u, is is gonna leverage to then delegate parts of this overall
[11:41]
process to this little army of agents we're gonna end up with over here. So, when we're
[11:46]
orchestrating the, okay, we wanna create a quote. So, now each of these resources like our CRM, for
[11:52]
example, our CRM is going to need to become an MCP host. Um,
[11:59]
and it's gonna need to spawn an M an MCP service very similar to down here. We're in kind of a
[12:05]
client server architecture, so not much is changing in terms of the overall framework. Ah. But
[12:10]
now that we have this MCP service, we can kind of spawn, we can kind of spawn
[12:16]
our, you know, we can spawn our, our agents. This might be our master agent. And then our master
[12:22]
agent. We're gonna get a little fancy with the artwork here. Sorry. And then our master agent, you
[12:27]
know, may spawn agents, one and two that that are really well trained on how to
[12:34]
dive through this MCP ser ah service in this hosted area. And these two, these two agents are really
[12:41]
good at evaluating where we are. So this first agent might be able to evaluate, okay, and identify, hey,
[12:48]
we're in the right step of the process to create a quote. So, this agent may say yes, we need
[12:54]
to create a quote for this specific ah opportunity. Um. And then agent two might be a grabber
[13:01]
agent. So when agent one says yep, this is the this is time to generate a quote, um, a message may be sent
[13:07]
to agent two. And agent two may go in and grab, you know, customer name, customer number, um, customer
[13:14]
address, um, and then may pull documents that have been attached into the into the CRM workflow to
[13:20]
pull out, you know, what products and services have been discussed with the customer um and what is it?
[13:25]
What does it look like we need to pull together for a quote? That information then comes back and
[13:31]
can be checkpointed in the in the orchestration layer, so that now these agents, agent one and two
[13:38]
have now done their job and they're released and they go away. What we've done is we've cached some
[13:42]
context data so that now we launch um, our sort of
[13:49]
captain, our captain agent here, um, now, now launches agents, you know, three and four.
[13:56]
And agents three and four. Try to keep my colors the same. Here, this
[14:03]
is this, this is now an MCP. We're gonna need all of these things to be MCP
[14:09]
services. Okay. So that our agents know how to talk to them. And so, basically the agents three and
[14:15]
four are gonna dive through the MCP of, ultimately, a data source. So now instead of lots
[14:20]
of tools, we're describing really specific data sources around product SKUs and product catalog.
[14:25]
So, agent three maybe designed to go take a look at the data that we fetched that
[14:32]
agent two fetched and then say, okay, I know how to take the data that agent two fetch, and I know how
[14:37]
to go in here, and I know how to interpret the product SKUs so that now I can pull together a
[14:44]
list of SKUs that I believe satisfies the the requirements for this customer. And then the those may
[14:51]
be handed off to agent four. And agent four may know may know how to come in here and navigate the
[14:56]
product catalog. So, agent four says I've gotta go onto the product catalog, and I've gotta check
[15:00]
and make sure that the list of SKUs that agent three gave me all work together and that we can
[15:06]
ship them together. So I might be checking compatibility. I might be checking, um, you know,
[15:11]
legal constraints or commercial terms. Um. I may even be doing really higher-level logic like checking
[15:17]
against whether or not these SKUs um, are aligned with the current, um, product goals, sales goals,
[15:24]
etc. So, really, this can get quite complicated and and can can become much richer than what we used to
[15:31]
think about, you know, down here in in robotic process automation. So now the these agents three and four
[15:37]
come back, you know, and they add some more data into our into our story up here. So, over here we
[15:43]
got some like customer info and and and and a list of what we thought needed to be in the quote. Over here we
[15:48]
started adding useful stuff like we got we got SKUs that we think we need to go in the quote. And we
[15:53]
did a lot of checks around whether or not these SKUs go together off the product catalog
[15:58]
and again, sales goals. And then we move forward and say, okay, now we're moving into this financial,
[16:02]
legal Ts and Cs area. So again, our our little master agent knows where we're
[16:09]
at. Three and four are released. And now we move. We need green again. And now we move
[16:16]
into agents, um, five and six. So we call in our agent five and six team.
[16:24]
And again we dive into the MCP service, into our financial and and legal, um, terms and
[16:31]
conditions catalog. Okay. Now our legal department, you know, may have already had has already set up this
[16:37]
data source for us and has vetted it. And our financial, uh, teams have already vetted, um, you know, our
[16:43]
pricing, um, etc. Now, our first, uh, agent five may come in and knows to take
[16:50]
our, ah, SKU list and come in here. And and agent five prices our SKU list. Okay. So
[16:56]
agent five is is pricing. And this probably involves, you know, inferring or looking at
[17:04]
sort of some notion of quantity. Um, and then agent six may come in and understand how to interpret
[17:10]
our legal catalog. And so when agent six comes in and interprets the legal legal catalog, we pull
[17:16]
together, um, you know, the the the terms and conditions and those sorts of things. So now the now agents five and
[17:22]
six can make up here. And we've, you know, we've added some data. I guess I didn't write the word
[17:26]
data, but this is all data up here. So we've added some, you know, we've added some pricing and we've
[17:31]
added some legal Ts and Cs. And now you can kinda the story is kinda writing itself at this
[17:37]
point. Right? So now, you know, what what can we do over here? Well, we we know the customer
[17:43]
name. You know we know the customer address. We know, um, you know, we
[17:50]
probably, uh, we probably uh know some of the details about what's been agreed between us and this
[17:55]
customer. Like, do we have a MSA in place? Um, and so then over here in a quote, we move forward and
[18:02]
we start to have, um, number and SKUs of what we think we need to ship. We've already
[18:09]
done some checks against product catalog and sales goals.
[18:17]
And lastly, you know, we get into pricing and legal Ts and
[18:23]
Cs. And, you know, just for fun. And um,
[18:30]
just for fun, you know, we'll say that there was an agent seven to top this off. And, you know, agent
[18:36]
seven maybe maybe operates up here. And, you know, when we get when we get when we get
[18:43]
our three processes completed, agent seven creates our
[18:50]
quote in a really nice format in a way everybody's gonna like. And that's how
[18:57]
we meet our goal. So, looking at this example, I think what we can see is that both approaches, the
[19:03]
the agentic orchestration and the robotic process automation, are ultimately geared to increase
[19:09]
productivity by automating these low-value tasks so that our teams can focus on the
[19:16]
more high-value ah goal here of increasing revenue. so, you know, going back to the to the
[19:21]
conversation that I had with my friend, when you really start looking at the richness of what can
[19:26]
be done with agents and orchestration versus RPA, we really see this as a paradigm shift in what's
[19:33]
possible as opposed to an incremental step forward.