Orchestrating Complex AI Workflows with AI Agents & LLMs

[00:00] Artificial intelligence is said to transform IT. Yesterday I asked AI how many AI

[00:06] agents are being created every day? The answer was 11,000 per day based on news releases and other

[00:13] public sources. At that pace, we'll have over a million new AI agents deployed this

[00:20] year. While there's no way to know exactly how many AI agents are being created each day, chances

[00:26] are that you will be asked to work on a project to develop AI agents, or to begin working with an

[00:32] orchestration platform to work with orchestration of complex workflows in your environment using AI

[00:39] agents. There's some good news. Ah, orchestration of workflows with agents are in

[00:46] many ways an extension of some of the established frameworks and tools that most developers have

[00:52] worked with before. Today we'll take a look at how agent orchestration is fitting into the

[00:59] existing IT ecosystem. Okay, now the big new new kid on the

[01:05] block, uh, are these GPT models um that are that are allowing us to have these

[01:12] large language models. And what we're what we're seeing with the LLM is that we're bringing

[01:19] in a, a strong language um faculty that is allowing us to open

[01:25] up the kind of logic that we're dealing with ah when we're automating a business task with

[01:31] technology. Right. So we know that the LLMs are trained on massive text datasets. They understand

[01:37] human language. Um. And and so this is a really nice um component

[01:43] to to pull into our design framework when we're when we're building software. So, some of the

[01:49] really, um, you know, some of the really common software that we're, that we're dealing with is

[01:55] broadly assistance. Okay. And,

[02:03] agents. And the simple thing to remember about assistants and agents um are

[02:10] that they're actually very similar. Um. But assistants are going to, to really be driven in a, in a prompt

[02:16] response framework. Right? So, you know, we ask a question. It's called a prompt. And then we get, you

[02:23] know, basically we get an answer uh, a response. Okay. Now, with agents. Um, agents

[02:30] don't really need to be prompted in this way. Um, so with agents, what we're really doing is

[02:36] we're talking about defining goals. And then what we want out are

[02:43] outcomes. Okay. And so again, assistance, we're sort of asking

[02:50] questions and looking for answers. Agents, we're defining goals and we're looking for outcomes. Now, the

[02:56] the really big um, the really big difference here is down to the definition of the word agency. An

[03:03] agency means that we are giving the software agency to actually take action at its

[03:09] discretion within the boundaries that we set. Whereas an assistant, an assistant is just gonna

[03:15] sit there until it's prompted. Right? And then it's gonna be at the ready to answer us when

[03:20] we prompt it. Okay? So, remember that when we're developing assistants and when we're developing

[03:26] agents, um, there's a lot of news but new buzzwords there's sort of small language models, large language

[03:32] models ,constrain language models. Um, you know, with agents, there are a whole bunch of different

[03:37] design decisions that that we make. Um, but at the end of the day, it's it's very important to remember

[03:43] that this is software. Right? And so experienced software engineers should come into this space ah with

[03:49] some confidence that you can bring all of your best practices, ah, and past project experience. Um.

[03:56] And and all of those those same things are gonna serve you very well. So, most of the developers, um, that

[04:02] that I interact with have have have mentioned that ,once they really jump in and get started

[04:07] working with assistants and working in these agentic frameworks, that they're able to have,

[04:12] they're able to make progress quite quickly um, and have a little bit of fun, as well, with these new

[04:16] with these new technologies. So, I was having a conversation ah recently, maybe a little bit of a

[04:22] debate on with a friend of mine, and we were talking about agents and orchestration layers.

[04:29] And my friend said, hey, why? What is different about an orchestration layer? Is it really just

[04:34] robotic process automation with LLMs added into it? And ah, you know, I thought that was a that was an

[04:41] interesting take. Um. So interesting that, you know, I think, when we're talking about people who are

[04:47] building things and and talking about things, we often like an example. So, let's let's imagine that we

[04:53] have a business process here with three steps. And and, obviously, the three steps here are gonna be

[04:58] supported by technology. And, let's talk about

[05:06] what it would look like to bring an orchestration layer

[05:14] into this process. And so, the the first thing we want to remember when we're talking about

[05:20] orchestration. So we have we have a flow like this. And, when we when we talk about orchestration and

[05:27] agents, we just said a moment ago that we're bringing, we're bringing, um, we're bringing LLMs into

[05:33] place. What I'm gonna do is I'm gonna create a little triangle agent here. This is an agent. And

[05:39] we said that, you know, agents have LLM capabilities. And, we we stressed that that we're

[05:46] talking about goals and outcomes. Okay? So,

[05:53] let's say let's take an example of a process where, you know, the thing that we want out is a

[05:59] customer quote. So an actual quote that we can send to a customer.

[06:06] And the thing that we're putting in is a goal. Okay? And the goal is to create

[06:13] a quote a create a quote. So we want to create a commercial quote

[06:20] that's good enough so that our sales team can send that quote out to a customer and not get us

[06:26] in trouble with all the finance people and the product people. So, the the the way to think about

[06:31] orchestration versus robotic process

[06:38] automation.

[06:44] Okay? So, you know, ultimately, they're similar in

[06:51] that we wanna understand our process. So, in this case, it's kind of left to right. We wanna

[06:55] understand our process. And we want to boil this down into a really, really tight job story so that

[07:01] we understand, if we want to create a quote, what has to happen so that we get a quote on the other

[07:06] side that's conforming to all of our, ah, to all of our desires. So, in the old days, robotic process

[07:12] automation would typically have to interact with, let's say, the first step here is maybe we

[07:19] have ah an application like a CRM. And this is where we kinda understand, you know, where we are in

[07:25] step in a sales process. And this is where we know when we need to when it's time to create a quote.

[07:32] Okay. Now, the API would would probably have a difficult time um, discerning

[07:38] when to when it's time to make a quote, unless we had a really strong like programmatic step built

[07:45] in here where we click, you know, make a quote or something. It would have to be really obvious. Okay?

[07:51] Because we're gonna through the API, we're gonna be able to access very specific actions,

[07:56] like almost like calling a so a software process. And, we're gonna be able to access like very, very

[08:02] specific structured data. So, usually like data tables, you know, or keyword pairs or things that

[08:09] are very structured in the RPA framework. Okay? So, after we after we go in here, let's assume we can

[08:15] go in here and we can detect when we need to create a quote. And let's say that we can also

[08:20] retrieve valuable information, like the customer name and the customer address and some of the

[08:24] other, you know, particulars that we're gonna need along the way here to get over here ah, where we wanna

[08:29] get to to our quote. So, then let's say the middle process is something like, um, you know,

[08:35] product or product SKUs. Um, and, and maybe and maybe

[08:42] product catalog. Okay. So here maybe we have a database of ah of

[08:49] of approved product SKUs and that maybe we have a separate database that's kind of in this

[08:54] container that's going to give us detailed product catalog and more, more descriptions um of

[09:00] those SKUs. And then lastly, you know, over here, this might be some kind of like ah sort of financial

[09:07] application or fric financial container that knows how to look at what we pull together and sort of, um, you

[09:13] know, correctly put prices on things. Um, and then maybe there's also sort of a legal container ah,

[09:20] or legal information in this container where we kind of know our, our Ts and Cs that we wanna

[09:26] attach ah based on the based on the combination of SKUs that we put together. Okay. So, so this is kind

[09:32] of a very simple cartoon of, of a process where we're trying to, you know, when we wanna create a

[09:37] quote, we wanna leverage, you know, a CRM application, um, a data source that has product SKUs

[09:43] and product catalog. then we wanna get into some kind of financial application that can price

[09:48] SKUs and and and also apply legal T's and C's based on those SKUs. Okay? Now, in the in the RPA

[09:55] world, you know, what we're gonna need is we're gonna need APIs um, and and kind of really, well,

[10:02] well-defined data tables into each of these

[10:08] resources. But we're probably gonna run into some real problems when we when we try to

[10:15] do this use case, um, because we're gonna really need to make sure that all of our, our

[10:20] applications are configured in ways that are very, very highly structured and that really provide us

[10:26] explicit triggers and explicit structure um around this problem. So, probably not

[10:32] impossible. But if any of you have tried to use RPA to do something like creating quotes for your

[10:36] sales team uh, drop it in the comments and tell us how it went. I'm sure there there I'm sure there's some

[10:41] interesting challenges that you've run into. Now, when we go to the orchestration, we suddenly have

[10:46] agents. Okay? And the beautiful thing about agents is that we can have many of them, okay, that are

[10:52] going to be working together. So, with the agents, we're going to we're gonna create, ah, you know, in

[10:59] reality, let me make sure I have my agent colors still the same here. Um. In reality, we're gonna

[11:03] make, you know, we're probably gonna make an army of of little agents. Right? So, agents, you you know do

[11:10] really well when we keep them kind of narrowly defined and and and keep their job stories really, really

[11:16] tight. So, because we're gonna give these things agency, and and that means we don't want them

[11:22] to get off the rails. We don't want them. We don't wanna hand these things LLMs and give them a big

[11:26] scope and, you know, and have these things doing ah a lot of things that are not useful. So, we're gonna

[11:31] create ,we're probably gonna have kind of a master agent for this process that the

[11:35] orchestration layer is gonna u, is is gonna leverage to then delegate parts of this overall

[11:41] process to this little army of agents we're gonna end up with over here. So, when we're

[11:46] orchestrating the, okay, we wanna create a quote. So, now each of these resources like our CRM, for

[11:52] example, our CRM is going to need to become an MCP host. Um,

[11:59] and it's gonna need to spawn an M an MCP service very similar to down here. We're in kind of a

[12:05] client server architecture, so not much is changing in terms of the overall framework. Ah. But

[12:10] now that we have this MCP service, we can kind of spawn, we can kind of spawn

[12:16] our, you know, we can spawn our, our agents. This might be our master agent. And then our master

[12:22] agent. We're gonna get a little fancy with the artwork here. Sorry. And then our master agent, you

[12:27] know, may spawn agents, one and two that that are really well trained on how to

[12:34] dive through this MCP ser ah service in this hosted area. And these two, these two agents are really

[12:41] good at evaluating where we are. So this first agent might be able to evaluate, okay, and identify, hey,

[12:48] we're in the right step of the process to create a quote. So, this agent may say yes, we need

[12:54] to create a quote for this specific ah opportunity. Um. And then agent two might be a grabber

[13:01] agent. So when agent one says yep, this is the this is time to generate a quote, um, a message may be sent

[13:07] to agent two. And agent two may go in and grab, you know, customer name, customer number, um, customer

[13:14] address, um, and then may pull documents that have been attached into the into the CRM workflow to

[13:20] pull out, you know, what products and services have been discussed with the customer um and what is it?

[13:25] What does it look like we need to pull together for a quote? That information then comes back and

[13:31] can be checkpointed in the in the orchestration layer, so that now these agents, agent one and two

[13:38] have now done their job and they're released and they go away. What we've done is we've cached some

[13:42] context data so that now we launch um, our sort of

[13:49] captain, our captain agent here, um, now, now launches agents, you know, three and four.

[13:56] And agents three and four. Try to keep my colors the same. Here, this

[14:03] is this, this is now an MCP. We're gonna need all of these things to be MCP

[14:09] services. Okay. So that our agents know how to talk to them. And so, basically the agents three and

[14:15] four are gonna dive through the MCP of, ultimately, a data source. So now instead of lots

[14:20] of tools, we're describing really specific data sources around product SKUs and product catalog.

[14:25] So, agent three maybe designed to go take a look at the data that we fetched that

[14:32] agent two fetched and then say, okay, I know how to take the data that agent two fetch, and I know how

[14:37] to go in here, and I know how to interpret the product SKUs so that now I can pull together a

[14:44] list of SKUs that I believe satisfies the the requirements for this customer. And then the those may

[14:51] be handed off to agent four. And agent four may know may know how to come in here and navigate the

[14:56] product catalog. So, agent four says I've gotta go onto the product catalog, and I've gotta check

[15:00] and make sure that the list of SKUs that agent three gave me all work together and that we can

[15:06] ship them together. So I might be checking compatibility. I might be checking, um, you know,

[15:11] legal constraints or commercial terms. Um. I may even be doing really higher-level logic like checking

[15:17] against whether or not these SKUs um, are aligned with the current, um, product goals, sales goals,

[15:24] etc. So, really, this can get quite complicated and and can can become much richer than what we used to

[15:31] think about, you know, down here in in robotic process automation. So now the these agents three and four

[15:37] come back, you know, and they add some more data into our into our story up here. So, over here we

[15:43] got some like customer info and and and and a list of what we thought needed to be in the quote. Over here we

[15:48] started adding useful stuff like we got we got SKUs that we think we need to go in the quote. And we

[15:53] did a lot of checks around whether or not these SKUs go together off the product catalog

[15:58] and again, sales goals. And then we move forward and say, okay, now we're moving into this financial,

[16:02] legal Ts and Cs area. So again, our our little master agent knows where we're

[16:09] at. Three and four are released. And now we move. We need green again. And now we move

[16:16] into agents, um, five and six. So we call in our agent five and six team.

[16:24] And again we dive into the MCP service, into our financial and and legal, um, terms and

[16:31] conditions catalog. Okay. Now our legal department, you know, may have already had has already set up this

[16:37] data source for us and has vetted it. And our financial, uh, teams have already vetted, um, you know, our

[16:43] pricing, um, etc. Now, our first, uh, agent five may come in and knows to take

[16:50] our, ah, SKU list and come in here. And and agent five prices our SKU list. Okay. So

[16:56] agent five is is pricing. And this probably involves, you know, inferring or looking at

[17:04] sort of some notion of quantity. Um, and then agent six may come in and understand how to interpret

[17:10] our legal catalog. And so when agent six comes in and interprets the legal legal catalog, we pull

[17:16] together, um, you know, the the the terms and conditions and those sorts of things. So now the now agents five and

[17:22] six can make up here. And we've, you know, we've added some data. I guess I didn't write the word

[17:26] data, but this is all data up here. So we've added some, you know, we've added some pricing and we've

[17:31] added some legal Ts and Cs. And now you can kinda the story is kinda writing itself at this

[17:37] point. Right? So now, you know, what what can we do over here? Well, we we know the customer

[17:43] name. You know we know the customer address. We know, um, you know, we

[17:50] probably, uh, we probably uh know some of the details about what's been agreed between us and this

[17:55] customer. Like, do we have a MSA in place? Um, and so then over here in a quote, we move forward and

[18:02] we start to have, um, number and SKUs of what we think we need to ship. We've already

[18:09] done some checks against product catalog and sales goals.

[18:17] And lastly, you know, we get into pricing and legal Ts and

[18:23] Cs. And, you know, just for fun. And um,

[18:30] just for fun, you know, we'll say that there was an agent seven to top this off. And, you know, agent

[18:36] seven maybe maybe operates up here. And, you know, when we get when we get when we get

[18:43] our three processes completed, agent seven creates our

[18:50] quote in a really nice format in a way everybody's gonna like. And that's how

[18:57] we meet our goal. So, looking at this example, I think what we can see is that both approaches, the

[19:03] the agentic orchestration and the robotic process automation, are ultimately geared to increase

[19:09] productivity by automating these low-value tasks so that our teams can focus on the

[19:16] more high-value ah goal here of increasing revenue. so, you know, going back to the to the

[19:21] conversation that I had with my friend, when you really start looking at the richness of what can

[19:26] be done with agents and orchestration versus RPA, we really see this as a paradigm shift in what's

[19:33] possible as opposed to an incremental step forward.