Build With LLMs Program Deep Dive — Full Breakdown & Transcript

[00:12] LinkedIn and YouTube as well. Um a very, very warm welcome to everyone everyone joining us there from different parts of the world, I'm sure. Um it's great to see a lot of people joining us. Hi Jay, you are joining us from

[00:29] um Emmy, there's Aishwarya, software engineer. Great, Samantha from Okay. All right, we have um John from Las Vegas. Someone from Scotland.

[00:46] Okay, from Houston, Arturo. Amazing. It's great to see um you know, people from different domains and different parts of um the world joining us today. My name is Ananya and I would be your host and moderator here for the session

[01:01] and I will be bringing your questions to our guest speaker and faculty towards the end of the session. Um today is a very, very special webinar for us because this is the official launch webinar for uh Build with LLMs from

[01:13] Systems, a program from Carnegie Mellon University's School of Computer Science Executive Education. Uh you will be among uh the first to see the program in depth and you will hear about it

[01:28] directly from the faculty member who teaches it. Um and uh great to see more introductions coming in. I'm sorry, like you know, we can't uh read out all of uh the names. Uh but here's a quick context on why this program and why this

[01:42] webinar. A lot has changed about AI and LLMs uh in specific in the last three to four years. Um getting a model to give a good answer consistently has now become everything around the model. Like what context it sees, which tools it's

[01:57] allowed to call, how its output gets checked, and how it holds up in the real world. And LLM work has moved from writing clever prompts to engineering reliable systems, and that shift matters a lot. And that shift lays the

[02:11] foundation for, you know, today's topic and by extension the program as well. And while I say that in very, very, you know, a brief way, I'm very sure that more depth, perspective, context, and direction to this. So, let's um dive

[02:25] straight in. Um a very few quick ground rules before we begin. First, please put all your questions in the Q&A box and not in the chat box. Um that This is the only way that I would be able to, you know, make

[02:38] sure that your questions reach the faculty, and we will take as many the session. So, please use the Q&A box effectively. Um secondly, the chat is thoughts, your reactions, your experiences there, and please keep it

[02:52] related to the topic only. And totally, if you would like a recording and the deck of the webinar, please stay with us until the end. We will email both to everyone who attends the full session.

[03:04] All right. So, before we get to talking about the program, we definitely have to talk about the amazing institution that is bringing you this. Um so, this program comes from Carnegie Mellon University School of Computer Science.

[03:17] Um and I think uh for this topic, it's hard to think of a more fitting home. Uh Carnegie Mellon has been central to artificial intelligence since the field began. Some of the earliest AI programs were built there in the 1950s. CMU has

[03:32] also created the first machine learning department at any university and later the first undergraduate degree in AI in the US. And that history shows in the recognition that you see here on the slide. US News and World Report ranks

[03:47] CMU number one for AI programs in the US and among the most innovative schools in the country, and 13 Turing Awards are connected to the university. So, now um specifically the School of Computer Science Executive Education brings that

[04:02] research and expertise to working professionals through programs taught by the same faculty who are pushing um you know, the entire field forward. So, with that as the background, um here is a quick look at what we're going to cover.

[04:16] We'll start with how LLMs have changed the move from chatbots to agents, and why context engineering has become a core skill. Then we will go inside the path, the projects, case studies, the tools you will work with, and what the

[04:30] overall learning experience is going to look like. Next, we will look at the uh look at who the program is designed for, and what you need in order to get started. I will then quickly cover the admission details like the program fee,

[04:43] financing, and key dates, and then we will close um with a live Q&A with our faculty. So, we strongly encourage you to start sharing your questions as we go along, um and we would like the session to be as interactive as possible, so

[04:57] coming. Okay, so um on to a very exciting part of the session. It's my absolute pleasure to introduce our faculty for today. Uh Dr. Travis Breaux is an associate uh professor of computer

[05:12] science at Carnegie Mellon, where he teaches software design, engineering, and privacy. He also directs the requirements engineering lab, and his research there focuses on aligning software systems with legal and

[05:25] regulatory requirements across privacy, security, natural natural language understanding, and machine readable policies. I'll add a few um more things that you will not find in the bio here. Um in 2019 at Carnegie Mellon, he

[05:39] co-introduced the course Artificial Intelligence for Software Engineering. In 2024, he created CMU's course on prompt engineering, and both these subjects, I'm sure, carry through uh into the program as well that you will

[05:52] see today. His recent research applies large language models to his own field. It includes work on in-context learning and on using LLMs to generate and understand software goals. His background, I think, is especially

[06:05] relevant to today's webinar as he also leads the build with LLMs program. So, you will hear about, you know, about the entire program from the person who's going to be leading you today. Professor Travis, like very, very

[06:18] have you here. I think over to you now. Anything that introduction and take over the session, that would be great. introduction. Um now, we'll just go ahead and get

[06:32] And thanks, everybody, for joining today. I think it speaks to the interest in this topic and how fast this topic is moving. If you just look through the chat, if you have the chance to do that, you can see all of the different

[06:46] are coming from. And so, this is clearly a global phenomenon. It's uh it's changing how we do work. It's changing how we live our lives. And for some of us, we actually have the opportunity to build these

[06:59] systems. And so, that's what this course is really about. is really about. So, um how has the world changed? The world has changed because a few years ago, quite literally, three or four

[07:11] years ago, right, we were chatting with ChatGPT. Um and that's GPT 3.5. And we were all amazed at the fluency of what was coming back after we would ask our questions. In hindsight, what that really looks

[07:25] like to us is a very sophisticated search algorithm that has much of the world's knowledge um encoded in the pre-training of that model. Over time, what we found is there were

[07:38] gaps. There were problems in how we would apply these models to real world applications. So, for example, hallucinations were both fun to play with, but also a little disconcerting if it was, for

[07:50] example, your customer chat agent telling all of your customers that they were going to get, you know, full refunds on the products they were happy with. So, what that meant was we had to introduce um grounding into how these

[08:04] models respond. We had to control the input context to reduce those hallucinations. At the same time, model manufacturers were working on better ways to pre-train these models so that they wouldn't learn

[08:17] But, that introduced a number of um rag-based techniques. Since then, a few years ago, there was um there was a real innovation in chain-of-thought prompting. This started

[08:30] as organic prompts where you would prompt the model to basically think And it was observed in really small settings with math problems where all of problem that without that prompt, it couldn't solve that problem. And what

[08:45] people noticed was the reasoning that the model would generate was actually the key to the solution. And then, researchers started applying that same concept in other domains beyond math. And what we generally saw was an

[08:59] improvement in performance. And so, industry latched onto that concept. And language models, which is usually what we all talk about, but many models and most frontier models are actually large reasoning models. These are models that

[09:15] reasoning tokens, which you've probably heard about and you've probably even interacted with some of them. And so, what that did was it opened the door Oh, back one slide. It opened the door to um

[09:29] to how we can create agents. Essentially, the concept now is that if Essentially, the concept now is that if models can observe input and and uh information from the environment, they can also reason and then make tool

[09:42] selection calls. And so, that's kind of where this slide ends up is on the agent where this slide ends up is on the agent space. Thank you, Ananya. about is shifting away from writing prompts where you might, you know, tell

[09:57] the model what it is you want to the kind of response you want and thinking more about engineering the context. That is the input context. Um with the intent of being able to control and guide that output context. And so, a number of

[10:11] topics start to emerge in context engineering. For example, tool selection. Um we'll cover that topic in the course, when you're trying to get models to reliably select the right tools, being

[10:25] able to retrieve information through rag, being able to condense information and summarize information so that the model doesn't get overwhelmed by a really long context. And where things get really challenging is when tasks

[10:37] become increasingly more complex, we actually need agents to remember what it is they just did recently because they otherwise don't have a memory. So, we memories, especially over what we call long horizon tasks.

[10:53] long horizon tasks. Next slide. All right. So, one of the things we're trying to do is we're trying to um avoid failure. And this new form of failure can come up in many ways. It can show up

[11:06] can come up in many ways. It can show up in overconfidence if the model um often responds in ways that we think it knows something when in fact these models don't really know in the same way that we know. Um there's also challenges when

[11:20] you're using rag where they retrieve the wrong information for the task at hand. And understand that you really have to get into the internals of how uh retrieval augmented generation works. Or

[11:32] if they take the wrong action, why was that? Was it because the description of one tool looks too similar to the description of another tool given a particular context? You could actually learn to identify those potential

[11:44] failures in advance. And the problem with agents is over time these errors just compound. So once you call the wrong tool, you get the wrong output, and the problem the agent starts spinning out of control. And that's the

[11:56] kind of failure mode we want to avoid. in general, there's really kind of like multiple steps that we're going to talk

[12:09] about in terms of scaling up in this class. And so for example, it's going to be taking the right context to generate the right reasoning to perform the right action. And then where you would come in as an engineer is thinking about how to

[12:22] verify those actions. And if you can't verify them, you can't guarantee the model's performance, how to introduce safeguards and fallbacks to prevent the model from failing in its application area.

[12:42] So let's take a look at the program from the inside. around eight weeks over essentially nine modules. And so what this is going to cover is a number of topics that I think all of

[12:58] you have been seeing in the press. For example, we've talked about rag and tool example, we've talked about rag and tool calling, MCP, multi-agent systems. Many People are interested in them, but the the devil's in the details is what we

[13:11] like to say. So in the course modules, we're going to go in depth, and we're going to look at not just what the topic means, but what the research has been saying. One of the benefits of taking courses from CMU faculty, I like to

[13:24] think, is that we are all have our heads immersed in the latest research. We are following papers that are being published in archive, not even published yet, within the last few months, to understand the future direction of the

[13:37] technology. And we take that those ideas and we build them into our classes. And so this program is actually modeled directly from a CMU course. Um and that's one of the benefits. Now, in this version of the course, we're

[13:51] going to cover over 17 different tools and frameworks. What that will do is prepare you with the state of the art in tools that people are using out in Um it'll also give you a leg up because you'll have heard about some of the more

[14:03] modern tools. Now, that's not going to be the end-all be-all. We're hoping to equip you with the vocabulary and the knowledge so that when a new new tool comes out, you'll be able to understand quickly how it fits into your toolbox

[14:17] and how you can pull it out and apply it to a problem in practice. And I think the fun part of this course is it's got a lot of hands-on material. So while I'll be lecturing about some of the conceptual ideas behind these models and

[14:30] how they operate, and showing you the evidence of how they fail, your coursework will actually be focused on how to try these ideas out in practice, and our TAs will help guide you through that process.

[14:48] mentioned some of the topics, but the topics are scaffolded in a particular order to really help people layer their learning because many of these topics are independent interdependent. It's hard to make them linear, but we've

[15:00] this. So the course will start out with what we call the foundations and evolution of large language models. We're actually not just going to talk about LLMs. We're going to talk about SLMs. So we're going

[15:12] to talk about small language models and how they're created. As I mentioned, reasoning models. We're going to cover the gamut in terms of what the models are that are out there. So when you see a new model, you'll kind of know what

[15:24] anticipate the capability that you can get out of that model. we're going to be begin with in-context learning and reasoning. This is kind of where the world started in about 2021, so to speak. So, after the first large

[15:39] language models, people started thinking about what is this model doing? We're going to cover it from the beginning and bring that forward into reasoning where LLMs are operating today. We're going to look at augmentation.

[15:51] Augmentation is the general category of all the ways you can bring knowledge into that input context, and how you would engineer um the methods to get that information in, because the output or response you get from a language

[16:04] model is only as good as the content coming in. So, we're going to cover rag, tool calling, MCP, all of that in module In module four, we're kind of going to

[16:16] step things up a little bit. So, instead of just using a single model to answer a single question, we're going to start looking at pipelining models, or it different models, depending on their capabilities, but this is where that

[16:29] error starts to compound and um and actually escalate. And so, we're also techniques. Then, we're going to move into module five, where the natural conclusion is to pull these four modules together into AI agents. And so, with

[16:45] agents, we are going to introduce the react loop. We're going to talk about how agents uh basically perceive information, how they reason, and then how they call tools. We're going to talk about memory, and the need to basically

[16:58] about memory, and the need to basically store information across long contexts. some basics on collaboration to show you what that really means, and what we really think it is. After that, we're going to do a deep

[17:11] because these are critical topics, right? They're in the news all the time. We're hearing about agents running off and hacking companies without prompting. happens and what we can do about it. And that'll bring us to LLM security. And

[17:27] then lastly, we're going to focus on AI for software engineering. This is where the future really stands, right? Because as agents get better at writing code, um we will rely on agents more to write code for us. And we might even build

[17:41] code-writing agents um to supplement some of those larger models. And so, our goal is to really get you through this entire process. And by module nine with the capstone project, you'll get to actually implement your own agent. And

[17:54] you'll be able to test and evaluate and see how well it works and also build in some of these controls um to protect to protect us from these agents getting out of control. All right. Next slide.

[18:12] project, but in fact we'll have a few different project options. Um some of these would include things like the MCP-powered order assistant. So, if uh be doing things like ordering, you want a human in the loop to do approvals. Um

[18:28] another concept would be, for example, um securing an AI copilot through red teaming. Um so, there's a number of project ideas um that can be covered through this course. Next slide.

[18:43] And then there's also going to be three case studies. So, case studies are nice because what they do is they take the course concepts and they ground them in day-to-day um throughout the day. So, for example,

[18:57] looking at things like advanced context engineering for coding agents. So, there are a lot of coding agents out there. What is actually happening? What are people doing? That's what that case study is going to look at. Context

[19:09] engineering for effective agents is going to look more at um how these prompts and tool selections come together for reliability. And then the last one, um which I like, is really about um looking at this Ripple database

[19:24] um incident. Some might call it a a fiasco, but situations where agents can make mistakes and cause uh very catastrophic errors within an organization. And so it's good to see these errors because from these

[19:39] failures, we start to learn what we need to anticipate. So the role of the case the broader environment beyond the course. Next slide.

[19:51] All right. So just to summarize kind of the program experience here, um you've got myself. There's going to be a lot of hands-on AI work that's being done. I'm going to give you the lectures and some of the concepts and be your basically

[20:05] state-of-the-art research over the last three or four years. with our TAs and they're going to help you um basically apply these concepts to some of the uh the course assignments that we'll have. And then of course, you

[20:21] You'll get that from multiple directions, but also you're going to be part of a global classroom. And this is where I think things get pretty cool. um you know, that are positive that bring

[20:34] us all together and I think LLMs are actually one of those situations. I think you would be um not surprised to realize that the problems you are facing are being faced um on the west and east coast of whichever

[20:48] country you're in or whichever continent you're in. And this is a really great with other students and build some of these relationships. And then lastly, I don't want to minimize like the coolness of a prestigious certificate. This comes

[21:03] from CMU. It has our name on it. It's branded by what we offer. And so, as I said, the content in this program will connect to our actual course, and you'll connect to our actual course, and you'll be able to speak to that personally.

[21:21] And so, I mentioned the 13 skills and 17 tools, and this is just some examples of kind of what you'll see. You probably know some of these tools already, like LangChain. You know about Hugging Face, where many models are are sourced and

[21:33] offered. Um, know, I like to use Ollama. If you power, you can run small models on your laptop. It's really important to together in order to get you the most power out

[21:49] of LLMs that you can. And so, we'll introduce you to a broad range, and then you can learn to select them when you need them.

[22:08] even though we have a building piece to this, where you build and implement uh some of these ideas, these projects are actually and and assignments, they're really scaffolded. So, if you come to the course with a

[22:22] little bit of an understanding of how LLMs work, and maybe not a very strong ML background, I think you'll still be able to get something out of this. But, the intent is really to focus on people who are in kind of a data analysis or

[22:36] engineering role, right? So, as software engineering agents like Claude Code and engineering agents like Claude Code and uh GitHub Copilot have been advancing, even the best software engineers in the world are starting to offload their work

[22:50] to those agents. What that means is if you have enough direction, if you know what it is you want to do, you can rely on those agents to help you build something. And that's why I think in this course, we're focused on such a

[23:02] broader audience. We're not just looking at experienced software engineers, some of whom are probably trying to figure out how to use agents first hand, but also people with deep ML expertise

[23:14] that maybe don't know a lot about software engineering. And so issues like system reliability and high assurance, those may be concepts that are less familiar, but the problems are very familiar

[23:26] to like a machine learning engineer. With data science, the challenge becomes how do you orchestrate all of these tools that you can use to analyze data? And in fact, you know, some of the techniques we'll talk about can help you

[23:39] build pipelines for doing that. And then lastly, these are people who used to be engineers, sometimes they're people with MBAs who want to learn more about the topic. And so I want to just highlight

[23:51] course to really understand the underlying internals. And I know that because the CMU course we teach, we have students from across the university through the College of Fine Arts and Business right back into the

[24:04] School of Computer Science. So anybody who's working with a specialized role that might involve an LLM, which is increasingly everybody, this course, but the course is generally focused more on the engineer who's going

[24:18] that's the target audience we're going to be speaking to. All right. So in terms of eligibility criteria, we tried to design this course

[24:33] to make it accessible. And what that means is we expect everyone to have a functional knowledge of a Python programming language. If you don't know Python, but you know Java, you're probably in a you're going to be

[24:45] okay. Java is a more typed language, and so you probably have better programming discipline coming from a Java background than a Python background, but we do expect you to be able to write Python.

[24:58] to have some familiarity with machine learning concepts. Some of this you might be able to study on your own, uh but in general, concepts like what we mean by fine-tuning or what we mean by training models, we're not going to go

[25:12] into the deep details of that, but we will use those words frequently. So, if you find yourself less familiar with some of the vocabulary, and that will happen because we eventually will get into the deep vocabulary of LLMs, don't

[25:25] hesitate to reach out to the TAs and also to do some own background reading on your own. The problem with this topic is I honestly don't believe there are any good books you can just pick up off the shelf that are going to be current.

[25:38] Most of them have been edited at least a year ago or 6 months ago, and by then the technologies have already advanced. And then lastly, I want to point out 3-plus years of experience in IT or a technology role. This is really

[25:51] important because I think what this does is it helps you understand the domain software, um to libraries that we'll be using. Next slide.

[26:04] >> Yeah, perfect. Thank you, Professor Travis, uh for the complete, um you know, preview of the program. Um we have a few very interest- few questions that have come our way, and

[26:17] uh read them out, um and I think, like, you know, after the question segment, we can probably uh take people interested in enrolling in the program. So, we have uh quite a few questions from Ashwarya, so I will read them one by one, and um

[26:32] you know, you can take them forward. Will we have the opportunity to choose our own capstone projects based on a specific industry application or area of interest? And will we be allowed to publish the projects on GitHub as part

[26:45] of our professional portfolios? >> Yeah, so um that's a good Those are great questions. I'm going to have to work with the um the Capstone team to get the answer. In general, I'm the kind of person that

[26:59] projects because it's personal. It allows you to go a little deeper um on the project and if you can relate it to your work, that's even better. Um to my Um though if the projects come from the course, then maybe the course tries to

[27:16] hold on to the intellectual property for obvious reasons. follow up with you. So make sure you leave your contact information so we can >> Sure. Sure. Sounds good. Uh we will ensure to like, you know, get the answer

[27:30] to um Aishwarya. Okay. So second question. Um will we have the opportunity to receive feedback on our project ideas, technical decisions, and architecture throughout the course? Will we have an opportunity

[27:43] to incorporate the feedback to improve our projects? >> Yes. So this is a very feedback course and um feedback open course and I'm also um I like discussion. So you'll have at least two opportunities to do that.

[27:58] During the course lectures, as long as it relates to the course content uh for that lecture, I highly encourage you to ask questions. In addition, the TAs will also be running live sessions and those are also great times to connect. Those

[28:12] TAs that you'll have are actually my TAs for my course. And so they've already the material. Um they're not strangers to to this area. So they'll be able to answer those questions.

[28:28] required to use a predefined set of tools and frameworks for our for our projects or will we have the flexibility to choose the technologies that best suit our project requirements? >> You'll have the the freedom to choose um

[28:41] your own tools. And so and there's good reason for this because everybody uses different tools, right? Companies require their employees to use different tools, and so we want you to choose what's available to you.

[28:54] Um there's also a general question of like, "Well, wait a minute. Can I use AI to complete this course?" And what I can say to that is that yes, you can use AI to complete this course, cuz we use AI for everything. But there is a

[29:08] cautionary tale I would advise you. That is the increasing use of AI to solve problems we haven't ever solved before, that creates a a certain personal risk.

[29:20] all kinds of things. Sometimes we offload really monotonous tasks, like, this weekend." The last thing I want to do is search the web for 8 hours to find

[29:32] something that I can do for 2 hours. Let's offload it. But when it comes to building a system with a language model, language models are really good at generating code, and the risk is they'll generate so much code you have no idea

[29:45] what it does. And at that point, you're not really learning. So this is unfortunately going to be something that you have to self-regulate as a student. You'll have to look at the opportunities where you can use these agents to do

[29:58] something, to complete a task, but where then you can also either verify in depth what they've done, or allow yourself the time to try to attempt the task yourself independent of the agent, and then see what the agent does. So either way, the

[30:12] more you engage with the problem that will be assigning in this class, the from it. But you're free to do You're free to use AI. >> Okay. Very well put. Thank you,

[30:25] Professor. Um one more question from the same person. If we miss a live session due to work commitments, will there be opportunities to catch up through recordings or make up for the missed session? Would missing

[30:37] a session affect our eligibility to receive the certificate? Um I believe, Professor, the sessions are recorded and it would be accessible um later for a anything else you'd like to add on to the

[30:50] um answer, please do. >> No, that sounds great. >> Okay. All right. Um, okay, that is done.

[31:04] Okay, great. So, these are the program specific um one more question regarding Uh could you clarify whether the intermediate projects contribute to the capstone grade? Also, is there a specific scoring method or rubric used

[31:18] to evaluate them? >> I Yes, so we have rubrics um for how to evaluate all the coursework. Um but generally what happens with projects is because they're a little more open-ended, the rubric allows

[31:33] more open-ended, the rubric allows students to be to explore and uh to develop extensions to the original ideas. And uh and so we have to basically build those rubrics and we have uh to be that kind of flexible to

[31:46] be that flexible. With respect to the assignments that'll be taking place um during the course, those are more focused. And so the reason there is to really give you a practice experience where you we know that you're touching

[32:00] on certain topics that we want you to see first hand. And so the rubrics for those are fairly straightforward. Um and in that case, I think there's going to you're doing. And you're going to get feedback while you're doing it.

[32:13] Okay. Um and are we going to be exploring multimodal for data set data exploring multimodal for data set data sets across text, image, and audio? >> Yeah, so every model that we're going to consider in this course is going to be a

[32:28] multimodal model. At some points we'll be talking about models historically, talk about um you know, text-to-code models. Um,

[32:40] but in general, all the models now are multimodal. So, ever since GPT-4, and, you know, the equivalent models for Claude and so on, Gemini, um, it's all multimodal. Now, there is a distinction between multimodal models

[32:55] going to make that distinction stand with a wrong prompting style, you'll get the wrong results. So, we want to make the wrong results. So, we want to make that clear.

[33:10] question regarding the program. Um, sorry, my apologies. Uh, two more questions. How will a working professional be able to participate and complete assignment plus the course

[33:22] Um, how would the engagement, how frequently and TAs? Um, and so on.

[33:35] what we can do for you is we can make the schedule, um, as clear as possible with as much advance notice as possible. So, what that allows you to do is to give the heads-up to people you're working with, right? Whether you need an

[33:49] um, and you need to shift your work around to be able to attend. I highly encourage that if you can do it. Um, because the downside is if you miss the live experience, you'll still get the recorded experience, but it's not

[34:02] able to interact. Now, some people are okay with that learning style and that's their preference, and that style is supported. Um, but as much as possible, I would try to be able to anticipate when you want to be participating in the

[34:16] course and see if you can, recognizing that things are going to come up at work and you're not going to be able to compromise. Your work comes first. So, uh, we understand that, um, and what we're going to do is just try to support

[34:28] you by making it clear, um, when the courses and when the, uh the the virtual lab sessions are happening. >> Okay, perfect. Uh the last question is, um, are the sessions live or recorded? If live, can you tell us the schedules,

[34:44] timing, etc.? Um, this is from an anonymous person. So, Professor, if if you don't mind, I'll take the question. Um, the sessions are live, um, completely live. And, um, but that said, they are also recorded

[34:57] for you to be, um, accessing it later if required. Um, the cohort uh dates, we will just let you know in a short while. Um, and the timings and schedule and

[35:09] everything, um, our team would be able to help you with, um, that and, um, you right. So, we have a few generic questions as well, but we will keep

[35:21] that, I think, um, in the next, uh, segment. Um, okay. So, we had a very segment. Um, okay. So, we had a very clear look at, um, you know, um, the program and, um, you know, uh all that it entails. So, for everyone thinking

[35:34] about joining the cohort, here are the investment details. For learners from the USA, it is $4,000, which is currently being offered at an 11% discount from $4,500. There are monthly installment options

[35:48] available with low APR and no hidden fees. And if you're interested in this fees. And if you're interested in this way forward, our advisor, um, can get in touch with you and definitely help you with that. So, you can scan this QR code

[36:00] for more details. You will also be able to, um, you know, see, uh, link in the resources tab, um, to the program page, so you can click on that. You will be able to download the curriculum there, book a call with our advisor, and and

[36:14] learn a lot more about the program. Um, so, the next cohort runs from November so, the next cohort runs from November 5th, 2026 to February 11th, 2027. It is an 8-week program, uh, as Professor said, um, that spans across 14 weeks you

[36:29] have Christmas and New Year holidays in between. And since the cohort starts in a short while, now is the perfect time to take the next step forward and start your enrollment process. So, with that I am going to launch a poll to take your

[36:44] am going to launch a poll to take your interest in enrolling in the program. Um interest in enrolling in the program. Um all you have to do is click on yes in you're interested in any of the details that we spoke about today. Um even if

[36:58] details, you need to talk to someone about this, you can click on yes because our expert advisors will be getting in touch with you in the next you know few hours to discuss with you and guide you

[37:11] on the way forward. Um okay, so while we have the poll live for a little bit more time, we can take a couple more questions, Professor. Um

[37:27] all right, so this is not specifically about the program but there are there is a lot of interest about like you know AI and the job market and a lot of new Um so I'll read out a couple of questions

[37:41] regarding that. Um first question, models, tactics, techniques and procedures, TTPs as they're called, are evolving rapidly. How can we effectively keep pace with these continuous changes that keep happening?

[37:54] >> That's a great question. So, one thing the course is going to cover is the history. So, when you're in the moment of our lectures, you're going to be wondering like, "Wow, this this paper is from

[38:06] like, "Wow, this this paper is from 2021. That seems like decades ago in research time, right?" Um but our intent of showing you the Um but our intent of showing you the data points on technology evolution over

[38:18] data points on technology evolution over the course of the last five or six years is to help you build an intuition for where the challenges are and where the opportunities are. Some things about LLMs have actually improved and some

[38:31] things haven't changed at all. And the problem is, I think as an end user, is we just get wowed by the output and we don't understand how they work. And so this course is really going to pull back the curtain and show you where the

[38:44] where they're not and we're actually going to see what people are saying about them uh from their own research as recently as 2026. And so uh so I think you'll build that intuition and while we can't prepare you

[38:58] future, just like we can't prepare ourselves, what we can do is try to reduce the amount of surprise you'll experience.

[39:13] can you talk a little bit more about the future of job market and LLMs and where future of job market and LLMs and where the jobs um in this field will be? Will the job market for this technology mostly be used by companies in Silicon

[39:25] Valley and Texas? >> Yeah, that's a great question. Um so it's hard to predict the future, but what I believe will happen is we will start seeing LLMs become more of a compa- a commodity component that you

[39:41] would include in any kind of AI-driven software. We'll probably even stop talking about AI at some point. Right? The idea is that most software will have some form of built-in intelligence. And somebody will have to maintain that.

[39:54] Um what does that mean? It means companies um in every industry will probably have some form of small language model that's helping to assist with different parts of their tasks. And uh and so I see a lot of opportunity

[40:07] there. I also think that that means that software engineers, um however their roles will come to emerge in the future, will have to be able to navigate this space. They'll have to be able to build systems, probably using coding agents,

[40:21] have to have a good intuition for where the failure modes are going to be. we are done with the questions now. There is one more.

[40:36] Um, okay, so does the program cover system design for LLM applications like architecture trade-off, scaling latency, and cost optimization, or is it mainly and cost optimization, or is it mainly focused on building and prompting?

[40:49] >> So, um, great question. We do talk about model reliability in terms of system reliability, where you're talking about the system that you integrate the LLM into. Um, unfortunately, that is a deep, deep

[41:04] topic topic in and of itself. So, for example, to have a very reliable system, you need to collect a lot of telemetry about how that system is operating. about how that system is operating. Think DevOps, for example. Um, we won't

[41:18] really get into those topics. That's more of a software engineering course. On the other hand, we will talk about scaling. We will talk about latency and this course is going to have you measuring the cost of your different

[41:32] uses of LLMs and the prompts that you're using. And we will talk about orchestrating the models together to cause data to flow between models and the effects of what that means. So, I wish we could cover the systems part of

[41:46] the of the problem, because that's also a problem that's dear to my heart. Um, but we had to cut this off at nine martials. >> Okay, perfect. All right. Um, okay, so with that, I think we are done with um,

[42:01] all of the questions now, Professor, and we've gotten quite a lot of engagement in the poll as well. So, I will um, end the poll right now. Um, and I think that brings us to the end of today's session.

[42:16] Uh, Professor Travis, thank you so much for a very thoughtful look at what it walking us through the program in such detail. uh, do you have any parting words for us? Um, any advice that you'd like to

[42:29] share with our uh, learners? >> Well, just thanks everybody for showing up. This has been great taking the time to talk about this course together and I hope to see most of you there.

[42:41] Perfect. All right. And thank you to each and every one of you who has uh, um, as well as on LinkedIn and YouTube. As promised, the recording and deck will reach your inbox very shortly. If you

[42:56] write to us at uh, [email protected]. Um, and uh, final reminder, admission for this cohort closes very, very soon. So, we hope to see many of you in the program as well. So, thank you and have

[43:13] program as well. So, thank you and have a great rest of the day.