Prompting 101 | Code w/ Claude

[00:05] Hi everyone. Thank you for joining us today for 

[00:10] of the applied AI team here at Anthropic. And 

[00:15] AI team. And what we're going to do today is 

[00:19] practices. And we're going to use a real world 

[00:25] a little bit about what prompt engineering is. uh 

[00:30] bit familiar with this. This is the way that we 

[00:34] it to do what we want. So, this is the practice of 

[00:39] the model the context that it needs to complete 

[00:44] arrange that information in order to get the 

[00:49] a lot of different ways you might want to think 

[00:53] the best way to learn this is just to practice 

[00:58] a hands-on scenario. Uh, we're going to use an 

[01:03] with. So, we've modified what the actual customer 

[01:08] case of trying to analyze some images and get uh 

[01:14] Claude make a judgment about what content it finds 

[01:20] that this content is in, but luckily Christian and 

[01:25] to Christian to talk about the scenario and the 

[01:30] it's uh intended so so to set the stage, imagine 

[01:35] and you deal with uh car insurance claims on a 

[01:41] you have two pieces of information. Um we're going 

[01:45] see on the left hand side we have a car accident 

[01:52] before the action accident actually took place. 

[01:57] um sketch of how the accident took place as well. 

[02:03] going to try to pass on to cloud. And to begin 

[02:08] into a console and just see what what happens. 

[02:13] we can actually do this in a real manner. And 

[02:17] shiny beautiful entropic console. We're using the 

[02:24] setting temperature zero and having a a huge max 

[02:30] that there's no limitations to what CL can do. In 

[02:34] just setting the stage of what Cloud's supposed 

[02:38] um intend to review a an accident report form uh 

[02:45] in an accident and who's at fault. So you can 

[02:49] run this let me go to preview. Uh we can see here 

[02:58] skiing accident that happened on a street called 

[03:05] Um and in many ways you can sort of understand 

[03:10] prompt we actually haven't done anything to set 

[03:15] So this sort of first guess is not too bad but 

[03:20] bake into cloud. So if we switch back to the 

[03:27] prompt engineering is a very iterative empirical 

[03:32] have a test case where Claude is supposed to make 

[03:38] environment, nothing to do with skiing. Uh and in 

[03:43] to make sure it's actually tackling the problem 

[03:48] we'll go through some best practices of how we we 

[03:54] recommend others to do so as well. So, we're going 

[03:59] a great prompt. Uh, first we want to talk a 

[04:04] looks like. So you might be familiar with kind of 

[04:09] and forth having a more kind of conversational 

[04:14] like this, we're probably using the API and we 

[04:19] and have it nail the task the first time around 

[04:24] Uh, so the kind of structure that we recommend is 

[04:30] Claude, "What are you here to do? What's your 

[04:34] today?" Then we provide content. So in this 

[04:38] the form and the drawing of the accident and how 

[04:43] might also be something you're retrieving from 

[04:47] is. We're going to give some detailed instructions 

[04:52] how we want Claude to go through the task and how 

[04:58] some examples to Claude. Here's an example of some 

[05:03] should respond when given that content. And at 

[05:08] that's really important for Claude to understand 

[05:12] information with Claude, emphasizing things that 

[05:17] go ahead and do your work." So, here's another 

[05:23] bit more of a breakdown, and we're going to walk 

[05:27] show you how we build this up, um, in the console. 

[05:33] going to talk about the task context and the tone 

[05:38] task context, as you realized when I went through 

[05:43] elaborating what what scenario Chlo was actually 

[05:48] tell that claw doesn't necessarily need to guess a 

[05:53] our case, we really want to break that down, make 

[05:57] and also make sure we understand what's the 

[06:02] as well, we also make sure we add a little bit of 

[06:08] Claw to stay factual and to stay confident. So if 

[06:14] we don't want to guess and just sort of mislead 

[06:19] and in our case we want to make sure that we can 

[06:23] sure that assessment is as clear and as confident 

[06:27] what we're doing. So if we transition back to the 

[06:34] here. So I'll just navigate to V2. And you can see 

[06:41] we didn't really do that last time around just 

[06:45] what we're seeing here, this is the car accident 

[06:51] going through what actually happened. You 

[06:55] both on the left and right hand side. And the main 

[06:58] Claude can understand this manually generated data 

[07:05] uh corroborated by if I navigate back here to this 

[07:11] case, the form is just a different um data point 

[07:18] want to bake in more information into our version 

[07:24] a lot more on what's going on. So, you can see 

[07:29] supposed to help a human's claim claims adjuster 

[07:34] Swedish as well. Um, you can see here we're also 

[07:39] the incident and that you should not um make an 

[07:45] And that's really key because if we run this, 

[07:49] settings as well. Clo my new shiny model zero 

[07:54] here what actually happens in this case. Um, CL is 

[08:02] accidents, not skiing accidents, which is great. 

[08:08] marked on on checkbox one and then vehicle B was 

[08:13] still tell that there's some information missing 

[08:19] of who's at fault here. And this is great. This 

[08:24] make anything any claims that aren't um uh factual 

[08:30] when you're when you're confident. But there's 

[08:33] um regarding the form uh what the form actually 

[08:39] we want to want to bake into this LM application 

[08:44] adding it to the system prompt which Hannah will 

[08:50] the next item we're going to add to the prompt 

[08:56] and images and here as Christian was saying we 

[09:01] going to be the same every single time. The form 

[09:06] type of information to provide to Claude to tell 

[09:10] be looking at. We know that will not ever alter 

[09:15] filled out will change, but the form itself is not 

[09:20] um information to put into the system prompt. Also 

[09:25] considering using prompt caching. This will always 

[09:29] spend less time trying to figure out what the form 

[09:35] it's going to do a better job of reading the form 

[09:41] So another thing I want to touch on here is how we 

[09:47] really loves structure, loves organization. 

[09:51] standard structure in your prompts. And there's 

[09:56] understand the information better. I also just 

[10:01] lot of really great examples. So definitely take 

[10:05] don't worry. All of this content is online with 

[10:10] guys to check it out there too. Um anyway the uh 

[10:17] tags also markdown is pretty useful to Claude 

[10:23] specify what's inside those tags. So we can tell 

[10:29] going to read some content and these XML tags are 

[10:33] tags is related to the user's preferences and 

[10:38] maybe at later points in the prompt. Um, so I 

[10:44] actually do this in this case. And Christian's 

[10:49] keeping everything about the other part of the 

[10:54] case to put this information in the system prompt. 

[10:59] it in the system prompt here. And we're going 

[11:03] this form. So this is a Swedish car accident 

[11:07] this title. It'll have two columns. The columns 

[11:13] about each of the 17 rows and what they mean. 

[11:18] Claude was reading individually each of the lines 

[11:23] that information up front. And we're also going 

[11:27] how this form should be filled out. This is also 

[11:32] like, you know, humans are filling this form 

[11:37] People might put a circle. They might scribble. 

[11:42] be many types of markings that you need to look 

[11:47] give Claude a little bit of information about how 

[11:51] of this form is. And all of this is context 

[11:56] um do a better job analyzing the form. So if 

[12:02] So we've kept the same user prompt down here. 

[12:08] the we have the same user prompt here. Still 

[12:14] And we'll see here that it's spending less time. 

[12:19] about what the form is because it already knows 

[12:23] of bringing us that information back. It's going 

[12:28] checked, what the sketch shows. And here Claude 

[12:33] additional context that we gave to Claude. Claude 

[12:39] at fault in this case based on this drawing and 

[12:44] improvement in the way Claude is analyzing these. 

[12:49] at the drawing and at the list that vehicle 

[12:55] Uh so we're going to go back to the slides and 

[12:59] really using in this prompt um but can be really 

[13:06] and making it work better. Exactly. I think um 

[13:11] I think examples or few shot is a mechanism that 

[13:18] imagine this um in in quite a non-trivial way as 

[13:25] even in this case concrete accidents that have 

[13:31] But you with your human intuition and your human 

[13:37] conclusion. Then you can bake that information 

[13:42] examples of a the data that that it's supposed 

[13:47] you can just base 64 encode a a an image and have 

[13:53] into the examples and then on top of that you can 

[13:58] of how to break that down and understand it. This 

[14:02] in how you can sort of push the limits of your 

[14:08] into system prompt. And this again is sort of the 

[14:12] sort of always want to push the limits of your 

[14:17] it's going wrong and try to add that into system 

[14:21] of mimics that u takes place it's able to actually 

[14:27] as well, this is just a little example of how we 

[14:34] structure that we we um we enjoy. It's it gives a 

[14:38] fine-tuned on as well. Um and it works perfectly 

[14:43] doing this just because it's a simple demo, 

[14:47] building this for an insurance company, you'd have 

[14:52] difficult, maybe in the gray, that you'd like to 

[14:57] to make the verdict next time. Um, another topic 

[15:02] in this demo, is conversation history. It's in the 

[15:08] that the enough context rich information is at 

[15:15] your behalf. Um in our case now this isn't really 

[15:21] happening in the background. You can imagine for 

[15:25] system some data is generated out of this and then 

[15:29] end. If you were have to build something much more 

[15:34] history that would be um relevant to bring in this 

[15:40] because it enriches the context that Claude works 

[15:47] we do is and the next step is try to make sure 

[15:55] So, now we're going to build out the 

[15:59] and that's coming back to the reminder of what 

[16:03] about any important guidelines that we want it 

[16:09] a preventing hallucinations. Um, so we want Claude 

[16:16] this prompt, right? Or not finding in the data. 

[16:21] we don't want Claude to take its best guess or 

[16:26] it's not. If the sketch is unintelligible, the 

[16:32] and even a human would not be able to figure it 

[16:36] so these are some of the things we'll include in 

[16:41] Claude. Uh remind it to do things like answer only 

[16:46] refer back to what it has seen in the form anytime 

[16:52] vehicle B turned right, it should say I know this 

[16:57] or whatever it might be. We can kind of give 

[17:01] back to the console, we can see the next version 

[17:09] going to keep everything the same here in the 

[17:13] background context that we gave to Claude about 

[17:17] out. We're not changing anything else about the 

[17:22] detailed list of tasks. And this is how we want 

[17:27] key thing that we found here as we were building 

[17:31] example is that the order in which Claude analyzes 

[17:36] analogous to way you might think about doing this. 

[17:41] at the drawing first and try to understand what 

[17:45] a bunch of boxes and lines. We don't really know 

[17:50] additional context. But if we have the form and we 

[17:55] talking about a car accident and that we're seeing 

[17:59] doing at certain times, then we know a little 

[18:04] in the drawing. And so that's the kind of detail 

[18:08] "Hey, first go look at the form. Look at it very 

[18:13] checked. Make sure you're not missing anything 

[18:19] in that. And then move on to the sketch. So after 

[18:24] out of the form and you can say what's factually 

[18:30] can gain from that sketch. keeping in mind your 

[18:36] you've learned from the form, you're trying to 

[18:40] you're going to arrive um at your final uh at your 

[18:52] And here you can see one behavior that 

[18:56] it to very carefully examine the form. It's 

[19:01] it's telling me each individual box. Is the box 

[19:06] thing you'll notice as you do prompt engineering. 

[19:11] claw decide how much it wanted to tell us about 

[19:16] it carefully examine each and every box, it's very 

[19:21] might not be what we want in the end. So, that's 

[19:26] to give me these other things that I asked for 

[19:32] accident summary so far. It's going to give me a 

[19:37] that vehicle B appears to be clearly at fault. In 

[19:42] with more complicated drawings, more uh less 

[19:48] thinking for Claude is really impactful in 

[19:54] Uh, so I think we'll go back to the slides and 

[19:59] piece that we might add to this um to really make 

[20:05] you so much. So, as Hannah mentioned, uh, we sort 

[20:11] really acting on our behalf in a right manner. 

[20:16] end of this prompt that I'm going to show you in a 

[20:21] part as well. just strengthening and reinforcing 

[20:26] important piece is actually output formatting. 

[20:30] on this LM application, all the sort of fancy 

[20:35] you want your piece of information to to 

[20:39] wherever you want to store that data. And the rest 

[20:44] its verdict isn't really that necessary for your 

[20:49] for your application. So if we transition back to 

[20:54] a simple importance guidelines part. And again, 

[21:01] behavior that we want out of cloud here. Want 

[21:06] and accurate. Want to make sure that nothing 

[21:11] apart from the data it's analyzing. And then 

[21:15] in my case here, I'm just going to ask Claude 

[21:20] actually going to ignore for my application and 

[21:24] that is I can I can use this if I want to build 

[21:29] Or if I just want a clearcut um uh determination, 

[21:35] here, you'll see it's going through the same sort 

[21:39] it's much more succinct because we've asked 

[21:43] more straightforward manner. And then finally 

[21:48] output in these final verdict XML tags. So you 

[21:54] a skiing accident to sort of unconfident insecure 

[22:01] version to now a much more strictly formatted 

[22:08] application around and actually help you know a 

[22:15] U finally if we transition back to the um slides 

[22:24] putting words in CL's mouth or as we call it 

[22:30] parsing XML tags is nice and all but maybe you 

[22:36] uh it's JSON serializable and you can use this 

[22:42] is quite simple to do. You could just add that um 

[22:48] format. This could be for example a uh open 

[22:53] or even in this case that we see in front of us 

[22:58] case it could also be that final verdict XML tag. 

[23:04] how Claude is supposed to respond. Um, without all 

[23:09] that is also key in shaping his output to make 

[23:14] that we wanted. So in our case here, we would just 

[23:18] afterwards. But you can use prefill as well. Now 

[23:24] here as well is that both cloud 3.7 and especially 

[23:31] model meaning that there's extended thinking at 

[23:36] to highlight because you can use extended thinking 

[23:41] you can enable this to make sure that Claude 

[23:45] tags and the scratch pad. Um and the beauty of 

[23:50] to understand how claude is going about that data. 

[23:55] it goes through step by step of the scenario 

[24:00] ways there you can actually try to help claude in 

[24:04] not only more token efficient but it's a good way 

[24:10] don't have our intuition actually go about the 

[24:14] it's quite key in actually trying to break down 

[24:19] and with that said, I think uh I'd like to thank 

[24:24] So if you have any questions on prompting, please 

[24:29] You want to learn more about prompting in an hour. 

[24:34] have an amazing demo of Claude plays Pokemon. So 

[24:39] we'll be around all day. So, I know we 

[24:42] but uh please come find us if you want to chat.