Okay, this looks good. Just going to test it. That works. Okay. We have codeax cloth code. Okay great. The sound. Perfect. So, just going to check this. Totally could check this video out. Okay. Yeah. So today uh I wanted to do like a comparison between uh running GPT5 in codeex. So I kind of hooked up my subscription status. So I'm kind of signed in with chat GPT here in Codex now. GPT5 effort So, I wanted to try that out, but I haven't really tried it too much. Hello, Moaba. How are you? If anyone has tried codeex with um GPD5, let me know. So, I thought I could do like a I want to do like a fair test. So, I'm going to I think this is my last day on the Ultra subscription from Google One. So, I'm going to ask 2.5 Pro deep thinking to create a PRD. We can test cloud code versus codeex with GPT5 uh ID API from replicate. So I need to upload this. That is my plan. We're doing a comparison between codeex with GPD5 claw code with sonet and opus. We need a fair prd to test them versus each other. Not too complicated, but an interesting challenge that includes backend front end using the ideoggram API from replicate. I think I need to come up with something else too. So that is kind of the main thing I want to try out today comparison between um Gypty with codeex and cloud code with sonet opus. Hello Fabian. Did you get the chance to test out codeex? Didn't you talk about that or what did you want to test out? There was something you were thinking about testing, wasn't it? So, it was really easy to Did you? I'm going to try it now. What did you think? So um it was really easy to sign in with chat GPT. You just had to upgrade Codeex and run codeex login. So it's pretty easy. Pretty good with medium thinking. Okay. Yeah, you can see I'm on medium now. Reasoning effort medium. So I'm just going to keep it there. GPT5 medium. Right. Uh I'm on the plus plan. I didn't even know uh high was also good but with complex stuff stuff a little over top. Okay. So I'm just going to keep it on medium then. So what I want to so what I've done now is I give I have given uh cloud code and codeex the same uh environment or directories. They have an API key and they have the ideoggram documentation. So Claude code has this and uh codeex. So I'm going to ask Gemini 2.5 Pro deep think to come up with a PRD so we can test them versus each other. But they're going to we're going to do a challenge that includes both the back end and a front end using the ideagram API from replicate. The API key is in the ENV in root. Your task is now to create a PRD that challenges both models on a few different uh what should I say on a few different uh tasks. I think I was an opus level. It was an opus level for your test. Bit better. You will see. Yeah, I'm going to try it. I don't know how it's going to go though. So, Gemini 2.5 Pro is pretty slow. So, I'm just going to start this at least deep think. So, hopefully we get like a nice PR and um Hi Stor. Hello, Edgar. Nice to see you again, Stor. Edgar, did you have the chance to try out the codeex with GPD5? I'm gonna try it out now and compare it with cloud code. Feels good. Cool. So I signed in uh with GPT plus plan GT5 effort is medium. I'm going to start on that. I think I know you probably talked about high. Okay. I'm just going to keep it on medium. Just when it gets very complex, it gets slow and make some mistakes. Okay, just use it as it is. Yeah, I'm going to just keep it at like this. But what we're going to do now is we're going to get Gemini 2.5 to create a PRD that's going to use the replicate API to create a PRD that is going to be a challenge for both models. I'm just going to take the PRD, send them to both and see what I like best. So the ideagram is this ideoggram character API. So you can do inputs and you can get different outputs of the same character. Hello JS, nice to see you. Welcome. So this is the project they're going to work on. They can take an image in and they can use it on something else. So we're going to see how that turns out. 2.5 Pro deep think is very slow though. So, it's going to take some time, but um had a lot of fun this morning. I played around with um I made a video on it. It's really fun. Uh output styles. This was really cool. for open source. I also dropped uh a video uh prompt engineering by dropped a video by prompt engineering some interesting os results and how they differ in the link app. Did I call it? Okay, good. I also had this video. David dias, nice to see you back. Welcome. Uh, I I kind of like this guy Matt here. He's pretty good. He says, "Can Claude GPD5 keep up with cloud code?" I might watch this video. You test the GPD5 on open code and cursor too, but on codeex is a different level. Okay, that sounds pretty interesting. Hello logo cloud code hooks. I have checked it out, but I haven't used it too much. I kind of want to get into it though. Do you have any like good um use cases? Yeah, it seems pretty interesting. Uh I have done a few things with hooks, but I haven't really found something I need every single since I don't work in like a big code base or anything. Yeah, Edgar, I can drop the link. It's uh here I guess hooks are great for linting. Yeah, I heard that. But um okay, so we have the PRD now. It's quite big. Not too bad. So I want to check it out first. Uh the PRD uh for evaluating AI coding assistance codeex GP5 versus cloud code implementing a full stack app that integrates the replicate ideoggram API front end file handling backend security. Okay. user experience. Okay, it's going to use Express, React, Tailwind Node. It's pretty good. The user uploads an image. The front end validates the image, converts it to B 64. user provides three separate text prompts. Okay. The user initiates the generation. The back end receives the URI and prompts uh and concurrently initiates three separate calls to the API. The front end displays a completed three panel comic strip once all images are generated. Okay, that's can be pretty fun. Uh, I'm not don't think I'm going to go through the full PRD here, but uh, I'm just going to copy it. All right. And I'm going to save it as just PRD, right? Uh PRD. Yesterday uh I thought I expand my personal image to create image app to create story boards batch generation and create link in carol says. Okay, that's pretty cool. Yeah, the ideagram character is pretty cool. It's very good. Okay. So now we both have the PRD uh in uh cloud code. We have the PRD uh in codeex. So what I'm going to do now is just head over here and I think we're going to get started to compare these two. Hello Mug. Today we are comparing Claude code with Opus Sonet and Codeex with GPT5. So we just had 2.5 Pro deep thinking create a PRD. Hello Jamie. And we're going to take that PRD and give them to both GP5 and CL code and see which app we like the best. I'll check. The price is 15 cents per output image I think or something. It's really good though. So, this is the input image and you can get these images, right? This is pretty good. This is very good. So you input this, you get this out. That is pretty similar. So we're going to see how um this turns out. So I think we're just going to get started. Hello calling Larry. What's up Jamie? So, uh, I might record this into a video. So, I'm just going to do it like, um, at least coherent generated said carousel. Just threw five uh of the images into manus and try out vio anime of your carousel. Okay. Verick, nice to see you. Uh we are just comparing open now codeex with GPT5 versus claude with GP with opus set. Is there a plan mode in here? I guess we don't need a plan. I guess we already have that. Uh but I think I'm going to use claw code the way I usually use it. So let's do this now. So I'm just going to start set it going on codeex first. Can it be tag? There we can. Okay. So let's do uh we are um we have a task. Um it's in the PRD at PRD. Um start with a good plan to solve this then execute the task. I think that should be enough because we kind of already have the PRD. So, I'm going to start that on codeang GP5 and let's go to claude. I'm going to do pretty much the same. So, I guess I guess we could just grab this same input prompt, but I'm going to use plan mode here because I'm used to that. Okay. So, I want to check out here. So, preparing to access the PRD file, locating it. We are doing some uh tool calling here right the speed looks pretty good I would say um planning the yeah the speed looks pretty good hello uh we are comparing GP5 on codeex uh versus claw code on uh opus and sonet and we're doing the same project. So we're going to try to compare it. Okay. So Claude code has started uh working on the plan and since yeah I guess codeex with GT5 is already started and so far the speed seems pretty good. Generate the base image. make the story board. Uh yeah, you should try out um the ideagram character. I think it's pretty cool. Okay, so Codex is um already starting to adding some code here. We are still working on the plan here on uh cloud code. We are in plan mode still. Uh, so we're going to switch from Opus 4.1 to Sonnet when we're going to execute it. So the app we're going to do is a character comic strip uh implementation. A character comic strip generator. Okay. So here's the plan from Claude. I'm just going to accept that. I'm not going to interfere. Okay. So both are running pretty smooth. So I I kind of like that we can um we can link our chatbt subscription to codeex. Uh I think that's just better than having the API because uh then you don't have to worry so much about so much about the cost. So we have the to-do list here in codeex 2. So we have done one out of four tasks. The back end we're going to do the front end. We're going to do some proxy and course and validation documentation and run. Okay. Uh hello Sid. Yeah Jam I know. Uh but I wanted to try out the codeex uh because I've used open code. So today uh it's asking me a question here. Would you like me to request permission to install dependencies? Uh okay. Yes. I'm not going to put too much effort on speed today because uh I need to kind of go back and forward a bit. So, it's going to be result oriented, not speed. But, uh so far GP5 seems pretty fast here, I will say. So, Sid, we are just doing a comparison now with GPT5 and um Codex versus Claude and Opus Sonnet. So there is uh we have a PRD from Gemini 2.5 Pro Deep Think. It's going to generate a character comic strip generator where people can upload images. So we're going to compare them. My boss lost his uncommitted git changes codeex. Ouch. That has to hurt, right? Did it have much? Like was it a lot? Okay. So, we did install um npm install. We run a build. Okay. What did it It just keeps asking me questions. Uh but it's fine. I'm just going to say yes. Lots of files but nothing the AI can read in a short time. Codeex lost our trust though. Yeah, I can see that to be honest. Uh it would be a bad comparison as cloud code uh product wise pretty nice implemented. So, in the end, it's rather open AI versus Claude, who knows their model better. Yeah, we're just going to have to see. Uh, I just have to try it out anyway. So, I guess this is just my attempt of comparing it. So, how would you compare it then, Sid? Codeex wants to run. Is there something I should do here or I don't know. I'll give it some time. Maybe I have to stop it here. Maybe it's trying to Okay, so it has a sleep here after I'll give it some time. Just model specific would be use the API on open code. Uh it will be unified products and a bias of different products affecting our results. Uh yeah but uh let's say I'm testing out click enter maybe. Okay. Yeah. Thanks Fabian. Um yeah that's true. But you can say I'm testing out both uh codeex and gyptus. But uh I don't I don't disagree with you. So this failed it run them. If you want, I can proceed starting them on your machine. Um, okay. uh the is already innoving me a lot of questions but uh it's fine. I might spill my API key here but uh I'm going to revoke it. Doesn't matter too much. Um, you can also interact with escal code. Okay. But the most open thing codeex is open source. So it's a huge win on that part. Yeah, that's pretty cool. But so is Gemini CLI and open code. But for me personally, if claw code is much better or better uh but codeex is open source, uh I'm probably just going to pick the best. I don't really care too much. Price could change that. Of course. Is there a way to use different models from different providers via cloud code? I think you can set up open router to be honest. Haven't done it but I think there is some kind of router you can set up. Yes. From just a users perspective, I really like uh I got to use the open code. It's MIT Max Plan from Antropic. Yeah, it's pretty cool. Hello. Push the push the model. How's codeex going so far? You only used it one. I was terrible so far. Uh, seems fine. Keeps asking me a lot of questions though. Do you want me to understand what it's asking here? Pick what you think is best. I don't want my um I don't want to interfere here. That was the whole point. Clothe code router is also a project that makes this possible. How do you all feel about clin? I think a lot of people has used clinally haven't used it too much. Remember I said I need to do cursor for my professional work. Not anymore. You added an LSP to open code. I put the functions I need to NCPS. Now I can use open code for that. Well, that's pretty cool. So JS is reaching his limit on 100 pro. So you spent 5450 this month. I spent a lot too. I'll check it out after so far. Um, cloud code has been much more agentic. Uh, codeex keeps asking me questions almost every single thing here. I don't mind questions, but uh it is supposed to be like an agentic tool. Um it's really hard to but there's probably a way to avoid this. So it says something like if you like uh I can share a version and pick what you think is best. Okay. So we have an okay on local host 31. The back end is running. Start the front end. So, we're going to do a test on the API. Failed. Okay. You didn't pass an authorization token. Check out the prompts from the documentation. we had yesterday. Yeah, maybe. Uh how much trade or difference do you feel uh trade? How much trade or diff difference do you feel about open code with cloud models compared to cloud code with cloud models? Maybe Edgar can answer that. Open code with cloud models versus cloud code with cloud models. Hello AR. Yeah, JS, the new rate limits are just a couple of weeks away. That's going to be interesting. GPT5 does very much what you tell it to do, but you can tell it to make its own decisions more. Yeah, that was the idea behind the test here. Um, I don't really want to interfere too much. No difference uh from open code to closed code. Hey at F. Uh we can check. Uh, so lately it's been pretty much uh NPX CC usage monthly. Okay, so Claude Code finished first after starting last. So that's a plus, I guess, for Claude Code. So, okay, I spent a lot this month. So, in August so far, I spent $700 on code. So yeah, this week this month it's been a lot, this weekend I was working on a big kind of big code base this weekend and the tokens kept flying. But this is nothing compared to other people. JS I'm on the 100. Does Open Code allow uh custom flash commands? I know they have sub agents. I don't know about hooks though. Edgar, go to the daily view. Okay. So, it was this weekend. I spent a lot. Today, $20, yesterday 30. Uh on Saturday or Friday, 126. Yeah, that was the day I did that uh tool I was creating on the fork of open code. I I I need to help Codex here. So, uh what is the issue here? Uh the replicate what is it asking about versus look I don't understand this. I'm just going to say okay. I don't understand what this means. No, this is not what I've been charged. Of course, this is just uh in case you didn't know. This is just like the theoretical cost if I was do using the API. So, uh I have saved in August. You can say I've saved uh $500 in August. Okay. So, now I see what it means. I didn't think of that. We're not going to deploy it. We're just going to run it locally. Okay. So, maybe that was my bad. So, let's see now. Um, so we need to we have our back end and front end. How did we set this up? So, I guess we can create um create a simple how to use guide. Okay, so we got that here. That's pretty good. So we can do see the server mpm install. Okay. Uh see the see the client mpm install. Okay. So, let's do a new terminal. Let's do CD client npn rundev. Let's do the back end cd server npm rundev. Okay, maybe there's an issue with the maybe um like um is she Let's see for our other version. We're going to do okay. So, let's see. We can do see the back end mpm install, I guess. Did that work? Let's do see the front end and and it's set. So see uh port in use. Um okay. Kill port. So let's start with uh cloud code here. Okay. So this is from this was very bright. So this is the result from cloud code. So this is the character comic strip generator. So we're going to upload a reference image. We're going to add our prompts and we're going to click on generate strip and it's going to open up here. So who thinks this is going to work zero shot? I don't. So let's do Gemini. Generate a image of generate a portrait image of a female in her 30s. Black background smiling. I don't know. Hello, Maple Dev. What a wonderful time to be coding. Yeah, I'm just going to work out the things uh that about the app and then I'm going to try to compare them. So, let's just grab her. So, this is going to be a reference image. So, we're going to upload that whenever it gets downloaded. Okay. So, it needs to be big, less than one megabyte. Okay. So, I'm going to alter that a bit. When I run into issues with codecs for debugging, I just tell it to run all the necessary commands to fix the problem, including installing requirements and running test. Sunni, have you been um using codeex a lot? Do you like it? So, I'm going to pick the same reference image for both um for both um both apps. We might too smaller. Let's just Let's try that. Perfect. Okay. So, what's the what's the first panel's going to be? Uh, the character walking in New York. The character gets approached by a eager squirrel almost attacked. The final strip is going to be The squirrel chasing the character down in Central Park. I have no idea. So, the squirrel. So, I'm just going to see if this is going to work now. So, I'm going to watch the back end. Okay. So, we failed here. I'm going to see what the issue is. So, we failed on the back end. We didn't pass uh token here, the API, I guess. Okay. So let's give this error message to cloud code and to uh resolve this. Um the API key is in our env. While we wait for that, let's check out codeex on codeex. I'm going to do clear. I'm going to stop this. So, let's do kill port 3001. Okay, let's run it again. Yeah, I'm not quite sure about this. So this is the codeex version. At least um UI is working. So I can upload our image. And let's do female walking down New York street. Female gets attacked uh by a squirrel. Female gets chased in Central Park by a I don't even know how you write that squirrel. So let's see now. Okay, so we didn't get our error message. Let's check our back end there. I don't see anything. Our console log. Um, okay, that wasn't too bad. That was pretty quick. So, so far I would say Opus Codex did pretty well. Hello JS. So, this was our first panel. Female walking down the New York street. That was pretty good. Uh, female gets attacked by a squirrel. Maybe the prompt wasn't maybe that precise, but she looks pretty good. And the final is female gets chased in Central Park by a squirrel. Uh okay. Not exactly, but pretty fun. So, I guess the app worked first go. So, that was pretty good by Gypt 5 and Codeex. I was happy with that. Um, I'm just going to Let's try a different input image. Let's do who is chasing who. Yeah, that's the question. So, I'm going to do a different uh portrait image. Let's do a male in his 70s. So, that was pretty good by Codeex. We're going to compare them side by side. So, let's grab this. Uh, I need to make it a bit smaller here. Let me see how. Okay. So, I need to resize this. Copy. Thanks for becoming a member, JS. I appreciate it and welcome. So, we are doing um members giveaway each week now. So, we're going to give away a cloud code pro chat GPT plus subscription every week. I think it's going to be every Friday. So, at the end before the weekend, so people can try out the things on the weekend. So, that's going to be every Friday. That's the plan. So, check look out for that. Yeah, you can I'm going to post a video on YouTube. It's like a members video. You can find an instruction there. So, let's keep the let's keep the same prompt, but let's just change it to male. And let's try a different input image. So, let's try the guy. and generate strip. While this is generating, let's check back here. Uh, I'm not going to kill it now. Uh, switch to 3000. We don't need the same port. So, let's check out the strip here now. So, now we're going to take this input. Uh, the model is really good, though. So, we have him walking down the street. He gets attacked by a squirrel and he gets chased by a squirrel. Uh, the character consistency is good, but the prompt maybe isn't the best. Let's try a different prompt. Older male is hacking FBI gets into a secure database. FBI raids his uh apartment. black hoodie. See what happens. Okay, so we switched the port to 3000. That's good. So, the character consistency is pretty impressive with this model, I got to say. So, here is hacking. Okay, it gets into a secure database and he gets raided. Okay, so I didn't keep the black hoodie. Make a number of con comic strip configurable. Yeah, we can iterate, but I'm just going to have to wait till claw code is done. We can iterate but uh I'm just going to wait a bit. So let's do this more text. Yeah, we can try that. I would just want to get clawed code up and running before so I can do a comparison. Then I can do the same. No, the prompt doesn't have the context of the previous image. I don't think so. Uh, but the character consistency is great right? This was pretty bad though. You could you could you could set it up so we can keep the prompt from the previous input, right? Hello Ibraim. Uh, what model is this? This is the ideoggram AI ideoggram character. I'm going to do an MCP um video AI video. I'm going to do an MCP AI video with this video with this model uh someday this week because you can do crazy stuff with AI video using this model. It's pretty fun. So definitely check it out. Ideog character. Yeah, it's pretty good. So let's do like a consistent scene. So is eating breakfast, wearing a Nice suit. Uh, put on um, winter jacket, dark brown. hoodie and beanie and then you can do older male. We don't really need to say that in a winter jacket walking in five feet of snow something like this. Okay. So let's see if we can run cloud code now. So, this is Codeex. Uh, I'm going to stop the server here really. So, let's run a back end run. Okay. Um, kill port 3000 mpm run. So I want to try the same input on clothe code here. So let's grab the old man. Let's do the same inputs. It should be pretty similar though. Shouldn't be any big differences. I just want to see if it works. Okay, it looks better now. So, let's check out this older man is eating breakfast, wearing a nice suit. Older male puts on a winter jacket. He forgot the beanie here, but it is working in snow. Okay. Did you try animating those pictures using V3? That is what I'm going to do in the upcoming video. I'm going to use this model. I'm going to generate setup like scenes using this with the same character. I'm going to animate them. I'm going to build it into a MCP pipeline. So everything is automated. Uh I'm going to do that soon. So now we have Cloud Code working. It's pretty much the same app. So we don't see like any big differences. We got uh this and yeah, that was pretty much the same. So I would say both models did a pretty good job. They follow the PRD pretty much exactly how we wanted it to be. We got the three panel comic strip on both. Right, the input image gets nicely transferred from the input over to the we turn it into B 64 and a data URI and passed it onto the API with the three prompts at the same time. Uh, it's probably not async. It's probably sequential. I'm not quite sure though. It might be async because it was pretty quick. Tips for better prompts. The UI side I would give to cloud code. This is a bit boring but it works. UI is easier to use with codeex. I would say more compact. Yeah, I guess it's just uh it's just um what you are used to. For me, I like cloud code, but I'm biased right? So, does anyone have any iterations on the app we can try? It's just some protein stuff. it. I just came from the gym. So, does anyone have any iterations we can send to the models suggestions decorate? Uh, this is not uh this is the ideagram AI. It's a really cool image model. Just can't imagine giving up clothes code as the main driver. Yeah, I have a real time giving it up. So, I'm going to do decorate a video on this probably later this week, probably Friday. Uh, no, I meant this result. Cloud code is of course way better uh than codeex to use. Okay, I see what you mean. Uh yeah. Uh I guess I prefer that. Does replicate still give a $10 credit? If so, drop your link. I'm not quite sure. Clothe code is a great great at app architecture but weak in domain expertise. Codeex is the opposite. Strong in domain but weaker in architecture. The best approach is to combine both. Yeah, I agree. You shouldn't have any moat in uh AI at the moment. You shouldn't you should avoid being locked into something because I would say that's pretty stupid because everything can change, right? So, if you lock yourself into something, I wouldn't even buy like a year subscription. I'm not going to buy a yearly subscription on Cloud Code because things can change, right? Added a Google Drive link with the images. Okay, I'll check it out. I've been using Cloud Code to execute and Chachibd Pro account to serve uh as the research and development supervisor agent for cloud code. Yeah, someone else yesterday had the same approach. They use GP5 Pro to do a plan and stuff and um clothe code to execute likely switching to Gemini when three drops. Yeah, that might be it. Maybe everyone is going over to Gemini CLI or they are using uh open code with Gemini 3. Who knows? In like a month maybe everyone is over there. It's been a while since we checked out Poly Market now. So if you go to tech AI best model in August. So now it's 93% chance that Google is going to have the best model by the end of the month. So something happened here. I'm not quite sure. I think they tease Gemini 3 or something. Mahid bought a year sub back when they had a deal. Okay. So, if you had it for a while, you uh you probably have gotten some value out of it, but I wouldn't do it now. Who gets to decide which model is best? Yeah, this is just a prediction market, but they have some rules. So, it's based off on the chatbot arena leaderboard. So, is it best for coding? Maybe not. I don't think so. It's just overall. They have to pick something. Uh, yearly subscription is dangerous. I would avoid that at least. Yeah, I agree. That's my opinion. I guess Gemini is half off for ultra. Uh so I figured good time to get it before um Gemini 3. Yeah, I had ultra for 3 months basically. Basically um because um V3 No, I agree, Edgar. There's no best model. Uh you can base it in that Ella Maria I flaw. Yeah, I agree. I don't really but they have to pick something right better than GPD5 will be very huge. Looking forward to something great from Google. Yeah. So, I've been kind of paying a bit of attention on uh Reddit and um I was hoping there would be more pressure on Antropic to lower their prices, but I haven't seen like a big mass exodus from Claude and Claude Code and Opus Sonnet to to um GPD5. So, as long as there's not going to be like a mass exodus from uh Claude from Claude models to GPD5, uh they're probably not going to lower their price. So, let's hope for Gemini. Uh Gemini is going to push Antropic a bit more so we can get some cheaper prices. If Gemini can create some mass exodus from cloud code, uh they probably have to lower their prices. OpenAI introduced some new paradigms. Hope uh Google does the same. Yeah. Is Gemini 3 Deep Think is it really Gemini 3 Deep Think versus Gift 5 Pro? Yeah, but you can't really you can't use GD5 Pro for coding, right? That's just way too slow, isn't it? CL code with a PS5 controller. That sounds awesome. Sitting my sofa and uh using claw code with a controller. That's a cool project. Okay, so we need to iterate on the app. Hello, Pavle. Nice to see you. What have you been up to lately, Pavl? So, I'm going we're going to do some iterations here on the app. Um, if anyone had any suggestions for iterations. Um, okay. Going to have to think it. It's too fresh with codeex and stuff. I don't think Antropic will lower prices, but maybe new model soon. Yeah, we'll see. Way too slow to code with, but uh to make markdown files is key. Yeah, I think doing like a great plan with let's say GIT 5 Pro or Gemini Deep Think is uh superb because like I use Opus to do the plan and Sonet to execute. It's much easier to execute instructions than coming up with a good plan. Powell has just finished his series uh series of calls from your project privately adding some custom slash commands to codeex and open code. Cool. But the ladders uh team is working on their solution and should land this week. Nice. Yeah, we've been trying out codecs today with GT5. We made the same PRD from um so we had Gemini create a PRD for us. We had the same project. So this was the result from GIPD5 and Codex. Worked out of the box. So that was a oneshot. That was pretty good. Impressive. Cloud code had a bit better UI. Worked pretty much the same. uh had some small issues with the authentications or the API, but I basically think it maybe it was my fault. So, I would say they were pretty neck and neck here in my small test. Uh but I'm going to try an iteration. I just have to think of something. Uh yeah, I don't think I want to do video now. We could do we could chain this. Okay. So, I have a I have a suggestion. So, uh I'm going to do a compact here. Uh keep what we need for a iteration. Can we do compact on uh yeah codeex? So I think the iteration is going to be the user can pick one image and this is the new reference image. So when the user picks an image, we can continue on this reference image that is going to be the iteration. You can tell them to make a number of images configurable. Yeah, almost something like that. But uh that is pretty easy. That is just that's very easy. But I think I'm going to do what I said. So the user can pick uh let's say panel two. This is going to be the new reference image. And when the user picks this um this the prompts are kind of reset and it's moved up here and we can continue. Do I have whisper? Okay. So, uh we need to do some iterations. So what we're going to do is that um the user now can pick one image from the three panels. So you need to add a button under each panel. If the user selects that image, this is going to work as the new reference image. So when a user clicks uh select this image as new reference uh we're going to move that to reference and everything else should kind of reset so we can continue using that image as a new reference image. So I'm going to send that to both Uh working on automating your coding workflow using cloud code. Cool. Uh main driver plus hooks plus subs and local LLMs uh to offset cost prone code in a loop and run quality checks and code review. Cool. Sounds like a good plan. Decor GPT5 uh is the first model that feels like I'm talking to a legit genius. It's been working on your very large project and never really has an issues. That's cool, Aaron. Happy to hear that. Andropic offers claw via Amazon Bedrock and Google Vertex AI without its own infrastructure. Prices remain tied to these cloud providers. Yeah, but I guess Yeah, I see what you mean. But they must have some margin to go on, right? I should have used maybe plan mode here, but uh Okay, so it's going to be Sonet versus uh GT5 here. Let's see what happens. Yeah, I hope so. So, OpenAI is of course running on um the Microsoft data centers I guess and Google has their own TPUs. Meta probably I I never use meta but um okay so cold says This has made uh changes. So, let's uh head over to Codeex. We can restart uh at least the back end, I guess, and we might as well restart the front end. So, let's check out code access iteration here. So, of course, we need to create this. I'm just going to wait for uh OpenAI now cloud code. Okay, so it should be good to go. So let's restart code. So let's stop the front and let's stop the I don't know why we can't force this very uh let's do see the back end and run again kill And let's do mpm run down. So let's try this now. So we have from codeex we have close code. So we're going to upload the same image. Let's use the girl again. So I'm just going to come up with some prompts here. What is the verdict? Says David does. Uh so far I think it's pretty even. Maybe the challenge wasn't hard enough. Uh I would give the UI to cl code but that's just like by a margin. Uh the speed I think codeex was a bit faster and claude code. uh had some small issues like uh API issues, but that was basically pretty much my fault. Why would they lower their price? So, I was talking about like if Gemini can put some pressure and you see like a mass exodus from Claude, then they probably have to do something. JK, hello. Codex is nice. I've been using him with your plus subscription. Yeah, I'm trying it out now. Seems pretty good. Old backend was still running uh in the cloud code version. Didn't I restart it? I think I did. Uh David, uh you went to your API check limits and I discovered that they have reset your credit. Okay. You have donated them 250. That's annoying. I think I I know they do credit resets. That shouldn't be allowed, right? Okay, I'll check. Fabian, do you mean the front end or the back end? So, we are running the back end now on port 3000. Let's check. Codeex is running on 3001. Is that what you mean? Front end. Yeah, I don't think that matters, does it? Because this was busy. 5174. This is now switched to 75. I think it's fine. Yeah, they expire after a year. That's pretty I think that's a bad business pract practice, right? That's annoying. I'm going to ask uh Google 2.5 Pro deep thinking to come up with uh uh generate three prompts aka one for each panel. that challenge us the app to keep the consistency. Uh but still uh makes sense. By the end of the year, if we can make uh a detailed markdown file and pass it off to fully agentic coding system knowing it will be executed correctly, everything goes vertical from devs. Maybe interesting. Yeah, they shouldn't expire. I agree. Uh, I wanted to try animated We will get this uh community there sooner. Yeah, I hope so. There's a lot of people here that has a lot of good knowledge. So, that's pretty cool. Just got to find the best way to share it. It's not It's not that easy. Isn't that exactly Transformers? For me, each markdown file is about four hours of work. Wow. So, getting clo up with the LLM to create the agentic system. Uh, but you need the right prompts and scaffolding to make it reliable. Yeah. Since I don't work I don't work in like big serious projects. I don't have any like big serious projects. So, it's a bit a bit out of my league. So, I can't really comment a lot on that, but I believe I believe you're going if if that's the case. Okay. So, let's do this. So, we're going to follow these instructions. Uh, I'm going to change up the input image here. I'm going to do the I'm going to try both actually. This was a bit strange. We can try it out. Let's try it out with the woman first. So the idea now, the iteration was that we first going to generate a panel of three images. So let's run that. Then we was able to pick one of the panels and set them to the new next reference image. That was what we're trying to do. So let's see if we can succeed with that. So the first input image was the character looking worried, biting her lip. That was pretty good. Second one was Is it a comic style in a a comic rendered in the style of a Jack Kirby comic book book? Uh, characters yelling and leaping over a fireball in a destroyed street. Okay, pretty good. The third one was Studio Gibli style watercolor painting. I'm super impressed by this image model. And we have the button select this image as the new reference. Pretty good. Let's check out Claude Code. So, she's biting her lip. We have this Jack Kirby style and the same uh Studio Gibli. So, I'm going to select this as new reference. Did it work? Need to check the back end. Doesn't look like anything happened. Okay. Uh don't think that worked. Let's try this one. Okay, so that did work. So here I would say codeex did the better job. Let's try and see if it works. So that's pretty interesting. So definitely I will give the iteration to codeex here if this works now. if we can use this now as the new reference image. Uh I guess we did. Yeah. So if we pick uh this one now we run the same. So I would say that if that works, the iteration should be better here on the codeex GP5 version of the app. So I'm not going to even bother doing debugging here. Yeah, it's definitely working. We have this as a reference. And yeah, that's pretty cool. Pretty impressive. That's a good app. Uh, let's see if I find an image of me here. just going to wait for us. as expected. Yeah. Um, yeah, I got to say I'm pretty I have enjoyed Codeex so far. I'm going to try it out more since I already have the the subscription. I might as well, right? So, this is me. Look at this. So, this was my input image. This is me uh as a professional Pokemon catcher. I'm out here. I don't know what I'm doing. Holding some kind of camera. And maybe this wasn't the best. It doesn't really have my hair color. And this was a bit strange maybe. But this was pretty good. This kind of looks like me as a professional Pokemon player. Catcher. Hello Jacko. How are you? Yeah, Hunter, that's no problem. Uh, you got it right. Game controller. Yeah, that's great. Yeah. Uh I've been using PayPal for the giveaways and a lot of people want stable coins, too. So, that's super easy to just use um uh cryptocurrency. I kind of prefer PayPal or cryptocurrency. Cryptocurrency is cheaper because uh um yeah, I don't have to pay a fee here. So, this is me as a Pokemon catcher. Here I am as a UFC fighter. And here I am as a mountaineer. This is a cool model. I'm definitely going to try it out. So, what if I do an app that has a webcam that can screenshot the user, pick that as the reference image, and you can generate like in different situations. That could be pretty cool. Bus cut. Yeah, I can try that with codeex, but I kind of want to use MCPS for that. Does anyone have any like good experience with MCPS in here? Does it even have support for that? Uh, we need MCPS though. Does it have a help? Maybe not. It does. Yeah, I will need MCPS for that. This model is so good. This looks exactly like me. Maybe not that, but this does. Okay, that's the config. Okay. So Edgar said he has some uh files here. I'm I'm a bit worried about Okay, so it's some animation files. Um I'm going to check it on the other desktop just to be sure. Okay. So, it's like animated. I see what you mean. Pretty good. I can play that. System online. How can I assist you today? Processing your request. Information found. Task completed successfully. System online. How can I assist you today? Processing your request. Information found. Task completed successfully. Here's how I can help within boundaries. Yeah, that's pretty good. Did you use the Did you use uh I totally forgot the ideagram? Did you use this audiogram or is is this another model ide? Oh, so GP1. Okay. Yeah, I've used GP1, but I got to say I think idiogram seems better to be honest. Seems a bit better. The one thing about GPD1, it's so slow. Takes forever. It's so slow. So, I don't really like using it too much. It's good, but it's so slow. Haven't tried it too much with text, but uh the idoggram is like a character model, more like character consistency model. It doesn't really have so much to do with text, I think. But we can try. Yeah, I don't even know how to try it. Playground. Um, add the text uh model behind her. Okay. So, let's see. It does text, but uh let's see. Haven't tried it with text before. Yeah, this looks pretty good with text. I like that it's kind of it's like this blurry effect, right? So, we kind of get this uh it's kind of blurry in the background. That's pretty cool. It's really impressive. Let's see if it changes to 1990s what the difference is going to be. Yeah, I can do that. Hello Dab. What's up? What is up? So now it changed into 1990s. Still pretty clear though. Uh what did the hold a sign? Holding a sign. Ai I need a job. Ai took mine. Okay, let's try that. What the hell happened here? Oh, I can't send money. What's up, D Web? That's pretty good. All right. She She looks sad, too. I don't think you can do it better than this. Yeah, I would say I to be honest, I think this model is better than Gypy one. Uh, it's much faster and it's just going to be better. And look at her finger over the Look how good it is. It kind of cover the letters here. That's very impressive to get this detail right. Yeah, this is a great model. It's probably probably my favorite uh model at the moment for images. Yeah, Sula is We can do 1970s, right? Um film grain. It's really good at instruction following. Yeah, you should be sold. It's really good. I would recommend it. The instruction following is pro crazy. It's really good. Well, let's do it. It is very adherent. So, we can probably turn turn it down a bit style type. Uh, I guess we can't do that. Well, it's pretty good though. Uh David does this is the ideoggram character. Uh I run it on replicate for the API. Yeah, there's probably a lot of stuff you can do here. But yeah, um got to say I enjoyed um Codeex. Pretty good. Definitely going to use it more, try it out, see what I can do with it. Since I kind of already have um I have the plus subscription, I might as well use it on codeex for some things alongside with cl code. Could try to make it work in the same directory. It's going to be messy though. Yeah, this is pretty good. This failed very hard. What the hell happened here? Vex. So, I think the final thing I want to do today is uh watch this video from Matt. I think he got some good takes. I did watch the Indie Devdan video this morning and it was really cool. I recommend it. Okay. Yeah, this is really good. very impressed. So, I'm going to watch that video from Matt. So, I just need to So, he's going to see uh can keep up with Claude Code his full test. So, uh I just need to I want to grab my headphones wherever they are. more question. So, I'm going to grab my headphones and I thought I can watch this. Um but um yeah, before I watch this video, uh I'll be right back. Just going to grab something. Okay. So, um I'm going to test out the sound. So, please let me know if it's too loud. uh need to link codeex and clothe code maybe through GitHub and use the markdown files of context made of communication with uh one other. Yeah, this is something I want to explore a bit more. Uh I have done like a parallel uh parallel CLI workflow before and it worked pretty good. I had one working on the front end, one on the back end, the same PRD. Uh it did work though. So, it is possible. Is it efficient? I'm not sure. Uh, but I'm going to play this video now and just let me know if we need to turn the volume up or down. Uh, I think I might promise using files as context. I don't think so. Uh, I might check it out later. So, let me know how the sound is. Is GPT5 better than clogged code? That's today's question, but you'll be surprised that it's not as easy to answer as you might think. This was the hot mess I got yesterday when I used GPT5 with this application in cursor. So, let's see if today goes any better. By the end of this video, you'll know which model's more reliable, which one surprises with creativity, but most importantly, how to get the best results for yourself. But first, let's take a look at what we're building to give you the context. I did this yesterday, previous video where I was just using Sonnet and GPT5 in cursor. And if you don't understand the nuances, that's okay. But that previous video, is GPT5 better than clogged code? That's today's question. Yeah, hopefully it's good now. So, is GPD5 better? Uh, yeah, we'll see what he thinks. I guess it's his. But you'll be surprised that it's not as easy to answer as you might think. This was the hot mess I got yesterday when I used GPT5 with this application in cursor. So, let's see if today goes any better. By the end of this video, you'll know which model's more reliable, which one surprises with creativity, but most importantly, how to get the best results for yourself. But first, let's take a look at what we're building to give you the context. I did this yesterday in a previous video where I was just using Sonnet and GPT5 in cursor. And if you don't understand the nuances, that's okay. But that previous video covers a lot of ground in more depth. I highly advise looking at that one. This time what we're doing is we're using those models kind of native idees. We'll be using claude code for sonnet and opus and we'll be using codec cli for the that is exactly what we did today. To be honest, I didn't know that he has done that. I haven't watched this video. Sounds sketchy, but it's true. But I guess it's a standard way to do this, right? Uh you don't want to test it in cursor, you kind of want to test it in uh since uh codeex is created for gypt. It's uh created for u anthropic or cloth. OpenAI GPT5 model. Is GPT5 better than something like Sonnet or Opus from Anthropic? Today we're going to find out. But first, let me walk you through a few of the documents. The initial build, what we're showing here is a PRD that I shared and I will not go through this again. Thanks for that, Edgar. I'm going to move the camera. Yeah, that's true. I don't want to be exactly over where he is. We can just do this previous video. This is kind of a what the application should be the major elements inside of the application. It does not go into design. It's really the highle use cases that you want from the system. That's what this PRD is doing. I'm also sharing an architecture document really defines some of the naming and theming that I use in a lot of my systems, but this will also help us really clearly identify when the models are diverging from the request. Both of these documents were sent in to all of the models. So, every model got exactly the same prompt to get started and kicked off. And what we're about to see is what they came back with after the first response from these inputs. I say let's dive in and take a look at what they returned after getting these two documents. Okay, first impression. Yeah, I think that's a good way to do it. That's basically what I did. I guess he had some more information around the architecture stuff. Uh that's pretty smart, too. But you don't want to give it like directions how to design stuff. Basically, I think you just want to give it to the model to find the best ways really matter. And I will say Sonnet almost never fails on first impressions. It does a very good job. It's a very creative model making up quite a few things. Maybe that's good or bad in certain circumstances. But again, what I'll show you here is we have a left to right sonnet and then the opus results and then GPT5 is the last one that we have here. Now GPT5 massively outperforms the last time that we built this yesterday in cursor. So this is kind of a fantastic find. That looks good. The GPT5 UI here is superb. Frankly, very interesting that codeci turned this out and cursor turned out what we're about to see. But it's worth saying these are probabilistic. Maybe I could have run it three different times in cursor and gotten this one of those three times. So, I'm not going to put too much weight in this, but this first impression from GPT5 is great. In fact, I'd go so far as to say I don't know if it won this round, but it definitely definitely is in the pack this time. And last time, I was ready to throw it away. So let's take a very brief look at what it was like last time. Okay. So from GPT5, what we're seeing is yesterday's video, video one findings are over here. This was after initial build at exactly the same point as we are today. Probabilistic. I'm still going to call that out here, but definitely dramatically different. I will say that I did move into this a little bit in that video or when I was running those experiments to try to get it to clean up a little bit for some of this, but it really stuck to this database design kind of interface. it was pretty terrible and it stuck with that. So it unless I was very explicitly telling it what to change, it was not going to change it. And what we find this time is it has a good starting point and it kind of sticks to that throughout the system build. That's kind of an interesting finding. If we look down really briefly at Sonnet again, yesterday's video and today's video, what we're seeing from a Sonnet standpoint is Yeah, it looks pretty much the same. It has done a few things different, right? It found a couple of things that it missed yesterday. Like it found that there were actions that needed to be taken on these different tiles. So it's interesting that it found those details and then yesterday it did not find those details. That Yeah, that's a good point. Arson, great app architecture, weak domain expertise. Codex is the opposite. Strong in domain but weaker in architecture. You might be right. I haven't tried it out enough to know. That's kind of interesting. But it's worth also saying if we come back to Sonnet and it did a pretty good job. Again, it created its own little tile to do these things. But if we try to go into Superman as I click, nothing happens. It missed half the application. This was not just uh some kind of linkage that it missed. This does not exist in the project. So half of the information that I described inside of that initial PRD, it just didn't do. It just left it on the floor. And that's kind of starkly telling. That's a very important detail. Opus did not have this problem. Opus made a detail page for us and put a C. But did he use Okay, so I didn't I guess he didn't put in the PRD the architecture and run the plan mode from Opus and then send it to Sonnet. He probably just put in the PRD the architecture plan on set from beginning till end. Did the same with Opus and probably the same with Chip 5. fair number of the features that I was asking for on that detail page. So, it did a good job. And GPT5, if we look at the same also created the detail page, but had some problems. It is worth saying all three of these, none of them stood up from ground zero. The moment they were born and I launched them, all of them had this kind of problem for me to have to address so that I could actually see what they had built. They just had errors. It's not uncommon to have these kinds of copy and paste errors, but it was surprising that all three of the models ended up having very different but problematic errors that wouldn't let me see it at the beginning start. The initial build is only so much. What if I gave it a very explicit PRD to build from just for the detail page? The information that I gave it was information about the show detail page. Just the detail page, nothing else. So, I wanted to keep the kind of blast radius very small but be very explicit about what we wanted. there should be a large banner 16 by9 dark gradient overlay to have text legibility there's a movie title that's a certain size so I started getting rather prescriptive this time saying if I give you this much information will you build this more conformantly than you did last time really important detail what okay so now it's going to give like precise uh instructions right so then you can kind of check if every single single instructions was uh followed in uh from the PRD. We're really saying is can you do it with words only and at the end you'll see that I'm going to use images. We're going to use some of the screenshots that are in here. You'll see one right here which is the detailed screenshot. It's kind of a mockup of what we might want to build. I give this to the model at the end using this. Can you build what we're trying to do? Does that help you? Okay, let's begin again at sonnet and take a look. This I would say as much context as possible is always better or maybe not always but high signal context at least is after giving that PRD in and saying please go build all of this explicitly when we go into the details we now have a detail page which is great. It does have some challenges. It doesn't have a dark overlay. It's using its white page background. It is full bleed as I was asking for which really means edgeto edge I'm using for this hero section but I will say there's a little bit of crunchiness to some of the stuff at the top but you can see that it found the cast circle avatars that was described very clearly the where to watch kind of streaming a yeah I agree Edgar so hello Jeffrey nice to see you so the context uh like I think about context like a much much high signal as possible and leave out all the noise, right? So you don't really want to poison your context with like u So that's why sometimes when you scrape a website into your context, it kind of messes up things because you you you get sometimes the full HTML like the full DOM or something and you basically all you're looking for is like the high signal and that's just going to mess up your context. So, uh, I'm doing a project on some FPL, uh, football league fantasy manager with AI there. I'm really specific about the high signal context buttons that you would be able to go and find a place to stream it. That's called out very explicitly. The order of these items was called out very explicitly. So, it conformed quite well. I did say that I wanted a specific type and it didn't quite conform to that. Okay, coming over to Opus. You can see I think Opus did a very good job. Light years better than yesterday. So this again is claude code with Opus. Claude code with Opus comparatively much much better. Um I also will say that it's very conformant down toward the bottom. Let's look at GPT5 which is really light years away from what happened yesterday which looked like a database application almost the whole way through. Um but again we're giving a PR to these. This is not what we did yesterday. The reason I decided to add it today is it felt like GPT5 was leaning toward being very explicit. When you spoke to it in a very explicit way, it seemed to conform quite well to the instructions. And GPT5 maybe has a pattern of if you tell it explicitly what you want, it does better. And this, I believe, is my first finding in that regard. I gave it Yeah, I think they tried to make it um very good at instruction following. I think that OpenAI's aim was to make it really good at it in instruction following and uh yeah I think so far uh I think they got but that can be like like we talked about yesterday it could be like not a negative thing but it could kind of hurt your old ways of doing things if you if you were a bit sloppy on the instruction you could kind of get away with it. But uh if the model is doing exactly what you tell it to do, uh it could be some a bit difficult sometimes. It kind of doesn't read between the lines right? Uh big on context uh shared philosophy and logic uh of the approach uh and reasoning from your human perspective. Share your wisdom and enjoy better results. Yeah. Voice to prompt is good. What I do is to create a markdown file or your project information is recorded. Include modifications. It's a way to build contextual memory from future decision discussions. Yeah, I would say as long as it's relevant and high signal, it's pretty good. Very explicit instructions. And I think that it did not hit the mark perfectly. It it missed a fair number of items here. Frankly, as you can tell, it's still leaning back into its database roots. So it's not great comparatively. It is night and day. But okay, as I mentioned, let's take a look at how conformant they were in a place that it really matters. When you start saying which one's the better coding model, it's not all UI. We've been looking at a lot of UI and I spend a lot of time looking at all of the code itself. So let's just do a little bit of that and see if we can compare and figure out have things changed from the first video. Okay, here's where clean architecture and conformance really starts to separate these models as coding models. And it's worth really paying attention to some of this because this frankly is the one step that can really save you going forward. If you recall when we gave the first PRD, we said here's how we want to lay things out. Here's the naming structure. Here's the separation. It's very explicit in there. Let me tell you, looking at sonnet, it did not conform even at the top level. I at this point have gone in and said, "Hey, go to the detail page." So this whole directory and refactor all of it. This is the second time. This is not the original initial build. We're not looking at that one. I have explicitly come back and said go to the show detail page in that in that folder. I want you to refactor everything paying very close attention to our naming structure and separation rules. And those separation rules are the ones that we're going to explicitly look at. Sonnet went through still has pretty clean code in in here. It's actually separated. But what it didn't do is if you look at the rest of this code line, it's got areas where it's got showing an error or u no results. It's got a loading panel up here. All of these could be separated very easily. And worse, I will say it's got all of its handlers up here, some loading mechanisms that it's dealing with. And all of this should have been separated so that it's an easier thing to puzzle about what the show detail content itself is doing. Let me show you Opus who did do exactly that. So if we take a look. Okay, that's pretty interesting. So, for me that hasn't used opus too much because I always run into um I always run into rate limits. Uh I think this is nice to know, right? So, basically what it's saying is that it did get a better result on this at least with Opus. Look at their page and we start at the top and go to the bottom. It is only 52 lines long long and there's a lot of double spacing in here. So, great work to Opus that they really not only broke apart just like we just saw, but they also separated the different things into hooks or utilities or some other mechanism that is a controlled unit that you would be able to test. If you break that out, that controlled unit or element you can actually go and test. It's it's useful in a hundred ways. I would say go look at the last video. I won't kill it here, but it's much more useful to separate your code into these more isolated component elements rather than seeing them here in the page. So, this did a great job. I will then very quickly jump over and say similarly in chat G in GPT5 does a great job. It's much shorter, 27 lines, but I think largely because it doesn't have the double spacing. It's ostensibly doing the same kind of work. I think this one is clean. Both of them followed and conformed very well. Opus being a really okay that's pretty cool. Yeah. Interesting. Really big model even compared to GPT5 to be fair. However, if we go just one level deeper and we jump into one of the features here, if we wanted to go look at the cast section and said, okay, did it continue down the stream as it was creating this separation? Did it then go inside of each one of those objects and separate appropriately? There's some work here to kind of deal with the response information that's coming back to be able to deal with caching mechanisms and things like that. This is all on this page. Feels like a concern that should have been tucked away into a hook or some other methodologies. But then you'll start seeing things that we saw on sonnet. The loading panel is sitting here. However, the one thing that is a to me an enormous transgression is you'll see these colors. I'm very explicit in my document that I don't want colors in any of or any kind of stylistic information any what we call magic numbers that are laying around uh inside of the application because it gets very difficult if I said oh I want that green to be a little bit darker the system has to go find everything in in every scenario where that green exists and that gets really tricky you want a single source for a lot of these things and so I have said make sure that you that up into a style think it's going to be a while before. I guess this was a oneshot. I'm pretty sure it's going to be quite the while before you can oneshot like a full detailed PRD. It's going to surprise me if he gets every single thing uh correct the first time. It might be in a some time but uh I'm not expecting that yet. Are you guys do you expect that? Uh you watched this video earlier. Okay, it seems pretty interesting at least for me. CSS variables is the way to go. Cool collocation. Yeah, maybe. Yeah, at least I don't expect everything. I feel that's just way too early. The models are not going to do that yet. System and it does have a style system to do that. Sonnet did that quite well. So even in the areas where it is describing things like this foreground color here, it doesn't have text white for example, that's our foreground or our background color, whatever it might be. But if you come over to GPT5 and go into the same Castro object, and you could say, "Yep, they have tucked away any concerns." So they're only passing in what's needed here. It doesn't have any of that other caching and everything else concerned, which is great. However, in here and throughout what they've done, you can see that how they're doing their colors is using some kind of CSS uh variable situation. So I know this gets kind of nerdy, but it's worth saying. GPT5 consistently conforms better to my instructions for the engineering practices. That's across the board. It's worth understanding that that's a difference. But this is definitely a clear finding and I found this through all of my tests uh for this this project. But as impressive as it is on engineering, one last challenge, and this is the big one. Can we give these things screenshots to design from and come out with a better application? Let's take a look this time. Yeah, that's pretty fun. I don't know about you guys, but I get 255 to check cloud codes work two to three time per patch. I never assume they're oneshotting. Yeah, I I don't get why everything has to be oneshotted. Like uh that would be like the dream, but this seems very strange. If you add the sign rules for your code and add a sub agent to lay out the plan for each individual component before implementing with the main agent, most of LLM issues is not implementing stuff. Yeah, that's pretty cool. Um yeah, I just think if you you have too high expectations, if you think everything is going to be oneshotted, debugging uh is like a big part of using edgentic coding. I think it's going to be for a long while. Like the previous video, instead of just sending in cold screenshots, I decided to also create a PRD. I have included that. Once again, you can go look at the the repo itself to understand what I sent in with it. But I go so far as to even call out that there's an accent color. It's even defined in this PRD saying this is the accent color. Use circular badges for the ratings on the tiles. So, I go into a little bit of depth both for the discover or homepage and then the detail page. And so, I've sent in this request, I've sent both screenshots. This is one of them, the detail that we've seen previously. And this is the front page essentially. And you can see that there's three different tile designs that we're working with. Three sizes, two tile designs, and a carousel on the front page. All of that's described very well in that PRD. So, let's take a look at how these did and see who did best. Uh, yeah, I like this. I think this is cool. I I use uh screenshots a lot, like additional context. I think it's pretty good. Micro steps is the key. Yeah, that's a good way of doing it. Small patches, quality checks, use a lot of uh git uh version control. Uh I when I do like let's say I worked on my tool, my AI red teaming tool, I use version control like crazy and before every single update, I try to do version control because it's so annoying when uh let's say set messes up everything you had worked on. So I I'm very I try to do that at least. Hello Will. Uh he can't expect an agent coder to code like he does. If he wants done in a certain way, he should set it up first. Uh all the zero shots approaches are basically useless. Yeah, I agree. Even if your agent does several steps, it's also illogical. Uh if I give a human one PRD and let them work without checking back and educating beforehand. Yeah. And I just feel like all of these small details here, like it's just going to get lost in context. Like you're not going to get every single detail here. Maybe if it did it started with 1.1, 1.2, 2 1.3 step by step maybe. But uh if all of this is getting into the prompt uh and it's not going to happen, I think. But we'll see. Let's see. If you're not building the Iron Man suit or a genetic system you that you drop these models into, there's no way to leverage them anymore. agents now not LLM's uh you have to uh have an OS uh you can do it with one shot full approval uh add in the prompt to execute all necessary commands and test your projects um normally at the end you get the results in single run yeah but you don't you're not going to get I don't know if you saw this detailed PRD here it's just so detailed if you expect every single thing here to be on point. That just seems ridiculous. But let's see. Maybe he figured it out. Do you have a guess? And GPT5 has a good strong image here at the top. Interestingly, it conforms to the width. So, it's kind of a responsive image expression. So, really from a design standpoint, you'd like for it to have a max height that it kind of stops at doing this. It does have a carousel at the top that you can click through. Not entirely obvious. Does not have buttons at the bottom to travel with. That's fine. It does have the toggle between using popular information and trending information. I wanted to show you one here which is their add to list button. Does not work. So, it looks like it's got a drop menu next to it and will do something if you click on it. Doesn't. As we scroll down, found the different tile type. I'm impressed with this. And you'll see that it has different size tiles. Keeps the circle that we're looking for. So, this is all really nice and strong. I would say this is a pretty good entrance into the changes that we were looking for. Let's go back to Opus and take a look at Opus. Opus hit one thing and it's the only one that hit it, which is this carousel is autoprogressing. It doesn't quite work perfectly, but at least it is moving through the different images on this strand on popular. It has this button that I can click and it'll go to trending or popular. Interesting way to do a toggle. These buttons do not work. So nothing else on this carousel works. You can't click on the buttons at the bottom. You So typical carousel actions. It didn't quite hit. And then add to list is is inactive as well. Doesn't do anything. There is a rule that when you add it to a list, you should be able to select what list to add it to or how excited you are about it. It just seems too much to ask for in one shot. Like for my experience, that's just never going to work. But it's interesting to see how much of it was implemented, I guess. Um, I guess as the models get better, zero shot will be the only expectation in 2030. 2030 maybe. Yeah. Uh, now probably not. At least I'm not expecting uh one shots for this complex instructions. I would say Opus kind of did rather poorly on some of this. I'll call out that they are the only ones that got the logo right. They're the only ones that basically got the entire header right and that was only from image. Their full bleed is very strong. So, they did some really good stuff, but they also really didn't do some Let's take a look at Sonnet. And then Sonnet basically has this same thing. So, they have it, but it doesn't do anything. Um, they have this add to list button. And when I click their add to list, you can't quite see it because it's it's clipped by this top image, but they do have the idea of you being able to select and add it to different they're the only ones that hit that mark. You can also move around in the shows. Excellent. You can click the dots as you'd expect. It doesn't auto progress, but I didn't ask for that. So, they really hit the mark best maybe on the carousel. It's That's pretty funny. First it was Opus was best, then was GT5. Sonnet was horrible. Now, Sonnet is the best. And Opus, and then GT5. So, of course, there's a lot of like uh nondeterministic uh based on chance here. Uh, of course, sometimes a model can overperform, sometimes underperform. Uh, I guess some people kind of mis don't think about that. Sometimes you could get really lucky and made your dream app in one shot. Everything lines up perfect. All the uh yeah, the stat statistical picks by the attention uh model is going to be perfect. Um, since it's nondeterministic, right, in everything could happen. So, you could get really lucky and get a perfect app the first time. Uh, I bet the models will oneshot most things by the end of this year. I don't think so. And I I think the way that they've bolded their fonts and kind of built this UI to some degree is maybe the strongest expression. Also, their first section of tiles is maybe the strongest example of these tiles also. So, I thought this was a very impressive. I'd be happy to use this system. So, really, Sonnet gets strongest marks, frankly. But this is what I was talking about. these anthropic models very often will start making some of their own decisions even though they're not out ofthebox ridiculous decisions that make it feel more like typical systems that you would have seen and used. So I think that's what we're seeing here. We're also seeing a progression of tile sizes which is excellent. So I think they hit a lot of the marks starting at sonnet. Let's look at the detail pages. The detail pages are Yeah. I was just thinking like with the detail instruction he has. I don't know if people can chime in on this like um add this explicitly font or something like to exactly this button in the like attention of the LLM. Isn't that going to be like really close? So let's say courier serif is going to be really close to yeah some kind of other font and sometimes it picks that font like it's maybe 90 44% versus 42% chance to pick that text or that token. So the token prediction could get really close if the details are very uh specific. So you can't you don't get the same every single time, right? I don't know if people know what I mean. Uh I would bet if I give five developers the same PR, none of them will have the details in the first attempt. Yeah. And there's like small changes like uh if you think about the the token prediction uh in the attention like in the model if the the details is so specific uh the picks the token it has to pick could be really close. So sometimes is what you're looking for, sometimes it's not a bit more complex. Frankly, uh they had started, if you recall, we've already given it a PR to get it pretty close, and this was a follow-up to that to try to help guide them into the final design that we were really looking for. So this might be considered as the same PRD going in a second time. It's not quite, but it's somewhat similar. They used the circle graphic that we had on the tiles, which is a nice call back to that. I think that that's a nice tie-in. Again, this is Sonnet. Um then it has the user score and the critics score which is very nice. They all have the watch trailer mechanism. Uh they have the budget. Yeah, I agree Fabian. Like with all if you send this in like a oneshot uh I I don't know. I don't think the attention mechanism can and the context can have everything at once. If you did like step by step, I think it would be better if you did like small steps because then everything in context could kind of hone in on the detail you're looking for and you you don't get so much poison in your context when it's going to pick the next token prediction right? So yeah, I think it would be better to do this modular and less uh high signal context. That seems just like a better way to do it. But it's a cool experiment at least and it's very impressive revenue. But you can also see something that I'll make a call out. They are using kind of human readable numbers here. They also have as they had before the structure correct and the different tiles correct. They are however using the tiles they were using before and in my PRD it was very very clear I want poster only tiles with no other information on them in these sections. So it really completely missed details like that. What did Opus do? Opus gives us a very interesting looking really. Yeah, Fabian. Yeah, you need to keep everything clean and start new chats from time to time. Yeah, that's what I mean. So, if you just keep the context like everything is going to send be sent into the attention layers, right, in the model and um the more context you have in the attention layer, that's going to affect the next token prediction. So if you keep that smaller and on point, I would say you can get a better token prediction if you have like a specific um a specific instruction. So clearing is really important I think and keeping the uh context window under under control. Personally, I look at this and think, well, that's kind of interesting looking until I realize. But if you don't care about like I do when I do UI, I don't really have these specific instructions. So, I don't really care what the token prediction is going to be. Is the button going to be in courier or courier serif? I don't care. So, for me, it doesn't really matter. But if you have instructed it to be in courier serif 45 then it can get conflicted with the attention the context and the token prediction. Right. Well this is one score. This is another score. They're shown different ways with percentage and some kind of progressive pie chart. So it just doesn't work that you're telling me the same information in two different ways. They don't have the detail of budget. And they've introduced these panels or what card layout might look like on the background. That's not in the design. It's not desired. They do have full bleed still. So that's a plus. Let's go to the last one. Is it a winner? This is our chance. Did GPT5 using codec seal high did it crush it? Okay. So I will say disregard this bar. This was something that was there from the very first build and did not index that out. Again, this is one of those things that unless I'm explicitly saying remove this from your design, I think gets stuck unless it has explicit instructions. I haven't learned the model well enough to know. I don't like that it's there. So, I will ding it for it, but at the same time, just disregard it for now and pretend that we basically had this kind of expression, which has the same sort of relativistic size issue that we saw before. We see a button with kind of color problems and light versus dark text. Do you think he's like a front- end developer or like a full stack? He has to be. I don't know so much about UI stuff, but he seems to be very knowledgeable about this. Uh, you can start a new chat or clear contacts anytime when you use uh files as context prompt to-do list. Yeah, I agree that uh I thought GT5 was supposed to simplify air usage, but now even power users and developers are struggling with it. Yeah, it's going to be interesting to see, but um I don't really want to I don't want to go back to like the prompting from two years ago. Do people really want that? You have to spend like ages structuring your prompts, making everything perfect. I don't want that. That's just annoying situation going on. They have two different scores. Interesting. And so they also indexed in on the score circle that was being asked for. But you'll also see that their numbers are sometimes collapsed and sometimes not collapsed. So it didn't do quite as good a job about kind of understanding the information that it was distributing. It also didn't these next to one another which you would think that they're related enough that they should be near one another. It stayed indexed on its database background. Some reason its roots are in database uh application design. So I would say that they didn't this they also have the old tile bunch of things. It is better absolutely better. And if you remember the last video this is a world better. So this would take me 2 or 3 hours probably of reasonable if I was just prompting to get it into the shape that I want. Whereas so he's saying that chip 5 was much better using it in codeex than using it in cursor like he tried a few days ago. So yeah, that's why I picked Codeex today for my test. Yesterday I I was willing to give up on it because it just would have taken way too long for me to get it into shape. So those are the kind of findings. That's kind of the bleeding eyeball version of let's look at absolutely everything they designed from having a very discreet PRD and screenshots to go along with it. All right, so what does this all mean? That's the real question. Okay, so that's a really deep flythrough of the same application five times now. And I hope to not build it anytime soon once again as I would love to build something else for a moment. But what is this all about? What's our finding here? Is GPT5 better than Claude code? Is cla code the answer? Uh so obviously my answer is going to be clear here. If you've gotten this far, then you know my it's going to be no. No. Answer is likely to be sometimes. sometimes not. It depends on how you use it. Really the winner here is us in clearly understanding. Number one, the best thing you can do is give a clearer signal. Give less context at a time. These are really obvious. I know a lot of you Yeah, that's exactly what we say. You need high signal, less context at a time if you're going to do aic coding. Perfect. Going, yeah, yeah, yeah, yeah. But this is experimental proof of those evaluations instead of just assuming it. Yes. Don't give it your entire application to begin with and imagine that it's going to be able to conclude that. Build it smaller. Build it your prompts and what you're asking it to do like you would any project. Let's build this first and see if that works. Great. Now let's go build this. You're going to be much better off. Also, the more detailed your prompts are, not to a fault I would think, but every kind of extra detail probably helps in your prompting. So, don't leave it so generic such that it's trying to guess what you mean. With GPT5, my finding is you have to be explicit. You really need to be explicit. Like we saw those things that it would not remove even though it wasn't in its PRD saying you have to meet this. This is what the detail page is. and it wasn't meeting the the visual definition of it either with a a visual example. Yeah, maybe you have to think a bit different on GT5. Um, yeah, I might try that. Um, you have to reit revisit your uh results by building to make adjustments if your UI. Yeah. Yeah, I agree. Agar sucks on cursor. Uh we create prds and markdown uh fields. Uh now we don't prompt anymore. So Decor uh have you doing a lot of research on PRDS and markdown files to kind of set up your projects? Could be interesting if you can share something about it. I haven't spent too much time on that. Hello Richard. Cursor have rag or long-term memory now? Uh if you want to make sure it implements your PRD, I think cursor uses uh rag to index long-term memory. I'm not quite sure about that. Uh I haven't been on the pro on cursor for a while now, so I'm not quite sure. Maybe some other guys knows. Uh that's what I like about GT5. Uh it doesn't do what you're not asking it to do. Sometimes that could be fine, I guess, but that requires more of your prompting, right? It's pros and cons, I would say. Mostly pros. Poly market tool. Uh, it's coming along. I haven't been playing around with it for a while now, but uh, it's pretty fun. I've been doing some trading with it, the poly market trading. It's fun. Uh I've lost like $20, but still fun. GPT5 is more or less proved that while we always wanted a better instruction following, having it now comes with its own consequences. Yeah, that's true, Edgar. Totally agree. So, uh you can get u contradictory instructions like you you say something early in the prompt and later in the prompt you kind of contradict yourself and uh that kind of messes everything up. So, you got to be more careful I guess. Uh if GP5 doesn't ignore the contradiction, you can get into some issues. Uh each markdown file for me is around one to 2,000 lines for every given planned uh page or component faces of dev broken down into steps and micro steps. Do you write them yourself decor or do you use like claw to help you write them? Client has something called memory bank. Uh it's a folder with markdown files that create summaries uh as you move along building your project like uh yeah long-term memory. Yeah. So, more like a scratch pad. Yeah. Pretty interesting. But, uh, what if you have to redo something? Uh, Richard, does it erase that? Let's say you have to drop a feature or something or like a small thing. Those things should have indexed it out and it didn't. So, I would say that with GPT5, if you're using that brand of model for a while, you may want to kind of experiment with giving it a much cleaner set and a much more explicit set of instructions than you might have to with other models. Maybe you're already doing that and you're getting great results with it. I've had comment uh conversations with multiple people in the comments to these videos and been getting really good feedback. I really appreciate it because that gives me a clarity on what people are seeing in the world so that it's not just my perspective that I'm sharing here. So, thank you for that. And a lot of people are reporting that they're getting great results with GPT5. And I agree. I am not trying to bash GPT5. Again, I think the finding is from an engineering standpoint, it seems tuned to give very high quality engineering output. And yeah, I kind of agree. Uh we we used GTF 5 today. It did a really good job and I think it's a fine model. Pretty good. It's not better than Clo Opus, but it's at least on point, I would say. So, just good for all consumers. There's nothing negative about that. Uh, biggest win uh will be uh not messing up working code rather than one-shotting. Yeah, I agree Hillary. That's the most annoying thing. Let's say you work on something and suddenly it changed something that was already working. Agents work about four hours for each markdown file. Wow, that's pretty crazy. If you can set something up, we can go through decorate. That would be pretty cool. I can go through that on stream. If you have maybe some blog post or something or like a video, you don't have to do video like a blog post or something. I'm eager to learn more about PRDS and the planning part of agentic coding. Uh I think at some point having an efficient graph uh grow along the code base could be helpful. Yeah. So Edgar, do you mean like not logging? Is that like or just a list of all changes and stuff? Use GP4 5 Pro and Google deep thing to write the files. Opus 4.1 to refine and tailor. Damn, you must have a lot of uh a lot of um expensive uh subscriptions. Decorate. So, GD5 is better than Sonet. I don't know. Probably. Yeah, that'd be cool. Decor, has anyone used Cairo? Really good job at planning task, implementing a task. It's slower but accurate. Uh, I haven't access yet, Aaron, but uh, if I do, I will probably give it a shot. No, literally a graph database with nodes and connections. Okay, I see what you mean. Yeah, that could be cool to just keep the structure knowledge structure context. Let's finish this video. We're only using essentially the sonnet version of GPT5. We haven't used GPT5 Pro. That's not out there for me to use yet for me to really kick the tires and see how much better it is. But I would expect very similar to the son of an opus. It has better behaviors in some places and doesn't really matter in others. Sorry, wishy-washy every single time, but the outcome of this is I hope this helped you make up your mind of what you think is better or in what places it's more ergonomic for the kind of code you're doing or the way you work. It certainly helped inform me, frankly. The next thing I'm going to build with, not surprisingly, is clawed code. But I do want to do a little more kicking of the tires of of codec cli just in case it's me and the way that I'm prompting it. I need to learn that if that's the case. All right. Well, thanks for coming along for the ride on this one and I'll see you in the next one. Yeah, great video. Uh, a bit maybe a bit too much on the UI for me, but uh I'm I'm going to like it. Very good. Uh, I am subscribed. Uh, yeah, I'll share the video. I guess. Yeah, pretty good video. I like his um Yeah, that's the Bman method. Yeah, I'll check that out. I guess a lot of you know about this guy, but uh let's share the video anyway. Was pretty cool. So, this was the video if you want to watch it again. Uh, Archron by Cole. Yeah, we had a look at that. That looked interesting. That was pretty cool. Uh, I see people use PRD task manager, memory bank, maybe overkill. Curious to hear other people's experience with that. Yeah, me too. I kind of want to hear maybe like it look like Decor has a lot of uh experience with this. And Edgar has some stuff working on like um some agentic setups we might have a look at next week. That's going to be pretty cool. Uh I want to show you how I you use the computer use preview model from OpenAI. Sorry, I got it again. I want to show you how I use the computer use preview model from open air to do human work on desktop uh ABS workspace that I launched 100 at a time. Yeah, that sounds cool. Uh maybe you can come on u we can try to do you can do some screen share if you want to comment. You don't have to be on the video if you don't want to. That's up to you. You can pre-record screen share or something. We can play it on the stream. Okay, pretty cool. That that that is uh that means everyone that's here, if you have something you want to share uh with other people and it's best to do it uh on either like come on the stream, we can do like um an interview or just hang out and you can screen share. That would be cool. Uh I'm open for that. That'd be really cool because now it's just me talking, right? But if other people have something they want to share, uh yeah, let's do it. Uh I maybe we need to set up something. So maybe I can uh this app here I have uh I can do some iterations on it and we can set up uh some um people can sign up to if they have something to share on the stream, they can just come on and do their presentation or something like that. Uh, I might do that. I know Edgar wants to come on. Do Web, I think he says he wants to come on. And now Decor and probably a lot of other people to have some cool information to share. So, I'm going to try to set that up. Uh, it's not going to be tomorrow. Uh, I'm doing some traveling tomorrow, so no stream for me tomorrow. But Thursday I should probably be back. Yeah, I think we need a Discord. It might be that. Uh then I got to think about mods and stuff. Uh but I'll try to figure it out live panel discussion. We can do that then. I guess we need more people than just me. But uh we'll figure something out. I think we just need now we just need a way to organize it. So if I do some iterations on this app here, uh we create a second page where we can sign up and um that could be pretty easy. Keep discord simple. Yeah. Yeah, I've tried it before. Uh I will look into it. I know you're very eager eager on Discord, David. Yeah. Yeah. There's a lot of people are here with a lot of knowledge, right? And not everyone is here every time, but if you have like um a setup so maybe Edgar and Decor and other people here could uh join this day, we can kind of set it up. So, I'm going to try to figure out a good system to do this so we can plan. We can just have it totally open so people can see and some people can do screen shares, demos. We can just have a chat. Uh, not so interesting always. It's just me talking. Could be a bit boring in the long term, but if we get more people involved, I thought it could be more interesting. So, Richard also want this card. Okay. So, uh, I'm going to try to figure it out. We should plan it though not to tank the quality of your content. Uh the quality is pretty low still. It's pretty low. So don't worry about the quality. Uh but uh the most important part is the sound. I have done everything I ever wanted on YouTube. So I don't have any aspirations anymore on YouTube. Everything I ever wanted from YouTube I have completed. So doesn't really matter anymore to me. So we have a single UI that allows the human and 03 model perform playright. Okay. Uh but can you really do that on a live stream decor or do you need to pre-record it? aim to be able to say that one day. Yeah Leonardo you get there. Uh yeah, for me, I just haven't don't have any more ambitions on YouTube. Like everything I ever wanted with YouTube is uh far expected my expectations. But uh you can uh come on and share your content, Leonardo, if you want to. We can watch some of your videos if you have anything cool. You can set up a Discord in five minutes for chatting and a voice voice channel in our 3 minutes. Yeah, Richard, that sounds interesting. So, you think you can do it on the live stream decor? Cool. From experience, when we had a podcast guest and did not uh did not least talk about the episode beforehand, we created Okay, I see what you mean. Yeah. Yeah. You should probably have a plan what's going to be so I can do the title and headlines, right? That at least pretty good. Yeah, decor if you kind of know what's if you think but we can't um if you're going to do it on stream, we can't wait like 40 minutes for the VMs to get ready, but you know that better than me. So, like I said, I'm going to set it up. I'm going to look into Discord uh if people want that. Yeah, sounds good. And I I'm going to fix something here on the open app where people can sign up. If they have something demos to share. Um or just if they have just articles, we can go through on the stream, we can fix that. So that could be pretty interesting. And I can do a community post on it and maybe people see it. I can do a members video on it, too. Even mention it in my next video. So yeah. Uh but uh I need to go now. It's been a long almost three hours. So, thanks for a lot of interesting conversations today. We had a look at uh GPD5 with open with codeex was uh I was pretty impressed. Codex CLI cloud code is worth it. $20. Yeah. Uh remember at the end of the week we're going to do a members giveaway on the YouTube channel. We might even do a stream stream uh giveaway. We'll see. Yeah, Leonardo, if you have anything to share, we can look at it. D Web, thanks for tuning in. Arony, as always, Edgar, great to have you here. De Cory, appreciate your information today. Richard Davos as always. Thank you everyone for tuning in and I hope you see you again maybe on Thursday. Tomorrow I'm off. s 8F some good information. Uh yeah, do that. Enjoy your day and yeah, hopefully see you again on Thursday and yeah bye-bye.