Testing OpenAI's Structured Outputs and AI Predictions for Software Engineering

hello hope you can hear me if you are here I'm just going to test stuff right yeah it looks pretty good okay so so it's been a while since I did a stream so I don't know if there are people here but we'll see we're going to give it some time anyway so I'm basically going to talk about um I'm just going to have to refresh this I think I could see there are some people here

yeah we'll probably work out after a while I'm going to try the shat again yeah seem to be working I can see some people answering but I can't see the shat down here but um uh hopefully it will sort itself out can you hear me fine let me know if you can hear me okay hello IIA 360 camel Penny maker uh yeah I can see my chat now so I think we're ready to go hello Bruno

nice to meet you so yeah uh I've been looking at this today so structured output in the API so I guess that was my title so yeah that is kind of going to be what we're going to look at today uh I think it's pretty cool I don't know I'm going to give it some few more minutes before I look into it uh I kind of want to fix the shat first I don't know what's happening

here but uh it's a bit strange uh we'll see so yeah I'm just going to test something real quick while I wait here uh let me just uh test this hello hello hello uh okay so let me open that uh I'm going to explain that yeah that seem to be working uh it's just very annoying about the shat part here but I think everyone can hear me so that's good hopefully it fixes itself after a while

um so I'm giving going to give it like three more minutes before I'm going to look at U what I wanted to check out today uh I'm just going to try to grab the shat URL and put it here somewhere so I can see it yeah that seem to work hello uh ramunas nice to meet you yeah I'm good to go now so yeah how's everyone been it's been a while since I did a live stream

are you enjoying summer I guess for some of you it's been a lot of news lately like when I was on vacation a lot of stuff happening like with llama that I haven't really covered with uh pretty cool stuff but I was kind of interested in these structured outputs because I think it's um kind of interesting and this morning I just made some stuff so I was testing out how we can extract different kind of structured

data uh using kind of the new model from I guess it's not a new model it's kind of a new structured output so we're actually using this client beta shat completion. pars so that's kind interesting and we are using pantic to kind of set what we want to extract so we're going to do a few different um tests I think so I'm just going to give it a couple of more minutes before before I kind of

dive into what I wanted to take a look at today so I don't know if you read through this AI Studio has adjacent output too curious to see the use cases yeah like open AI has had the the Json mode for a while now uh but I think if we look at um there are some differences uh I think it was in their documentation they kind of listed out the differences down here I think uh where

was it I think it was somewhere around here so structured output versus Json mode so I think they trained this model on like uh they said something about training the this model on uh I'm going to have to find it uh I think it was made in the article that they train this model on something something else uh we're probably going to take a look at that uh but you can see adheres to schema so Json

modes kind is no and here is yes and there are a few newer models that are compatible so GPT 40 mini is compatible with this new structured outputs that is kind of cool support parallel function calling so this is no that is a bit sad but they might change that in the future I guess uh there's a lot of stuff to dive into here we're not going to cover everything today I think we're just going to

look at what I have tested out this morning and yeah I was really impressed to be honest so I think I just wanted to start to look a bit at their blog post we're not going to go too deep into it but uh we are introducing struct outputs in the API reliably adhere to developer Supply Json schemas so this is not something new like I've been using this in my videos but I haven't gotten it to

be like perfectly reliable it's been pretty good at least with Json mode but uh I think this way it's a bit easier to use at least for what I have been testing so you can see last year at the was it wa last year at De we introduced Json mode a useful building block for developers looking to build reliable application with our models while Json mode uh improves model readability for generary Value Json output is do

does not guarantee that the model will response uh the model response will confirm to particular schema yeah that is kind of my experience too it does not guarantee so today we are introducing structured outputs in the API a new feature designed to ensure model generated outputs will exactly match Json schemas provided by developers so I think that's pretty interesting and they say something about how they did this I can't remember there was something about the the

way they trained their model I think so generating structured data from un generating structured data from unstructured inputs is one of the core use cases for AI in today's applications developers use the open API to build powerful assistant that have the ability to fetch data and answer questions via function calling extract structured data for data entry and build multistep agentic workflows that allow llms to take actions yeah so developers have been long working around the limitations

of llms in this area via open source tooling prompting relying on requests so this was horrible in like the early days of like gpt3 and stuff it was almost impossible to get structure data output so it's got it come a long way I guess so it's much easier now that it used to be I remember trying this with gpt3 like preat gbt was almost hopeless structured output solves this problem by constraining open AI models to match

developer supplied scheme and by training our models to better understand complicated schemas so I guess they did something here they trained the model on maybe some big data with complicated schemas I guess so the evaluation is that with structured output scores are perfect 100% uh in compared to yeah I don't quite see in the evals at least following Json schema GPT 4 uh the old GPT 4 scores less than 40% so you can kind of see

them so now they have 100% when strict is set equals to True interesting and they kind of go into a bunch of examples here uh hello digs nice to see you welcome so how to use structured outputs I'm not going to go too deep into this uh because I kind of made some examples I thought we can go through today uh but it is interesting I'm going to do like this uh and I want to take

a look at uh later after that we want to take a look at this uh Russell Kaplan tweet 1.4 million views uh it's basically his predictions for the future of software engineering and I read through it and I got to say I agree with a lot of this uh of course there are some bias here because he's the president of cognition Labs uh cognition Labs is of course Devin right so there are of course bias here

but I kind of think was interesting and I think we can go through that later I think it's going to be pretty cool but yeah let's take a look at how I um I tested out this structured output this morning uh so I wanted to kind of try two different things I wanted to the first thing I wanted to do was to take a YouTube video and kind of set what I want to extract from that

YouTube video into like a structured output by using whisper so you can see uh I'm just using um I'm just going to use um whisper now to kind of I'm going to use the API whisper not the local one so we're going to run this client audio transcriptions uh and the idea is that we can enter our YouTube url download the full video and we want to extract some structured output from that video that could be

entities right so what I thought we could start with with was uh extract shocking statement and names from the transcript um hello camel I want to know how to use open API with assistant using gp24 Min apparently there's a second version thanks uh yeah that shouldn't be too hard do you mean like there's a second version of GPT 40 Mini so yeah like I said uh what I want to ex try to extract from this video

that we're going to look at is uh shocking statements whatever that is and the names right that is mentioned in the video so basically it's not going to be it's going to be a transcript right so I kind of created two different uh kind of U scripts to do this okay so you say there's a second mini version that kind of H uh let's take a quick look so if there is a new version we should

see it here in models right GPT 40 mini uh this one is not very new I can't see any new GPT 40 mini but uh there was a new GPT 4 o model yesterday yesterday do you mean this from the one we're going to try today maybe I'm not quite sure so let's just try this now so I'm I'm going to show you so hello track friend so I I found some videos um so let's go

to let's just let me just open like an in Incognito here and let me just turn off yeah we don't need any sound here now so I saw there was this news clip GPT 4 Oh Long yeah uh I've looked at that and I was hoping I had access but I didn't uh we might do I might do a video on that actually it's pretty interesting and you can also see if we take a look at

the new model now uh the new GT4 o model you can see the max output tokens is 16k so it's four times more than uh the previous version so that is interesting interesting right I have been complaining about the MOX output tokens for a while like when we use 3.5 Sonet uh the annoying thing is that sometimes you can't get any more output so it just stops you have to do like continue but I like like

the to see that the providers are extending the output tokens too not just the input and the gb4 O long version that has like four that has like um it's not that long output I think it's called can't really find it here whoa here it is chipd 40 um can you bypass output limit now that output is structured is to depend I don't think you can B bypass it can you I think it's like a hard

hard set output use perplexity yeah you can do that it's good as long as the quality yeah I was thinking about that too Tech friend um I was want I was I rather have better outputs than how yeah yeah that was what I wanted to test like to see if the attention uh gets weaker if you have a very long output but they say here gp4 all long output that it can handle up to what was

it 64k output tokens per request it's very it's quite expensive compared to GPT 40 not crazy but it's more expensive but if you think like big code bases and stuff this could be interesting to have a longer output model uh we can try this afterwards but I don't think I have access so we're going to check that out but first let's take a look at what uh I was looking at this morning uh with the structured

output so remember now uh I found this video here it's basically just a new Vice press and speech from last night so I'm going to copy this right uh and I have a few different scripts here so we have two I I didn't combine them so we have one script that just takes this um URL so you probably can't see this but let's just do python YouTube transcribe and we can paste in the YouTube url right

and this is going to download download and transcribe the video into text right and then we're going to try to so I'm going to show you the output and then we going to try to extract something from that uh video uh I tried this morning and it worked pretty well so you see we got to transcribe this into chunks because it's too big so we have to divide it into three different chunks and I think the

other script I created for this was called uh python YouTube extract but I thought we can look at the text file first so how long was this video it's 19 minutes right okay so we got it uh transcribed whoa so you can see these are all the text parts from the video right and now let's try to extract some um structured output from this so remember I wanted to extract what I just called shocking statements and

some names that I mention in this speech right and we are running the new model from yesterday and the system prompt is going to be extract the shocking statements and name from the transcript and of course we want this in a Json file right so let's try it so I think this could be interesting if we can get it like very reliable okay that was pretty quick so the shocking statements we can zoom in a bit

Donald Trump weakens our economy and strengthen his own hand Donald Trump marcks our loss uh JD van wrote the forward to the architect of project 2025 these guys are creepy and yes just weird as hell there are some statements here I think that worked pretty good and we extracted some names Tim wall Doug Josh Shir vice president Harris JD van Donald Trump so yeah I was quite impressed by this Gwen hope Gus I don't know about

this I can't really fact check this because the video is quite long right uh but I think we can get this into a nice structure now so if we reload this you can see we get this and I think this is very useful so we have the shocking statements par parameters here and all of this and we also have the names right so pretty easy to use Simple to set up and it's very easy to switch

between this so I had this other video uh let me see if I can find that I think it was this one copy that so so this is the video from AI explained he's a great Channel hello night spider nice to see you again so if you haven't checked out uh his channel I guess you have he's great so let's copy this video and let's do some changes to the extraction part here so now I wanted

to see if we can uh just change this up a bit and instead of extracting hello Hy baby nice to meet you now I wanted to just try to extract AI model names right so we can just change the here we can just use pantic and kind of extract what we want right we can kind of add some stuff let's say we wanted to extract some numbers we can just do integer right but let's try to

extract the names a I models let's just try that because I think uh Philip here is going to mention a lot of models GPT 5 probably chat GPT so let's see if we can extract all the model names he will use in this video I think we are good to go here now ai model names mentioned so first we kind of we can just clear this first we got to download the video again so let me

do that copy this just going to enter the video download it so this is going to transcribe the video at the same time so these are just two chunks right so I'm going to think this week uh I kind of want to try to build some kind of app or something by using this function I saw some ideas like things you can use this for uh I might just make a video about it I think it's

pretty useful to be honest okay so we transcribed the video now let's try to see if we can extract the model names okay we got an error here oh shocking statements what did I miss here yeah uh we forgot to do like this right let's do that again okay so here are the model names here Philip mentioned in his video GPT 5 GPT 4 I think this is gp40 GPT 40 mini CLA Sonet CLA Opus clae

4 okay llama three llama four and yeah pretty easy to use like I haven't fact checked this but I think it does a pretty good job of extracting what I wanted here let's try to do names so let's just do names maybe let's see if we can just do names uh something like this okay so camel according to documentation assistance API with gd4 Mino is in open Beta assistant wi2 okay I haven't checked that out I

might look into that I haven't looked into it yet uh okay so let's try to extract the names from the transcript now so you can see we just feed the transcript in here in the content right um print names so I think this is very easy to use so be honest it's not very hard to set up using pantic super easy to add stuff here so let's see if we can get some names here now since

we already have transcribed the video we don't have to do that again so he mentions Greg Brockman Jon Schulman Yan Lea Ilia satua Sam Elman Elon Musk suckerberg Brett ad Brett adock so there are some I guess the transcription isn't perfect but we kind of get Sam ulman I think that was funny so I'm ult man but yeah pretty easy to use and if we open this now yeah we get this into like a good structure

format that is easy to use right but again I guess we could use gp4 to fact check these names if you wanted to because they are not correct right so you can just add like a layer here maybe something simple right uh I'm not going to go into that now but I guess we could do that if we wanted to so we can just fact check the names and correct them or something that could probably work

and we can I think we can add like I I don't know dates maybe is that a string maybe that's just a string let's try to add dates names and dates so something like dates uh date dates how can I train my large language model Gerald you mean like from scratch or fine tuning like a like a new language model okay maybe I don't know how what is the best way to do this let's ask uh

Claude or something oh I want to extract dates in the correct format I haven't really used pantic too much so I'm not 100% sure what is the best way to use it but definitely kind of interesting oops so field description list of dates in ISO format okay let's try this I don't know let like a make a new one um what's the project today uh BX and I uh we just GNA we're just going to test

out the new structured output so since open AI came out with this I think it was yesterday uh I've just trying it out this morning and it is pretty interesting right I think I want to add data that I want to a model like f tree or Gemma you mean like uh structure data or just uh context or company data I think I'm going to do the full let's try companies names and St so I think

then I'm going to dive more into this structured output it's going to be uh what goes wrong here I think there are some just I think it could be pretty interesting I'm just going to see what's wrong here so let's just see the output is in a strange format please fix so they have some there was one example here I was looking at I think it was uh down I wanted to try yeah this is the

one so here is kind of like a Chain of Thought example they have so they add these reasoning steps so this looks very interesting bxn I yeah I think it's pretty cool and it is uh very nice if you want to do build something that uh is has has to be reliable right okay I didn't see it so I think it's pretty interesting right so let's see now if we get yeah so here you can see

we have the names and open AI was mentioned anthropic character AI Google meta Deep Mind weights and biases so it is pretty accurate easy to extract do you have any ideas what we can try to extract that is quite difficult uh I think I'm don't think I'm going to play the video but I kind of want to extract some data that is kind of hard to collect but I think kind of the shocking statement was pretty

good so we can try that again but then you kind of have to Define what is a shocking statement right so let's find another video or we can try the one we have now sorry so let's just mix it up a bit here and structure data to knowledge graph graph R actually for specific use cases yeah that is interesting Rome lead welcome by the way so I don't know if you saw this example they have here

on open ai's blog post where they add in this reasoning steps so they ask this question I don't know if you've seen this 9.11 and 9.1 which is bigger how about a video that does a list of suggestions like top ai2 like something like webs names and websites yeah if people like that I might do that uh think it's less about extraction but the additive Parts like you extract some keywords uh but then add layers like

sentiment tags okay that's cool yeah so what do you use Json mode for like an app or something or just some personal stuff or at work maybe in a political debit it's useful to point um contradictions point to contradictions useful to point so you I'm not 100% sure what that means testing structure data extraction okay yeah I see no it's Spa yeah could be interesting uh so Bruno do you mean like uh to extract contradictions in

like between the in in a debate yeah yeah I think I see what you mean yeah that could be work too and like in financial data at work yeah extract transcripts categorize add to database sentiment yeah that's cool no no problem BR it's just I'm not English too uh yeah that twitch thing yeah okay I might have to fix that I'm not uh I haven't really looked into twitch there might be some error in my settings

but for next stream I'm going to try to make sure that twitch is working too uh I haven't really looked too much into it so uh Nexus looks generate some ideas in favor argument against yeah sentiment I guess that's pretty good for like uh reviews and stuff if you have like a massive and add things to database I'm definitely going to try this to add to like a database right that could be interesting yeah thanks M

that's nice to let them know I I didn't know that I've been just using something called restream uh and I'm not 100% sure how it works I haven't really looked too much into it so yeah I guess I have to try to make sure that both stream are working uh I think I have to do that uh but where where we here now yeah I wanted to try to extract um some more shocking statements but that

is kind of that that is something that is non that is something that's non deterministic because what is a shocking statement that is kind of objective right not objective but that's kind of it's not a fact right but we could do like Finance statements maybe or something like that Finance statements that could work Finance statements yeah I don't really care about the so let's do Finance Finance statements something like this and let's ex let's download the

other video again I think I just searched um let me run this okay but yeah I was also wondering um it's a bit strange uh at open AI lately all people leaving and stuff I don't know if they're leaving because they got better job offers or if there's something they look like they are in some kind of trouble right uh think I was most surprised to hear how confident open AI is using this I guess I

heard from all the Big Spenders that this is a high requirement yeah I 100% agree Nexus because I think right now most most people are using llms for data extraction and data processing text extraction and things like you do at work that has to be one of the biggest use cases uh next to coding generation I guess it has to be right right and if you can get it like in a very structured format but I

guess the 100% how did I say it with structured outputs achieves 100% reliability in our evaluation perfectly matching the output schemas that is high confidence but I guess theyve tested it a lot and from my testing it seems pretty good did did you say you tried it out Nexus this structured format versus Json mode do you have anything like do you think it's better worse same okay so we transcribed this let's see if we can extract

um some Finance statements did I change the prompt yeah uh what happened here let me see just trying it now okay let me know um yeah let me know what you think hello this modern day have you looked into instructor no I haven't but I heard of it instructor blinding my eyes here let me go here instructor oh there's not dark mode kills my eyes is it this I've seen this I think it's open source models

like uh pretty base maybe can be at that level to the fraction of the cost yeah so like for me that is not doing like massive data I have I don't really notice the cost uh but they did bring down the cost uh I think they brought down the cost for GPD 40 by 50% again that's the one okay Brandon sorry about this there's no dark mode here I don't know I'm going to look into this

what moders okay so they are on open Ai and it looks pretty I've heard of this I think I heard it on a podcast or something instructor yeah Pro structured output for all lmos in a variety of languages cool yeah so they have Google anthropic the I guess they support 3.5 then click at the okay sorry I didn't see it looks pretty easy to use I guess looks very easy so they have kind of the same

setup here with the pantic right so you just set your desired uh structured output I think it's basically the same almost actually it looked as easy yeah thanks pong pong using an Tropic yeah this looks nice but then again is it as reliable as open AI claims here with 100% that's interesting uh what was this again line 43 okay I see uh we need this yeah structured output takes inspiration from excellent work of the open source

Community yeah and there's instructor right it has to be because it looked so similar they using pantic okay so here are the fin statements Donald Trump weakens our economy he drove our economy into the ground racing cost repeal the Affordable Care Act get social structure and Medicare sat his country CB in maralago wondering how he could cut taxes for his rich friends so yeah I guess that is pretty good let's find another video we can test

right let's do some kind of stock thing market crash I don't know let's do this one and Let's do let's transcribe this so these are more General Finance statements uh let's see if we can extract uh stock tickers from menion companies use scrape graph Ai and I finance with structured output mode yeah cool but let's see if we can do it like very extract the stock tickers from the mentioned companies yeah it doesn't really matter what

we name this I don't know if this list string is the correct format but let's try it what about key points of a product review do you mean like from a YouTube video we can do that we can try that let's do this ticker first uh okay okay so we transcribe this and let's see if we can get some tickers here I got to look a bit more into pantic right uh that maybe didn't work too

good so we got Bank of America UBS uh let's find uh I like this channel MK Let's uh Samsung let's find a iPad Pro let's try this video iPad Pro so with pantic you can also use fields and provide examples okay I'm definitely going to look more into pantic if I'm going to use this I haven't really tried that too much use fields and provide examples okay cool that could help right okay so night SP what

about key points from a product review let's try that extract the key points uh from the product review I don't think we have to do from the transcript extract the key points uh let's see if we can get it like in a structured output I don't really care about um I guess we could do key points doesn't really matter but let's do it I think it doesn't matter uh key points okay so we transcribe this [Music]

key points is that all I think so hello an what do you think about the ulia programming language I haven't looked into that have you tried it out can't really make any comment okay so here we got some key points it looks to be working pretty good um Apple announced iPad Air has been upgraded from one M1 to M2 ship 128 GB of storage new iPad Air features spec bump and introduces new color options iPad Pros

are now the thinnest Apple device ever iPad Pro introduce an OLED display all right we have some key points here I think despite improvements the usability of the new iPad hinges on the future software upgrades so I can't really uh yeah was pretty nice but I can't really okay Anil I I can't comment on that but yeah go for it if you think it's useful maybe some other one in the chat knows has knowledge about that

can't really comment but yeah this seems to work pretty good but again I haven't watched the video so I can't really it's kind of hard to rate but from the first look looks pretty good let's try uh extract the key points let's try sentiment should work too right sentiment mixed so that didn't help too much because this kind of relies on the model to right since this is not like a fact uh I use pantic and

instructor to actually automate creating crew crew AI agents okay that's cool like I have um some contact with the crew AI I think it's the founder his name is uh is it Jose I think so I might do something with crew AI this fall some kind of video I think could be pretty cool I always said I wanted to wait a bit on this crew AI agent videos because I kind of want the models to be

cheap and very good before I start diving deeper into this agents uh but I think we are on the way it might take some time though oh yeah sorry not Jose but wow I don't know how to pronounce that but uh he's a super nice guy and yes so Matthew Matthew bman he kind of set me in contact with him he's also a nice guy he's got a big YouTube channel Matthew bman check it out so

yeah I think I kind of tested out this structured output uh I think I'm pretty happy with what we've done so far uh but I'm going to look more into pantic what we can do with this so I might just I might make a video on it and there are some other stuff I want to try out I want to look into this um Chain of Thought um reasoning steps part uh I want to test um

the output window that is now 16k I think autog gr seems like an interesting tool for making crew a teams yeah Auto Rock that's cool no not me neither but uh I have access we could take a quick look here I guess or I was going to do something else so let's just wait a bit on that I think um if you are in the crew AI Discord I can link you the examples I shared okay

cool uh I'm not in the crew AI Discord I think at the moment uh do you have uh I have a Discord though but uh you can find my mail address if you want to send me the link that could be cool you can find it on the the YouTube channel is llm the only way for AI maning llm are uh has delayed real AI well that's a big question annel I I I think llms uh

are just a part of AI I don't think it's the only part I think it's one part of AI but how do you define AI there's a lot of trouble there right what is AI but yeah I think it's a part of it not the whole thing but that's kind of my idea uh but yeah I wanted to kind of look at this Twitter tread that's got a lot of traction so I hope you can read

this fine now so like I said in the beginning uh 1.5 million views uh 4,000 likes 9 uh 800 reposts uh but again you have to read this with kind of some bias because he is the president of cognition Labs that again is the maker of Devin so yeah uh some bias here of course but I kind of wanted to go through this post because I think it's pretty interesting so Russell Kaplan predictions for the future

of software engineering so it makes a lot of points here right so the first point is models will be extraord ordinary good at coding very soon research Labs like uh cognition labs are investing more in coding plus reasoning improvements than any other domain for the next smaller generation their efforts will bear fruit that could be true I guess uh I guess he is correct about investing but again there are some bias here right because it's from

cognition and they are a research lab but I guess research lab he kind of means maybe like anthropic I'm not quite sure but uh there are a lot of money going in of course to model uh generation with the coding generation right and it kind of follows up with why so besides General AI progress coding specifically has a unique Advantage potential for superhuman data scaling via selfplay models can write code then run it or write code

write a test and check for self consistency so I have a video coming up where uh I'm going to test a bunch of Open Source models proprietary models that's going to take a problem write a code write a test and see if they can complete it so that is going to be an upcoming video uh but I kind of agree with this I think selfplay could be important so basically you can have like a Sandbox so

the models can just write a bunch of code run it test it see if it works uh that could be something uh or write a code write a test check isn't that what Devon is already trying to do uh does any of you have access to Devon I don't I don't think so many people have uh but I think this is kind of describing Devon right are you testing deeps coder in your future videos I heard

it's one of the best deeps coder yeah that's great I'll try that uh but I haven't done any video on it I think I just tried it on live stream but uh yeah I might do something I want to try super human too um nice super Maven I think it's called that is like a open source GitHub um not GitHub open source GitHub co-pilot for code generation yeah only human species know no language in Animal Kingdom

L might be a part of yeah uh okay so let's go back here so then he follows up with this type of uh automatic supervision is not possible in most domains which are facing data walls in post training as we approach the limits of human expertise code is different it can be tested empirically and automatically yeah I guess I agree with that uh but sometimes can the models solve a new problem right a problem that has

never existed before that's going to be interesting to see uh I guess it can maybe do that well like I mean like a real new problem that could be interesting uh yeah deep sick math I think that's pretty good too but I haven't done any extensive testing so then he follows up with as a result oh that's great this modern day cool I'll take a look at that so then he follows up with as a result

software Engineers will look radically different in a few years through coding agents we which do task end to end will complement today's AI co-pilots the experience will look something like giving every engineer an arm e of interns yeah I heard that talked about right so you can have this true coding agents so you're kind of the manager I don't know what you could call it like a the the manager of these agents and you kind of

steer them in in the direction you want right um yeah I think it talks about that so in this new world every engineer becomes an engineering manager yeah you will delegate basic task to coding agents and spend more time on the higher level parts of coding understanding the requirements architecting systems deciding what to build so basically that is what I've been doing with my small projects but how will this work in like a big big project

that is what I like to see it works fine now to use like let's say clae here or something for like a small project so you can see here is kind of my Chrome extension that was even not a project but yeah I've done some project here uh so you can see we have some simple project works pretty good but if you're getting like these projects of 100,000 lines of code I don't think that's working now

no way I don't know what you think but like have you seen an llm that can do a code base of a million lines of code probably not but who knows in like a few years hard to tell uh so everything that is prediction is just guessing if you ask me so then it kind of follows up with this will lead to an Era of unprecedented software abundance software has historically been difficult and expensive to create

yeah that's true uh it will soon be 10x more accessible I wonder where they get the 10x from all the time it's always 10x we'll see how Prof proliferation of single use software one of app and websites that are now are only now viable yeah I've been using it for single use software so you can say what we did here is a single use case that is true one of apps websites yeah that's true uh his

use of army of interns and basic coding task is not uh innocent we're still far away from having a system that can code something yeah I 100% agree ma Marina I haven't seen anything that can you take like a 100,000 2001 million code base lines of code and create something like that I don't think so but sometimes language itself Paradox of statements there's no truth which leads to Paradox yeah oops that's hard to tell um but

we'll find out in a few years I guess we know more in a few years but uh I haven't seen anything up to this point that makes this work but I can't really say what's going to happen in the future right uh there will be way more s engineers in the future than the present the job will just be very different more English less boiler prate coding Engineers will adjust like they did from the transition from

assembly to python I think that's true right there might be some change but uh is it going to be like a drastic change that could take years right hello Adam true but today you can iterate Oreo code and make it better with AI right now uh make get better with AI right now which is still valuable yeah 100% agree this doesn't mean that this is not valuable right it's super effective and we seen this polls that

people think they are more effective using llms like debugging and iterating over your code but I'm just skeptical to this huge projects how it's going to work but uh yeah I think it's valuable right and we are still very early right it's been just a couple of years of um even though we didn't get any like new models I think still there's a lot of improvement right just like we got this yesterday this is an improvement

I think so I guess we got some kind of new model but it's B basically gbd4 but if this works as they say it is an improvement same with Claude right the artifacts that is an improvement right even though the model is not maybe that much better but it's still an improvement there will also be a substantial second order effects for startups beside the immediate productivity gains for one company that uh markets to Developers soon start

marketing to coding agents as well after all your agent might decide what um Cloud you use and which database you choose agent friendly uiux often good CLI will be prioritized yeah maybe um I know some of you said they worked with crew AI here so uh I can't remember one of there was one of you uh but I don't know about coding agents yet I guess Devin it will be interesting to see when they come out

with a new version of this how good it actually is right I'm sure they're going to make an update soon you can see we haven't seen anything from them since basically from March 12th that's almost six months ago so I guess they're going to do like an up date soon and the prices are just coming down so could be interesting I guess the bar for product quality will also rise Half Baked or feature incomplete MPS are

less acceptable in a world where developers can ship so much faster yeah testing infrastructure will be much more important and prevalent with the rise of coding agents both because the coding agents will write more tests and also because they will depend on these tests and check uh their work switching cost will decline as a mode for tech companies as agents make migration easier companies will uh even start building out migration assistant coding agents where you can

buy their products and streamline adoption uh yeah I think that was pretty much this regardless regardless of specifics the macro is clear there never been a better time or more productive time to be a builder I can agree with that I'm excited to share that I join cognition labs to build Devin so again it's a bit of a bias right uh it is I think there's pressure on AI companies to show Enterprise level value hence the

structured output announcement I think we'll start seeing companies beginning to understand understand how to use it yeah it's just going to be interesting to see like how uh I feel like we've been on this it's been a big hype but I feel kind of things are like cooling down a bit and I think in this cooling down part and when things kind of calm down uh then we can kind of start making better progress maybe when

everything isn't that hyped up uh so I'm kind of looking forward to kind of the hype dying down a bit right uh I think that's good uh so next is you have started looking at task flow AI or recr AI yeah so have you tried this he talks about coding Ag and here right uh I haven't really I guess I've set up some code buing agents I don't know 100% I guess it just means like one

agent is building like the back end one is building the front end you have one agent that kind of controls the whole flow uh I think and that is kind of describing Devon right so or is Deon One agent I can't really remember now but uh uh yeah it could be something I'm sure these agents will do something in a few years but uh I think they are depending on the price that is coming down drastically

and they are dependent on the yeah the model I guess they could use vertical models that could be better uh but I kind of do agree with the that coding is a bit different right since the models can both write a code and write a test for it uh that is a bit different than other things like uh science and stuff like that because I guess when you do like some kind of science I guess you

have to do like physical experiments to test if it's correct let's say you have an AI model that creates some kind of material or a battery or something like that then you have to physically test it you can't just I guess you could simulate it but yeah code should be different I agree with that yeah I've tried some basic one in vs code uh field I just get in way a bit yeah still prefer to do

you copy paste yeah I kind of agree these agents they seem to get stuck and it's a hassle to kind of get their back on track but if they keep improving it might get better right I agree with that Nexus what do you think of quantum computers are they going to be in everyone's hand whoa sorry uh quantum computer so cubits and in everyone's hand that might be I don't know if I'm going to be alive

but who knows if like we could use some kind of I guess you can say some kind of AI to speed up the progress of quantum Computing maybe in my generation it's so hard to be a predictor so I don't like to do like these predictions because in the end it's just going to be guessing but it's fun though I hope I hope to have a quantum computer even though I can't really explain what they do

right um uh but yeah I think this was this thread was kind of interesting uh a lot of traction that is kind of interesting I want to see if someone made some responses I wonder if we see a search in pure uh FP very good type system languages like hll because of the testing INF for reasons or whether we fake it in Python land the alpha mat lean stuff uh is one signal towards put more stuff

in types typing should become even more popular also easier to migrate your an type code to type once you have a coding agent typ okay there's an ads here uh FS like taste or Aesthetics will become human input choosing between best solutions from the almost infinite space of generated Solutions we become a differentiator yeah large layoff machine imagine thousands of token per second writing and testing full CHS of code reading stack traes modifying so this is

kind of the the dream right but software engineering has barely been about writing code for a long time it's rare that one needs to spend long chunks of time writing a bunch of code most of the work is about building out the Integrations cost from Technologies and design remain one of the most important aspects of engineering 100% so the IDS will still be valuable right yeah maybe night spider I feel like today's llm are pretty smart

the main problem is they can't handle big task remember things for long because of there's mul Contex yeah meta it's too early I think for big tasks it's not just it's not viable the attention is not there I guess the context wi know is getting there but uh it's going to be interesting but yeah what uh what he said was like even though you can use llms in the future to do your syntax I think uh

the IDS you have will still be valuable right I guess you could get some good IDs from llms but I think IDs will be the most valuable thing right let's say you want to build an app software your ID is going to be the most important part right it's not how you write the syntax right uh but I guess some ideas require very complex code other than uh simple code but if you have like a simple

code it's maybe too easy to copy and you might need complex code to actually have some mode I'm not quite sure uh since I'm getting more into this now like creating small apps and stuff so I don't really have any experience with that part right uh but it is interesting time right so again uh I'm going to make I think I'm going to do either a live stream or a video where we are going to test

out output tokens to see how long of an output we can actually get and then try to look at and then try to look at what uh how the quality is right all of my IDs require complex code yeah right that is why software engineering is not easy right but what if your IDs if you can um explain to like Cloe your ID very well and it could write the code for you that'll be great right

but then again you get the mode issue someone could just steal your ID um but I do think like we can see the what do you call it like a one person unicorn in the future maybe in 10 years let's say we have like this solo entrepreneur that has this company that's worth like is it 1 billion or something I don't know what they call it but unicorn I guess that could be pretty cool if you

can run like a company software company on your own it's a bit optimistic though but maybe I think mid journey is just a few employees maybe like 50 or something 30 50 there are acceptance of course but we're still um creating huge programs or do we smaller applications that communicate with each others yeah that might be interesting too Arian I agree with that maybe there's a new way that applications can communicate using some kind of llm

or some other system so maybe we don't need those big code bases interesting idea I Like It Cool probably sooner to have that solo PR probably sooner to have that solo preneur exist I'm actually working to be this person yeah good luck Brandon me too uh I've been struggling to get a nextjs toui app made by I've been struggling to get our next JS T app uh made with buildin AI for a couple of weeks now

totally jacked my AI progress okay um cool I have this on my website let me see if it's still working so this is something I built into my website where you can upload an image right this is on the back end so you can upload an image and you can ask what is this so this is using G 40 so this is running on Firebase cool AI hello welcome there's a new Anonymous chat Bo by open

ey okay could be CH GPT 4.5 or five that would be cool have you tried it uh okay so you can see here yeah this works so this imagees a pixel art representation of a samurai so yeah it's pretty easy so now I can kind of can we test it in lmc's Arena I might look into that but it's kind of hard to know what kind of model it is so do you think this um AI

features like llms will be on people's website it's pretty easy to set up right so I can upload an image here let's do my thumbnail what is this so this is just running on the back end on my website the image to be a promotional thumbnail for a video or content related to a Super Nintendo style video game created with AI yeah that's true so yeah pretty smooth hello Petro welcome as a member uh you can

send me an email uh if you want access to the GitHub and the Discord just send me an email or just join the Discord you should find the link where you joined nice to meet you so yeah I don't know if You' seen this so this is my website I created this you say in CLA 3.5 tested it uh so that's a new model I guess slightly better than GT40 how do you manage uh token using

your with your website to keep cost from ballooning so I've just set limits on open AI so you can kind of set limits on the I think you can set limits maybe on each API key and I have input limits so in this text prompt I have I think it's maximum two th000 characters or something input and the image can be maximum 2 megabytes but yeah I think the hard set limit on open AI is the

most important one so I can only spend like 20 bucks hello Andre nice to meet you uh but yeah night spider it can be expensive but uh so this is running on gp4 or mini so the text generation is pretty cheap but the in the image part is a bit expensive so what you could do on let's say this was your website and you can just add some some knowledge in the system prompt and people could

ask you stuff right from your website about imagine this was about your product you were selling something and people just ask you stuff right so I think websites in the future will have some kind of feature like this uh I've seen overall limit but I didn't know per API key limit uh I can't confirm that uh but I will check after the video that might not be true so don't take that too seriously yeah so I

also added this uh you can join the newsletter but you have to be able to answer these three questions first so what is rag short for what is the most popular AI programming language python can an llm learn from Context yes so this is like a small quiz you have to be able to answer to be able to join in the newsletter right so this is just a fire based background and yeah pretty simple uh I

did add this so you can support me I just tested stripe uh but it's a bug now so it doesn't work I'm going to fix that uh but yeah pretty cool I enjoyed working on this was pretty fun so it's a react website so it's pretty responsive right and it's just one big page um but enough of that I just want to take a quick look here I'm not going to spend much time on it but

I was hoping I don't know if you guys know do you know if Gro is going to have the 405 model they do have the Llama three 170b right so that works but I haven't seen the 405 model yet write uh python code uh with a snake game so let's see how fast this is now boom was what did I do I didn't see it right 250 tokens per second thanks Andre that's nice appreciate it let's

test this snake game from from llama 317b perplexity has the four mod 405 model okay have you tried it out I haven't been able to try that yet there's an error here hello um hash nurri hi is it possible to teach AI to manage token tokens in dependent so do you mean like set a limit or or just manage how many tokens is going to Output or manage tokens independent if you could explain just a bit

more let's copy the error here I just want to try 7 Tob so add new message and error there's some bugs here what happened now strange uh let's have the temperature lower okay so here's the corrected code that was quick it is fun to play around with this right okay so here is our snake we died okay it's pretty smooth God I I'm horrible at this yeah that's working okay but I haven't really tested llama 70b

too much so I can't really make any comments on it but it is quite fun to use Gro and 8p that is insane fast right um python on snake game for me in colors look how quick this is now it's instant so 750 tokens per second it's pretty funny right I don't think it's going to work though oh so I try to fix it no I give up but yeah I think we covered some stuff today

um I really enjoyed checking out the structured output uh either in a stream or yeah in a video I'm going to be looking at uh the output window extension that gp40 has gotten uh also clawed has gotten that extension uh on the output so we're going to try to see if we can do longer outputs think that could be interesting uh and I have a bench some kind of benchmark I created to test the coding abilities

of different models this is going to be my personal Benchmark uh that's going to test out different models I think that's pretty cool so yeah I got some stuff planned and it's yeah stuff happening all the time so uh I'm going to like I have kind of decided direction for my channel and I kind of decided that it's going to be about yeah code generation building stuff using AI models that is going to be like my

full Focus uh I'm going to leave behind anything that has to do with like images General stuff I'm going to do some AI news but very little so it's going to be mostly focused on using AI to write software to create apps and other stuff around that code generation yeah so that is what I want to do and I think people like it because when do this video people seem to enjoy it yeah thanks nice SP

thanks for tuning in everyone was really fun to do a stream again so I'm going to do more uh I might do one later this week maybe Friday if I have time I will announce it of course first uh and I have some videos coming up so check it out if you want to uh but yeah as always pretty cool to interact with people a lot of good questions in the chat that was very fun and

since I've been a away way for a while it was pretty cool to do a stream again thanks ma thanks for engagement pretty cool and yeah I enjoyed it camel thanks for sticking to the full stream enjoy that so yeah I wish you all a great day and yeah like I said I will announce when I have like a new stream coming up of course Nexus thanks thanks for engagement pretty cool so yeah everyone we speak

soon yeah uh I announced them on my Discord but you can also see them on YouTube when they kind of pop up uh I I will do like an announcement when I'm going to do it so yeah thank you everyone for tuning in enjoy your day and we speak soon bye-bye