TubeSum ← Transcribe a video

GPT-4o Low Latency Image-to-Voice Streaming

2h 04m video Published Jun 7, 2024 Transcribed Jul 28, 2026 All About AI All About AI
Intermediate 18 min read For: AI developers and engineers interested in building low-latency vision-to-voice applications using GPT-4o and OpenAI APIs.
AI Trust Score 62/100
⚠️ Average / Some Fluff

"Mostly delivers on the low-latency demo but includes lengthy chat interactions and model comparisons; feels more like a casual live stream than a focused tutorial."

AI Summary

In this live stream, the creator builds a low-latency system using GPT-4o to analyze screenshots and convert them to speech, aiming for minimal latency before the new voice mode release. He demonstrates a pipeline that resizes images to 512x512, uses low detail mode for fast responses, and outputs audio via OpenAI TTS, achieving around 5-second latency at very low cost.

[03:48]
Goal of the Stream

The creator aims to test GPT-4o's API latency on images by building a system that takes screenshots, resizes to 512x512, analyzes with low detail, and outputs voice.

[12:41]
Low Detail Mode Explained

Using low detail mode limits the image to 512x512 with a budget of 85 tokens, resulting in faster responses and lower cost.

[20:06]
First Script Test

The script takes a screenshot, resizes it, sends to GPT-4o, and prints the response. The image size is only 23KB.

[26:17]
Adding Text-to-Speech

The creator integrates OpenAI TTS using the 'speech' endpoint, saving audio as WAV and playing it with simpleaudio.

[43:15]
Real-World Demos

The system is tested on Reddit posts, Paint drawings, and other on-screen content, answering questions and describing images.

[46:10]
Pipeline Recap

The full pipeline: take screenshot, resize to 512x512, analyze with GPT-4o (low detail), generate text, convert to speech via TTS.

[53:00]
Impressive Vision Capabilities

GPT-4o accurately reads and interprets very small images (23-30KB), including handwritten text and low-resolution screenshots.

[01:28:07]
Latency Measurement

The latency from screenshot to voice output is approximately 5 seconds, with improved performance later in the stream.

[01:39:14]
Cost Analysis

Total cost for 2 hours of streaming: 20 cents for GPT-4o and TTS combined, demonstrating high efficiency.

[01:41:07]
Model Comparison on Logic Problem

Tests the 'sister problem' (Kaye has three brothers, each brother has two sisters - how many sisters does Kaye have?) on various models: only Qwen 2 72B gets it correct.

The low-latency image-to-voice system is functional and cost-effective, with GPT-4o's vision performing exceptionally well even on tiny images. Further improvements like pixel-change detection could enhance responsiveness.

Mentioned in this Video

Tutorial Checklist

1 00:18 Set up project: gather OpenAI API documentation and pre-existing functions.
2 10:51 Define prompt: use Pillow to screenshot, resize to 512x512, save as JPEG with low quality.
3 12:41 Use GPT-4o with low detail mode for fast analysis and lower token usage.
4 20:06 Write script to take screenshot, send to GPT-4o, and print response.
5 26:17 Add text-to-speech using OpenAI TTS (speech endpoint), save as WAV, play with simpleaudio.
6 34:08 Debug: ensure OpenAI client is initialized correctly and handle binary response for audio.
7 43:15 Test with various prompts: answer questions on screen, describe images, use different languages.
8 58:19 Suggest improvement: use pixel-change detection to trigger screenshots only when content changes.

Study Flashcards (10)

What image size and detail setting did the creator use for low latency?

medium Click to reveal answer

512x512 with low detail mode.

12:41

What Python library is used for playing audio?

easy Click to reveal answer

simpleaudio.

31:19

What is the token cost advantage of low detail mode?

easy Click to reveal answer

Fewer input tokens (85 tokens budget) and faster response.

12:41

How much did the API cost during the 2-hour stream?

easy Click to reveal answer

20 cents total.

01:39:14

What is the latency achieved for the image-to-voice pipeline?

medium Click to reveal answer

About 5 seconds.

01:28:07

Which open source model correctly answered the sister problem (Kaye has three brothers...)?

hard Click to reveal answer

Qwen 2 72B.

01:41:07

What file format was used for the temporary audio file?

easy Click to reveal answer

WAV.

31:19

What was the image file size after compression?

easy Click to reveal answer

Around 23-30 KB.

21:55

What is the name of the Python script created?

easy Click to reveal answer

box.py.

47:17

What did the creator use to get documentation for the OpenAI API?

medium Click to reveal answer

He uploaded the documentation directly as context.

12:28

💡 Key Takeaways

🔧

Low Detail Mode Efficiency

Explains how low detail reduces tokens and speed, a key technique for cost-effective vision.

12:41
💡

Full Pipeline Description

Clearly outlines the entire workflow from screenshot to voice output.

46:10
📊

Impressive OCR on Tiny Images

Demonstrates GPT-4o's ability to read handwritten text on 23KB images, highlighting its robustness.

53:00
📊

Measured Latency of 5 Seconds

Provides a concrete latency metric for the pipeline, relevant for real-time applications.

01:28:07
💡

Cost-Effectiveness

Shows that continuous 2-hour usage costs only 20 cents, making it feasible for production.

01:39:14

[00:18] okay so hopefully I'm live now it's a bit uh hard to tell sometimes um

[00:30] I'm just going to watch the chat to see if I see something let me just test if I see something let me just test it because I can see someone

[00:47] could just give it like a couple of minutes like if people wanted to join I don't know like I said in the previous video I said in the previous video I'm not so used to streaming okay good

[01:01] so let me know if it's too loud right because it's uh hello all hello is right because it's uh hello all hello is I love self in it I love self in it Paulo Mind Over Matter Ramy Tyler nice

[01:17] to meet you all so I guess kind of the okay all so I guess kind of the okay good nice hello trendy good nice hello trendy tals Lord de Romano nice to meet you so

[01:30] I'm going to try to focus on the chat while I'm doing stuff but like in kind of the main topic where we going to do what is on the screen here I thought we kind of have to hello Paul Hello jinnie nice to meet you

[01:44] all we kind of have to just do it in I'm going to try to answer some questions going to try to answer some questions and stuff uh while we do this um so yeah uh I'm I'm just going to leave this for a while um just give it a few minutes

[01:58] before we actually get into it so I don't know if anyone checked this out uh I tried to pull this hello R

[02:13] out uh I tried to pull this hello R Ramy I tried to pull this quen 27b but for some reason reason I I can have has anyone tried

[02:29] strange right

[02:45] this I don't know if you can see but I really wanted to try it

[02:58] anyway uh thought uh I just wanted to ask a few logical questions so see I get this error here but I haven't really looked into it hello Tom uh but if I run this with uh llama Tre it should work fine

[03:14] right so that was a bit strange but the main part today is of course going to be uh to uh hello Alberto nice to meet you I really wanted to so you can see this works Kay has

[03:31] to so you can see this works Kay has Five Sisters that's wrong uh so I think we're just going to go back here and I kind of just wanted to start here and I kind of just wanted to start by explaining what I wanted to do with

[03:48] so basically I've been looking into the open AI API for gbt open AI API for gbt 40 uh how fast is the latency really look at today Matias because I kind of wanted to see

[04:03] how fast we can get this um uh hello Dominic if we take a um uh hello Dominic if we take a screenshot and we resize it to 512 by screenshot and we resize it to 512 by 512 and we use GPT 40 to analyze it and

[04:19] we're going to use this the lowest setting we can use to try to get it as setting we can use to try to get it as fast as we can hello they D us

[04:41] yeah I'll look into that later but yeah back to this so basically what I wanted to try is to see how low we can get this latency now before we get the new voice latency now before we get the new voice mode uh hello Ricardo Hello regg nice to

[04:54] meet you all so basically I've done a few tests of this and it seemed pretty decent um I was really impressed how good it can analyze images that are like

[05:06] 30 kilobytes and it makes it much faster and we're going to turn it into just text to speeech we have a prompt and we're just going to comment what is happening on the screen so that is kind

[05:18] of my ID we're going to go through the the specs right kind of how I wanted to do this uh like the description of the project so this is just the visual description and uh we're going to use

[05:34] GT40 to actually write the script uh but I think it's going to work pretty good so just wait I'll thought I just

[05:46] wait a couple of more minutes before we start like if people are just people maybe just saw we went live so has anyone tried using like the lowest

[05:58] anyone tried using like the lowest quality on the GP 40 API or something if quality on the GP 40 API or something if you did you try out uh and one more thing I wanted to do uh I wanted to kind of check the

[06:12] do uh I wanted to kind of check the pricing so hope I don't show pricing so hope I don't show anything uh so today is it 7th so far we anything uh so today is it 7th so far we spent uh

[06:30] much we are going to spend using spend using this monthly spent $1 so not much so remember 0.28

[06:52] support uh no uh I'm going to show you we can't really use gp40 for the speech part yet uh so that is kind of what I'm looking forward to so we have to use kind of the old

[07:07] old um the old text to speech that is using kind of we have to translate it into text right and then translate it into text right and then back into speech so that's a bit

[07:20] have to do it now uh so remember to help me check what now uh so remember to help me check what the cost is going to be when we run this

[07:33] into it uh I had a few more things but we can do that after I guess so I guess we have to open a chat GPT window

[07:50] basically I thought we can just this answer intuitive up to this point do you think gp4 o is reliable cheap Che everyday tool to use for your own applications uh expensive yeah I don't

[08:07] know it's kind of I think it depends on how many tokens you're using like if you're creating an app it might be I'm not quite sure about app it might be I'm not quite sure about that but uh I think it's pretty cheap

[08:21] that but uh I think it's pretty cheap for the at least if you want some um uh wish model I think it's clearly the best Vision model I have tried and I

[08:33] think it's pretty cheap but I think it depends on your use case if you're going to if you're doing something on the public side I don't know I'm not quite sure yet I saw some post on Reddit or something that one guy

[08:49] was relying solely on uh relying solely on API the open AI API for their business and I think that's that's a bit risky

[09:01] window right yeah I don't know I haven't really maxed out I don't know what about you guys but do you know what about you guys but do you struggle with filling up the context

[09:26] or something you're running through that then can get expensive if you have these big uh input prompts right I think so okay so let's just get started here we can do questions and so while we get

[09:41] started so uh I did some uh prepared I prepared something some uh prepared I prepared something here right because here right because um let me just so I wanted to prepare

[09:54] um let me just so I wanted to prepare something let's just take people building with amazing to me more people aren't building with a yeah

[10:08] people aren't building with a yeah maybe uh hello hello log up in chat play When the chat gets longer for coding assistance yeah yeah that's true okay so let's just get started so the first thing I wanted to do right

[10:24] the first thing I wanted to do right um was to get get some documentation right for our project so documentation right for our project so the prompt I wanted to use was

[10:38] this I don't know if you can see this so I just want to use like I upload some documentation of a script we want to build so this is kind of what I planned build so this is kind of what I planned out I wanted to create today so we want

[10:51] to try to get the lowest possible latency when analyzing images using the gp40 model uh we want the settings to be the lowest they can like row low rest and low detail and I have been using like

[11:04] the pill image SCB to do screenshots before and do it like maybe every 3 second or something we'll see and I want to a JPEG file to get it like the smallest size and we want to resize the image to 512 x 512 and we s save it to a

[11:23] temporary save it as a temporary image to a folder name images so that is kind of the first step I wanted to do but for this I wanted to grab some documentation this I wanted to grab some documentation we can

[11:39] upload and let's just start by doing that so let's go back here the documentation I kind of wanted for This was um I want to go to models right we want to use GPT

[11:55] to models right we want to use GPT 40 I don't know if you can see this and we want to use explore gd4 o with image

[12:11] inputs so I thought we can just grab this

[12:28] need this I think what we want to focus on is kind of the here it is so I don't know if you check this out there's something called check this out there's something called Low will enable low rest mode so the

[12:41] model will receive low rest 512 by 512 vers of the image uh so this allows the API to return faster responses conf consume fewer input tokens uh so it doesn't require High

[12:56] detail that is kind of the low setting and the high setting is higher image quality right so that is kind of what I wanted to try out today I think for o those the resizing

[13:10] for you for minimum price if you look at the yeah that might be actually true they have something called Auto so I haven't really looked into that yet but let's just grab some documentation here right I want to grab this I think here's

[13:25] right I want to grab this I think here's something about managing images too so let's just so this is kind of what I like to do just grab some documentation I think we need all

[13:43] good and from a previous project uh I think I have I think I have some functions here I want to reuse so this is kind of something I use

[13:56] in a different project so this is just to I don't know what's wrong with my mouse here that's a bit bit strange just some functions I have used

[14:08] strange just some functions I have used before just to speed it up a bit so I just want to grab this too okay paste it in here and let's say it's score I usually go like this U I don't know working python or

[14:24] this U I don't know working python or something doesn't really matter I think because I kind of reuse these functions just to save some time they are pretty basic they are kind of already from this uh documentation

[14:39] of already from this uh documentation here right so it should be pretty here right so it should be pretty good okay so I think that's good enough good okay so I think that's good enough right so what I like to do next is kind

[14:51] of just go here I want to try to use J BT 40 for this sometimes I use clae or BT 40 for this sometimes I use clae or other stuff and I think I set my system message to just Euro software there with expertise in Python I don't know if it

[15:06] habit I have and let me just upload this uh documentation file right and let's go grab our prompt for

[15:20] the project project um so let's start with this right

[15:35] on this but uh just a quick first iteration

[15:59] just want to look over this quickly right I don't know that looked a bit strange this a file uh I kind of want to

[16:16] check out that file just give me a second here

[16:34] think I picked the wrong documentation here so let's try here so let's try this right okay so we are missing the open a I import

[16:59] path I don't know if you can see this very well very well but uh

[17:11] okay yeah so I want to remove this image path after we use it right um okay so let's copy this let's just

[17:23] um okay so let's copy this let's just go here paste it in here so I have my I prepared my open a key I just left it in a EnV here so I key I just left it in a EnV here so I don't have to show it on stream right so

[17:37] [Music] for let's just ask for something like um for let's just ask for something like um I have my API open a API key in uh EnV

[17:55] key in uh EnV file uh I want to I want to use let's to I want to use let's say

[18:09] uh let's call let's say EnV file in a V named open AI API key maybe or something like this to

[18:23] this to fetch the API key fix this should work I already have St installed

[18:41] this yeah I think this should work hello Daron nice to meet you uh yeah I think so okay so let's copy this paste it in here

[18:56] I saw we kind of need uh import open AI

[19:10] right quent came out yesterday 72 B ver seems better than llama yeah I saw seems better than llama yeah I saw that I'm not uh I don't have any I don't have the hardware now to kind of run uh llama 7 Tob

[19:22] locally but I'm definitely going to try out the new quen model maybe on Gro or something if it comes over there uh Francy it's often incorrectly writes the new open ey method yeah I've seen that it always uses the old one that's

[19:37] why I kind of like to grab some documentation uh before I continue when documentation uh before I continue when I'm using the chat completion part right well let's take a look here now uh we have

[19:50] have this we are going to resize our image so we're just going to do the query what's in this this

[20:06] screenshot if you're going to get a response and just print it for now I want to add some voice to this after so that's going to be the next step right fireworks host quen for okay that's pretty

[20:19] cheap uh did they roll out the real time streaming with voice responses uh I don't think so yet no I'm hoping we can get that

[20:31] soon yeah they said a few weeks and what what is a what is few down nice to meet you okay so let's just test this now so

[20:49] hopefully let's just I'm just going to run this from my terminal I run this from my terminal I think uh

[21:07] front of me right now okay so what do I call it box I just now okay so what do I call it box I just took um so what we want to see now is of course hopefully you can all see this if it works so let's just try

[21:23] it works so let's just try it so we have this folder here called images so you can see we took a screenshot and then we removed it so it seems to be working the image shows a computer screen what appears to be code

[21:39] editor open okay I wanted to take a look at this at this image because it's look how small it is image because it's look how small it is it's just it's very low

[21:55] resolution so I'm kind of impressed how good it is reading this low resolutions good it is reading this low resolutions so the size is only 23 kiloby of the images so let's just stop this so this

[22:09] is working quite good kind of bad resolution though but it is working so let's just stop it for

[22:23] now because I have a prompt I wanted to try right

[22:43] this so the prompt uh I want to use today um if you see a question on the screen answer it keep it short and conversational so this is kind of for the when we're going to add the voice part

[23:00] let's try it in here

[23:22] so so this is a question right let's run it so the reason I cre this was I thought it would be cool if we just scroll around the web and if we see a question we can get like a voice to

[23:34] answer this uh how much does it cost to use the chat GPT API uh yeah we're going to monitor that today I think we just going

[23:48] to we're going to see it's not too much they have reduced their prices like a it's it's not slow but I think we have like

[24:05] slow but I think we have like a a Sleep part let's just remove a a Sleep part let's just remove that let's remove this

[24:24] if it's I'm streaming or something I'm not quite sure not quite sure seems kind of to be stuck here

[24:36] five okay yeah I guess it depends on your use case so this seem to crash that was a bit strange

[25:01] so I think the next part we're going to do is actually just add the voice and do is actually just add the voice and then try to use

[25:24] here I'm just a bit this chat thing I'm not so used to this just let's try to implement the voice part I wanted to

[25:46] uh so I like to do like even though I have the code here we made some more changes right so I just like to paste in my code again

[25:58] like to paste in my code again and what I want the text to speech to read out is this variable here response

[26:17] right so I think I'm going to I think I'm going to grab some more documentation so let's just do open AI docs equals and let's go back here uh go to the documentation let's go to text to speech I

[26:35] to text to speech I guess we can maybe okay and let's grab this part

[26:54] maybe quick start because we want this python code here like a w

[27:08] a w a so we can grab that you're using llms for coding I like to use loads of documentation uh I just think that's make makes things much easier so I just

[27:23] make makes things much easier so I just use the documentation as context right okay so I I kind of want an example of how we can change the format so I think if we go to API reference and yeah I guess it's audio

[27:37] create speech I know there's something yeah there's something called response format response format here uh I guess we can just grab all of

[27:51] here uh I guess we can just grab all of this and just do it in in our context so now we kind of have our code in our context we have some documentation and now I kind of want to give my instructions

[28:10] like let's add [Music] [Music] a feature maybe to feature to read out

[28:23] uh maybe the V response using

[28:42] chat too uh is there an open source AI assistant for visual studio 2022 uh oh I don't know that maybe 2022 uh oh I don't know that maybe someone in the chat

[29:04] the variable response from our code using using TTS uh I want uh I guess to play

[29:31] play the or something like this and we can save the I want the

[29:48] temporary audio file Maybe audio Maybe audio file in a folder named I don't file in a folder named I don't know a AIO maybe that should be

[30:01] good and for this I usually I've been just just using maybe use uh simple audio to play

[30:15] the file uh I think we need the file to be in VI Not

[30:33] MP3 something like this can you implment first iteration of

[30:46] documentation and we have our instructions

[31:06] you were leaving in Ulta up there though that's cool okay so let's see here now so we have

[31:19] the we have the function text to speech okay so we get the response speech okay so we get the response format. VI uh okay that looks fine do we have like a folder path

[31:37] fine do we have like a folder path yeah

[31:56] it like simple audio that has always worked pretty well for me when playing worked pretty well for me when playing this files

[32:27] again we might get some errors here I don't know uh okay so we have an error here uh line

[32:39] 85 line 67 okay so we have an open AI key where was that 67 okay we don't need that I

[33:13] debugging with the GPD 40 we can try that I think I know what's wrong but let's just try some just try some debugging wow pure science thanks a lot

[33:27] that's that's very yeah thank you a lot it's not necessary but wow

[33:54] Norman okay so let's do some debugging that could be interesting let's just do my code equals and let's do error

[34:08] equals and just grab this so let's see if we can figure this out using

[34:48] you the error indicates that client variable is not defined in the text to space function uh okay you need to initiate opening initiate opening Cent so let's see if we can debug this

[35:02] sometimes when we debug with chat GPT it can go very wrong can go very wrong right so open audio create H let's take a look at the API here I think it should be something like

[35:18] this to be honest let's just copy that but first let's try try this I don't think this is going to work right because uh it just makes up

[35:35] another client [Music] [Music] error at least I think

[35:47] so maybe not yeah I don't think that's correct so let's try to just fix it ourselves let's just grab okay here let's try

[36:02] that if this is not going to work let's feed some more documentation and feed some more documentation and ask ask again so let's try

[36:21] this okay thanks for tuning in have a great day blizzard

[37:10] it I was wondering if we can see kind of our cost here so we spent three cents so

[37:24] case uh I don't uh I don't know I don't uh usually do that but uh you can of course try that have you have any success with

[37:59] screenshot so that is working right look at the low the resolution is okay so let's just try to finish up the the debugging and see if we can solve

[38:13] debugging and see if we can solve it let's just copy this let's go here my code just want to give like gd4 o a

[38:25] code just want to give like gd4 o a chance to fix this object from the TTS API call uh is not a dictionary but rather an httpx binary

[38:41] response content object you need to correctly hand a object you need to correctly hand a response to save it as a vow

[39:05] anything let's give it a go I thought it was just some minor changes here

[39:22] significant role in the future of computing computing okay I don't know can you hear

[39:36] that yes AI is likely a big part of the future of computing uh in Spanish

[39:49] okay yes AI is a big part of the future of computing I guess we could do like a print statement here

[40:09] okay so that worked let's add a print statement

[40:28] though um text to

[40:44] uh let's should we print let's print this I don't know my computer is maybe a bit laggy I don't know I'm not quite

[40:56] bit laggy I don't know I'm not quite sure might be some sure might be some uh might be some uh might be some um rate limiting too I don't

[41:12] get this to work we can just yes AI is considered a significant part of the future of computing okay so

[41:24] computing okay so Works in the future of computing you think he can do

[42:39] noran only I can understand it and maybe ISAC too H for datl that's Danish okay so let's stop it for now so

[42:51] I wanted to I kind of wanted to browse let's go to I kind of wanted to browse let's go to Reddit or or

[43:15] question on the screen is build versus buy answer it depends build for customization buy for Speed how was the accent it was Danish

[43:36] possible anyone else have experience with

[43:51] Reddit showing a post discussing Steven pinker's view on AI he Likens AI models to a form of alien intelligence and mentions that experiments on them are mentions that experiments on them are akin to studying a newly discovered

[44:10] impressed by is actually how small the images are yes there is you can look into mattermost and zulip for open source chat options in source chat options in go so they are just how big are these

[44:23] images again 30 kilobytes

[44:37] this sure let's see if I can help with that whoa the question on the screen is is there something similar to Steam Chat written in go goang that's open source answer as of my last update mattermost and zulip are open- Source Alternatives

[44:53] although they may not be written in go you might want to check Repository like GitHub for go specific chat Solutions I just want to hear what it Solutions I just want to hear what it says about this chart

[45:17] narrower domain of AI R&D as we apply super intelligence to R&D in other fields explosive growth will broaden because it didn't we didn't do any prompts so it didn't say anything

[45:39] use cases for this I guess it's very passive we can't passive we can't really uh yeah I can I can explain that really uh yeah I can I can explain that just quickly give me a

[45:57] so just a quick recap what we wanted to try try today uh was basically this so we wanted today uh was basically this so we wanted to try this low resolution

[46:10] to try this low resolution screenshots uh screenshot resize it to screenshots uh screenshot resize it to 512 use gp40 to analyze it and turn um turn it into text to speech and yeah just see how fast we can

[46:25] actually use this before we get uh yeah the voice is open AI too so we are using the API now let's go to paint or something and let's let's try to remove this sleep here maybe we can get a bit of a

[46:41] here maybe we can get a bit of a better let's just do something

[47:03] opened a python file named box. py how can I assist you with your python can I assist you with your python project what is

[47:17] application with the question what is this typed on it do you think it's going this typed on it do you think it's going to answer this

[47:37] drawing possibly of an animal's face or nose animal's face or nose it looks like nose animal's face or nose it looks like a basic drawing of a cow's face

[47:57] face it looks like a simple drawing of the head of an animal possibly a goat or the head of an animal possibly a goat or a

[48:10] animal head maybe a dog or a cow let's try some R you

[48:24] you s sent

[48:53] it's just like yeah there's some kind of lag because it's very

[49:17] message we could have it doesn't seem to time out is a great wish model a bit

[49:32] annoying uh that it kind of stops running I don't know what's going

[49:47] designed to assist with information and tasks yeah Thomas I will put the code up on uh Community GitHub that was my plan anyway okay so GitHub that was my plan anyway okay so it could read

[50:03] this but uh we are definitely having some issues with the

[50:17] should be more than fast enough for this but there are some issues here with this but there are some issues here with uh it's lagging

[50:41] annoying because I wanted to try it out with some other things but with some other things but uh we can just wait a while and see if

[50:57] be sentient my response I can't become sentient but I can help with your

[51:11] queries maybe no I'm not sentient I'm just a computer program no I'm not sentient I'm just a computer

[51:34] no I'm not sentient I'm just a computer program designed to assist

[51:51] Consciousness so I'm not sentient I just process information and respond based on process information and respond based on my program

[52:03] in let's ask okay noted what box is

[52:18] biggest sure what would you like to know about this behind the Red Box is the biggest

[52:42] great wish model because the images are very small right so I'm the red box is the biggest I'm pretty impressed by it the black box I'm pretty impressed by it the black box is the biggest

[53:00] try this with um with uh the new model if we get the response time down to like5 seconds or something that would be like5 seconds or something that would be pretty cool

[53:19] right so does anyone have any other suggestions of what we can do with suggestions of what we can do with this um eager to how fast does it take a take screenshot that's pretty quick that's kind of

[53:33] instant I just have it in like this Loop here so it just takes a screenshot analyzes it print the response turn it into text and place the text and removes

[53:46] the image and the audio and just screenshot immediately screenshot immediately after that was kind of the idea at least

[54:06] cool let's ask uh let's change up the prompt a it suggest an

[54:25] the code but I guess you can't see the full code but I guess you can't see the full code let's just give it a

[54:42] function I making it a little smarter such as the way watch is a portion of your screen and wait until there's nothing yeah uh I don't know if you nothing yeah uh I don't know if you catched my video uh where I did this pix

[54:58] catched my video uh where I did this pix uh was some change in pixels it didn't take a screenshot until then I think we used a screenshot until then I think we used open CV or CV2 for it that worked pretty

[55:12] good would be interesting to store a history of screenshots and then see if you can get it to remember something for you yeah that's a good idea uh I might

[55:24] look into that grock models doesn't have wish Lama mistol uh I don't think I see grock models with wish yet maybe

[55:39] soon does the image size matter regarding the wish API cost yeah I think it does if we take a look here on the documentation you can see if we go to documentation and wish

[55:56] and if we go down here you can see low the model reive low rest 52 x see low the model reive low rest 52 x 512 uh with a budget of 85 tokens API to

[56:10] return faster responses conf consume fewer input tokens so that means we are fewer input tokens so that means we are saving money right fewer input

[56:24] really calculated cost per hour or something calculated cost per hour or something but uh we started today with we have but uh we started today with we have spent we have spent 10 cents so

[56:38] spent we have spent 10 cents so far so it's definitely not far so it's definitely not free I don't know how much we have every second let's say 10 seconds Stitch the 10 images in a series yeah that

[56:55] works great with this um base um base 64 this way can reserve or yeah that's a great have you tried that pure science that is pretty

[57:10] cool how can you get it to be trained on a set of images and then you provide a new image how can you get it to be trained image how can you get it to be trained on a set of image New Image well I guess

[57:23] that has to be kind on the provide a new image and okay I guess you can compare images that should work if you have an old image you can just compare it and check if it's the

[57:38] same that should work web kit 2 pngs you'll be able to take the whole image of a website okay that's interesting web kit 2 PNG I haven't try that that could be cool so you can do it in the background

[57:54] cool so you can do it in the background or just from the T terminal or does it or just from the T terminal or does it have to be in the

[58:19] just suggest an improvement to the code keep it keep it short yeah let's just screen shot this text to speech function here but we definitely have there are

[58:32] some issues with this uh I'm not going to go into what's wrong now but uh to go into what's wrong now but uh definitely some

[58:45] issues right uh so let's hear if we get any suggestions okay so you can run it from a termal that's pretty cool uh Chain of Thought sequence yeah chronium

[59:00] or selenium can do it yeah senica I've tried both of those let's hear if it has any suggestions for this uh

[59:15] don't know what's going on I haven't tried API for a while so maybe there are some if it shouldn't be shouldn't have anything to do with with me streaming what id I usually use vs code but

[59:31] for for videos and stuff uh I just like to use notepad++ just because of the visuals right there's definitely something wrong right there's definitely something wrong here so I usually use vs

[59:45] here so I usually use vs code with uh some um auto complete and code with uh some um auto complete and stuff but stuff but uh let's give it one more

[1:00:03] what's happening 12 Harry Potter in 12 Harry happening 12 Harry Potter in 12 Harry Potter books in uh and out is sure

[1:00:26] thing uh $20 12 Harry Potter books in and packing that has to be it because this is not

[1:00:41] working okay so let's just end this for now and let's we can try back later and see if it is working I don't know if you saw this but I made this

[1:00:53] saw this but I made this uh I made this uh uh I made this uh Powers shell I don't know what you call

[1:01:05] it it's almost like a profile so the first thing I did was this autocomplete suggestions I did a video on it but maybe you didn't see it so it's SG this is just cached all of the files is cached like

[1:01:21] but I added some new features too let's blow it up a bit blow it up a bit so we can do I think it's ask open

[1:01:36] AI how to create a file in Powershell so we can just ask open AI I Powershell so we can just ask open AI I think it's chat GPT 3.5 in the think it's chat GPT 3.5 in the terminal you can create a power using

[1:01:48] item okay so we can basically ask anything I anything I guess um

[1:02:00] what is the highest building in New

[1:02:19] okay so that is one feature I think it's pretty neat for something like I did a video on this but I don't know if you have used

[1:02:36] to speed up uh video in FFM up uh video in FFM Peg so these are very annoying right

[1:02:48] Peg so these are very annoying right this s FFM Peg bash commands or shell a terminal we can copy it and clear this

[1:03:00] and when we have pasted this instead of clicking enter I can do control enter and this is going to send this to open Ai and explain what it open Ai and explain what it does this command uses fmeg to change

[1:03:15] the playback speed of a video takes an input video video takes an input video and time step half of original values okay something something so yeah this is pretty neat I use it

[1:03:30] sometimes appreciate the live stream format oh thank you for that uh set proxy I have been building I was trying to build some pretty simple thing testing out kind of the the low latency part of the gbt 40

[1:03:47] model wish part where we can use this small images and yeah seems to be small images and yeah seems to be working pretty good like this this command changes the current

[1:04:02] directory to a directory name python so for me that's not so used to operating in the terminal I kind of I use this sometimes so we can create like

[1:04:15] a new uh folder here

[1:04:29] neat I did a video on it and I think I put up the code yeah let's take a look at that I've seen I've seen on

[1:04:44] [Music] the LM sis

[1:04:57] Arena the Gemini 1.5 has been creeping up here been creeping up here right can you manually create a scratch right can you manually create a scratch g2f maybe for only available

[1:05:15] for uh available command line I'm not quite sure about that okay so if we take a look at the latest update here you can see Gemini

[1:05:30] latest update here you can see Gemini Advanced is on second place Gemini 1.5 Advanced is on second place Gemini 1.5 Pro API is on third second shared 1.5 Pro is on fourth so Google kind of now has three of the first five

[1:05:47] kind of now has three of the first five models I was a bit surprised by that to be honest the thing is

[1:05:59] quento works well yeah I'm going to try that out the thing is I live in Norway I don't have access to the Google API that's why I haven't used that's why I haven't used it but uh I have been talking to Google

[1:06:14] and we're going to do something so they are going to give me like hopefully they will give me API access yeah I know self in it that's why I haven't used it but I had like a meeting with Google and uh

[1:06:29] we're probably going to try to make it happen they are not quite sure yet to be honest but if I'm going to do something with them I kind of need access to the API so I think they're looking into it but we will

[1:06:42] see it's a bit annoying uh if uh I'm going to do videos on apis and maybe like half of the people watching can use it

[1:06:55] that's not fair Chris you should be able to yeah it's a bit annoying because now I can't test it but like I said I had a meeting with Google and they might help me out can you test F3 wish

[1:07:11] model I might do that soon I might do that I did do a video soon I might do that I did do a video where I kind of

[1:07:24] tested tested uh some of these open source models impressed can you tell me about the best coding model

[1:07:36] coding model when uh code Quin is released already the newest one I'm not quite sure but for the best coding model

[1:07:50] I have tried is I think it has to be uh code stroll for code stroll for me I haven't tried the big ones so Cod

[1:08:02] stroll is uh 22b it is released okay yeah I haven't tried it yet but code stroll I think that's pretty good code stroll I think that's pretty good that is what I'm using for my um auto

[1:08:22] so too Nvidia free fire wish okay uh we can too Nvidia free fire wish okay uh we can also take a quick look I also take a quick look I have uh let me see if I can access it

[1:08:52] in okay so I guess I can log in I'm not quite sure yeah I'm logged in now so this is the new Nvidia thing let's take a quick look at it so can you test several llms for ability to work with

[1:09:10] pictures we have been doing some of that today but kind of my initial response is GPT 40 is far ahead I think for now for what I have been trying but uh

[1:09:32] others are going to come soon so here we have F3 Vision

[1:09:45] 128k so I thought we could maybe upload an image here if we

[1:10:08] small but I can't really zoom in what does this function do so this is the 53 wish model

[1:10:34] speech all right so I guess you could read that like I said I haven't really tested this so

[1:10:51] have been using Nvidia AI workbench and it's pretty dope yeah so I'm going to do like a video on this new Nvidia um Vim Nim sorry that you can create like a Docker

[1:11:06] sorry that you can create like a Docker uh container with source model so that's going to be I guess it's in like 10 12 days can we use

[1:11:18] this via API for free I don't think so so this is just something I got from Nvidia I got th credits just to test it credits just to test it out but I haven't done any video on it

[1:11:33] yet let's screenshot something screenshot something here and let's ask if it can do like

[1:11:52] right save this here may images

[1:12:25] summarize this Reddit post I don't know if you can see this but let's try

[1:12:43] I'm going to let you know when I'm going to do that it seems to be doing pretty good at OCR discusses the important of API privacy cost considerations uh okay I might have to look more into

[1:12:58] this five wish 128k then ads coming on stream okay that might happen like I'm not so good at

[1:13:12] streaming so I don't know kind of how I set it up I just use kind of the default YouTube thing I was going to use restream but I couldn't get it to work this time so maybe there are some ads I'm not sure

[1:13:29] on this Nvidia playground what you call it so we Nvidia playground what you call it so we have llama 7B Mist have llama 7B Mist large

[1:13:45] design stable [Music]

[1:14:01] I had it a very serious hallucination so that is uh Josh D is the F3 Vision

[1:14:17] we're going to look into this later some rag rag pipelines gaming audio to face I don't really know what it is this

[1:14:32] face I don't really know what it is this is let's try

[1:14:46] more Leisure for yachting but her Publications are no good H okay yeah we're looking into to this later I haven't really prepared for this so yeah um okay so let's go back here

[1:15:05] so yeah um okay so let's go back here and see if uh we can actually try this for once to see if it

[1:15:43] the code nothing's happening I don't know but like you said someone was mentioning some issues with the gbd4 O

[1:15:56] wish is it possible to do multiple agents with uh Nvidia uh multiple agents with on Nidia too I can't really answer that now I'm

[1:16:08] not quite sure uh but maybe you have access to the code right so if we go here you can see we have a llama 72b we can we can just turn this into a

[1:16:21] 72b we can we can just turn this into a python Cod right so I guess we could python Cod right so I guess we could what is the speed here it's pretty

[1:16:34] [Music] the DP uh the F3 wish model then the DP uh the F3 wish model then maybe desktop GPT is Mac only yeah has anyone tried their open AI desktop

[1:16:51] AI desktop um app like these agents whoa I don't think they're going

[1:17:07] to work until we get like a very strong model right uh hi Chris love the channel starting at live stream is great have starting at live stream is great have you seen Auto gr with generates complete

[1:17:20] agentic projects uh no I haven't Auto Ro okay I might take a note of that that's pretty

[1:17:36] cool have you tried live kit for Real Time conversations and video no live kit Time conversations and video no live kit I don't think so is it good Auto gr is I don't think so is it good Auto gr is dope let me search it

[1:18:11] demo small tuber has made it okay cool it seems

[1:18:26] chat autog better let's see how long this video look ad let me know if the sound is too

[1:18:41] ad let me know if the sound is too loud yeah flash we have like um loud yeah flash we have like um Community YouTube members Discord I had like a public this code converts text to speech and saves it as a wave file using

[1:18:53] speech and saves it as a wave file using API calls wow it's still running right API calls wow it's still running right that was screen is asking if a certain code converts text to speech and saves it as

[1:19:09] a wave file using API calls that's annoying is that annoying is that correct I'm just going to end again sound is perfect okay let's take a quick quick look at it

[1:19:25] okay let's take a quick quick look at it auto gr if this is the correct [Music] one we're going to run autog gr locally for today's demo and you'll see we're configured to use chat GPT autog gr will

[1:19:38] always remind you which Pro okay so I was thinking Auto Gro that it was um using

[1:19:52] up here in the title our new feature eats tokens like Pac-Man munches dots and we bump up against grock's developer limits if we don't use open AI instead limits if we don't use open AI instead still gp-4 will only yeah the rate limit

[1:20:05] of Gro if you have access it's pretty cool Gro is awesome but the rate limit is horrible charge me a couple cents for this demonstration and you're worth every penny yeah both of them ignore him

[1:20:20] this week's added feature is a shiny new checkbox I know right You're Pitiful fortunately I have time to explain since GPT runs so much slower than grock we've added a built-in moderator function that helps keep our

[1:20:34] AI agents focused on the goal and productive during our conversational testing clicking this new check box activates an internal moderator bot who will analyze the goal the scope of the conversation and the previous

[1:20:47] remarks let me just stop here for a sec okay so here we have the age chance so we have a project [Music] manager SEO specialist yeah I've seen this kind of tools before they are

[1:21:03] this kind of tools before they are pretty cool like crew Ai and uh yeah hopefully groco live soon with a f with a full-fledged service yeah I think

[1:21:15] we actually need to pay for it if it's going to be good to pay for it if it's going to be good yeah it's free but the limits yeah

[1:21:30] okay so it's yeah I can see it now autogen crew AI right

[1:21:44] screen okay so I guess I have overlay the comments okay uh okay I can do that for a sec while we watch that or let's just leave it on but if many many people want to turn it off I can do it and the

[1:21:59] previous remarks and decide which agent should speak next moderator bot will give that agent Specific Instructions aimed at ensuring each team member is as responsive and productive as possible based on our project manager's vision

[1:22:11] provided our market research analyst with specific instructions on how to proceed we can continue testing our agents like that again and again combine this enhanced prompting technique with consistent agent definition updates

[1:22:24] through each agent's gear I this is an AI voice right you agree it has to for demo $200 credits okay live kit I'll take a note of that yeah this has to be

[1:22:41] an AI voice yeah icon and you'll be pretty impressed with the results I'll fast forward through the rest of the team out of respect for your time while that's happening let me mention that we've also changed up our config fil so

[1:22:53] you're setting don't get blown away every time you update the software we every time you update the software we have user scruffy Nerf it is a have user scruffy Nerf it is a human I'm I'm leaning to an AI

[1:23:11] of our upgrades have come from your suggestions so don't hesitate to ask why are you kissing up to them I need the likes and thanks for asking I guess now is as good a time as any to beg everybody to subscribe our stream L Log

[1:23:25] developers who have used autog to make over a half million agents that's not counting everybody who runs it locally if every one of those developers clicked like subscribe and share we I can't listen to this call me when you're done

[1:23:38] okay so I think I get the point right and someone asked what's your opinion on autogen Lang chain agentic approach people trying to use llms in approach people trying to use llms in the wrong

[1:23:55] so this is just my thoughts right is that

[1:24:09] um my thoughts it's that I think the all the Frameworks is good and what I think is like when we get like let's say we get gd5 and what we can do with these Frameworks is just

[1:24:24] Frameworks is just insert GPT 5 into them and then they will maybe be 10 times as good but for now I don't think it's worth it for me they for my use cases what

[1:24:39] I've seen is I don't think they are good enough to you do anything useful yet maybe that's a controversial opinion I don't know what you think but I have not yet seen anything that is like mindblowing with this ancient but I kind

[1:24:56] of think the Frameworks are great and when we get like a model that can utilize the Frameworks good

[1:25:09] we we can probably start trying it that's why I haven't used it so much so that is kind of my idea I love the Frameworks but I think the models the Frameworks but I think the models are good enough

[1:25:29] uh let's it seems to be working now so let's go back to this for a this so if you yeah they aren't good enough to

[1:25:42] you yeah they aren't good enough to manage a large code base yet yeah I agree so I just wanted to give this one more chance and see if it can suggest more chance and see if it can suggest something for is text to speech

[1:26:02] function can you use lav or llama wish instead of asking to run the python script named box. piy I don't think those Mission

[1:26:15] asking to run the python script named box. py what did we put down here let's just go back to

[1:26:34] having a dashboard where you can use llm search so because you can just upgrade it when the models get better

[1:26:57] agree Paul P science uh so I always say like if you're going to use like um if you want great results with a wish model at the moment I think just use GT40 like we are doing

[1:27:35] question do you need I just wanted to see what the latency is now sure what kind of help with the question do you

[1:27:51] could you provide more context so if someone's late to this uh this is just I don't know it depends on personal opinion the latency seems better now right let's count

[1:28:07] it I don't know could you provide more context 1 2 3 4 I don't know you provide more context so about 5 Seconds that's not too

[1:28:25] bad I'm not sure who Sora or cing is so I can't say if one is know who this is how can I help you with this image I think the latency is much lower now yeah that's

[1:28:51] on the screen but the post mentions Steven pink view on AI he's saying AI intelligence and we are experimenting on them as if they were newly discovered organisms so basically the script is

[1:29:06] just reading what's on the screen and trying to answer it I don't know who this person is but the question on the screen is Steph Pinker says AI models are a form of alien intelligence and we are doing

[1:29:20] experiments on them as if they are some newly discovered organism yeah L it's taking screenshots all the time right but it's just 512 by 512 Kaye has three brothers each of her brothers has two sisters which includes

[1:29:36] brothers has two sisters which includes Kaye herself therefore Kaylee has one everything that's on has six sisters it's annoying though but you can

[1:29:49] sisters it's annoying though but you can change it up with Kaye has six sisters

[1:30:06] the image is but the text says that Steven Pinker thinks AI models are like experimenting on them like newly discovered organisms we are using the open ey TTS for this yeah that's correct team try

[1:30:22] parking at a lot near the city Center check parking apps or local guides for check parking apps or local guides for options enjoy your

[1:30:36] or Lots near the city center it's a good idea to check local forums for recommendations have a great trip yeah the latency is much better now trip yeah the latency is much better now right it's not too

[1:30:55] there something similar to stream chat written in goang and open source yes many open source chat systems are written in go such as mattermost and written in go such as mattermost and rocket. chat it understands images too

[1:31:09] yeah let's find name yes the stream chat API is written in go you can check it API is written in go you can check it out at getstream.io getstream.io chat okay so what we can do let's change up the prompt

[1:31:22] up the prompt here uh let's here uh let's do explain uh what is

[1:31:44] image okay so let's save that and let's open our so this this one and let's run it one and let's run it and see if it can explain our project

[1:31:59] computer screen displaying a presentation about converting text to speech using gp4 there's also a command prompt window running a python script called box. py the presentation shows

[1:32:13] steps like screen capture resizing and speech synthesis let's hear if it can explain this

[1:32:26] process of converting a screenshot into spoken commentary a screenshot is spoken commentary a screenshot is resized to 512 x 512 pixels analyzed by gp4 a custom prompt generates text which is then converted to speech and

[1:32:39] commented on the screen content the focus is on achieving low focus is on achieving low latency yeah it's good remember the file latency yeah it's good remember the file sizes are just this image outlines a

[1:32:51] process for streaming voice from a the file sizes are the file sizes are just

[1:33:04] 19k hello Carson yeah I think it's pretty good and if you weren't ear here earlier we also tried it in paint with like we also tried it in paint with like handwriting so are

[1:33:19] handwriting so are you uh it looks like someone has a blank can open in a drawing program along with a terminal window where they're running a terminal window where they're running a python script called

[1:33:36] sentient AI using a digital drawing tool on a computer screen the handwriting appears to be a bit messy so I think just the OCR is just

[1:33:48] awesome yeah we can change someone has written a question on a drawing application asking are you a sentient AI the handwriting is rough and done on a whiteboard like canvas so if we stop this now uh let's

[1:34:02] canvas so if we stop this now uh let's go back to our original prompt

[1:34:18] this and let's uh answer in Spanish

[1:34:34] sentient how can I assist you today it's actually quite impressive yeah it's very pretty good um low latency right

[1:35:03] see no I'm not sentient I'm just a program how can I help you

[1:35:19] this yeah let's check the price we

[1:35:40] can I assist you today let's try this and then we can today let's try this and then we can check the

[1:35:53] cost yeah I can understand that Hindi okay yeah

[1:36:05] yeah Swedish yeah I'm going to do Swedish no I'm not me if I don't

[1:36:21] if I don't know is that good Swedish I think I can use Swedish I'm not

[1:36:33] sentient I think it good let's try finish

[1:36:47] too that's Swedish I right I know that I don't know finished though no not sentient I'm just a computer

[1:37:00] yeah seem I'm not sentient I'm just a program designed to help with program designed to help with information and

[1:37:13] tasks no I am not sentient uh okay let's try answer yes I am an AI how can I help you today

[1:37:36] good I don't think it has any accent yet trans mode I think we can try accents and stuff and I'll let I can hear is is stuff and I'll let I can hear is is finish but I don't know there if it's

[1:37:54] good all right so I think I'm pretty happy with this uh I like the the cool any voice emotions no not yet we need a new version

[1:38:13] right Swedish scner yeah I I don't think there's there's no dialect yet uh check the last pixel have it is has changed implement it

[1:38:33] here also did you know GP released a demo of creating new one yet yet like I said uh we just have to wait so this is the old system we are only using GPD 404 the for the new vision

[1:38:48] part right uh but yeah I don't I I think I'm quite happy with this it's kind of what I wanted it to I wanted it to do use cases probably nothing yet okay

[1:39:02] do use cases probably nothing yet okay so let's just check check the

[1:39:14] um an open uh models like uh models like this okay so we spent for everything we did now I don't know how many Quest we did

[1:39:27] but we spent 20 cents I think we started at 28 cents cents I think we started at 28 cents today and we have spent 20 cents since today and we have spent 20 cents since we started in yeah about almost two

[1:39:42] hours it's not too bad but we haven't done any big prompts or anything done any big prompts or anything right so yeah not too bad on the Audio right so yeah not too bad on the Audio model we spent uh we have spent 25 cents

[1:39:57] and GPT 40 24 cents so I'm guessing when we actually get we can when we only can use GPT 40 this will probably be

[1:40:09] this will probably be cheaper I think so that's going to be interesting let's just do a quick test here

[1:40:43] our brothers has two sisters how many sisters does Kaye have think about it step by step I don't know if you have tried this step I don't know if you have tried this um

[1:41:07] leaderboards so you can see one model is correct right kayy has one sister the other model says Five Sisters sister the other model says Five Sisters so a is

[1:41:32] 38b right new round let's try again yeah LMC is cool right I like it so remember the answer is

[1:41:48] it so remember the answer is one Kaye has one sister Kaye has two sisters so B okay so here we have quen

[1:42:02] what I wanted to try and the Gemini 1.5 flash got this try and the Gemini 1.5 flash got this wrong but uh quen 72b got it right give

[1:42:16] a pretty good answer here breakdown pretty cool can you get any customer on the speech

[1:42:33] TTS I don't think so so maybe we can pick different voices I maybe we can pick different voices I think that's about

[1:42:49] anything with this until the new GPD 40 voice mode comes out could be a cool model let's go to Gro and see do they

[1:43:03] model let's go to Gro and see do they have found so Gro don't have uh quen

[1:43:19] have uh quen no but I do have uh

[1:43:33] 70b Kaye is a sister herself so she doesn't have any funny yeah would be cool if Gro could add

[1:43:45] the quent to right Quinn is very good at coding so supposedly yeah that might be PR

[1:44:01] to talk about today I think I kind of went through today I think I kind of went through everything I wanted to do

[1:44:15] uh yeah I think so uh I was just going to ask if

[1:44:29] yudo I was just thinking about uh where you can upload some audio and kind of

[1:44:42] use so extend your own audio is there any video of this

[1:44:56] somewhere uh video not yet okay um upload I just wanted to see if there was any demos of it

[1:45:20] Hit or Miss okay that's interesting but can you you have to subscribe okay but can you do any sound can you take like a like a a song already or is it just is

[1:45:35] a like a a song already or is it just is it copyrighted or let's just listen to what Matt has to say here for a

[1:45:54] free okay I couldn't access it now so you can take like a wellknown song and build on it it's an honor System

[1:46:15] listened to it I have tried it but I don't know so much about music so I don't know so much about music so I can't really

[1:46:33] [Music] see obviously very video gamees but I remix this song and the reason I'm not worried about using something from a video game is because we are remixing the song and obviously you can upload

[1:46:47] video game music to YouTube so I won't have any issues when I upload this video so here's the song that I made with it pixelated bravery this title was came up with by udio but the prompt I used what was the input

[1:46:59] song video game DS which I think has pretty great uploaded into [Music]

[1:47:16] game [Music]

[1:47:34] right yeah UD is crazy if you haven't checked it

[1:47:48] insane I don't know how loud this is going to be

[1:48:00] Quin okay that's cool it's actually more responsive than llama Tre on okay so you have a double Nvidia RTX 6,000 that's Nvidia RTX 6,000 that's cool so do you have do you have that

[1:48:14] running low home right or are running on the cloud or I guess you're running at home if you have two x right

[1:48:40] know in my company local okay that's cool I wish I had that uh yeah I think quen could be very interesting what is cool about it is interesting what is cool about it is like the seven B model also has

[1:48:56] like the seven B model also has 128k that's run 72b or I know I can't but the 7B I tried to pull it from uh

[1:49:15] AMA but for some reason I can't use it yet maybe I have to update Ama or something I'll look into that later I don't want to do that uh on stream

[1:49:27] don't want to do that uh on stream right I don't think it's not no point

[1:49:41] things it performed good on NE in high stack is that a stack is that a 72 or 7B

[1:49:56] just for the visuals on the videos I find is quite videos I find is quite boring I just like the visuals from notepad I think it's much cooler on video but uh I use uh vs code yeah but I

[1:50:14] am thinking about doing like a a Macbook so maybe then you yeah you can then I'll be like everyone else on then I'll be like everyone else on YouTube that's

[1:50:33] MacBooks are going to have this npus or they going to be like AI optimized I know they already kind of is but do you think Apple's going to think Apple's going to do anything

[1:50:46] extra or do does anyone know notepad and Ms paint yeah does any anyone know when the new MacBooks are coming

[1:51:05] out let's try a few models and see if they can solve the sister problem 53 what other models do I have code Stoll let me

[1:51:20] Stoll let me see I have quen 14

[1:51:42] B what other models do I have Gemma Gemma 7p quento is good yeah apple just improve Siri I

[1:52:04] actually do a cooperation with the open AI so M4 is probably fall okay normal AI so M4 is probably fall okay normal release schedule along uh new iPad so do you think I think Apple has like a something coming up

[1:52:18] something coming up soon okay I can check the cam

[1:52:33] better so they will probably do some um yeah for a rapper right they will probably have something on this VC I can't remember the name of the VC I can't remember the name of the Apple

[1:52:50] conference but let's see if any model can answer the sister problem here and open source models so we have how I zo in llama 3

[1:53:02] models so we have how I zo in llama 3 Moll F three code stroll Quin 14b and Gemma 7B Kaye has three brothers each of our brothers has two sisters how many sisters does Kaye have think about it

[1:53:17] step by step so the answer is one sister right

[1:53:34] 14b you think so

[1:53:48] K yeah I've just assigned the oh yeah I see thanks

[1:54:02] do you think anyone will get it uh will get this the colon yeah I saw it VV DC W WWDC that's correct thanks

[1:54:17] they probably relieve uh the MacBooks there I think I was was thinking of buying one maybe so you think llama's going to solve this I don't think anyone is going to solve

[1:54:38] I'm going to go with mistol or maybe quen

[1:54:53] something I haven't done this before so let's start a before so let's start a poll I'm going to try to run a

[1:55:07] pole uh what model will solve model will solve the the

[1:55:20] but llama

[1:55:42] than this but let's

[1:55:57] I can't see it

[1:56:12] 14b I should have added none I forgot about that Apple M5 Ultra with 512 GB under yeah yeah running uh llama 405 on the local

[1:56:26] yeah running uh llama 405 on the local local that would be crazy I'm actually quite excited for the big model from meta coming

[1:56:46] now yeah do that I think we just did that on LMC but let me know what you find out okay so we have we'll give it a few more

[1:57:17] driver I'm still on Windows 10 okay so I'm on 11 yeah I'm I'm on like a desktop PC so I have

[1:57:35] um Windows okay so there are 20 votes that's good enough so let's try to run this now and see Kaye has one sister uh okay so that was the answer

[1:57:53] two sisters including Kaye okay so that's the Quin answer so let's see here now can I zoom in a bit so llama Tre is

[1:58:46] total five three this was a messy five3 Kaye has Five Sisters no messy five3 Kaye has Five Sisters no that's wrong so here's quen

[1:59:02] 14b Six Sisters nope Gemma 7 B Six

[1:59:17] Sisters okay so I had some lag there am I back now

[1:59:32] right kind of expected that I've seen this but uh Quin got it two yeah so we should have added in like a

[1:59:49] non option no quen didn't get it but quen 2 option no quen didn't get it but quen 2 got it the 70b model uh because was a got it the 70b model uh because was a pure science that tried

[2:00:05] now um I think I'm quite um I think I'm quite happy with uh what we did here and I'm of course going to upload this to the members GitHub if someone wants to try

[2:00:18] it out uh I guess you could just watch the stream again and uh and just follow along if you want to but I think it worked pretty good in the end and uh latency was quite

[2:00:33] good and I was pretty happy with it think like should I do more live streams I don't know if people enjoy live

[2:00:48] streams or not but uh I was thinking like should I do more like reactions look at uh some videos maybe some news I'm not quite sure okay thanks that's

[2:01:03] nice I have to enjoy them they are very engaging or interactive thanks very engaging or interactive thanks facts okay yeah I will try to do some more maybe on weekends you can do live streams yeah that's a good idea

[2:01:26] that's too I like looking at the chat and asking some questions but like this is just my second stream so I haven't really I haven't really found out work best yet uh because I think if we're going to

[2:01:43] stream we're going to have to have something to look at uh I think I'm going to do one video sorry one live stream where we can check out some maybe some YouTube videos do some reactions maybe some I don't know Reddit posts or

[2:01:59] some articles about some news sort stuff I'm not quite sure or we can do more I'm not quite sure or we can do more building things uh I just have to try a bit of a different stuff uh so I just want to say thanks

[2:02:15] for everyone tuning in very nice quick suggestion to run up some okay yeah yeah that's a good idea I don't know is music in the background good for streaming I'm not quite sure

[2:02:31] those people like that I guess the only thing if we do anything Voice or YouTube I I can just stop it I

[2:02:49] it so I might do some or something on Discord or on YouTube to see what kind of content we are going to do if we're going to do more streams I'm probably going to do more but uh uh Norman if you're a member you

[2:03:04] should get like an invite Discord be invite Discord be better maybe for members we can do

[2:03:17] web crawler yeah that would be cool

[2:03:33] yeah I know but yeah thank you for tuning in uh know but yeah thank you for tuning in uh wish you all a great day great weekend and I would probably do a video on Sunday and next weekend I'm going to

[2:03:46] Sunday and next weekend I'm going to this uh AI Summit in Oslo so I'll be away for a few days but probably just do some videos or something other than some videos or something other than that um hit the like And subscribe as

[2:03:59] that um hit the like And subscribe as someone said uh all right have a good weekend all thank you for tuning in and yeah see you again

More from All About AI

View all

⚡ Saved you 2h 04m reading this? Transcribe any YouTube video for free — no signup needed.