The Future of AI: Why You Need to Run Models Locally

So, this is running at home. This is my uh GLM-5.2. It's like a custom compression that I did for it. But, if we go here, like this is the same model, right? This is a compression. It's uh it's compressed 80%. And so, you know, this is 3D, you know, the Flappy Bird. And uh this was a single-shot uh attempt. This is another one. So, this is running DeepSeek-V4-Flash. Uh there are there are going to be two

sessions left and right that are running. So, you can see it's running pretty fast. It's able to do like a concurrency. It's doing its tool calls, reading. This is a like private inference that I'm running at home that is capable of doing everything that the like frontier models is capable of doing, at least for work, right? So, I'm using 374 million tokens a month locally. >> All right. So, Seto, you're one of the top voices when

it comes to like local models, open-source models on Twitter. What's your thoughts on GLM-5.2? >> So, I've been using GLM-5.2 for a few days now. And uh well, actually, I got access to it sometime last week. They usually give me early access to these models. Uh the entire GLM series lineup from 4.5 up until now has been phenomenal. They're very good at agent work. They're uh excellent at coding on back-end systems. They're really good at Docker

and like devops type of work. They're good at GPU programming uh and like anything related to AI ops, like ML ops. Like it's the only model that will always just reverse engineer anything. So, if you just ask it, like here's a thing, reverse engineer it. It is more than happy to to work on it for like 8 hours plus to get it to perfect. And uh I really love it. >> one of the first models where

people don't feel like it's been optimized for benchmarks, because that was been one of the main critiques of the Chinese models, right? That like usually they perform like really well well on benchmarks, but then when you use it, they don't feel as good. So, do you agree with the statement? And do you think like there's something different about this model? >> Um I I don't agree with the statement. I have never really believed this statement. You

could say the same about the OpenAI models or the Anthropic models, that they're benchmarked. And um I've found that like the Chinese models tend to make trade-offs. For example, ZAI's trade-off is that they don't have vision in their like coding models. Without vision, it gives it more like space to learn how to code. Because if you want to train it on vision, you're going to have to like have like a vision section, you're going to have

to like add training data related to that and it's going to require it to be bigger, which means it's going to be harder to run and self-host and use even deploy for them. So, I don't really agree that they're benchmarked like benchmarks. You know, OpenAI and Anthropic tend to release like a full package. So, you take the model and everything just works and it's like it's like really refined and and the marketing around it. Like Americans

are very good marketers. So, we are you know, like think of Hollywood. Think of like we're very good at convincing people, you know, our stuff is the best. So, I would say that's the case. But, it is better significantly than anything that's come before it. That's for sure. >> Yeah, I do think like the point about marketing is valid. The Chinese models a lot of people just don't want to use them because they come from China

even if they're the inference is like in Europe or in USA. So, I do think there's a like a marketing issue when it comes to all these open source models. Like how do you fix that? Is it just like better UI, better package like better tooling around them, better harnesses? >> Yeah, I think I think mainly it's like distribution. So, right now if if an American wanted to try the GLM models, it's not going to be

available on cursor. It's not going to be available in like Claude Coder, Codex without you having to go to this website that like is typically not named the same thing as the model. You know, get get a card connected, you know, listen to people telling you it's benchmarked, listen to people telling you the Chinese are trying to hack your computers. Like it's I think I think it just needs better distribution and so a company that I

would like to call out that does this well is Factory AI or Droid. So, they have like these Droid core models and they use the Chinese open weight models. I think they host them themselves or they partner with some American company. It is hosted on American soil. But, they've been they've been able to get these models over to more people, which is something that I really respect and appreciate. >> at OpenAI and Anthropic as like a

research labs and all these are like technical tech companies and you know, scientific efforts, whatever. But, they're also excellent in marketing, right? Cuz like Anthropic, they kind of hyped up their models like they're going to replace 50% of jobs, they're going to be dangerous to society. And then when the US government slaps them, they're acting all surprised. But, even though like they had it coming. So, maybe maybe that's a pivot to like Fable 5 and what's

your thoughts on the model and on the whole situation around it. The web is full of public data. The problem is actually getting it. The moment you try scraping at scale, the websites fight back with CAPTCHAs, rate limits, and IP bans. But, it doesn't have to be that way. With Oxylabs, you give Oxi Co-pilot a target URL and describe what you want in plain English. It builds a scraper, the parser, all of it. Oxylabs literally handles

the proxy rotations and CAPTCHA solving for you. And even on the toughest of toughest sites, it has over 99% success rate. If you're building an A10 like I do, the Oxylabs AI Studio node plugs right in. You describe what you want, it scrapes the data, and your workflow handles the rest. Under the hood, Oxylabs has over 175 million residential IP addresses from over 195 different countries, giving you real-time data for nearly any website. Plus, they have

a new fast search API that can drop organic search results straight into your AI pipeline. And by the way, Oxylabs gives you 2,000 scraped results for free. Just try it at oxylabs.io/david. And make sure to use code david to get 20% off any paid plan. Thank you to Oxylabs for sponsoring this video. So, I have I used the model the entire time it was out. I think like in API credits I used about $1,500 in 2

days. And I've tried like various types of prompts on it. So, I have this directory on my computer called personal it's just personal. And it has all of my medical records, all of my social media posts, all of my Google data like I just download everything from all of the sites which you can do pretty easily. Like in X settings you can go and download personal data. Google Takeout lets you take all your Google, YouTube, Gmail,

calendar stuff out. And so I download this stuff and I rag it and I put it on my computer. And it's super useful in my opinion cuz I get like good search and I can you know have like conversations and extract insights. But I put it there and I just had it like go over things and then I told it just give me like some insight or like some topic to make my life better. It found

essentially like 25 out of the 27 or years of my life. And then it said like it wants to talk to me about that missing 2 years. And that was the first model that's ever done anything like that. And then it just like it like it was like acting like some kind of psychiatrist, psychologist. Like it was very good at human communication and and portraying empathy. Which is rare. Like GPT models don't do that at all.

They will shut down these types of conversations. The Chinese models tend to be like more tailored towards coding and they're not necessarily that great at this kind of thing. So, I really loved that part of it. That was like mind-blowing to me. But there is this important thing that I've tried. So, >> [snorts] >> there is a game called Bloodborne where you just like go and it's like an a Japanese action role-playing game. And you're the

the first level is like you start off at like a square and then you have to make it through the fireplace, fight these enemies, then go on a bridge, fight the enemies, and then go on to fight like this massive werewolf type creature. So, I give this prompt to all the models. I just say rebuild Bloodborne basically. That that like very short prompt. Starting from Gemma 331B, they will basically get the components of it right. So,

but everything is just this like a block, like a square block. You know, the gun is a square block, the weapon is a square block. So, there there is no detail in the outputs. And now, as [snorts] the models get larger and larger, you start seeing more and more detail in the same functional thing, right? It's like the same thing that they're outputting, but there's just like so much more detail to it. And I found that

the like the Fable model outputs for that specific game benchmark that I do to be really like true to the actual game. Like the types of enemies that there were were the same, the types of like behaviors and game mechanics were the same. And so, I think that the model is capable like the size of the model makes it capable of extreme detail. And I really like that about it. I think it's a shame that we

don't have access to it right now. >> Yeah, I mean, I had the same experience for all the projects I wanted to build, I just like you can see on my GitHub those four days or whatever, just like way more contributions. I just got way more done. And also, it like the difference I would describe it as like a you know, open 4.8 and 5.5 Pro or like X high are more like tools. They really feel

like tools. You need to exactly explain what you're doing. But with Fable, it felt like an actual contributor. Sometimes I didn't have to explain what I was doing, but it understand why I'm doing it. Like the the deeper intent behind it. And yeah, it felt like like something else is working on this project with me. Not like I have to be constantly the director and like give all the details. >> Yeah, I understand exactly what you

mean. So, the the main thing about that model that I think we should keep in mind is that there is a honeymoon period to everything. So, whenever a new model comes out, everybody's excited, everybody's like, you know, talking about it. It's in people's attention. And people are hyping it up. Like any model that comes out. Look look at GLM 5.2. People are hyping it up. And so, what this this situation has done is it denied people

from the come down of the the honeymoon phase, right? It denied people from taking the time to see the flaws in the model. And so, whatever like conceptions that we have right now, I I would say like I'm making a bet or a prediction that when the model does return, and it will, uh that it is everybody's going to call it like, "Oh, it's not the same model. This is like uh this has been lobotomized." And

it's going to be the same model. So, this is a bet I'm making. I don't I don't know if it's going to happen or not. >> Yeah, it always always happens even when when no changes are made. Uh but, you know, that's what goes viral. Real quick, if you want all of the materials from this video, it's in the second link below, including the model matcher skill, which analyzes the computer, sees what specs you have, and

which models you can actually run locally. So, again, all of this is available completely for free in the second link below the video. But, I guess what are your thoughts on like the situation, like the ban? Uh like I would say this is the closest thing. Like, you know, I I don't know we can talk talk about like AGI and what are your thoughts on like super intelligence, but like, you know, this is a step in

that direction, and I think it's a huge disservice to all of humanity, no matter if you're American or not, that this model is not available, right? Even if there's a honeymoon period where like we are hyped about it, it's new capability, you know, new step change, but like, personally, I could feel the productivity increase, and I know that if everybody had access to it, or even better if it was open source, that like the progress of

society would be way faster. So, what are your thoughts on this ban? Do you think it's justified? Do you think like this this even makes it more clear that the future of AGI needs to be open source? How do you think about this? >> So, like we can we can take two paths here. Like, path one is that the model is truly capable of like extreme cybersecurity risks and biological terrorism risks. I don't think that is

actually outside of the norm of reality. If the model is capable to get you closer from like let's say rat DNA to some kind of like virus that is going to hurt people, then it is actually worth taking a look at the model and seeing how much it's being used for this kind of stuff and I guarantee you it is like just like every model. Uh people are asking how to make bombs, people like this is

the the reality of the situation and we need to be able to confront that. There's also the risk of hacking. So, now like imagine this, you have an open-source repo and they have a pull request. They don't say what the pull request is about and there's a lot of changes in it, but it's like critical software like let's say Linux and they're probably hiding like some uh RCE like CSV I forgot the the code for it.

But like there's they're probably hiding a fix to some kind of security bug which they do in every release. So, you can have the the models like that are really intelligent. You can have them just sit and loop on all of these open-source repos and try to identify essentially fixes for security bugs. And now until that PR is merged, now you have a zero-day exploit. So, I'm just giving you a scenario that somebody has told to

me which I I think is is really true. I used to work in financial analysis and blockchain analysis and one thing that these people do is that they'll have a model just sit on the blockchain and [snorts] watch every transaction and then find something to exploit. Like they they'll find like some opportunity for making money and they'll just sit and try to exploit it. So, now you're you're pouring fuel on that fire. Now, I don't I'm

not saying this is actually why they banned it. I don't think that's the case because you can do this with other models as well. I think what's what's happening like it could be that, it could be that they're just like they just like open their mouth too much to the government or it could be that this is like a way for them to shift the narrative from them being a SaaS company to them being a weapons

manufacturer which is much easier to raise a trillion dollars for on an open market. If you think about like American companies that are also weapons manufacturers, they can get close to those numbers. So, I think it's an easier sale. Now, uh last last uh point on that is regardless of what happens, I think we are going to a place where like maybe we get one more generation and that's it for uh improvements for civilian usage. I

don't think we can keep pushing past this point without it being a big issue. So, the Chinese companies Moonshot, Deep Seek are either partially or fully owned by the government in some capacity and they are uh like they they So, it's like not a choice anymore. It's like this is what's happening and we have to figure out how to accept it and then build an alternative so that when they do take it away, we can still

do what we're doing. We can still like learn and you know, uh contribute to science and uh I think it's super important. >> Okay, so this is fascinating perspective. I definitely want to go deeper there. So, you think basically the governments, whether it's American government or Chinese government, which you know, are the two countries really at the lead of cutting edge AI, you think they're just going to shut it down in the next one or two

generations? >> I don't think that they're going to shut it down. I think they're going to stop any type of like progress that is going to improve the models in intelligence. So, the models maybe like there will be newer models, but they're going to be more like efficiency gains. Like how can we do this with less tokens? How can we do this with less costs to the user as opposed to how can we make this model

do 20% better on Deep Swear or whatever because it's getting to a point where if you just keep increasing the intelligence, it really does become like a like an existential risk of like a cybersecurity. Uh we're not going to have like private like homes anymore. Like your home network can be hacked, anything can be hacked and uh that's going to cause a lot of problems. So, this is what I'm thinking is going to happen. Uh I

read this online, of course, and that's influenced my opinion a lot uh because it makes sense. But may maybe it's maybe it's not the case. >> Okay, but the counter argument would be that, you know, let's say the latest generation that is available, open source, you can have like any country, whether it's Iran, Russia, or just any random country, download these models and like put their best engineers on them and start fine-tuning them and start reverse

engineering them, start training better models based on them. So, like how how do you think about that? Cuz like the progress, technological progress, I I don't think it can be stopped really. >> I don't think it can be stopped, for sure. I just think it's like it's very easy to slow something down. So, going back to the example of blockchain, cryptocurrency, is it full of scams, full of people robbing each other, full of like corrupt behavior?

100%. And it's even more so because it's a financial thing. But, it's still like open-source financial services, right? Like if you are in a country where you cannot like you I've known I live in Warsaw, Poland, and I have known many Ukrainians who have fled from from, you know, their country. And they had to live in the forest for weeks, and then they have to bribe people to get to the other side. And so, the of

this is happening in crypto, right? A lot of donations back to people are happening in crypto. Whether that's good or bad doesn't matter. The point is like this is a necessary service, but governments can still ban it. It's still happening under their hood, it's still happening within their organizations, within their government, but they can ban it for like the average person, right? Like the average consumer, which is what I think is like probably going to happen.

Now, I don't know, again, like there is a chance where what I'm saying is wrong, but this has happened with a lot of technology beforehand right? >> Yeah. I mean, I guess nuclear is the biggest example. I think it was a like a tragedy that we had nuclear reactors and, you know, this magical form of energy and then got shut down. And now like, you know, Germany and other countries are going back to coal, basically back

to the Stone Age, like literally reverse progress. But, when it comes to AI, like it's a lot simpler, you know, it's runnable locally. Not every Not everybody can build a nuclear reactor, but like people can set up their own rig and start fine-tuning. Like I think it's a lot harder to ban it, no? >> Yeah, so this is this is why I'm so excited about like local AI, open-source AI. So partly is because I love this

technology. It has truly made me a better person in every way possible. Like I am smarter, I am more capable, I am better like as an adult. I can pay my taxes better. I can like figure out how to be, you know, normal, right? Like it helps me like schedule my calendar, helps me learn, apply for visas. It's very hard stuff that I was not capable of doing before AI. I was just like too scatterbrained. And

now I can do a lot of this. And so it's helped me. And I you can run it at home right now. Like I think the Qwen models like the 27B and the the other one, the 27B and the 35B are truly better than Sonnet 4 in every single way on all the benchmarks. So it is it like I I and I would also challenge people to cuz you can still use Sonnet 4 through the Claude

code subscription. I challenge people to like try and see will you get better performance on a MacBook or on Sonnet 4 from a year and a few months ago. I think you will get better performance with the Qwen models. So uh but that requires people use it, that requires people want to protect it. Every single right that we have, we have to kind of fight for. Personal computers, the internet, right? Like the the internet freedom thing

in 2016 2014 like they were trying to shut down like the like make it very hard like so in America they were basically trying to stop people from going on 4chan and other websites from their mobile subscriptions. And and we have to fight for that. Like freedom our own freedoms. So if people start installing these things, if people start running these things, there's nothing the government can do to stop us. But if people don't start doing

this within the next 2 years or 3 years, what they can do is go to the manufacturers or go to the distributors, right? Cuz Hugging Face is a Western country. You can go to Hugging Face and you can say these models are deemed a security risk, they're on the sanctions list, you have to take them down. But if everybody has Qwen on their machine, it's not possible for them to do that and it's not worth the

effort of going to Hugging Face. So, I would I would say that this is kind of like a we have to fight for our freedom uh thing, you know? Obviously peacefully and nicely and respectfully, but we we do have to fight for it. >> Yeah, and I think what a lot of people don't realize, the reason why it's a severe situation, is that, you know, if people remove your access from AI, you're done. Right now, people

don't see it because, okay, you would go without ChatGPT, Claude, blah blah blah, whatever the average consumer uses, you would probably survive in the world. But if we extrapolate 2 years, 3 years, 5 years in the future, where AI is super intelligent, really removing your access from AI will be worse than if someone took away your internet, took away your electricity, took away everything, like all your technology, and like put you back in the Stone Age.

It will be 100 times more crippling in 5 years to not have access to AI. I'm certain of that, and a lot of people need to like, you know, go into long-term thinking and see, okay, if the future, if AI keeps improving, and it is improving, how bad will it be if the government takes away your AI access? It'll be insanely bad. You will just get unable to function in society, unable to compete in business, anything.

So, if people start to realize that, I think they'll realize the gravity of the situation and that there cannot be any centralized control. Whether it's Sam Altman or Dario Amodei, whether it's some like group of small companies or a few governments, it doesn't really matter. There cannot be centralized control. This technology is just like too important for the future of humanity. >> I love the example. This is only really relevant for Americans, uh but American healthcare

system. So, in in the American healthcare system, you have two options. Option one is you get your healthcare from the government. So, you you get a massive subsidy from the government and you pay in still like 12 plus thousand dollars a year. Option two, on top of your taxes. Option two is you get it from your employer. And when you get it from your employer, you get better healthcare, you get your you know, you get a

dentist, you get eye doctors. So, what's going to happen in the future in my opinion is you will have a subsidized national AI in countries that are more like socially leaning. So, like if you can think of Europe. Like they'll they'll probably have a deal with Mistral and they'll like distribute it for free for people. But then every single thing that you say is going straight to the government. Everything. And they're going to be able to

like mass analyze it. Like in connect all of these dots and like decide who's who's worth, you know, giving the smarter models to, who's worth like literally throttling on purpose or ruining their life on purpose. This is going to happen. This is already happening. It's just going to be scaled up. So, and in the US, if you like for example, you go to a company that's really strong, they're going to give you the equivalent of Opus.

And if you're really good at your job and if you're really like kiss ass to the boss, maybe you'll get Fabled. This is what I think might happen. Like it's kind of like a one of those nightmare situations for me personally. Um Does that make sense? >> Yeah, it's happening already. Like not not necessarily like the government surveillance, which I think also might happen. But like the companies, it's happening, right? Like people at my company, they

they can use basically unlimited stuff as long as they're building something useful, right? Like when when it comes to like $200 a month for Codex subscription, Claude Claude subscription, Open AI or whatever. Like as long as they're they're doing something useful and they can show their output, like I'm happy to cover it, right? So, it's already happening with like companies that understand the importance of it. They're going to give you the inference and you're going to

be able to run way more than you could afford personally. And yeah, I think a lot of people are also going to look at that the same way they look at, like you said, health insurance, right? A lot of people don't want to work with companies that don't provide any health insurance or whatever. It's going to be the same thing with AI. If a company cannot even cover like ChatGPT Pro subscription or Claude Claude subscription or

give you some like budget for open source models that or like you know some local AI rig if a company doesn't have a local self-hosted supercomputer then a lot of the top talent will be against dissuaded by that and go to companies that will give you like unlimited AI budget. >> Well, you can already see this with Anthropic like many many many smart people are leaving from other companies or from being independents and going to Anthropic

because they objectively right now have the best model that is available on the market that we know of, right? They have the best model. There's no There's no doubt about that. Um maybe OpenAI can compete in two releases. I don't think the next one is going to be that level. I think they might shoot for the one after that uh to be like, you know, either that level of or above it. Uh but Anthropic has proved

that they have the best model and so uh the the incentive to go work for them as opposed to work for yourself or do anything else um is enormous. It's It's enormous. Even if you're like a local maxi or an open source maxi um you know, it it If you can do your life's work, like if you can truly be the best you can possibly be by going to Anthropic, why not? Why why shouldn't you do

that, you know? It's It doesn't make sense. So, I think we we we just need to start like figuring out how to um spread the intelligence. Spread it out so it there's not these like massive bumps where uh you know, things centralize and and things uh like you you have to basically surround yourself with with how Anthropic wants to do things to get access to intelligence. We should have it available everywhere and and um I I

I think you know, it should be considered a human right, honestly. >> Yeah, absolutely. I agree. Um on the topic of Anthropic, I guess you mentioned like the top people prefer working for them even though they're like open source maxis, but why? Like why would people go to this closed source company and you know, maximize like these people who also have like tens of millions of dollars in career earnings, right? Like why do they go work

for Anthropic? Like do Do believe they have like recursive self-improvement? Why don't they like work on the open source side of things? >> Because if if you believe that Anthropic has the most intelligence in the world right now, and you are somebody that is motivated by intelligence and by the understanding of intelligence, I I don't think that it makes any sense not to. Like I think that that model, right, and we got like a kind of

a nerfed version of it. I think that that model is a a complete step up, like a completely different thing than we But you know, again, I am might be biased. But that's what I think, and I think that's what people like Andre believe is like this is a huge step up. I'll go in there, I'll learn for like 5, 10, 20 years, and I'll come back and I'll spread the spread the the intelligence openly because,

you know, if he was working for all these companies that weren't able to hit that level, then that means, you know, there's something he can learn there. I think I I at least this is like I'm I'm projecting my beliefs and my my my thoughts onto him. I uh you know, he's he's a he's a leader in the space, and like a lot of people respect him. I don't think he's malicious per se, I just think

he's motivated by learning and intelligence more than anything. >> I see. So basically it's like the need to understand, you know, the whether it's like scientific need or like more like sci-fi like universe, these top people, the top talent, they just seek to have that knowledge and to, you know, interact with the closest thing to super intelligence that we have, and to have basically unfettered unlimited access to it. And yeah, like you said, Anthropic is kind

of that place. By the way, everything we discuss in this video, you can get absolutely for free by clicking the second link in the description, including a custom rig planner skill that allows you to plan your own hardware rig at home, depending on your budget and depending on your requirements. So all of this is available in the second link in the description, completely for free. So go grab it now. Do you think that's the main reason

why they are able to keep all their co-founders, all their top employees, or like why does Anthropic have such a low employee churn compared to all the other AI companies? >> So if you look at like the like the way that they call their employees, everybody is chief like what is it? Like member of technical staff, right? So that's one thing. Another thing is that they're building something that is very artistic and like in my opinion

what they're trying to do is like embed soul and character and persona and and like something that is deeper than just like an a task executor. They're trying to embed that into the model purposefully. So they write the like Claude constitution. So I think a lot of this is more like moral beliefs and like the types of people that also work at Anthropic are very like morally driven. They're very like I've met and spoken to many

of them. They have like basically helped me run a Claude code meetup in Warsaw. They have been nice people to meet in like in person, right? Every single one of them. And I think that you know, all of this stuff goes into creating an atmosphere. It's like you're developing really fast. You're innovating on a lot of things. You are working with people that are like very morally and ethically driven. And there seems to be like a

little bit of an equality within the company. Like there isn't that much of a hierarchy where this person is more important than this person, you know? At least in the way that they're titling things to the public. And I think all of that kind of goes together to create an atmosphere of like a you know, people say cult, but I would say like like an imagine an open source project that is like really great and people

love. I think like that's kind of the same mentality is that they're doing it because they like it as opposed to doing it because they want to have a good job or but maybe maybe I'm wrong. I don't know. I'm just assuming. >> I mean yeah, the the work environment for sure is probably on point, but I'm wondering like whether they don't see any of the risks, right? Like They're all trying to it's it's kind of

funny, but they're all trying to like create the AI automatic researcher, right? So they're all kind of trying to like replace themselves out of a job. And you know, whether that will happen or the company will keep hiring more people. There is other risks. Like for example, getting nationalized, right? Like that's not out of the realm of possibility. So like are these like researchers just blind to these like practical geopolitical risks or like are they just

like all about like I I want to have access to the best AI? >> I you know, I think everybody that's doing anything thinks that they're doing the right thing. Or like it's very easy for people to you know, the majority of people think that whatever they're doing is the right thing. There's this article called Meditations on Moloch. It was written by Scott Alexander in 2014 and it talks about how whenever there is like a a

pocket of an incentive. So, like imagine you have you know, the the cells on your body, they're all independent creatures that come together to create a whole. And if there's one that figures out well, they could just exploit the resources to keep multiplying and keep spreading their own genetic line, they're going to kill the host, but they do it anyway, right? So, there's this like incentive mechanism that is transcendent of like humans or like it's you're

a creature that requires sustenance and stuff to survive. And so, if something is going to give you that like at full force more than anybody around you, you're going to follow that pocket. And if you don't do it, someone else will and that's enough. Like if if if I know that I don't do something and somebody else is going to do it, I mean, I'm I'm going to have to do it. Like why would I let

myself be on the sidelines when I am capable of doing this and it's going to happen anyway? And you know, the way that they market themselves, the way that they speak on these things is is this mindset. Like they have this like mindset and this belief system is like it's going to happen anyway, it's better that we do it because we think we know better. And maybe they do, maybe they don't. Like I don't know if

time has proven out what they're saying. It's going to take a like five five years plus maybe to see what you know, the consequences of whatever is happening right now. >> I see. I mean, yeah, that logic does make sense, but like how does that look like practically? Because like you said a few minutes ago, you think the government will stop giving like access to the best models. So, at that point we'll just like the the

closed source become even more closed and it's like literally just a internal company and some like project like Glass Wing, you know, 200 enterprises and that shrinks to even less and the government and that shrinks to just the government. How do How does that look like? >> Well, if you think about like the economics of it. So, I I I run my own rig at home. I have 4 6000s. Before that I had I had eight

3090s and I have like a like the sparks and stuff at my house and I'm doing inference like 24 hours a day. >> [snorts] >> So, the cost of the GPUs, they are using mostly B200s. Like they are paying I think a billion dollars a month to to Elon, right? So, a billion dollars a month to Elon and they are selling mostly these subscriptions and these subscriptions like I can go look right now how much I'm

getting back, something like 4000 to 8000 dollars a month. Every month from my $200 subscription. And I think that that is an accurate relative cost, right? Like it's probably 2x uh more expensive. They're selling it for 2x or maybe like uh 3x the price. That's possible. But then if you think about like the taxes, they have 2500 plus employees, they have to deal with all the regulation, they have to also like give money to the government

so that they can get their way. Like if you start like thinking about all the costs, like it gets lower and lower and lower and lower. And if you have somebody who like me who in one day can spend 4 billion tokens like the with with OpenAI, I spent 4 billion tokens. That's eight B200s running for 24 like uh 16 B200s running for 24 hours just for me. And 16 B200s cost about $120 an hour and

I'm doing it for 24 hours. So, you could you could start to see like this is completely completely unprofitable. The way at least we, the end consumers, are getting. What is profitable is enterprise contracts. So, when they deal with enterprise, they remove subscriptions. So, it's it's it's token usage only. So, because of that, that's profitable. So, their incentive is going to be to give money give out this free money to the end users, to the people

on the internet, get them really excited, show them what's possible, then they go tell their companies, "Hey, let's get an Anthropic subscription." And then the companies onboard onto a subscription a year ago. It was $120 for like even more usage. But then once they onboard and their entire company is built on Anthropic, then they switch off. Now it's it's token usage and then they see, "Oh, wow. Well, this is pretty expensive." And so now you just

like filter off all the companies that can't afford to pay you. And you have a profitable business, which is where they are right now, and which is what I think OpenAI struggled to accomplish. Like they had it in reverse. They had enterprise first and then they went down and like, you know, targeted the developers. I like I I like their message better. I like the fact that they're pushing intelligence for everybody. At least they're saying that.

Um but yeah, so does that does like the economics of it don't make sense unless you do it that way. >> But the question is like if the if the generation or two generations after that cannot be released how does that work? Or or will it only be released to the top companies that the government approves or how do you see that? >> I think that's what's like yeah, it's it's likely that the the government is

going to have like a sanctions list of all of the types of entities that can't get access to Fable and like Fable or whatever Fable 2. Um and then there are going to be companies that don't fit into that like sanctions list and they can use it. So, uh they'll like for example sanction all people that have less than whatever a thousand uh employees that are or like they have more than 20% of their employees are

non-American. Like you could find a million different rules >> Yeah. >> to do the same thing. >> Yeah. >> Uh so and and that's going to be very profitable for Anthropic, of course, because the companies already have money. Already have like the pre-filtered best customers, including the government. >> So, what do you think like what will happen as the technological progress will keep going? Like, you know, let's say, you know, AGI, then after AGI, closer to

ASI, will that be really in control of the government or like how how do you see that going at the current trends if nothing changes? >> So, the way that I feel about the models right now is that they're they don't have Like, you know, we say agents, but they don't have personal agency. They need to be interacted with from the outside, right? Like, they need to be triggered. I don't know if anybody is building the

infrastructure that is required to have a model just be like an actual free entity where it can go around. And if they are, they require a lot of B200s or B300s or whatever the the new the new Vera Rubin stock or you know, a bunch of H200s. So, that like imagine that as a as an organ, right? Like, that is an organ of whatever it is the creature that we are trying to create. So, it's very

easy to target. So, if if if they create this like RSI ASI like independent entity that is running around the world, it's still running in data centers, at least for now. So, if it's running for data in data centers, people are going to be able to target those data centers, they're going to be able to wreck them. So, what we're already seeing this again right now. So, in Dubai, they were building star star something like stargate.

And Iran specifically bombed exactly that area, specifically so they cannot build the data center, right? Because they know that's where it is. And so, we have like stargate two or I'm not really sure the numbers. And also, I could be wrong. So, if if I am, I'm sorry, but this is what I heard on the news. And like we could kind of see like I don't know if it's ever going to progress to the point where

it's controlling the entire world. I think it's more like this is a a situation of convenience. AI is very convenient. It's very helpful. And so, we will give more and more of ourselves to these systems and you know, help others pilot these systems. Like Like human beings pilot like a huge data center farm and have access to all this intelligence, which is where the risk is. I don't think the risk is like something taking over the

world. It's like people kind of mess messing up, which is very obvious. >> Yeah, so I guess it's the same ideology as Bitcoin then, right? Like the decentralization of something highly important. This case it's not like money and financial system. It is just intelligence, which is even more important. So I guess what what's your thoughts on Bitcoin and like do you have the same ideology for like future of AI? >> I pretty much all of my

money is in Bitcoin and ETH. Honest like almost all of my money. I think I have some >> lost it in a boating accident, right? >> I lost it in a boating accident. Yeah, yeah. I don't have the private keys anymore. But the yeah, so but also it's like it's actually in a bank like in in bank safes. So the the the situation is like Bitcoin is just another evolution of free compute and like freedom through

compute. Cuz we first had these like time share systems. I read this in a book called We Programmers. Programmers used to have to go and like like put in a slip and they get access to a computer for 2 hours and then they have to write their program and it has to compile the first time or your your access is like useless because you're not going to get feedback for at least an hour of it processing.

And and so then computers started to spread around the world and spread more and everybody had a computer in their home. And and then the financial system came up and like we can start to digitize it. And so there were many like there was like these decentralized signs in the '90s where you can connect like an I think it was a device that like looked into the sky and mapped the stars. And you can share that

data. And then this is like one of the first decentralized science experiments. Then there was internet beans, which was like I think a plugin and you just like collect beans on different websites and they didn't have any value. And and that evolved into Bitcoin and and Bitcoin evolved into ETH. And and so I think this technology is just like an evolution of human freedom. We are trying to build systems that make us more free and I

I love Bitcoin. I think Bitcoin should should always be something that people consider holding even a little bit of money in um just for the fact that it is controlled by no one. Um and I think you know AI also falls in into the same categories. Like this is freedom technology and hopefully we can spread it out as much as possible. >> Yeah, I I agree. I basically have the same opinion and uh I I think

like people really need to realize that AI and they need to I guess think more into the future and you know realize that like whatever is right now it's it's not the same as taking away your ChatGPT description. It really is the same as like taking away all your technology and you know maybe it's more easier with like electric cars that like shutting off access where you can drive and you know it's easier for people to

visualize that. It's kind of harder to visualize that not having access to intelligence because people cannot again, you know, see the exponential. They cannot see into the future. Most people just like live paycheck to paycheck so they can't even project long-term. So I guess how can we like practically do something? Right? Like obviously we we can encourage our followers and viewers to like run AI locally, get into fine-tuning, download the model weights, but like what we

can do as like humanity society to really make sure like the future of AI is open. >> So let's let's make it actually like very clear to visualize what's going on. So right now in the United States we have Waymo and we also have um Teslas and they're developing the Tesla trucks and and it's been possible I think for the last 5 years at least to have mostly autonomous vehicles, right? So 35% of people in the

United States survive off of income from driving. They make their money in transportation, driving trucks, driving Ubers, driving cars and that money then is going into feeding the families by buying groceries, by paying taxes, by going to like events and then that so like the amount that driving touches is crazy. And the reason we don't have full self-driving like 3, 4 years ago and all the companies haven't moved to it is because the government is purposefully

trying to slow this process down. Because we are going to have like 15 to 20% of the male population unable to make money. So, if we if we just flip that switch on and we say it's fully legal now, go ahead, you can invest in it, you can you can work on it, uh it's going to be very hard for a lot of these people to like find a job. What are they going to do? I

mean, the you know, it's cheaper to use a robot to just drive around. So, then these people are not going to be able to pay their their neighborhoods and like you can start to see the cascading effect of like what will happen as soon as self-driving and it is it is officially in Europe now. And Europe has been really adamant on not letting this kind of thing happen. And now it's in the Czech Republic. Tesla's full

self-driving is active in the Czech Republic for the last 4 months, uh and it's spreading over to Germany. And uh you know, obviously it's in it's already in the UK. And so, we are seeing the ripples of this. Doesn't mean it's going to happen in a month or a year or 5 years, I don't know. Um but we are starting to see that this is developing. And so, what do we do when people cannot get a

jobs? There are very little entry-level jobs for men, right? I I I'm using men because I just I just remember my personal experience trying to get a job before I was doing programming. Like it's it's very hard to get a job that pays you anything that makes life worth living. And so, you're just being shoved out, shoved out, shoved out. Now programmers, like it's it's very hard for a programmer to get a job as soon as

if they're if they're starting, you know, college and they think that they it's similar to what was happening 10, 15 years ago where they can just put their CV up and and get a job. It's like more difficult. They have to be more like the average person now. And so, we're already seeing this happening. It's just It's just like we have to think logically ahead based on the past. What has happened in the past whenever technology

has been able to automate a bunch of jobs, where are houses and and all that? And what is the effect on it? In America, we don't have much uh warehouse like what are these called? Not warehouses, factories. Like all of the factory jobs have been outsourced to Mexico, China, right? And then you hear this from men now, 20, 30 years later, saying, "Well, we can't get a job. There's no union jobs. We can't afford a house."

Because we outsourced it, right? And there's nothing for you to do. So, unless you upgrade yourself, there's nothing for you to do. So, I think this is kind of like how we should visualize it. Now, if we take a a a step like towards positive, like how do we help? And so, I run three meetups. I have a meetup in Warsaw. And I just educate people. I just literally go and like, "Here is how you use

Codex. Here is how you use open models. Here is what this means." Um and I do this on the internet as well. I post these videos. Every single thing that I do, I literally post it online because when I was trying to start self-hosting, there was nobody putting this information online. I couldn't figure out like how much tokens a second I would get on a certain model on a certain hardware. Now, I can get that because

I'm making that data and other people of course that are also convinced. So, making data more accessible, educating people. And then last but not least is like, "What does an alternative economy where we help each other look like?" Instead of like me like trying to trick you and you trying to trick me, which is like a lot of the economy right now. Everything's like people trying to like gamble against each other, you know, it's it's it's

it's very corrosive. What does it look like if you and me can help each other make more money and obviously spread prosperity? Because I I think it would be very easy for us to figure that out together. It's just we like all of our our systems in the society is like I I Sorry, I know I've been rambling for a minute. I go out here in San Francisco. I usually live in Warsaw. But I go out

here in San Francisco. Um I have to get an Uber to go anywhere. I have to get a car. It's too far away. There's no public transport. Uh then I have to pay like a stupid absurd amount of money to get food. And then I get like this thing turned around and they're asking me for a tip. I go to a grocery store, they're asking me for a tip. Everything is so P2P. Like uh PVP, I

mean. Everything is like so aggressive, right? And uh I think I think that's really destroying our society completely. Sorry, I I I've been like so negative this this podcast. >> No, it's it's fine. It's fine. Also, I I would have asked about uh like why why Poland? Why Warsaw? Cuz personally, I I I moved to Katowice now cuz I used to live in in Dubai for 2 years and then it started getting bombed. So, I moved

back to Europe and you know, Katowice is close to Czech Republic where I'm from. So, I'm wondering like why Poland? Why Warsaw? >> So, uh why Poland? Why Warsaw? So, Warsaw well, one, I met my wife in the UK, I think like almost 8 years ago now. Um she is Polish. Uh then my UK visa ran out after COVID. Um I had to go back to the US and the process for getting her green card and

having her move would have taken 3 years. And if I didn't stay with them like I don't think a relationship could survive that long like on average. I don't think it's it's very hard for a relationship to last long like um you know, with people far away. So, we started exploring other countries to live in. Like we went We uh we went to we went to Mexico. Uh we went to Poland. We went to the Germany.

And uh I think I loved Poland like when I went there. Uh I really loved I loved the people. I think the people are like very libertarian-minded. At least the people in Warsaw that I know, like they're very like uh hard-working. Things don't close until 10:00 p.m. You don't see that in most of the world, right? Now, like leave like whether this should be the case or not. Like people work super hard. Um people are very

honest. Uh I think that's also really rare for people to be honest to you and like tell you things in your face. Uh and the government has been incredibly kind to me. Like everything has been pretty easy. I just uh I just put in a application. They asked me like, "Okay, what do you do?" Here's what I do. Here's your card and I'm good to go. Like it it's it's been really easy to interface with them.

Uh hopefully that answers your question. >> Yeah, it does. I mean, I chose Poland as well for similar reasons and you know, it's the fastest growing economy since COVID. It's like one of the safest countries in the world and I think it's on a great direction. Uh I just want to hear your thoughts on differences between San Francisco and Poland because personally I've never been to San Francisco. I think I need to go this year to

kind of experience it for myself. Um you know, you mentioned some of the negatives of it, but like what's what's your full opinion on San Francisco because that's like where most of AI is happening. >> Yeah, so I've been here for a month and a half. This has been like my second time to San Francisco. One thing about America and San Francisco especially is Americans they love opening doors for like open not opening like physical doors,

but like they want you to succeed most of the time. So, if you go somewhere like they will let you in. They will let you it's very easy to get into Apple. It's very easy to get into Nvidia. It's very easy to get like massive investments. Um it's easy to get connected to smart people. Um and uh it's because Americans in general they like they like the underdog story, you know, like David versus Goliath, the underdog,

the you know, like the Rocky uh like that kind of you know, somebody that is less likely to succeed succeeding because of like working hard. This is like an American uh fundamental like founding of the country story and uh I I see that everywhere I go. Um the city is extremely expensive. I think a hotel uh the cheapest hotel you're going to get is going to be $6,500 a month. Uh forget about Airbnbs um because I

would not stay in an Airbnb uh with other people. If I and if you want to do it alone, it's $8,000 to $9,000. >> [snorts] >> Food isn't extremely expensive. Um like it but the city is it it's like a slice out of heaven. Unless you go to the like middle. So, that you know, you have the top, heaven. The bottom, heaven. The right, the left, heaven. You go to the middle and it's like a zombie

town. And every time I come back here, it's shocking to me like how we can accept this being a reality in this country. Like people just like it's like a zombie land. I've never seen it anywhere else in the world and I've been to so many places. It's it's a zombie land. So, that that's a bad thing. But, there's nowhere you're going to be able to succeed more than America. Every time I come to the US,

my income like almost doubles. Um just and seriously, it's like it's it's just such good thing for business. Last thing is that the energy here is insane. Like the people here, like they're in a bubble for sure. Like in in their you know, in they're in a bubble. But, that like they are some of the smartest, most like driven, high talent, high agency people on the planet. I think you I think you would have a wonderful

time here. Just make sure yeah, to be ready like for the financial situation and for food to like keep your body healthy. It's going to it's going to require like some attention. But, I I I think it's one of the best places you can go, honestly. >> Yeah, I feel I feel I need to do it just for the vibe and you know, to see like really the difference. Maybe not for permanent living like you said,

you know, I think I'm happy in like Central Europe. I think it's actually one of the best places in the world. It's kind of crazy because I've been born here and I only see it like you said after traveling to different places, right? And like experiencing everything else that I realized how how blessed I was where I kind of spawned in this world. But, yeah, I guess a lot of people want to get into local models,

self-hosting, all that stuff. Where would you recommend them get into? Would you recommend them spend couple thousand dollars? Would you first tell them like try running something locally, whatever your computer can handle? How should someone start? >> So, okay. Any computer right now is able to run a model that is capable of tool calling and doing stuff. Every single one. So, try that. Working like I I LM Studio is an excellent start for most people, whether

you're on a MacBook or on a Windows or LM Studio is going to help you get something running, right? It'll tell you what can run, it'll help you get it running, and then you can see that. And then if you start thinking like, "Okay, I think I want to invest in this. I think I want to spend money on this." What I would recommend you do is go on to RunPod or Lambda Lama not Lama, the

other one, Prime Intellect. Yeah, Lambda and just go and find the device that you were interested in purchasing. Rent it for like two to four hours, run a model on it, and then see what kind of performance you get out of it. See if it like if it feels right for you. Once you get like experience tinkering on your own machines, and then you can make a decision from there. I think there are like some standards

that I would say if you are already convinced like you want to buy this, like the the standards are if you want a lot of memory and you don't mind speed, Mac is a good option. Or the DGX Spark. If Yeah, but that's a little more expensive. If you want a lot of speed, like frontier level speed, you need to go Nvidia. Unfortunately, that's the only option that makes sense. Maybe the AMD, there's a 7900X card.

Hopefully, that's right. But maybe AMD will get you close on some models, but I would recommend Nvidia. For the Nvidia chips, the 3090s if you can get them for $1,000 or less are excellent cards. The 6000s are the next best option. I would not touch the 4090s, the 5090s. There is a card called the 5000 Blackwell. So, it's similar to the 6000, but it's half memory or 72 GB. So, you get like increments that are the

same price. It's very expensive. The tiers basically the Quen cuz we have a web we have a a benchmark that we've been running. So, we took the 23 most popular models and we benchmarked them and all of their compressions, 4-bit, 3-bit, 2-bit, 8-bit on five benchmarks like terminal bench, sweet bench pro, these like agentic benchmarks, GDPEval, the best ones in our opinion. And what we found is that from zero like from let's say $250 to $9,000,

the only model that will make sense is either Quen 3.6 27B if you want really smart and slow or Quen 3.6 35B. The Gemma models are good like they have world knowledge, but they're not going to be able to do anything like cloud code in my opinion. Um then after that when you have $9,000, you can get two sparks and you can run a model called step 3.7 flash. I would say that that is like truly

phenomenal. So, not 4.5 level. Like it's like a big step up from from the Quens. Um and that would run on $9,000 of hardware with full context and like maybe two to four concurrency. And then after that like it's going to cut like you're going to spend another $10,000 to get the next best model. And that'll be something like MiniMax or Deep Seek V4 flash. And then last but not least like where I'm at is like

$50,000 and you can run GLM 5.2 with like compression. You can run MiniMax M3 at like 8-bits. So, you start getting some good like actually good models that are frontier capable. But that's the current price. It's 50,000. What's exciting is that last year it was 100,000 for the same like tier of intelligence. So, it's one getting cheaper and then two it there's more like tiers that you can jump through. So, keep that in mind It's like

if you have less than $10,000, I would just go with the whatever can run quant 3.6 with full context, maybe two to four concurrency, and then you have a bunch of options on how you would do that. >> Okay, so 50,000 for like >> 46,000. >> Yeah, the best best open source models, but roughly what tokens per second? >> Uh so, with uh GLM 5.2, I'm getting between 60 and 80 for a single stream and up

to 250 for like uh four, you know, like for yeah, for four it's 200 tokens a second uh for four like concurrent. Yeah. >> That's really really decent. Okay. So, how do you think about the setup that's like built in a way that's easily upgradeable? Like Like personally, I would be, you know, interested in like maybe like the 20K range. Like what would you recommend me first of all the hardware and like how to think about

it? How do How do you think about it when you build a hardware? Like are you like planning, okay, in the in the, you know, in 6 months I'm probably going to have a 100K worth of hardware or How are you planning about it in a way that's like you're not buying things that like get bad very fast? Just tell me in general how you how you think about it. >> So, uh can I share my

screen? Is Is that okay? >> Yeah, of course. Of course. >> Okay. So, uh basically like there's a few things I would go through. One is like the type of hardware again that you're going to get. Uh I'm going to go to bookmarks where I have a lot of stuff. So, this is like this is what we currently have for the benchmarks. Um each one of these speed is like the actual uh So, for the benchmarks,

it's a sample, and each sample is like between 64,000 to 200,000 tokens, uh and it takes 5 minutes to solve a sample, and it solves 85% of the samples. So, uh we're breaking it down like this. So, next is like when you have a budget, so if you have like let's say uh 20K, uh I'm going to go to hardware here. So, what when we have a budget of 20K, what you can spend is like you

can get four uh DGX Sparks. Uh the reason I say this is like uh each DGX Spark can link to another one. Um and you can hack it to link so you can daisy chain uh up to four. You get about 1.5 x speed up per node that you add in. So, and with four DGX Sparks 6000 uh DGX Sparks, each one is 128 GB. So, if you take 128 * 4, that's 512 GB. Um you

take this, let's say * 2.5, you get like 600 uh GB a second memory bandwidth, give or take. Uh and then you can see like what models are running and how good they are um for this range and it tells you like so that this is just essentially how I'm trying to walk through it myself. Um the price point is like 17.6 thousand. So, I think about increments of some base component. Now, the base components would

be like a 3090. That's a base component that you can upgrade in increments of doubles. Um you don't want to have three 3090s. You want to have 1, 2, 4, or 8, or 16. So uh each one has 24 GB. I know this is a little a little bit math heavy, but it's like there's not much of a better way to explain this. Um so, each one has 24 GB and let's say costs $1,000. So, if

you take that and you multiply it um by four, you get 96 GB and you have to spend $4,000, which is a little less than half price for what you would have to pay for 96 GB with the RTX Pro 6000. And you get half the speed, of course. Um also, uh so, the increment of update is very important. Like, do you want to have 16 3090s in your house? How are you going to cool that?

How are you going to get power to that? So, if you want models that are like 512 GB, whatever, I would not touch the 3090s. This is going to like limit you at I think 96 to 192 GB and you're it's already a problem. So, um and please stop me if at any any point you have >> No, no, keep going. Keep going. It's great. Keep going. >> Okay. So, this is like a little chart I

made in the past. I'm going to open it in a new tab. So, this is kind of like what you Now, we just have to 1.5x the cost cuz right now all of this all of this is like 1.5x the cost as as when I did it. Um for a Mac Ultra, you get a lot of memory. You pay $10,000 or $15,000, you get 512 GB, but it's slow. So, any model that is large and has

high active parameters is going to be slow. So, take a look at this this chart. I think uh it's if we can understand this like it's better for our decision-making. Uh 1 second. Okay. So, we have uh MiniMax M2.7, which has 229 billion parameters, 10 billion active. We have Kimmy K2.5 2.6. It has 1 trillion parameters and 30 act 30 billion active. Um the GLM 5 has 744 billion parameters and 40 billion active. So, the more

active parameters, the more intelligent the mod like the more intelligent uh intelligence you get per token that is generated. >> [snorts] >> Um with Qwen 3.627B, it has 27 billion parameters. So, it's actually almost as intelligent as a model like Kimmy. It just has less range. Like it knows less stuff, but per token it's as intelligent as Kimmy. Um so, thinking about it that way, so either you come in with hardware that you already own, and

then you need to be matched to models and configurations that work for your hardware. Or you come in with a price point that you want to spend, then if you come in with that price point, you need to look at the models that are available and see what can fit at what speeds. Then you can come in with a model that you want to run and then that dictates everything else. So, if you want to run

GLM 5 and you want to have more than like 15 tokens a second, you're going to need to get Nvidia hardware and you're going to probably need to get the 6000s because anything bigger than that is way more expensive and anything smaller than that you would need to build out like a a 24 um 24 card rig that is just not sustainable. Uh does that kind of answer your question? I know it's a lot of words.

>> For sure. No, it's amazing, man. Uh I I have so many follow-ups on that. I guess I I would approach it in a different way. I would say like, how would you advise me to go in the direction where like in 6 months I would have strong enough of a rig to run the best open-source model in 6 months at at at the best speeds possible. So, like I don't care how much it costs, right?

Obviously, not like millions of dollars, hopefully. But like, how would you advise me to start building in that direction? What would you buy right now that like would be still useful right now? Like hardware that I can run, you know, smaller models, but in a the main strategy being that like in 6 months I don't mind spending 100k to get the best model in 6 months to run it locally at the fastest TPS. >> So, I

think the answer is the RTX Pro 6000 for you. If you think in 6 months I'm willing to spend $100,000. Each card is worth like, let's say, $10,000. So, in 6 months you can have six of them. Uh let's add another 2 months. Like you have eight of them. If you have eight of them, you have 768 GB and you can run every single model, all of them. Um and you can run them at like high

precisions with full context windows and at like 100 plus tokens a second. This is This is applies to Kimmy, this applies to GLM. So, all of the models that are the current best models, you will be able to run if you spend $100,000 and you have eight RTX Pro 6000s. Now, if your budget is, like let's say uh 20% of that, like $20,000, I would say buy the um the DGX Sparks cuz each DGX Spark has

128 GB and uh it's really good at training, it's really good at prefill, fine-tuning. Um it's slower at the token generation speeds, but if you run mixture of expert models with like 10 billion to 20 billion active parameters, um you could get maybe 20 to 40 tokens a second, um closer to 40 if you really sit there and optimize it. Um so, based on however whatever your monthly income is, you you should take a slice of

that and you should think I'm going to invest this in compute and then based on that you can make the decision. So, if your monthly budget is like you have $2,000, I would say maybe go with the with the the the what are these called the DGX Sparks. If your monthly budget is like $5,000, maybe $10,000, um you can go with the the RTX Pro 6000s. If it's less, then it's best to get like let's say

you want to start out and you have $2,000 over 6 months, get two 3090s and you can run all the Quen models and all the Gemma models. Um does that give you kind of a good idea? >> No, it's great. Like I have a good idea. I'm going to go with the RTX Pro 6000. The reason I don't want to start with the 3090s is because I already have a MacBook with 128 GB of memory. So,

like I feel like the two 3090s would probably be similar or even less uh less powerful than that. So, uh and does that even change like your your question if I have the MacBook? Like should I think about like maybe getting a Mac Studio to like like pool the VRAM somehow or not not really? >> So, the currently Exos is uh allows you to take like multiple Mac devices or Macs and DGX Sparks and you can

combine them together. Uh and you get two things. Uh if you combine a Mac and a DGX Spark, you can run prefill. So, this is prompt processing. You can run it on the Spark, and you can run token generation on the Mac because this one is better here, this one is better there. So, on average, you get double the memory, and you get something that's like maybe 1.5 to 2x faster just by having these two. Um

if you were thinking about like stacking Macs, you also get a little bit of a speedup. I think Apple has been really good at uh going faster in their uh a a like AI hardware development uh cycle. Uh the upgrades with Mac are cheaper, also. Like, you can you can get like 512 GB more for $10,000, $15,000, which is like a like you cannot get that with a a $100,000 uh you need $100,000 with the Nvidia

to get close to that. Um so, keep that in mind. Uh though, if you really want frontier speeds, like the only option again, it it is Nvidia. So, if that's if you want something where you can actually replace cloud code and not know the difference, or like code X and not know the difference, you need Nvidia as of now. Um there's one thing this is still experimental. This is in the research phase. Uh a lot of

people also knock this idea is So, if you have a mixture of experts model, like let's say Kimmy or uh we're we're going to use GLM 5.2. The 40 billion of those parameters are active. Now, there the the LLM process is two steps, prefill and then decode. So, prefill requires that the entire memory be loaded into like VRAM or high-speed memory. Um decode is whatever the active parameters is plus whatever the specific math of that model,

but it usually comes down to like 1/10 of the requirements of memory for the decode step. So, is if it's possible for us to connect an Nvidia RTX Pro 6000, for example, and uh let's say DGX Sparks together, and efficiently and quickly move the bits of the attention. So, this is the token generation part. If we can move it over to the RTX 6000, we can get Nvidia speeds with like much cheaper costs. Um so, it's

an accelerator. Now, the labs, the laboratories, like the real giant gigantic labs already doing this. So, when they have like AWS titanium and they have Nvidia GPUs, they can do the like stuff on the titanium um and get the speed benefit of both. Um but, we as consumers, as end users, we still haven't gotten to the point where this is like a viable option. Does that make sense? >> Yeah, I see. So, you're you would predict

the trend continues like the hardware gets cheaper like in in 6 months or in 12 months, it's going to be half again? >> I think the cost of running like much more intelligent models is going to be half again. I don't think the cost of Nvidia hardware is going down anytime soon. Um they just don't have enough like capacity to to fit the demand. So, it's like the price of hardware is not going to get any

cheaper from my opinion. The price of running like frontier intelligence is going to get cheaper. >> I see. I see. >> Yeah. >> So, basically, you if you are planning to buy hardware, you would do it sooner than later, basically. >> Um I don't I don't think I don't I don't think it's healthy to like say that to people because like maybe people can't afford >> No, no, no. Like I'm not selling the message. I'm saying

like for myself like how I should think about it. >> I think I think you should think about it. Like if you have the budget and you use this stuff and depend on it, like you should have a budget for it and you should have that budget go into ownership. Like a mortgage versus rent. If you're going to do something like your entire life, would you want to rent it the entire time or do you want

to just like purchase it uh you know, over increments? And I think that's that's a good way for us to to approach it. >> Yeah, actually, as you say that, like I kind of realized like my strategy is is kind of renting strategy where basically, you know, I'm spending like right now between 5 and 9,000 a month on open router subscriptions for myself and employees and everything. And like, you know, a slice of that could be

going towards building the rig, where like, you know, obviously I will not get the same quality and TPS right off the bat, but like, if I do that consistently for 6 to 12 months, I'm going to have enough hardware to run it locally, and then I can cut off, you know, basically all of those subscriptions. Maybe just keep a few to like experiment with the best closest models, but like fully own my local setup. So, like

when you position it like that, it makes complete sense. >> Mhm. Yeah, so companies companies like if we if we start thinking about us as people, like if a company is spending $300,000 a year on Anthropic subscriptions or like Anthropic billing, it is not beyond the realm of possibility. There are companies that are spending like tens to hundreds of millions of dollars on AI right now that are not in AI. So, why wouldn't those companies think

logically? Like, okay, our like do all of the requests that go to to Claude, Fable need to go there? Do Do they? I don't think so. So, can we then purchase hardware to like like basically reduce the costs long-term of what we're doing and also have like some some of our own private, you know, private inference, some of our own training data that cuz each company is just training data, right? Like everything that they do, how

they interact with the browser, how they use Slack, that's all training data. Can be installed and they can just train a model and make it more and more and more accurate on their own stack. So, I would think of it as an like as a long-term investment for your own personal sovereignty is is essentially the best way to to consider it. Last thing is like if you actually buy hardware and you have to fit it together,

you have to learn from the bottom of the stack to the top of the stack. And that knowledge right now is so expensive. That knowledge is like what people pay for, right? This is a giant industry that's like being born, and they need people that do this. So, you know, you ultimately will just have an advantage over the rest of people cuz you know computers better. >> Yeah. And like there's going to be companies that like

will happily pay you big money to transfer, you know, their crazy Anthropic bill or crazy OpenAI bill towards like some self-hosted solution. Or maybe like even like help help you set up spin up some like GPUs in the cloud. Uh so, it's not like fully self-hosted, but you know, like you said, if you do it locally for yourself, you're going to have the knowledge that like companies are going to pay for. And I think this is

much better skill than like, you know, just going to a company like vibe coding a front end for them and calling that like AI implementation. This is the serious AI implementation. And there is still many, many companies that cannot really use frontier models because of, you know, terms of service, private data, financial, health care, you know, legal data that literally they legally cannot send to OpenAI API or Anthropic AI. And they don't even want to, probably.

That these companies don't want to fall behind. Nobody wants to fall behind, right? They want to use great AI models, but they just don't know how. Or they don't know it's it's it's much more possible than they think. And a lot of companies could afford to spend 100K on a local hardware rig, but they just don't know how to spend that 100K. They don't even know it's possible. Even if they spend that 100K, they don't know

which models to run. So, yeah, like you said, that expertise, I think it's going to become a like a like a actual position that a lot of people go into these companies and implement these setups. >> I have a uh I have a good friend in Warsaw who created a company, and they basically helped like launch at least like four, you know, billion Polish uh złoty companies in like in Warsaw. Like they've had their hands in

a lot of the most successful companies in Poland um as developers. And they um they basically started buying compute a few months ago or like a year ago. Uh and now they have an entire data center. Like it's got like multiple DGX sparks, multiple 5090s, 6000s, like they have a few B200s. Like they have it they have everything that they might need and they use it. Like they're using it all the time. They they are like

giving it to their employees. I also met a company in Germany who is doing the same. Like they are like it's a software development firm and they are just buying 6000s, buying B200s, you know, incrementally over time and they are creating a data center in their own company. And they service other companies. So now they can use their inference, right? Like this is German private inference. This is, you know, we are your partners. We will never

take anything from you and they can trust that. Fable, if you want to use Fable, you're not allowed to keep your data private. You have to share the data with Entropic even as an enterprise, yeah. So I don't know, you know, it's like it makes more more sense. It's just so expensive to get into for the average person. Like people feel priced out. >> Yeah. But again, like people need to look into the future and, you

know, see where it's headed and that like access to intelligent is going to be essential. And you know, people already do this with like solar panels, you know, people understand I want to have my own energy. But that's because electricity has been around for 100 plus years. AI, you know, also has been around for a long time, but like useful AI is only like, you know, 5 years old or whatever. So people don't really see the

need for it. They don't understand the implications of it. But I think the issue is that it's advancing faster than any other technology. So I think they they do need to see the need for it. >> This is this is hopefully it works and it doesn't embarrass me, but I will launch this. So this is running at home. This is my GLM 5.2. It's like a custom compression that I did for it. Okay. But it'll run

in a minute. So then we have so it's running now. It's doing its thinking. Yeah, it's doing its exploration. It's fetching its skills and it's going to make me a nice website. But if we go here, like this is the same model, right? This is a compression. It's It's compressed 80% like uh on the original. Typically, you can get about 75% but I've like learned how to push it a little more. And so, you know, this

is 3D, you know, 3D, what is it called? The the Flappy Bird. Like this is usable, right? And this was a single shot uh attempt. Now, I also have like um I'm trying to This is another one. So, this is running DeepSeek V4 Flash. Uh there are They're going to be two sessions left and right that are running. So, you can see it's running pretty fast. It's able to do like a concurrency. It's doing its tool

calls, reading. Um this is a even smaller model that you can probably run on $9,000. Uh I told this to uh reorganize my downloads folder, which was a mess. Just like reorganize it. I don't really care how you do it. Um let's go over here. So, you know, it's reading things. It created a bunch of folders. It's figuring out how to categorize. And it's just going to start moving things into the right folders in a second.

You can see that it's like disappearing. Yeah, it's moving those files into the right places. And all of this is economically valuable uh work, right? That would have taken me maybe 30 minutes to 60 minutes to do. And now I can just like, "Hey, organize my downloads." I could go do something else. Uh I saved myself that time. Uh this is a model that would run on uh two RTX 6000s or maybe like uh two um

DGX Sparks. And it's doing its job. Um this is basically DeepSeek V4 Flash on both sides. One of them is researching like inference engines. One of them is researching GPUs. So, the capabilities again are significantly better than they used to be. And now this is thinking. It's doing its job. And so, I can also like launch another one if I wanted to. And like I'm going to do Pi here. Um model GLM 5.2. I can do

this with um uh actually, let's do it with Droid. So, Droid typically would cost you a pretty significant amount of money cuz they only do API-based billing. I think right now they they're experimenting with usage pools. Sorry, by the way, just a second. There you go. Uh, hi. Hello. What's in this repo? So, this is a um like private inference that I'm running at home that is capable of doing everything that the like frontier models is

capable of doing, at least for work, right? Um, so I'm I'm very happy with the with what I got. I think it's worth the investment. Uh, and yeah, it's going to be slow cuz this is like the largest model that you can uh practically run at home like from a active parameter count size. Uh, hopefully this kind of gives an idea. I could also run like these smaller models. I can run like I could give inference

to maybe like 24 people uh with the cards that I have. Go back here and you can see that the model is running. It's uh around 62 tokens a second. Um, GLM 5.2 and you can see how much VRAM it takes like uh, how much Yeah. It's It's It's a pretty great model. You can see my usage per month is like So, I'm using 374 million tokens a month locally. Now, remotely I'm using much more, honestly.

Uh but you know, it's it's it's viable. The Does that make sense? Like it's viable. It wasn't viable last year, now it is. >> Absolutely. Yeah, so like you know, with the all the techniques, quantization, all all the research papers from China, it's basically becoming more affordable to run useful models locally. Maybe not the cutting-edge ones because the hardware is, you know, becoming more expensive. But, yeah, people that they, you know, tried self-hosting maybe 2 years

ago, they should try again because they can get much better models, not like Llama 3, you know, 7B or whatever, that like was kind of fun to play with, but not really work usable. So, I guess the strategy Do you see the strategy is like larger percentage of your total token spent? Because I think most people watching this believe in AI and like the tokens are going to go up and like you know each usage like

my token usage is growing every month. So how do you see like do you see the strategy of like more percentage is going to be self-hosted? Like you you still use like some you know cloud. >> Mhm. Yeah, it's definitely more and more of it is going to be self-hosted. Like I'm going to share my screen one more time and just show you kind of what I mean. It's like right now we think of inferences you

know coding cloud code codex but there is a million things that you can do with it. You can set it up in a robot. You can have it control your lights. You can do you know you can have it be on your watch and like take care of your health. Let's see where is this. So this is Mario the creator of Pi. This is Gemma running locally. So the like what is controlling the computer is Gemma

running locally. So I'm going to see it follows him around you know it does like I don't know if it's going to move in this video. Hopefully it does. But this was a toy that he ripped up. He broke his children's toys and like put them together and it can play him music. It can like run around like So this is what I envision is going to be the interesting amazing future of like local and self-hosting

is like there's so much we haven't even touched yet use case wise that is going to be enabled by people just tinkering at home. >> Yeah and even in in like you know traffic lights like it's kind of crazy that like traffic lights are like zero intelligence but you know there's one way there's zero cars going and like the red light with like full lanes and like everybody's waiting nothing is happening. It's just dumb and it's

slowing down society because there's no intelligence there. I mean you could probably solve that without without an agent you know with just some some nodes and like if all statement but you know the point is that like everything can be improved by having some intelligence. Like for example you know I'm I'm tracking my calories with open claw or Hermes like whichever one I use and you know it's it's kind of obvious and some people make fun

of me for using like Opus 4.8 fast for that but like you can ask it custom questions based on like okay, I'm going to the gym. What are some quick carbs I can eat? How much calories I have left, right? Like I I have a podcast, I need to be sharp. I I need to fill up my protein for today. Like custom questions, right? You're never going to get that with a basic meal planner app or

whatever. So, yeah, intelligence is super valuable everywhere. And I'm also wondering like how you think about your own setup. Do you plan on like adding more 6000s or or yeah, how are you thinking about it? Or like are you limited by electricity? What would be the most of the limit? Let's say like money aside, obviously money budget is the limit. What's the limit? Like is just the public grid? >> Yeah, there there is there is So,

I power cap my GPUs at 40% capacity, all of them. So, they just work at 40% of what they're capable of, which is also slowing down like inference speed by like 20 to 30%. But the power gains for me are worth it. Um I already have like I'm already So, I already have like an upgraded house like the electricity has is upgraded. I'm going to upgrade it again this this year in September. So, I'm going to

get more power. I want more like four more 6000s. And I think after that like there is not much sense in upgrading. What I want to do though is this friend that I told you he has um he has like basically a data center now. I want to move my rig into his place. So, the the the current limits, it's loud, it's hot, it's it's expensive. Okay, I'll leave that aside, but the heat is a problem.

Like how do you get the heat out of the room if you're running like real inference on it, it's going to get hot, like it's going to get loud. It the electricity the power, you know, you're you're going to need two circuits to power eight 6000s. Or if you're in America, you need four circuits to power eight 6000s, which is honestly to me a little bit ridiculous. Um so, those are the main limitations. But it's it's

not that hard and I think it's worth it. So, like this is the report that the GLM model um, spit out while the what we were asking it one second browser. There you go. So, it made this uh UI and it's basically explaining to me what a what like the Pi agent is. It's telling me it has 73 skills, etc. Like it built this UI in something like uh 2 minutes maybe. Uh and yeah, it's like

for me like this is more than enough. Uh make me a uh make me a video game uh 3D maybe like a car game. Maybe a car game. Okay, let let it let it do that and then we go back and see the response here. So, this is Inworld which again you would have to pay a lot of money for um because it it is probably the best harness, but I can use it for free because

I plug my own inference into it. Um and uh this is this is this is essentially uh what I mean is like it's worth it. It's worth it. If you're spending more than like 2 billion tokens a month, it's worth it. Um So, I am going to upgrade, yeah. >> While that's building, I guess what's your thoughts on like uncensored models and uh and fine-tuning that you think the government will like come for you if you

have a uncensored model at home? Like how how does that work in your mind? >> Yeah, it it it depends on the scale. Like how much how much people are using it and like whether it's going to get into the news. Like at least in America I'm I'm only operating with America in mind cuz that's where I'm from. Our politicians are like driven by news. So, if there's news that something bad happened, doesn't matter if it's

true or not, it's going to get into politics. If somebody wants to give them money to like bring a topic into the limelight, it's going to be there. So, I think it would be useful to have like some kind of organization like the Human Rights Foundation that sets a budget to lobby politicians to keep them away because essentially they're just waiting for money, right? Like that's that's the truth of it. Like they they want to get

paid and they're going to get paid from here or here. So, you got to bid for their attention here in this country. Um Now, this this goes back to the conversation earlier is how big is the scale and when does it get into the government's mind that, you know, people are running this stuff at home and they're doing this this stuff. Um if that does happen before we get the chance to spread this, it's going to

be a problem. Uh but, I you know, I've used uncensored models. Uh what the one that I liked was Hermes 70B cuz they they train it from like not from scratch, but they like do a like heavy post training on it to to remove all sensors. So, you can ask it about pretty much anything and get an answer. Um and sometimes I need that because uh I took a I have a cactus that is a peyote

and I need to take care of it. Like I'm not using it as a drug. It's just a beautiful cactus that I was gifted, but I don't know how to take care of it. So, I'm trying to ask GPT. I take a picture. It's like, "How do I take care of this?" It won't answer. Claude won't answer. Like they're like, "Oh, we can't we we can't talk to you about I I try with Hermes. It's like,

"Okay, this is this this is that." You know, you you get the these types of rocks and you water it once every 2 weeks. So, I could get that information off the web. I would have to just have to spend like 30 minutes like reading random blogs and looking at ads and, you know, correcting my mind. >> Yeah, so it's just convenience. So, I guess the strategy is basically spread the message without like, you know, showing

too much of a crazy use cases. And to get as many people Um like what would be the first steps? What do you think people should like download the weights? You think like Hugging Face is at any any risk of being shut down or or you think that's overblown? It feels like that. >> Uh I there's a chance that I am misremembering this, but I had spoke I had the guy a guy from Hugging Face on

the podcast, Victor. He's the head of product and he told me that they have gone like the Again, I might be misremembering this, so I'm sorry if that's the case, but he said something about the government like the French government trying to take down data sets, like putting in requests to take down data sets. So, uh if this happens, it's probably going to be like they start taking down models and data sets, not that that they

take down the entire site. So, I think it starts like that, right? Like they'll they'll go after whoever is launching the models, or they'll ask uh uh Hugging Face to basically remove the models from the index um or the the the weights. Um I think it's worth installing the weights, mainly because, you know, I used to use torrents when I was a child, right? That's how I got everything, all of my games, all my shows, and

the the torrenting culture requires people to have the files. You just have to have the files. Um so, it depends like how much memory do you have? Are you willing to Is this something Do you care for the 1 TB that it might take to to to download a model? Like if you do, then um and you you don't care about this cause, like I I there's nothing I can do to convince you. But I download

them. I store them, you know, I have 6 8 8 TB now. More, like 12 TB actually. Um and I have like a bunch of models stored, and that's not even nearly enough. I need more. >> And storage is also way cheaper than the GPUs, right? So, like people can easily acquire 20 TB pretty cheaply, and uh download like thousands of data sets and they know all of the best models, basically. >> Yeah, yeah, and you

can you can you can seed the torrents if you want, like, you know, in the future if that if that's the case, or you could just have them and you can, you know, they're great research subjects. Maybe eventually you'll be able to afford to run GLM 5.2, and if you have it, you cuz how much how much smarter are they going to get? Like this is the this is the the the big core question is Are

we close to a limit, or is it like is it possible for 1 TB of data to be that much more intelligent? I I can't tell. Like I I I I don't have the knowledge and skill to tell. >> I mean, do you see it slowing down? Because I I don't feel that way. I feel like the the models, like the progress in the small models is even faster than than models. >> I don't see it

slowing down, but I'm in a I'm like in my own, you know, bubble of cuz I still ask people like, okay, why like people still don't like AI, normie people or even artists or, you know, they don't like it and I just don't understand Maybe maybe I understand, but like I just I don't get like how you don't see what the Maybe it Maybe I'm crazy. I don't know. >> Yeah, yeah. I mean, I don't know.

I feel the same. Like it's literally the greatest technology of all time. It's here right now. It's our time. You know, it's the biggest technological in history and uh Like it's the most important resource of the future is the intelligence and like it feels like nobody's really paying attention. And then when a new model comes out, it's this crazy. I've got to go three layers deep, right? Nobody's paying attention to like AI percentage-wise from the people.

Out of all of my like friends, I was shocked how many of them disregarded Fable 5. Like how many of them just didn't take it seriously, didn't spend like every waking hour of it. It's like, "Oh, yeah, new model. It's a kind of expensive for most workflows, whatever." I'm like, "Guys, this is increase. This is a step increase, right?" Like Are you not true believers? And then when it got shut down, I was also shocked by

the even smaller percentage of people who realized the implications of this. It's like, you know, if the government controls the future of intelligence, you know, once we have AGI and beyond, like it's over. It cannot be centralized. And again, each layer are just like less and less percentage of the people. So I agree with you. Sometimes I I do feel like I'm insane because like the percentage of people that really understand where AI is, what it

can do, and also like the implications of the future, like where it's going to be in two years from now and the implications if it's fully centralized, like it's such a small percentage of a fraction of the people. It really is crazy. >> I want to point like I I have to leave at the top of the hour. So also tell me if we've been going on for too long. I'm happy, but I want to ask

you like a few questions. I mean, you know, your audience is watching this, but is that like what got you into into going this hard cuz I've I've seen you for the last year on YouTube and yeah, I'm aware of like what you do and like what made you go so hard on this like industry? >> By industry you mean AI as a whole or like specifically like open source local self-hosting like what? >> I think

both. Maybe you can start with AI as a whole and then like we can talk about local. >> Okay, so I'm going to go back a bit more. So when it comes to like some of my my more libertarian values, like I always had them even when I was like 11 or 12. I was explaining to my you know classmates in a free lesson like centralized banking and how it all works and you know they didn't

understand me because we were like 12 years old. But when it comes to AI for me because I'm 24, it's really the first technological revolution I can take part in. I was you know dot-com bubble happened like when I was getting born or even before. Um then like social media wave is really like 2004-2006. That was the best time to start social media. Mobile mobile apps even like you know 7-8 2010s. Bitcoin best time to buy

was like you know since it's launched until 2013 whatever. So I was basically like way too early. Like I didn't have any money at like 11 years old, right? So like I was way too early to any large technological revolution and this is the first one that I can actually take part in. So that's why like when I understood this, I just dropped everything because before I was doing like gaming content on YouTube channel and I

was making like really good money like 20k a month at like 20 years old, right? Which is great money, but I I completely stopped doing that because I just realized this is it. Like this is my chance. This is my first big technological revolution and it might be the greatest one of all time, right? And I think it's going to be the the biggest one of all time. So that's like why I switched to AI. When

it comes to open source and you know running AI locally and uncensored videos, like you know I have like a lot of videos that are like 200k views on this topic and to me it's just I don't know. It is obvious that like why should Dario Amodei decide what I can ask? Who decides this? Is some European bureaucrat? Is it like Ursula von der Leyen? Is it some like American politician that's like you know lobbied by

somebody? It's like no, I will decide what I ask, you know? Like I I always had like these libertarian values. I also a huge believer in Bitcoin like you. Like to me this is obvious, you know? So something's like you cannot really explain but it's just obvious. Like I I don't think the the future the most important important technology of all time should be centralized by some politicians or, you know, a few groups of of company

CEOs. >> And what about your audience? So like how have cuz you probably read their comments, you probably interact with them. Like how do how do you feel like the average mindset around this is? >> Yeah, so I mean I you can I can maybe screenshot my channel because I'm running these polls and I the last two were on open source. So let me just screenshot that because the results are overwhelming in favor of like open

sourcing and uh and uh self-hosting. Let me screenshot here. Boom. Yeah, so here I I voted like uh who should be in control of the future of AI and you know, 82% of people said uh nobody, it should be open source. Like 6% private companies, 5% governments, 7% don't care. The latest poll I did why do you want open source AI? Cheaper free access, privacy and control, don't trust these AI big AI labs, big tech for

all. So yeah, I mean uh uh how would you I guess interpret this? Does that answer your question? >> You know, that does answer. I mean like if 2,600 people voted and you have like 400 like 300,000 like that's a good amount of people that are interacting. Yeah, you have 400,000 good amount of people. 1.5% interaction rates like are pretty high even on Twitter. Um so it No, this is this is good. I mean uh my

my cuz I'm really interacting with a lot of people from the closed source world. Uh I cuz I have a contracting business and I've worked with like pretty much every single company or like had some kind of business dealing with every company and I don't think people there see it the same way. Maybe it's because they're already inside and it's like it's hard for them to imagine like not being inside. Um so that I think that's

where it's like the the opinions of other people outside of our niche, outside of our like communities is is still so far away and like mentioning them closer cuz it sounds like a sort of scam pitch or it sounds like a you need to spend money on something a pitch, you know, like it's it's hard to sell it as like a freedom technology. Yeah, the meetups the meetups man. I do the in-person meetups and I think

that's where I I see like the most impact on people. If you can meet people in real life and like explain to them it makes a big difference. I also saw you like did some fine-tuning. I saw you did some like a local inference. You've tried it out, right? What's been your experience there? >> I mean only on my MacBook, right? So like that's why I purchased a beefy MacBook. Like if I could buy a MacBook

with 256 GB of RAM, I would just do it instantly. But unfortunately, they don't sell them. If somebody from Apple is watching this guys, please next generation of MacBooks, at least double the RAM. But yeah, like I don't know even when like Llama 3 came out, I was doing a fine-tuning videos. Obviously, nothing as impressive as you. But to me, it just always seemed cool, you know, having your own model. Like isn't that like cool? Just

your own model. You can have it the only person in the world. It's just the idea of it seems good. >> I went to this office in Germany of a company called Micro AGI. So their business like they have like a few businesses. One of them is Micro AGI, but their their core business is they pay people to do jobs and to like record what they are doing. So like if you're a what it like a

mechanic, you can wear this device and they pay you. This is like a contracting thing, kind of like Uber. It's not like a company forcing it's So it's more like an Uber deal like end-to-end B2C. So they they do that and then they use that data. So I went to their lab. So they have the Unitree robots. Are you familiar with the Unitrees? >> Yeah. >> So, they have the Unitree robots. They have a DGX Spark

and the DGX Thor. They have the What are they called? The Meta headsets, the VR headsets. And they are using the open-source stack like released by Nvidia. It's impressive how much open-source like real open-source models Nvidia's released, the training data, the scripts to run it, the environments, the actual end weights, the the base models. But they they they showed me how Gemma 4, like the the the the 4B, was able to control this robot to like

help it like make decisions in the real world. And they're training it like how to pick boxes up, how to like flip things around, how to connect cables. It it it is so impressive what you can do cuz the the robot is $20,000. The DGX Spark is $4,000. And the VR headset is like $1,000. And then you need like that's that's it. For $30,000, let's say, you can have a real robot doing real things at home

for free. Like nobody can stop you. You don't need to do anything special. And that's so exciting to me. I can't wait to like have little robots running around the house doing like chores and education for your kids. Like I have kids now and so like I think about like how do I want to educate them? And having their own personalized education that is like technical and detailed and and I can tune >> Huge. >> you

know, or I can It's huge. It's huge. Cuz they're going to use it in in education. They're already like trying to use it in education. And uh I don't know if you've seen, but like the kind of educational quality >> Yeah, I've seen some some schools completely crush like that used AI. Yeah, I I don't remember name, but I've seen it on Twitter that like it was just insane difference. >> Yeah, man. Yeah. I I'm so

grateful that you invited me on. It's been a wonderful conversation. Do you have any other questions before we wrap up? >> we can wrap up. It's almost 11:00 p.m. here in Poland. So, I also do need to wrap up. Appreciate it, man. All right, have a good day.