Solving Browser Automation for AI Agents: Surfagent

[00:01] this video, I want to talk a bit about something I built over the weekend. This agents. And yeah, I think it's just really good and I'm going to make it open source. So, you can just install it like NPM

[00:14] install G surf agent. I call it that. So, basically, today I just want to show you what this can do because I was really surprised how well it works. So, let's say we just open up Cloud Code here for example. We

[00:28] go to I logged into my Discord. And this is the advantage of this, we don't really need any APIs, right? So, I'm logged into my Discord. I can just say navigate to the Bossy server, read what's

[00:41] happening in general. So, now we should have all the tools after we did the NPM install surf agent. And I did some custom addition things to this package here. So, we can actually navigate Discord

[00:55] with this now. So, we should be able now, you can see we are navigating in the background here into the server, right? I guess we can just go here. We can check the general chat like we said. And we should be able to get all the

[01:08] context from the Discord, right? You can see we are looking up the channels. Now, we're going to general, right? General chat. And now we can grab all the context from this server and use it for whatever you want. So, it's super easy

[01:22] kind of doing a recon. You can see in the background, we are scanning all the chats here like the last 200 messages. And now we can use this as context. So, this could be on your own server if you want to catch up or something. And you

[01:36] can see here's what's going on in the general chat. You can see Discord worms, scam discussion, AI music generation. Yeah, you get the point. So, this is what I've been building and I'm going to show you a few things we can use this

[01:50] to whatever you want to. And like I said, this is open source now. So, you can just use it for free. So, let me show you a few things we can do with this. But before we do that, I want to tell you about today's sponsor, Freebuf.

[02:06] and that is going to be Freebuf. So, coding agents these days are 20 100 $200 a month depending on your usage. But Freebuf is just free. So, this is funded

[02:18] by text ads, no subscriptions, and they don't even trade on your code. So, I now. It's super easy. So, all you have to do, go to freebuf.com, copy this, NPM G is install Freebuf. Just go to your terminal and just run this, right?

[02:33] in. So, I can just say, "Hello." And one thing they have, you can see here if you something called suggested follow-up. So, we can do explore projects, new feature, or code review. This is pretty cool. I kind of like this. But what I

[02:46] want to do is connect this to my ChatGPT subscription. So, if I do {slash} and I do connect, you can see connect to your ChatGPT account. And you can see I can confirm this. Yeah, continue. Connected, perfect. If I go back here now, ChatGPT

[03:00] connected. And now you can see I can do plan, create a plan with GPT 5.4. So, plan to use FFmpeg to remove the silence part of an MP4 file. Okay, so you can just going to say, "Yeah, I want to implement the plan." And let's see if

[03:15] this works. Okay, so that was done, perfect. So, now let's test it out now. So, I have this video here. It's how long is it? 5 minutes long. So, let's just run this now. So, we're going to do the Python remove silence. We're

[03:27] Let's see how this goes now. So, we're going to speed up FFmpeg, cutting 45 segments. Okay, perfect. No silence. So, we went from 5 minutes to 2:20. Let's

[03:40] have a quick listen. So, all you have to do, go to freebuf.com. Yeah, that works. So, that was super easy, right? So, now we created this tool. Perfect. So, yeah, I would just go to freebuf.com. Go check it out. I'm going to leave a link in the

[03:53] yeah, a big thanks to Freebuf for sponsoring this segment of the video. Now, let's go back to the project. Okay, like I said, you can see I have done this as an NPM package now. You can see we only have to do like NPM surf agent

[04:06] or something like that. So, let me just grab this here now. And let's just start a new. So, we have like a new terminal here on Cloud Code. And we have like a cursor empty repo here. So, let's just

[04:22] start with and let's just do the NPM install, right? And now I'm going to see show you how easy it is to get this set up now. So, we just going to install the package with NPM install. And you can see

[04:37] Yeah. This is installed. And now I'm just going to tell it uh Yeah, it's a browser automation API for AI agents using Chrome CDP. Let me install it globally. Uh yeah, that is fine. We didn't really

[04:50] need to install it globally. But it's fine. We can use it for other things, too. Uh but now I just want to say do you know how to use it, right?

[05:05] information. So, now we can just start this. So, I'm just going to say this. So, I'm just going to say uh go to Hacker News or something like that, right? That should be enough. We are kind of

[05:19] let's see if that's going to be an issue. We're just going to do surf agent start. You can see we are already running on this port. Yeah, but that is fine. So, now it should just going to pick a different port and we can open up

[05:33] this, right? And now we are inside here. So, you can see we went to Hacker News, perfect. So, let me give you some different instructions here of what we can do. So, you can see it's using reading all the

[05:45] information here. You can see we have DaVinci Resolve. This is the top post. We got all the posts. You want me to click any of these articles. So, let's say we say click into number 10, distributed duck DB.

[06:00] instance. So, let's just say click to 10. And this should be able to do this now. Yeah, we got right right into the open duck DB, right? Perfect. So, let's just go back here and let's just leave Hacker

[06:13] News because now I want to show you that we can actually do Google Sheets with this. This is pretty cool. So, now I'm just going to say, "Yeah, you can see it kind of read the Google Sheets." Uh let's do like a quick

[06:27] research project now. Let's just try to do use surf agents to find all the API do use surf agents to find all the API prices for Opus 4.6, Sonnet 4.6, GPT 4.4, and Gemini 4.1. Uh then enter these into Google Sheets,

[06:43] So, let's see if we can do this challenge here now. So, it needs to go out, find all the prices, right? You can see it's starting. So, it should be pretty quick because we are using this recon call to you can see it kind of

[06:57] speeding up here to find uh everything it needs to click uh autonomous. So, you can see now we went to Claude, we went to OpenAI, we went to Gemini, and we are looking for those prices. And the final thing we wanted to do was to go to

[07:12] Google Sheets, fill in all this information. You can see we are probably just looking this up here now. Yeah, and it probably wants to go back to Google Sheets once we have all the information. And we can start putting this into

[07:26] uh the spreadsheets. So, let's see if we can do that. Just give it some time And now you can see we are going back to Google Sheets. And now let's see if it can actually navigate this. You can see we are trying now. You can see if we go

[07:39] here, we are using so commands, right? Value A1. Now, it's going to try to figure out how this works. You can see we could put A1 down there. And let's just see if it figures this out now. And here you can

[07:52] see it goes like it puts in all the rows here. So, we do model, input, output. Hopefully now you can see it have faster. This is skips to Opus. Uh yeah, and Sonnet 5.4. I guess we can do a bit like this.

[08:07] And yeah, maybe I messed it up now. GPT 5.4 fills in all the information. And it should do Gemini 3.1 Pro. Yeah, you can see this is fully autonomous. It kind of learned how to operate sheets. And now we can do this

[08:22] things autonomous here. We can also maybe ask it to do a graph or something. Let's do that. Okay, so let's just ask for a graph of the pricing. this. So, this kind of opens up a few more

[08:35] things because we can go out, do research. What happened here now? An error record occurred. Okay, let's see if we can fix it. But what I wanted to say is that we can go out, do some research, and go back, put everything

[08:49] into like a spreadsheet or something like that. Do some statistics and stuff like that. But one thing I got to say is that this is not this is not headless. So, you need some setup like I have. I kind of have a Mac mini that is running

[09:03] this and is able to use the browser, right? That is quite important. So, this is not headless. I haven't looked at that yet. So, we just reloaded page. Now, let's see if we can start to work on the graph we asked for.

[09:16] price here. But let's just see what happens. And yeah, let's just let it run for a while here. Okay, so you can see it goes for insert. And let's see if it's going to pick chart here. And yeah, there you

[09:29] can see we have it. We have the chart here, right? 1 million per token. We are kind of missing this for Gemini 3.5 Pro. 3.1 Pro. But yeah, we got the chart we asked for. Okay, that's pretty good. Nice. And it's going to yeah, that's

[09:45] And I think we're just going to stop it here. And you can see it is possible to operate like Google Sheets or something by using this system, surf agent here. But of course, the most uh useful thing is that we can now skip APIs, right?

[10:01] if you have like a a station that can run that doesn't need has to be headless, you can just log in to your page here. So, I'm logged in on x.com here, and I can just say something like, "We are on x.com explore searching for

[10:15] the latest news on Claude Mitous, right?" And this should now be able to to kind of adapt to the page we are on because we are running this recon uh command that kind of uh maps everything that is on the page, and we

[10:29] can quickly navigate to those elements, right? So, now we're just going to look up this, and we can just recon the page. We navigate everything on the page, and we can start, yeah, doing stuff on the page autonomously by using SurfAgent.

[10:44] So, we went to explore, right? I said we wanted to search for uh Claude Co- So, let's see if we put that into the search field.

[10:56] Yeah, Claude Mitous, we are searching for that now, and we can kind of look up Uh yeah, we have some small issues here, but now we can uh find context on X, right? We go to latest, and we start mapping out the latest news on Claude

[11:11] Mitous, and we can kind of build up like a context here. And if you wanted to do something more with Yeah, that context, because we can read everything on the page, and you can see we can navigate, we can go to users, we

[11:25] can write post and everything. So, let's say uh let's try to build a post here on Claude Mitous, and yeah, see if we can make it work. So, let's just do create a creative post on Claude Mitous, lower capital, short, and post

[11:40] So, let's see if we can adapt to that request here on X. We go to post, right? And we should be able to find this straight away using kind of the recon feature, and we should be able to just uh insert

[11:55] uh the post here, and just click post. So, yeah, you can even automate uh stuff like this if you wanted to. Uh I already done my experiments on this, so I'm not using this so much for this now. You can see here, we put everything in, and

[12:10] >> [snorts] >> And yeah, super easy, and it works pretty smooth. So, what I was most impressed by uh creating this over the weekend is the speed, because it's gotten just so much

[12:22] quicker when we actually run this recon uh command first to kind of uh map everything that is on the page, right? So, let's just do let's go to YouTube, find videos on Claude Mitous. It should

[12:34] be able to just uh adapt to that very quick. Just go to youtube.com, search query Claude Mitous, and you can see uh yeah, we find some videos on this, and it should also be able to just grab

[12:49] the transcription of a video. Let's go to Love Levels video here. to Love Levels video here. Let's go to Love Level uh grab transcript and write a summary, something like

[13:04] So, let's see if we can do that. It should be able to do that, I think, just because of the mapping we do with the recon. You can see we are going straight into the video here, and we are playing the video, but we should be looking for

[13:18] the transcript, right? You can see we are also scrolling, so it's very good at scrolling, and it says show transcript, and that should be here, right? On the And it should be able to grab the full transcript here, and we can use that for

[13:32] wanted to. Got the full tran- transcript, uh the core finding, Claude Mitous preview can autonomously vulnerabilities. Pretty interesting model.

[13:46] So, yeah, I think I kind of have showed just a small bit of what we can do now with this automation, and I just started with it. So, I haven't really explored all the possibilities, but uh it should be open source, so I

[14:03] made it like a npm package, but you can also go to the GitHub if you wanted to check it out, right? And I'm going to leave all the links in the description. So, here you can read a bit more about how it works, and how you can adapt it

[14:16] to your use case. So, yeah, hope you will check it out. And that's what I want to share today, and I'm going to come back with more videos on this and how we can use it to set up these automated pipelines.

[14:28] Uh I've gotten into doing some small passive incomes now with these pipelines, so this fits very well to some testing I have running at the So, yeah, go check it out, and don't forget to check out Freebird, that is

[14:41] also a link in the description. So, yeah, hope you enjoyed it, and hope you will get some use of this. Share it, leave some requests uh on the uh GitHub if you have any things we want to upgrade, because this is not perfect.

[14:55] some Claude MD files here, some agent.md, and there's a lot of things we can improve, but uh I've been doing some QA testing, but I can't do everything alone. So, if you have any issues, pull

[15:08] make it happen. So, yeah, thank you for tuning in. Like I said, check out the links in the description, and have a great day.