[00:01] how I actually control my claude code AI agent uh by using the browser. So I have this browser.js file that is kind of connected to my Chrome via this port. So we can kind of do commands like yeah check all elements, check all buttons,
[00:16] click elements and stuff like that. So I thought I'd explain a bit more about it today and yeah, maybe you can set this up yourself. Okay, so before we head over to the Mac Mini and I'm going to show you how this works in practice,
[00:28] let's just kind of go through how the setup is today. So, the first thing we need to do is of course launch our Chrome uh because I set this up for Chrome. It might be able to do this in other uh browsers too, but for me, this
[00:40] other uh browsers too, but for me, this is Chrome now. And I'm using uh this setup I have here. So, it's basically just a shell script or like a launcher script that launches Chrome. And it's going to do this in this debugging mode
[00:54] that has this port set to for me it's just 9222. So this going to open a socket we can use. Okay. So the reason we have this is that you can see we have this Chrome browser in this debug mode and this
[01:08] opens the sockets. We can kind of connect to our CDP connection here. So connect to our CDP connection here. So this is the Chrome dev tools protocol. So basically the part two here or phase two is kind of the remote control that
[01:22] is controlled by my agent by running this uh JavaScript commands. And you can see we have a bunch of lists here of what we can do. So if we do like browser js and we say something like uh yeah let's call it uh list uh it can see all
[01:37] the open tabs. It can do open and the URL. This navigates to this URL. It can do browser js uh elements. This is one I use a lot. So this list all the clickable things. And this is how we navigate. So basically it's not like
[01:51] most insane setup, but it is kind of nice to do this if you want your agent to kind of control your Chrome browser. So this is the way I did it. I'm sure there's a lot of other ways to do this, but for me, this is working pretty good
[02:05] and it's easy to kind of transfer these commands over to your skill MD. So, let's head over to the Mac Mini and I'm going to show you one example of kind of how our agent is using this uh JavaScript file. Okay, so I'm not going
[02:17] to spend too much time on this, but basically here we can see my browser.js file and here are all the commands we import into this. So, let's say I was talking about open. So, open is a URL command to open a URL we give it. So,
[02:32] you can see if we go to commands here and we find open. Okay, so we can check the TypeScript if you wanted to and we can check the JavaScript. You can see can check the JavaScript. You can see it's basically a CDP command to Chrome
[02:45] how to actually do this. So I guess the arguments are a URL that is basically all we need I guess. And for that we navigate the current tab. Uh if we go to let's say list here again pretty simple. And if you go to list this should only
[03:01] be uh tabs. So we only check what kind of tabs we have open. So we can navigate between those tabs. So we can just send a command and see what tabs we have. And if you want to close some we can use that. So basically everything in the
[03:15] browser.js file that a lot of you have asked about is kind of controlled by um by these commands here. Okay. CLI tool to interact with Chrome via SIGDP. So
[03:27] that is how I do it. So let me show you a couple of example how this works in practice. So you can see, okay, here was an automatic command that happened. That wasn't me, but that was like a setup I have for my chron jobs. But let me show
[03:40] you now. So if we head into CO, okay, we are on set 4.6. By the way, I think CL uh sonet 4.6 is looking pretty strong on the aentic stuff. So definitely worth a check out. [snorts] So I can say basically we have this skilled already
[03:56] honed in. But I can say something like use the browser uh JS command uh list to go to
[04:08] hackernews.com. Okay. So you can see use browserjs command uh list to go to hackernews.com. So you can see we are running browser.js list. Okay. One tab open. navigating to hacker
[04:23] news and of course we open this and you can see we do a ch we check if we are in this tab zero is now on newsshacker.com right so tab zero is kind of the index of all the tab controls we have from list okay so you can see we ran list
[04:39] here to check all the tabs and yeah I guess I can open this to expand so let me do a control O here and you can see this is kind of the way we
[04:51] you can see this is kind of the way we ran this we gave it the URL here in the open command right browser.js open and we went to this and browser.js open and we went to this and now we are in here. So if I say
[05:05] something like click on uh number one uh post for example we should have the context that we already are in this tab zero and we are on hacker news. Okay. And yeah, here we use the click browser command. Okay.
[05:22] Click zero. So if we open up again and here you can see uh first is thinking. So let me take a screenshot to see what's on the first page and then that the content to find the one post and you can see it runs the content z-s.
[05:37] This is the first command it does and it finds that the contact is this garment n notional language on hacker news the URL. And then we have the content, URL. And then we have the content, right? And then we run the click zero
[05:51] right? And then we run the click zero and this yeah content here and we get into this. So basically it's nothing magical, but it's very efficient. That's what I like about my setup. It's very efficient at navigating using these
[06:05] JavaScript commands instead of trying to use kind of a virtual mouse to hover and click. I think this way is a bit more effective and it can kind of adapt to all of um all of the pages we basically using
[06:21] Chrome. So that is the the great thing. I don't know what else to say here. I know a lot of people asked about this. So we can basically do whatever we want. We can also do uh let me come up with
[06:34] you how we can combine this with my X skill then. So I can just say use X skill then. So I can just say use X skill all right to compose a draft of uh
[06:46] skill all right to compose a draft of uh hello YouTube. Uh this is my skill hello YouTube. Uh this is my skill md.store page. Okay. So right now we're going to use the X skill. We already have this X skill have more precise way
[06:59] of navigating skills. So navigating the browser. So we kind of save some time. compose/post. We are in the X skill part. Now we can just do the Okay, I can open this up now. So now you can see we are using we
[07:15] used browser.js open. Now we are using like an Xdraft using some CDP scripting here to paste this in here. Hello YouTube. This is my skills and the store page. Perfect. And yeah, that should be it. You do a screenshot of this. It
[07:30] confirms that it's up and that's about it. You can see how fast it was, right? So yeah, that is the draft. So if we go to the skills MD page. So this is my skills M skills MD.store page. So this is where I kind of put up my X uh skills
[07:47] that kind of has this navigation. So if you want me to put up um I might find a way to put up this browser.js with all the commands. Uh give this video a like
[07:59] and I see how many people are interested in this. So if a lot of people is interested, I might consider putting this up on this page. So you don't have to set up your own browser.js file with the commands. But I don't know
[08:12] because there's a lot of other setup you have to do too. But uh yeah, hopefully this gave you some um insights of how I am using uh the browser.js setup, the Chrome
[08:26] debugger port setup with the socket to control the browser. So yeah, like I said, give this video a like, subscribe if you want to see more of how I run my AI agents, and yeah, thank you for tuning