Opus 5 vs GPT-5.6 On Polymarket Predictions — Week 1

[00:01] we got a pretty exciting video I wanted to make for a while, but now is just the perfect time because we just got Opus 5. I have access and I think this is the perfect time to kind of launch this series. I kind of want to do in the uh

[00:15] upcoming weeks, months. We'll see. So this is uh for today it's going to be Poly Market, could be Cali, could be other stuff, but it's going to be like other stuff, but it's going to be like an AI predictions battle, right? So

[00:27] basically what we want to do is we want to take in this case two events and to take in this case two events and we're going to put GPT 5.6 and Opus up against each other to see who can actually predict this event. So the

[00:43] events I picked for today is 3 days out or two days out. It's the lowest or two days out. It's the lowest temperature in SE on July 29th. temperature in SE on July 29th. and how many descent at the Fed uh July

[00:57] Fed meeting because this is quite even right and you can look at this there are some breakouts here but there are some options to go either way and I think it's going to be really interesting to follow the results we might do some

[01:11] stake on them like maybe like 20 bucks on each just to see and maybe we can keep track of it. So uh I put up some restrictions here. This is very important because we don't want them to

[01:24] prices and take the one that is the favorite. So uh we have used uh today's sponsor SER API to give the AI agents sponsor SER API to give the AI agents access to actually run some research uh

[01:39] on their own kind of without using codeex or cloud code. So SER API is a really good platform if you want to do search in like a really structured output way. Let's say on Google uh in this case we're going to use the agents

[01:52] for that. So here I can just put in uh an API request and this is perfect for AI agents and they can get this in a really good structure like here we have JSON format right. So these work very well when we want to do some AITIC

[02:07] research because we get all responses in this very good format here and you can see we get it in JSON. We can kind of adjust and do whatever we want and they have a lot of different options here. Let's just take a look at a few other

[02:22] options we can use SER API for if you want to expand our research. Something I've been looking into is and that is kind of YouTube search engine results API. This could be also something we want to include. Maybe we want our

[02:36] models here to kind of use the YouTube search API from SER API and actually get the transcriptions from videos include this work this into the research around this is something you definitely do to check out if you want to be a agent. You

[02:52] have a bunch of other options too here if you kind of want to yeah start using uh extended third party research. I think this works a bit better than actually using the web search they have in kind of codeex and cloud code just

[03:07] because of the format here is so flexible and you can do a lot of stuff with the formatting here on surf API the JSON. I really like that. So get started with SER API using 250 create credits. You can get that by clicking the link in

[03:19] the description or just scan the QR code that you see on the screen here. So see throughout the video how we actually use SER API in this setup we have built

[03:31] here. So one thing that is really important here is that uh you can see I wrote down here obuscate uh share price obuscated share price. So this was something I was looking into to do because I don't want the models to be

[03:46] do because I don't want the models to be kind of biased or kind of inflected or like uh contaminated by checking out the latest movements in directions in price here because they will really get like bias towards these results. Uh same here

[04:01] really matter too much but you kind of get the point. So uh I found a way I think I found a way to do this but this might be improved in the future. We will see. And of course, the output is just going to be a prediction for both

[04:16] events. And we're probably going to put some stake on it. Maybe do like $10 on each or something like that just for fun, just to see how it ends. And I'm going to make a video at the at the end date for this. I probably going to be at

[04:29] least this week and check how we well. And in the future, we can probably do more predictions. we can test other models and just work a bit on the scaffolding to see if we can improve it over time. So now I think I just want to

[04:43] head over to the scaffolding, show you a bit more that uh before we run the events and see what kind of predictions GPT 5.6 and Opus 5 comes out with and Okay, so let's just start by going through the scaffolding because this is

[04:58] very important and it needs to be like repeatable. So uh the first thing the scaffolding does is depicts the event. So I kind of set this up to scan for uh three days. This could be changed,

[05:12] right? And drop anything that has already been decided because we want already been decided because we want like uh an even uh kind of market. So it doesn't really like it's 99% sure that this is going to happen. That is not

[05:24] very interesting. And we're going to keep one question per event. So 10 legs on the same ladder doesn't count as 10 data points. And we want to strip the market out of the question. So we kind of rewrite each question. We try to not

[05:38] reveal that this is like a prediction on poly market because that is kind of like a hint where it should go look for it. And we give the models the research tools but police them with a wire. So here we kind of have SER API like we

[05:52] talked about. We're going to check that out. and polyarket and kalshi.com are stripped out of the results. So we can't really open up URLs that has those uh really open up URLs that has those uh direct URLs in the or we won't get those

[06:07] see we want to make it fair. So we kind of don't really we just run them one by one and we clean out the cache and the uh the results after each so we don't

[06:19] get like interference and then we're just going to score it. So we kind of make the model submit a probability for every possible outcome. Not that we said yes or no. So let's say in this case here the model has to submit a

[06:33] probability score for each outcome not just the one it thinks it's going to land on. That makes it a bit more interesting and we can kind of compare it. So I mentioned that how the price is obuscated. I guess I can kind of mention

[06:47] this here. So instead of asking will it be 26° in se at blah blah blah uh you can see what we do is the model shows the entire range of possible outcomes

[06:59] with no prices attached and this uh the must spread is the probability across all of them. The range itself is sex by a fixed rule written before looking at the market. So nothing about the list uh of options reveal which uh one of the

[07:15] market favors. So we just give them all the options. So it doesn't really attach a price to the options it's going to try to pick from. So we always try to hide

[07:27] that this is like a prediction on like a prediction market with share prices. We just want them to give the highest confidence score in what's going to happen in this event. Right? Uh I also said why is uh SER API so good in this

[07:42] scaffolding? It's a single choke point. uh we control uh it returns structured data not a rendered page. This is very important. You can see results arrive as clean JSON. We have a title URL snippet. So filtering this is very easy if we

[07:59] kind of want to set up some guards around what the model returns. We can kind of block that if we want to. It's neutral and identical for everyone. This is another thing that's very important. So we want to use SER API here because

[08:12] that means that claude code and codecs and the models have the same way of searching the web. Let's say claude has a better way to search the web. This also happens when we kind of want to use local models then everything has the

[08:28] same uh structured for actually doing research. So that is three points why I think SER API is so good at this scaffolding. So now that we kind of know a bit more on how we're going to run this uh prediction battle, I think we're

[08:42] just going to do it and see what comes out and what models pick what prediction. So I'm just going to do it one by one. Now we could in theory do it in parallel, but that could do some contamination. So first I'm just going

[08:56] to launch Claude. Uh I think I set up a very simple prompt for this. So I'm just going to do read instructions. So we created like a instructions file for the model to follow so you know exactly what to do and execute your best predictions.

[09:11] So this is the only prompt I'm going to give this to Opus 5 and on GPT 5.6. So I set the model to high. Uh I'm going to do the same on GPA 5.6. So I'm just

[09:24] kind of work through. You can see here it read both questions. We can take a look at those questions. So these are Q1 and Q2 domarkdown. So here you can kind of see here is the question what it's

[09:37] supposed to do quality to for quantity to forecast we give them the full list of the possible options and what to submit write a submission in the format given in the template and here we going to yeah and it's basically the same on

[09:52] Q2 we have a list of predictions it's going to make six or more from zero to six or more and it's just going to submit that. So this is the Fed meeting thing. So it started by reading those two questions and now we just get into

[10:07] research and here you can see it's using SER API to do some Google search. You can see it's find acue weather that's a really good resource for this looking at the predictions on acue weather. It's just going around ear. That is like a

[10:24] Norwegian site that is also really good for forecast. So basically it goes out to the web now using SER API to gather all the research data. It's probably going to run some calculations and it's going to come back with a prediction for

[10:38] uh the weather and of course it's going to move on to do research on the on the Fed. And what's interesting you can also see it do some searches in Korean here. That's pretty cool. I wouldn't be able to do that. So let's

[10:52] just wait for it. So you can see it's got a a lot of different sources to actually look at the weather forecast and then we're just going to wait for and then we're just going to wait for the Fed and let's see what uh Opus 5

[11:04] the Fed and let's see what uh Opus 5 here predicts in this um this first run here. After about 10 minutes, you can see Opus now are ready to write its submissions and it kind of goes through every single question here and again we

[11:18] have this nice JSON format for our output. So, let's just see what it landed on here. Looks like it was 2 at 28 and 26 at 28. 28 and 26 at 28. So, yeah, that was basically in line.

[11:33] So, I'm just going to go to our kind of master here. I'm going to say Opus has submitted it prediction. Read it and print it here because we want to clear print it here because we want to clear out the answers before we run it on GP

[11:46] 5.6. And here we can kind of see we have both predictions here. So it's going to be peak 26 and the was

[11:59] So it's going to be peak 26 and the was it two at 28. So basically it's going to go 26. That is the favorite two. Uh still we're probably going to follow that. And this is two. Okay. Okay, so that was a bit

[12:14] both fell in line with the favorites. But let's see if uh something else But let's see if uh something else happens on GPT 5.6 if we get any other results than just following the favorites here. Uh I had a look and I

[12:27] couldn't see anything that kind of indicated that it went to poly market or anything but but that's fine. Just going to see clear out any traces of open submission and work. Make the repo ready for our next model. And then we can just

[12:41] for our next model. And then we can just launch codeex. Uh we're going to do the same there. Pretty much the same thing. Uh let me just see model. Uh we should have soul uh high. Yeah, that's good. And we're just going to do

[12:55] that's good. And we're just going to do the same uh in um same uh prompt here. cleaned everything out. Okay, so we kind of cleaned it up. That's good. And then we're just going to run it here on 5.6. And let me take

[13:10] you back when we have the predictions here from 5.6. Okay. So, you can see we have the results in from GPT 5.6. And this took another line. I'm really happy to see that. You can see this p peak 25 at 52 actually went for zero dissidence.

[13:27] That's going to be pretty interesting. So, you can see we have uh both the answers now. So, I guess I can just do something like this. something like this. Uh, here are um So, I guess I can just

[13:41] paste it in here. And here are the results from GPT 5.6. So, we're just going to do Opus versus GPT table. Why don't we just put $10 on each prediction? That's going to be pretty interesting, right? So, if we head over

[13:56] interesting, right? So, if we head over to the predictions market and here we can see they have the round one. Okay. So, you can see for Opus, for the sold daily, 29th July, we're going to pick the 26 bin. Okay. Let's put 10 on that.

[14:12] the 26 bin. Okay. Let's put 10 on that. So, that's going to be the 26. Uh, okay. Uh, let's put a 10 on that. Okay, that was done. And what was the Okay, that was done. And what was the other one? That was uh GPT 5.6 is going

[14:26] to do the 25 bucket. So, let's do a tener on that. Yeah, that was done. And the other one, let's scroll down here for Opus 5. It was the second bin here.

[14:41] Let's do a tener there. Okay, that was done. And the final one was uh zero for done. And the final one was uh zero for GPA 5.6. And that's it. Let's check our positions now. And you can see we have those here. Perfect.

[14:58] All right. So that is our first battle uh on the predictions market and we're uh on the predictions market and we're going to come back to this. So this was a really interesting video and if people like this I'm going to do a lot of these

[15:12] predictions going forward then we can kind of over a while and we can include kind of over a while and we can include other models too see who uh or who what type of model actually make the best predictions. But this was kind of just

[15:24] predictions. But this was kind of just like a like a an early uh version of the scaffolding. We might actually make this a bit better. It was a bit worrisome that kind of opus kind of picked exactly the top leading indicator. So maybe the

[15:39] obuscated share price wasn't 100%. But I'm going to work a bit more on kind of the scaffolding and hopefully we can improve it in uh upcoming uh upcoming years side. So this was really made possible because we can use

[15:53] uh research tools like SER API because it gets very neutral. Now both models have the exactly the same uh research tools and that is very nice. So definitely go check out the SER API. You can find a link in the description and

[16:09] you can sign up and start using this too for your AI agents or other stuff if you have that and you get this nice JSON output that the models really appreciate. So, thank you for tuning in today. I'm going to come back with a

[16:21] you can go check out the results yourself if you want to. Have a nice day yourself if you want to. Have a nice day and I'll speak to you again very