Testing GPT 5.6 on Polymarket: Can It Beat GPT 5.5?

All right, so we have a new model in town and that is going to be OpenAI's GPT 5.6 Soul Terra and Luna I think it was that. So basically this just released today and I don't know if I got lucky but if I go to my codeex and I do my SL model you can see I have access here Soul Terra and Luna. So we are going to try out that today. So I think we're actually

going to try it on like really high reasoning uh just for the the simple case we have not simple case but the case I want to run today because uh I want to do some analysis on my huge poly market data set see if we can find anything interesting. If you have been watching my previous video, I have kind of been running like a project on polyarket trying to use uh codeex cloud code uh cloud and

GPT 5.5 to try to find some strategies actually to make some money on polyarket uh specifically on the Bitcoin up and down market. And you can see we I tried it. I just swapped when I got access and we had a a bit of luck uh right away. We got a nice $69 profit here. And as you can see, the last week has been really good. $213 in profit the last week. So that's been pretty good.

Uh what about 5.6? Uh I am not going to go into benchmarks stuff like that. Uh there are bunch of other people doing that. So if you want to look at benchmarks stuff like that, you can watch those videos. But uh we can have a quick look at the artificial intelligence index from artificial analysis. And this is really good, right? The soulmax is on 59. That's just one point behind fable. Uh but it it also what

did 5.5 do? Yeah. And you can see 5.5 is on 55. So like a huge numberic uh uh increase, but it is something right. So, like I said, I want to analyze Poly Market with this. And the simple way I'm going to do it is I think I'm going to compare 5.5 high uh versus 5.6 Sol. And we're probably going to do Why don't we just do the max since that was the artificial analysis. So, we

skip extra high. We go to max. And for 5.5, we can do extra high. Okay. And let me just pull up all the data sets I have so we can know what we can analyze and I want to see if we see any big differences in the analysis of what makes like a profitable trade uh strategy on poly market. So one other thing we're going to monitor is kind of how uh we use our uh usage

here. So you can see I am uh 5 hours uh I am on the 100% and for weekly we are on 87%. I am am on the $200 max plan, I think it's called that. And we're gonna see how much usage we kind of do with our one prompt we're going to run now. So I basically, you can, if you go up here, you can see I switched the model to GPT 5.6 Soul Max. Okay, I

think that's where the one I want to use. So, we're just going to count up our data. So, I kind of can show you what kind of size of data we have collected over the last 30 days on Poly Market. Okay. So, you can see here we have uh about 610,000 uh logged rows across 43 markets. And basically all they have corrected is like order book snapshots, we have fills, we have uh BTC USD candles, meta

data and basically all the data we should need to actually find a good strategy. You can also see we have some draw downs here, strategy compensation back test, we have some win rates and stuff like that. But basically I want to see now when we run my prompt both on 5.6 six soul max and we're going to try it on 5.5 extra high and just to see if we see any difference that is basically what I

want to check out today. Okay. So to avoid any com contamination of my previous strategy I just ask it to remove any tests that reveal our current or old strategy. We're looking for a new uh and we want to avoid any contamination. So basically now we just removed all of the previous back tests. We had the results from the back test. So we can just keep the data and that is going to be kind of our

starting point. So I'm going to clear this out afterwards and we can actually start running our prompt and we can compare the two models. Okay. So we kind of cleaned our workspace. That's good. So I'm just going to clear this and I have kind of prepared my prompt here. So this is going to be a slashgo prompt because I'm just want to keep it running because I have some stuff to do in the background. So, I'm

going to read this once, but this is going to be the same prompt as we're going to run on 5.5 high. Uh, we have a big poly market data set from the BTC 5m minute up and down in my data folder. You are a quant trader expert that has been given a task to find the most profitable strategy from this data. The only restraint is that we want to avoid slippage. We want to avoid fees if

possible and we need to run back test uh including the best practices from a quant algo trading uh yeah space like Monte Carlo simulations etc. Look at drawd downs work on this until you are confident you have the best setup to run a longunning profitable trade and that is going to be my goal. So the first run is going to be on 5.56 soul max just because let's say we get some rate limiting and stuff. I

just want to run it straight away and then we're going to do our run on 5.5 high and we got to compare them and see if there are any differences. Maybe we can ask 5.6 what the differences is and it can give some kind of rating or something. So I'm just going to let this run in the background now. Uh we have the timer here on pursuing our goal. So, I'm just going to let this run

and I'll take you back when we kind of have that and we're going to fire up the 5.5 and we're going to compare them. Okay. So, I just had to intervene here. I didn't actually see what the question was, but it it stopped and asked me this request requires additional safety checks which can take extra time. And I got the option to say allow or learn more. This has probably something to do with uh some cyber

security stuff. It should be pretty straightforward. I can see we are doing some API calls here. We're doing like a Binance curl command. But uh if I get it again, I'm going to record when I kind of actually get the question. But uh if nothing else happen, I'm just going to keep it running. You can see we've been running for five minutes now. So yeah, I will take you back if we get this same uh request

again. Okay, so just an update. I'm really happy with the path it went down here because it is actually going to pick the real time fair value model that we have kind of been using before. I'm interested to see if it's going to take like a different take on it, but that is uh exactly what I hoped to see because that is a strategy I has been working really good for me on polyarket. So, I'm really

excited to see how it's going to implement this. And so far, I think the speed has been pretty good to be honest. So, uh I also did start a 5.5 uh model here. We basically just going to run the same prompt, but I cleaned up the data. This is a new new folder. So, now we have kind of only the data and not any like old strategies. So, I think we just going to clear this. And

in parallel with um 5.6 Saul, we're going to run 5.5. I think we're going to do extra high. 5.5 extra high. And basically, we're going to run the same prompt as we did before. And that is just going to be our goal prompt. Uh if you check on the usage you can see we are still pretty good here 83% but we are down to 84 weekly but so far hasn't been too bad actually. Uh this kind

of project is of course running a lot of tests. So we do run a lot of local code here but uh let's see what happens now when we start up 5.5 extra high in parallel with the same uh goal. And yeah, we're just going to compare at the end, see what I like the best and maybe we do some kind of 5.6 review of both results or something. So yeah, that was the short update and I

will take you back when both models are complete. Okay, so this was uh what I wanted to show you. So I got this request here, additional safety checks. This request requires additional safety checks which can takes extra time. So I'm just going to do keep waiting. I don't even know what learn more is, but uh I'm just going to do keep waiting and then I don't know, didn't see anything uh when the first time we ran

it, but uh yeah, that's something new I haven't seen before. Okay, so that took some time because I was watching some World Cups games. I just had this running in the background. Uh, you didn't see this, but I actually did start a GPT 5.5 extra high strategy run two and we ended up with a GPT 5.5 strategy MBD markdown file. So, this is the strategy we landed up uh on using GPT 5.5. Basically, that's the old

model and let's see if it's finished. You can see we are writing the strategy here from um 5.6 soul or soul. I don't know how you pronounce it, but uh we ended up with two different strategies and I thought that was really interesting and I'm kind of already got to say I'm pretty intrigued by the strategy 5.6 actually picked. So, what we're going to do now is I'm going to load up a new repo, or not

a repo, but like um like a folder here. So, we're going to put the data we have here in this folder. We have the GPT 5.5 strategy. We have the 5.6 strategy. And I'm going to use 5.6. Um maybe we try the Ultra or something to evaluate and kind of rate each strategy uh based on the data we have. Okay. So we can kind of see what does Saul or Soul GPT 5.6 Soul think is the

best strategy here. So I thought that was just something we can do at the end. Uh let me see if I find this. So let's just go into this with soul tree. Okay. And let's change the model. That's going to be let's try 5.6 sol. And let's do ultra high reasoning or something. I haven't tried that. Let's see our also let's check up on our usage. So we have actually run this for over 5 hours because

I was watching the football. So now we are back at 96% but uh we didn't actually use up our full um usage. So that's fine. You can see I also have some limit resets if I wanted to. But basically let's use 5.6 six soul ultra to evaluate both strategies and see what it thinks is the most interesting one to run on the specific market on poly market. So the prompt I wanted to do is uh we

have two different poly market strategies in GPT 5.5 5.6 Six, we have the data in our folder. Please evaluate both strategies as an expert quantrader. We know some data is missing, but focus on the best potential strategy. Be brutal, honest, and give a rating from zero to 100 for both. And again, like I said, we are going to be on the ultra high uh reasoning setting for this. So I'm just going to let it run and

I'm going to take you back to see kind of our first result based on the new open AAI GPT 5.6 model. So so far so what I have seen is pretty interesting. I was quite impressed by the strategy it picked and the speed was quite fine. I didn't have any issues. I know they are doing like a slow roll out but uh the speed felt pretty good. Nothing like uh worse than 5.5 at least. So, let's

just end with the results here and just take a quick more look and talk a bit about uh what we're going to do uh with this model going forward. Okay, so we have the results back from uh Soul Ultra here and we can see the 5.6 did get a better rating. We ended up with a 47 out of 100. that is not great but uh I came into this knowing that we were missing some data and

it decided that this has best potential but uh it still needs more research and for 5.5 is an interesting hypo hypothesis in insufficient evidence so far. So both are pretty good but again you can see we are edging a bit up on 5.6 six and it kind of gives the advantage um on why it ranks higher better economic thesis chron chronological training validation and claimed untouched 20-day hold out. Okay, that's not bad. So, I'm not going

to go too much into the details in this video, but uh the recommendation is to allocate research effort for to 5.6 6 and keep 5.5 as a simple control strategy. So it definitely favored the 5.6 here. So I am actually going to continue looking more into the strategy because I haven't seen it from any other model so far and it was actually really interesting. Uh I'm going to cover more about what it the strategy exactly was

if I found it uh viable to go live it on polyarket. I'm gonna keep exploring GPT 5.6 probably next week. Uh I'm going away this weekend so I can't really play around more with it then. Maybe on the road I have some time. But so far pretty pretty excited. I don't know. Uh the first impressions were pretty good. At least the speed and it picked a strategy I haven't seen before. So pretty happy with that. But

I'm kind of looking forward to exploring this more. And let me know in the comments if you have tried it out. What's your first impression? Mine has been pretty good. Is it like a step change or is it just like a small incremental jump? If we just look at the ratings here, uh that is very non-scientific. Uh it is a jump at least. So really enjoyed it and I'm going to come back with more uh videos

on this topic probably next week. So enjoy your weekend and I'll see you again soon. Go Norway.