GPT-5.5 VS Deepseek V4 Pro VS Opus 4.7: I tested THEM on My KingBench 2.0 Questions!

[00:05] Hi, welcome to another video. So,

[00:07] Deepseek has launched Deepseek V4 Lite

[00:10] and V4 Pro. But they are not the only

[00:13] ones who have released a new model.

[00:15] OpenAI has also launched GPT 5.5 which

[00:18] aims to be better in all aspects and

[00:20] also fix the gripes of the old model

[00:22] which are mostly just front-end issues

[00:24] but there might be some more which we'll

[00:26] talk about later as well. Apart from

[00:28] this, I have also been working on

[00:29] Kingbench 2.0, which is a refresh of my

[00:32] old benchmark. The aim was to test a

[00:34] model on all aspects of coding. You

[00:36] might see some general questions as well

[00:38] in the benchmark, but that is trivial.

[00:40] Some questions in here require the model

[00:42] to be used with an agentic contraption,

[00:44] while the others don't. Basically, as I

[00:46] said before, the benchmark is supposed

[00:48] to be a benchmark that doesn't just test

[00:50] agentic behavior or something. It also

[00:52] tests other aspects and makes it a

[00:54] better benchmark all around. Currently,

[00:56] I don't have a UI to show all of this.

[00:57] I'm still working on it. So, you guessed

[00:59] it. We'll use an Excel sheet. By the

[01:01] way, how many of you have been following

[01:03] this benchmark since the Excel sheet

[01:04] days? Comment below. Anyway, I'll be

[01:06] testing three models here, which are GPT

[01:08] 5.5, Deepseek V4, and Opus 4.7. These

[01:12] models give you a good look into the

[01:13] worldview of models in respect to the

[01:15] resolve of these companies. Before

[01:17] jumping into the bench, I do want to

[01:18] talk a bit about GPT 5.5 and Deepseek.

[01:21] Though Deepseek has launched two models

[01:23] which are Deepseek V4 Pro and Deepseek

[01:25] V4 Flash. The Pro version is a 1.6

[01:28] trillion parameter model that is a

[01:30] mixture of experts with only 49 billion

[01:33] parameters activated in a pass. The V4

[01:36] flash, however, is a relatively smaller

[01:38] model at 284 billion parameters with

[01:40] only 13B activated in a pass. I don't

[01:43] want to talk a lot about the model's

[01:44] architecture as it can be boring, but as

[01:47] an ML enthusiast, I can't help but

[01:49] notice that it uses a muon optimizer. I

[01:51] believe Kimmy also uses it. Moonshot

[01:53] even has a paper on this where they talk

[01:55] about how it is actually scalable.

[01:57] Contrary to some of the prior beliefs

[01:58] that it is just good for small LLMs or

[02:01] something, I believe it might be one of

[02:02] the biggest models using Muon if I am

[02:04] not wrong. GPT 5.5 on the other hand is

[02:07] one of the most anticipated models. It

[02:10] apparently beats Opus 4.7 across the

[02:12] board and even on front-end tasks now. I

[02:15] mean, if the card issue with GPT models

[02:17] is fixed, then I can easily recommend it

[02:19] to anyone. It seems that GPT 5.5 also

[02:22] now works better with lower token

[02:23] consumption, which looks very

[02:25] interesting. You'd know that I have

[02:27] indeed tested the alleged DeepSseek V4

[02:29] when it kind of became available on

[02:30] their chat platform, but that is not

[02:32] something I can fully trust. So, a new

[02:34] test is kind of mandatory. All of these

[02:36] models are indeed available everywhere

[02:37] now including open router, kilo gateway

[02:40] etc. I mostly use my models with kilo

[02:42] CLI and it is available there as well

[02:44] which is quite cool to see. So let's

[02:46] look at the results direct to start. I

[02:48] have a question where I ask the models

[02:49] to build me an elevator simulator. I

[02:52] want to make a simulator kind of thing

[02:53] where there are floors and on each floor

[02:55] we can spawn a person and the elevator

[02:57] should take the simulated person to

[02:59] their floor and then keep doing that

[03:01] until no one is left. Each elevator is

[03:03] only allowed to take one person at a

[03:05] time. So this one allows me to test how

[03:07] good a model is at front end and complex

[03:10] backend at the same time. To start, if I

[03:12] show you the generation from DeepSeek V4

[03:14] Pro, then it is not good. The elevator

[03:17] positioning is not correct at all and it

[03:19] all seems very random. So yeah, this is

[03:21] not great. Then we've got the generation

[03:23] from GPT 545 and this is also not good.

[03:27] I mean, it kind of works, but it just

[03:29] flickers too much. And if you see here,

[03:31] then it looks really bad. In terms of

[03:32] UI, it still does the same bad card

[03:35] designs and stuff. So, yeah, not great.

[03:37] Next up, we've got the generation from

[03:39] Opus 4.7. In here, you can see that this

[03:41] one actually looks awesome, works really

[03:43] well, and it is just really good. This

[03:45] is what I imagine when I ask a human to

[03:47] do it. So, yeah, Opus 4.7 kind of nails

[03:50] it. Next up, I ask it to make me a 3J

[03:52] contact lens case that also opens up

[03:54] when clicked. This is also quite good

[03:56] because it is something that is not very

[03:58] much in the training data of the models

[03:59] and 3D is also tricky for models. So if

[04:02] we look at the generation from DeepSeek,

[04:03] then it is not great either. I mean it

[04:05] is fine but it looks like a brick with

[04:07] two holes. So this ain't good at all.

[04:09] Then we have the one from GPT545 and

[04:12] this one is also not very good. I mean

[04:13] it is fine but it isn't the best.

[04:15] Clicking it, it does get opened but it

[04:17] opens the cap on the left for some odd

[04:19] reason. So that isn't the best for sure.

[04:22] The one from Opus 4.7 does indeed look

[04:24] like one of the best, but the L&R are

[04:26] kind of flipped and the cap opens on the

[04:28] bottom for some reason. So there's that.

[04:31] not the best from anyone. Next up, I

[04:33] have a question where I ask it to build

[04:34] me a 3J's folding table. It should have

[04:37] a slider that should allow the user to

[04:39] fold it or unfold it accordingly. If we

[04:41] look at the generation from Deepseek,

[04:43] then it is pretty good. I mean, it's not

[04:45] the best, but it is good nonetheless. I

[04:47] can't complain much. It does indeed

[04:49] work. Next, we've got the one from

[04:51] GPT545, and it is not good either. It

[04:54] does look good when unfolded, but it is

[04:56] not good when folded. It looks out of

[04:59] place. Both partitions overlap each

[05:00] other and it ain't good at all. Next up,

[05:04] we've got the one from Opus 4.7 and it

[05:06] is kind of fine, but not very good

[05:08] either. So, yeah, there's that. After

[05:11] that, we have a comeback question where

[05:12] I ask the models to make me an SVG of a

[05:14] panda eating a burger. Well, all the

[05:16] models are wonky in this.

[05:19] The panda with a burger from Deepseek is

[05:21] as good as a rock, so yeah, this is not

[05:24] good. Similarly, the one from GPT545 is

[05:27] also very weird. However, the one from

[05:29] Claude is actually kind of good. It's

[05:30] not the best, but good nonetheless. The

[05:32] next question is to make me a bow and

[05:34] arrow simulator game. And well, this one

[05:37] is really interesting. So, the one from

[05:39] Deep Seek basically doesn't even work.

[05:41] It is quite buggy. However, the one from

[05:44] GPT 5.5 is actually quite good. You can

[05:47] aim and shoot and everything. It still

[05:49] uses that dirty card thing, but keeping

[05:51] that aside, it is quite fine. Now, the

[05:54] next one is from Claude, and this one is

[05:55] just too good. I mean, it looks really

[05:57] professional, good-looking, and it is

[05:59] just good. I mean, there's nothing more

[06:01] that I'd want from it, and it is one of

[06:02] the best for sure. Then there's a new

[06:04] mathematics question, and none of them

[06:06] pass it either. I also ask it to

[06:08] fine-tune a Gemma 4 model for me with a

[06:10] generated data set for Pandaax, and

[06:12] well, none of them are able to do this

[06:14] yet. So, this is it. I think Opus is

[06:16] still just a better overall model for

[06:18] most people, as it is just good overall,

[06:20] while GPT 5.5 is good at some things,

[06:23] but it isn't that great overall.

[06:24] Deepseek is not good either. Deepseek is

[06:27] just a model. It's not good. It's not

[06:28] the best and it's not bad. But this is

[06:31] not what I would have thought when

[06:32] someone said Deepseek V4. I also want to

[06:34] talk a bit about the pricing as well.

[06:37] Deepseek V4 is a 1 million context

[06:39] window model that comes at about 1.74

[06:42] and $378 for input and output per

[06:45] million tokens respectively. It is

[06:47] actually extremely cheap. Deepseek v4

[06:50] flash is also a million token context

[06:53] window model that costs about4 cents and

[06:55] your 28 cent per million tokens for

[06:57] input and output per million tokens

[06:59] respectively. It is actually extremely

[07:01] cheap. Deepseek v4 flash is also a

[07:04] million token context window model that

[07:06] costs about4 cents and euro 28.4 million

[07:09] tokens for input and output

[07:10] respectively. I mean, yes, it consumes

[07:13] fewer tokens, but those token costs are

[07:15] now high. So, the end user will end up

[07:17] paying more in the long term, especially

[07:19] the API users. All that codeex rate

[07:22] limiting will also probably go into the

[07:23] trash, similar to cloud code, and there

[07:25] is no denying that. GPT 5.5 needs to be

[07:28] a 16 trillion model to justify this

[07:30] price based on how much Deep Seek costs.

[07:32] So, yeah, that is a bummer. I do get

[07:34] that research and training costs more,

[07:36] but I don't like it nonetheless. I think

[07:38] Opus 4.7 is the best model, but it is

[07:41] not the best experience anymore due to

[07:42] all the limits and stuff in the clawed

[07:44] code plan. Codeex might be better for

[07:45] that, but it will also only last for a

[07:47] bit. So, that is about it.

[07:51] I think DeepS isn't a good model and I

[07:53] can't recommend it either. Overall, it's

[07:55] pretty cool. Anyway, let me know your

[07:57] thoughts in the comments. If you like

[07:58] this video, consider donating through

[08:00] the super thanks option or becoming a

[08:02] member by clicking the join button.

[08:04] Also, give this video a thumbs up and

[08:06] subscribe to my channel. I'll see you in

[08:08] the next one. Until then, bye.