Modern Warfare 4 Engine Deep Dive — Full Breakdown & Transcript

[00:09] Infinity Ward's R&D center is on a mission to push the technology of each Call of Duty title over the last. From discrete innovations to dazzling breakthroughs like ray tracing, it's here that much of the series technical

[00:22] wizardry happens. Visiting the studio 2 years on from my last trip to see Black Ops 6's PS5 Pro upgrades. If you'll recall, today it's Modern Warfare 4 in the spotlight. I was once again joined by IW Poland studio head Mau Drobot, who

[00:38] served as the series principal rendering engineer, and this time he generously showcased each new upgrade coming to the Call of Duty engine. It's unprecedented access, and much of what you're seeing here is debug footage plucked from very

[00:51] early development builds, typically using base PS5. So video quality will vary as we go. Most of it is set in a virtual test environment and we get debug diagnostics into the bargain as well. Oh, and it's worth stressing that

[01:05] being such early footage, there may be visual oddities that are simply not in the final game. So, first of all, let's dig into the new engine features debuting in Modern Warfare 4, like the ray marched fog and smoke, the weather

[01:18] simulation, and improved materials and lighting. Secondly, let's see the new rendering tech used to simulate ray tracing and path tracing using less resource inensive rasterized techniques. And finally, for the hardcore tech

[01:31] enthusiast, we'll be taking a look at some of the more deep-seated changes being made to the engine's renderer. Before we kick off, there is of course a Switch 2 version releasing day and date with the other consoles handled by

[01:43] developer Digital Legends. It's a momentous release because it marks the series first return to a Nintendo console since 2013's Call of Duty Ghosts on Wii U. For today's video, the focus is on broader engine upgrades on

[01:57] PlayStation 5, Xbox Series X S, and PC, though of course we'll be covering Switch 2 in a later video. So, with that in mind, let's get started. in mind, let's get started. [music]

[02:19] in Modern Warfare 4 is its ray marched smoke and fog. It's a brand new voytric system built exclusively for the game with a knock-on benefit to playability across its campaign, multiplayer, and of course the new open map DMZ modes. So,

[02:34] putting this in a practical example with actual gameplay. An obvious use case for this is smoke grenades. Previously, the Call of Duty engine used a more conventional visual effect here, transparencies, to fill the space with

[02:46] smoke after ignition, but this was never a truly interactive element. For Modern Warfare 4, though, at last, we get a fully voxalbased solution that allows for all sorts of interactions with those plumes of smoke via ray marched

[03:00] calculations. Hence, it's now possible to chisel through that smoke with gunfire where the trajectory of each bullet creates a precise hole, at least bullet creates a precise hole, at least briefly before it all dissipates.

[03:15] trivial feat, especially to make it consistent to all players online for competitive balance. Modern Warfare 4's focus on current gen consoles like PS5 and Xbox Series X helps, of course, and I'm told that adapting the tech for

[03:28] older PS4 and Xbox One machines in a theoretical last gen release would have been a challenge. Not impossible, but certainly an extra hurdle to jump on larger maps. Instead, Modern Warfare 4

[03:40] is a fully current gen release, and there's no need to scale back this time. So, how was this achieved? Well, Mau explains that this vauil system covers everything from grenades to height fog, clouds, air fog, atmospheric fog, all of

[03:54] which use a single unified vauil system. This is the crucial bit. This vauil method incurs a fixed cost to the game's overall render budget. It's a set ceiling in terms of GPU demands,

[04:06] regardless of how many grenades go off in one place, for example, or how much fog appears across the horizon. In this way, it has an advantage over the conventional VFX approach in its scalability. Whereas before, many

[04:19] overlapping smoke transparency effects might spike GPU load and therefore take might spike GPU load and therefore take a visible toll on performance.

[04:32] advantages. For one, it's possible to strictly contain the area of a smoke grenade spread. So supposing you unleash a grenade in a tunnel, the smoke's volume logically fills out that space with no spill through the collision

[04:44] mesh. And likewise for walls. Added to that, these are fully shadow casting and self shadow casting, meaning they receive and transmit shadows to the environment. And finally, smoke volumes also interact with light. Outdoors,

[04:57] you'll get streaks of light passing through that smoke. And impressively, lit billboards also transmit light through it. This is the multiplayer map, Nautical, where what's known as an animated texture area light projects its

[05:09] colorful details onto a nearby smoke volume. Now, the challenge for evolving Call of Duty's tech like this is that fair online balance is a priority. And so, everything you see here, the lighting, the fog dispersion, the

[05:22] shadows must manifest exactly for all players across the network. The solution to this problem is that the vaual grid determining where bullets create holes is generated both on the server and locally to keep every player in sync.

[05:36] This applies to every volutric effect, including overhead clouds. And this even factors in where their shadows project to the ground.

[05:51] 4's new weather systems. Yes, you guessed it. Vauxil save the day once again here to keep everything coordinated for all players online. Everything environmental based is new with vauels evenly scattered across the

[06:03] environment to allow server determined zones for different weather states whether that's sunshine, rainfall, snow, and so on. In this case, the vauels determine the light state for each area. And there's even temperature maps plus

[06:16] wind simulation modeled in 3D all handled server side for consistent online balance. For more open-ended modes like DMZ, it's a huge change at an engine level. Now, of all the weather states in MW4, the snow is perhaps the

[06:31] most involved at a tech level, as it's quite literally an extra layer added to a pre-existing level. Some elements of the map are inevitably non-interactive, like these static snow caps, which are added by artists in select spots, but a

[06:45] surprising amount is fully reactive this time around. As an extra flourish, you also get what's known as glint on specular highlights on the snowed, best spotted as the sun hits the surface while crawling.

[06:58] while crawling. [music] on in one video, and often we're getting refinements on what we saw in Black Ops 6 and 7. In that category, we have improved glass destruction physics,

[07:13] which gradually lets you chip away at the structure of a glass pane before it crumbles through. Plus, there's a brand new technique for projecting light from animated walls. It's known internally as [ __ ] projection. And in this example,

[07:25] using ray tracing in the environment. All the lighting from an animated surface is factored into the reflective flooring. Another big area of expansion for this year's game is in the use of voxels to track the weather's impact on

[07:37] objects. Simply put, Modern Warfare 4 expands the system we had in Black Ops 6 and 7 outwards to allow for even more weather states. So, if a soldier is crawling, as in this debug footage, imagine they're in an upright standing

[07:50] T-pose and how this grid of voxels would map to each body part. Different colors denote different levels of accumulation, whether that's for mud, dust, blood, charring from flames, snow, wetness, or even oil buildup. In other cases, it'll

[08:05] add raindroplets to a gun, water streaks down the car's chassis, or even affect the way raindroplets hit the tarmac. Also, to cram one more new feature in, for blood, known as skin bolted particles. It's not for the squeamish,

[08:20] but this allows blood to spew from the precise point of a bullet's impact, and that blood again leaves trails behind the player as they walk.

[08:36] in Call of Duty. Of course, path tracing is considered the holy grail in computer graphics to achieve realistic reflections and lighting across the scene. I can confirm that there are no plans to include path tracing in Modern

[08:49] Warfare 4 as it stands today. But on the upside, the PC version does ship with options for ray traced, global illumination, and reflections. As with Black Ops 6, path tracing is still used internally, but purely as a visual

[09:02] reference point for now. The line of thinking is, suppose you optimize an engine and the art to the point that you're able to get most of the way to the look of path tracing using less taxing rasterized techniques. Suppose

[09:14] you get it perceptually close while running at a fraction of the GPU cost. In that case, the team's work pays off for a much wider spectrum of platforms rather than a top small percentage of users with high-end GPUs. Clearly, path

[09:28] tracing is king, but the performance hit to get there is considerable, and hitting 60 frames pers and beyond will be a bigger priority for most players. The solution, Modern Warfare 4's rasterized pipeline, is now vastly

[09:43] improved over last year's Black Ops 7. We get an entirely new approach to screen space reflections and screen space ambient occlusion this time with a focus on avoiding occlusion artifacts in a smarter way. Inevitably, there are

[09:56] limits. It's worth saying straight up. For reflections, rasterized SSR still depends on screen space data to project a mirror image, meaning puddle details are apt to change as you move the camera. Still, it's a step forward for

[10:09] this year's game. And here's how it works. Modern Warfare 4 uses a neural light grid which contains the dynamic occlusion data for three aspects in probes. The ambient occlusion shading, global illumination, and bent normals.

[10:24] The key thing is the engine now uses a compression algorithm machine learned via a neural network that learns the representation of lighting within a scene to create more accurate results. Thanks to this, probes need not be

[10:36] spread densely across the scene to get highquality rasterized lighting. instead. And to save on GPU load, it's possible to spread them more sparsely at set points across an area. While characters also use between two to 60

[10:50] buffer in isolation before they're blended to create the final frame. There's screen space ambient occlusion, global illumination, and finally screen space bent normals, a representation of specular highlights. Ultimately, taking

[11:04] this new machine-learned approach, the team says everything to do with ambient occlusion and global illumination is higher quality and more dimensional. It integrates more correctly with the scene, too, even with a sparer spread of

[11:17] probes overall. As an aside, the use of this compressed light grid is said to be data saving, so there's a double benefit. Call of Duty installs have notoriously ballooned in recent years, and this aims to help minimize Modern

[11:30] and this aims to help minimize Modern Warfare 4's install size where possible. approach to reflections and GI compare to ray tracing and path tracing?

[11:43] Focusing on the reflections then in this multiplayer level, lithium, flipping between the rasterized and ray traced view shows a difference on the facto's metal piping there. Details under and around the pipe fall outside of screen

[11:57] space, which is an area ray tracing is able to better factor in, but both approaches look natural on their own terms. Moving to path tracing when compared to the rasterized approach, path tracing takes it a further step

[12:10] forward. You get more light bounces at a higher level of precision, reflecting more scene data across that piping. Again, no question the path traced approach is the more accurate of the two, but at a fraction of the GPU cost,

[12:23] the rasterized method is certainly convincing at range. As a useful alternative view, here's the very same shot, but isolating the reflection buffer of each. The team's new material capture tools

[12:37] also make this restized approach easier to engineer. BRDF birectional distribution function databases are freely available but in using its own capture technology here their scope for more accurate physically based materials

[12:50] across the game. One of the devices used internally is this one, the refletoter 2.0. Again, thanks to Macau for supplying these. Here's a few shots of the gizmo itself being used out in the field. The idea is the CT refletoter

[13:04] takes samples at multiple light gradients and then the camera snaps the BRDF. Overall, that gives the team 16 containing all the crucial properties of a material, including their diffuse,

[13:18] specular, and glossy values. Unlike photoggramometry, for example, this approach lets the team get an actual perpoint physical value of a surface, while a new version of the device is also work in progress. With this new

[13:31] material workflow, cloth has subsurface scattering, while the rubber band holding the soldiers radios and the tiny hooks of polyester representing Velcro hooks of polyester representing Velcro all benefit from more realistic shading.

[13:49] isolate just that GI pass this time. Comparing raster, ray tracing, and path tracing. This buffer shows the indirect lighting and shading of a scene in its rawest form. Similar to the reflection shot, each has their differences.

[14:03] Clearly, variations in how that shedding spreads. The rasteriz method is also much more noisy in this untreated state, which the IW engine's anti-alysis cleans up in a later pass. Even so, with the improved material capture and the new

[14:16] light probe system, it gets pretty close to the path traced GI in spots. Areas just outside of direct sunlight have a similar spread of shade. Though, if we return to the completely realized final frame, path tracing still has more

[14:30] accurate contact shadows on the railings to the side there. Outside of the think it achieves what it sets out to do. Modern Warfare 4's maps get rasterized GI and reflection coverage at an accuracy that wasn't so viable

[14:44] an accuracy that wasn't so viable before. we have new voytric systems, new weather, and a much improved rasterized

[14:57] pipeline. But the reality is many of this year's improvements go beyond what's visible to the player. Under the hood, the team at Infinity Ward Poland makes a sway of improvements here that all being well, the player shouldn't

[15:10] notice. The good news is that having early dev footage actually lets us visualize those upgrades for a change. Well, at least some of them. Chief among the optimizations this year is the way variable rate shading VRS is drastically

[15:24] improved over previous entries. For Modern Warfare 4, VRS is now applied through the entire render pipeline. Not just the pixels being rendered at the end, but also visual effects, transparencies, and post effects. This

[15:37] approach is unique and unlike most other engines on the scene today. So to demonstrate, we have a breakdown of a scene presented three ways. First off, there's a new debug view for meshlets, the triangles making up a scene. As an

[15:51] aside, all the games models are meshlets based, and this has been in place in Call of Duty for a fair while. But finally, there's a debug mode to properly visualize it. Meanwhile, in the middle, there's the VRS view. Green

[16:04] represents full resolution, while those slivers of red and yellow highlight areas where visual data is compressed. To achieve this, I'm told a hierarchy of bins is created to determine the priority of each frame and which part

[16:18] gets more pixels. It attacks the entire render pipeline and by the end VRS with the assistance of TAA ultimately aims to resolve to 4K resolution in this example. So, it's remarkably effective as a cost-saving measure. And as you can

[16:33] see in the final frame, those affected areas are tough to actually catch by areas are tough to actually catch by eye.

[16:45] effects. Despite the heavy reliance on voxalbased volumetrics this year, there is still a need for good old-fashioned transparencies as well. In short, a more optimal approach is adopted this year with multi-res transparencies that scale

[16:58] distance. So, for example, the smoke billowing in the far distance here is colorcoded with red representing a quarter resolution. This is a stepping the teams faced with Call of Duty for all these years. Order independent

[17:13] transparencies. The nightmare scenario is one where multiple transparent elements like glass overlap with one another. Add in effects like smoke in between and the GPU cost can be immense. But with Modern Warfare 4, there's a

[17:27] system that now dynamically calculates the resolution of an effect based on distance and also where they overlap. And the result is a more than 60% reduction in VFX cost. It's the first time the series has deployed this

[17:40] solution across the entire game, having been used in parts for Black Ops 7. The key point being, much like VRS, it should not be visible to the player in should not be visible to the player in any way besides a frame rate uplift.

[17:59] another big change now that it's moved to modern PS5 and Xbox Series systems. First up, a half precision floatingoint 16 renderer is supported now, doubling GPU efficiency while handling the game shaders. For context, FP16 support was

[18:15] not possible on last gen PS4 and Xbox One architecture where it was single precision only. For Modern Warfare 4, there's also a move to a new hybrid rendering setup allowing two renderers to run in parallel. First, there's Ford

[18:29] Plus, the classic GPU accelerated raster, but then there's a new softwarebased visibility buffer called V+. The idea is to have the two run simultaneously to shade opaque elements like trees and dense geometry, but the

[18:43] task there is split based on distance. So, for example, if a triangle appears large on the screen, it'll be run with the classic hardware F+ approach, if that triangle is too small in the distance to be shaded within a pixel. It

[18:56] then dynamically flips to the visibility buffer instead, V+. Muel explains that this solves a problem with what's termed quad occupancy, where the shading of distant triangles is a challenge for GPUs when clustered up at a sub pixel

[19:10] level. It's a neat optimization and for overlapping foliage elements in overlapping foliage elements in particular, it takes say 17 milliseconds in the overall frame time to handle opacities down to 14 milliseconds.

[19:30] machines, Modern Warfare 4 represents a more significant forward step for Call of Duty's tech than usual. Alongside the weather, its voxalbased fog, and weather, its voxalbased fog, and rasterized AO and GI, we're also seeing

[19:43] fundamental parts of its renderer being reworked here. Again, thank you to the team at Infinity Ward Poland for all this access and all this footage. The count in just one video, sadly, but hopefully I've covered enough of them

[19:57] here today. Taking a step back on a more general gameplay level, there's plenty of change being made to its momentto- moment action this year, too. A few points are certainly worth noting here. Movement is completely reworked this

[20:09] year, for example, with every footfall affecting your gun swaying motion. You'll react to walls contextually, and your stance even changes based on proximity to an enemy. All of this leads up to the moment you finally hit the

[20:21] trigger to fire, where a new physical simulation of a bullet's trajectory is now in place. It avoids what's known as bloom, where firing from the hip would previously randomly decide where bullets would fire. Today though, Modern Warfare

[20:34] 4 is the first in the series to move to a more realistic model where that bullet goes exactly where the gun tip is pointed. Factoring in recoil and the player's movement in the process. Alongside improvements to depth of field

[20:46] while aiming and tweaks to field of view, it's fair to say long-term Call of Duty players will feel a difference to its handling and animations. But that's all from me today. If you did enjoy this video, feel free to like or

[20:59] subscribe, and don't forget to hit that bell for instant notifications as any new video lands. But for today, thanks for watching.