You might be as surprised as I was to learn that Assassin’s Creed Mirage is one of the few games to use AI neural texture compression—the only one, I think

If you’ve been thinking, like I have, that we’re still some way off seeing AI being used in games for more than just upscaling or frame generation, then it’s time to think again. That’s because it turns out that Ubisoft‘s Assassin’s Creed Mirage is one of only a tiny number of games to use neural texture compression.

As someone who is very much a hobbyist in the world of 3D graphics, spending my spare time messing around in Unreal Engine and ShaderToy, I like to think that I have my ear to the ground about what games are using the very latest rendering technologies. So when I read a blog by Ubisoft (via Access The Animus on X), describing the implementation of neural texture compression in AC Mirage, I was genuinely caught out.

Up to now, I wasn’t aware of any game using it, and I’m a bit puzzled as to why Ubisoft has only now decided to talk about this. The company has done so in the past, at the 2024 GDC, but that was just about the tech in general and used Mirage as a demo for it. The presentation even has “Internal Tech Demo | Not Shipped | Not Future Update” plastered all over it. Yes, I am a bit slow to catch on.

Anyway, so what’s all this about? In short, it’s about replacing the use of multiple, high-resolution textures with a single, lower-resolution map and a small neural network. The replacement map looks a little bit like the original(s) but when it comes to using it, the engine runs the neural network to reconstruct material properties, such as how well it reflects, how rough the surface is, and so on.

Image 1 of 2

A presentation slide shown at GDC 2024 by Ubisoft, showing the visual and memory benefits of using neural texture compression in Assassin's Creed Mirage — (Image credit: Ubisoft)

You don’t get something for nothing in the world of graphics, though, and when it comes to neural texture compression, all that fancy AI stuff requires a fair bit of compute. So much so that Ubisoft only “applied [it] selectively to a subset of assets, focusing on objects with high instance counts and significant memory pressure.”

Although the blog doesn’t explicitly mention them, the images suggest that Ubisoft is talking about objects such as trees, furniture, and buildings. The game will have relatively few variations of them, but a typical scene will still apply lots of them around (which is what ‘instances’ refers to here). They’ll all be using pretty much the same textures, so there will be a constant VRAM usage just for these objects.

Enter stage left, neural texture compression to cut down on that memory demand. Given the state of the DRAM and SSD market right now, you can bet your last dollar that a whole heap of game developers will be looking closely at what Ubisoft has done to see if it can help out in their games, too.