In this news:
Generative AI is the new tech buzzword, and it's sent stocks of companies like Nvidia both soaring and crashing as a new global tech arms race has been sent into overdrive. Mere months ago Large Language Models (LLMs) were demonstrating decent facsimiles of human conversation, and now AI generated images, videos, and audio have become a hot topic. If I were a tech CEO that wanted to demonstrate value to shareholders ahead of an annual financial call, I would be showing the world what we've achieved with generative AI, criticisms be damned. And that's exactly what Microsoft has February 19, Microsoft unveiled Muse, a generative AI model "designed for gameplay ideation." It's not entirely clear what that means, but from the perspective of someone that has followed the gaming industry for decades and spoken to dozens of developers in a variety of positions over that period, what Microsoft has achieved with Muse is both impressive, and incredibly concept of an AI generated game is nothing new, and AI Minecraft from Oasis AI has been available to the public for several months. Anyone briefly familiar with Minecraft – or indeed video games in general – can try out the available demo and identify the problems. There's considerable latency introduced by the AI model having to dynamically generate each frame, the clarity of the image changes in motion, and if you pan away from objects in view, and then look back at them, they have likely disappeared. AI Minecraft has no object permanence, like a newborn developers that have dropped the cynicism and approached AI Minecraft critically have asked themselves how to solve these obvious issues. Latency will improve as technology evolves, as will the clarity of the image. Object permanence, meanwhile, is more complicated. One thing that solves all of these problems, though? Building a video game has identified its own set of issues it intends to tackle with Muse, and titled these "consistency, diversity, and persistency." Video games change depending on the inputs of the players – the viewpoint, the actions on screen, and so on – and Muse needs to be able to reflect that with diverse outcomes based on the same initial input, while still retaining the layout and mechanics of the initial world. This is what Microsoft means by consistency and diversity, and finally persistency tackles that lack of object permanence. If a character becomes obscured by a wall or another player, that character still needs to retain its position and appearance, which is something that AI Minecraft is incapable tackles these points to an impressive degree, as we can see with the prototype "WHAM Demonstrator," which stands for World and Human Action Models, which is what Muse is. Here we're shown input with an Xbox controller, and the AI model reflecting that input in real time. It is, despite its impressive result, still very slow to render. A publicly playable AI video game from Microsoft might still be a way all of these impressive advancements in AI tech ignore one key detail: the game being rendered here already exists. Muse has been trained on gameplay of Ninja Theory's Bleeding Edge, a 2020 multiplayer was given one second of real human gameplay – both visual information and controller inputs – and then another nine seconds of just controller inputs, to see how the model was learning to generate accurate visuals. After 10k training updates, the game turns into a smear in moments. After 100k, the game appears relatively stable, but mechanics don't work as intended. After one million training updates, the visuals finally start properly representing the real game, though character position accuracy is still clearly a bit are impressive demonstrations, undoubtedly, but the knowledge that an existing game was necessary is a bit harrowing. Even after a billion training updates, will it still only be able to recreate a game that already exists, one that it has been trained on extensively? If that's the case, can we not simply play that game now? I do understand that this kind of training is an investment in the future of AI models, but in the here and now, I don't understand what kind of value this offers to consumers, or when we'll be able to see the do, however, already understand the environmental impact that generative AI has, with MIT News saying: "Beyond electricity demands, a great deal of water is needed to cool the hardware used for training, deploying, and fine-tuning generative AI models, which can strain municipal water supplies and disrupt local ecosystems. The increasing number of generative AI applications has also spurred demand for high-performance computing hardware, adding indirect environmental impacts from its manufacture and transport."Impressive but impractical feels like the key takeaway, especially since – in a broader sense – playable AI-generated video games are already in the hands of consumers. Nvidia is one of the biggest names in tech right now, and that share price can be attributed to Nvidia's line of processors. Nvidia's GPUs being able to mine crypto currency more efficiently than competitors made them incredibly desirable during the bitcoin boom, and now Nvidia's line of H100 Tensor Core GPUs are considered to be industry-leading for AI, and are mentioned in Microsoft's Muse breakdown. All of those business applications are well and good, but let's not forget that Nvidia first became a household name because of PC has been industry-leading in terms of consumer GPU power and proprietary technology like DLSS (Deep Learning Super Sampling) for years, and modern Nvidia GPUs utilise real time AI heavily. DLSS' principle use case is upscaling low resolution video game visuals for high resolution monitors using AI, and now, Frame Generation technology allows a modern Nvidia GPU to output four times more AI generated frames than real (or "rasterized") DLSS technologies are low latency, and tick off all of the issues identified in both AI Minecraft and Microsoft's Muse, while reducing the amount of power consumed when compared to the same workload without DLSS. Of course, that's because of a simple fact: the games that utilise DLSS actually exist. AI technology is advancing at a rapid pace, and tech giants like Microsoft are scrambling to solve the glaring problems with this bold new technology. But if we already have solutions that exist in other technologies, why not combine them?AI can be used to quickly generate detailed game visuals, as DLSS proves, and all it needs is detailed initial input data. When rendering a game, the GPU knows what each model in the area looks like, where it is, what should show on screen when the camera pans around — all of the things AI struggles with are right here. Take that burden away from the AI, and instead use AI to improve the output results based on – again – detailed input data, and that's what makes DLSS the consumer standard for gaming on an Nvidia GPU. AI upscaling has become so important in gaming that Sony's PlayStation 5 Pro even has its own proprietary PSSR (PlayStation Spectral Super Resolution) AI upscaling trawling through Microsoft's Muse announcement and trying to uncritically find benefits, I came away with just one: "AI agents that could behave more like human players". This was mentioned by Ninja Theory's technical director, Gavin Costello. His full quotes read as follows: "From the hackathon that started it all, where we first integrated AI into Bleeding Edge, to building AI agents that could behave more like human players, to the World and Human Action Model being able to dream up entirely new sequences of Bleeding Edge gameplay under human guidance, it's been eye-opening to see the potential this type of technology has.""Potential" sums it up in a less critical way than I could manage. Muse is a genuinely interesting project, and I want to see what AI can provide to gaming experiences, but poorly replicated video game visuals – based on video games that already exist – feels like a creative dead end. Generative AI has a place in video games – it already has that place whether people like it or not – but instead of prototypes of pie-in-the-sky potential models that are impractical in the present, I want to see current-day applications of AI, like DLSS. NPCs that mimic human behavior is one avenue that gaming could genuinely benefit from in the here and now — though it could also cause some real isn't one of the most valuable companies in the world for no reason. DLSS technologies helped cement its position as the leading consumer GPU provider, and now Microsoft is using Nvidia technologies to introduce a demonstration of something that is out of reach and impractical. Again, I do understand why this has been shown off now – shareholders want confidence that Microsoft is competitive in the AI space – but as a consumer and an armchair analyst, I don't see anything here that benefits the players. The promise of bold new technologies is sure to make some people a lot of money, but there's nothing Muse has demonstrated that should make the people that actually play games excited.
