Nvidia has unveiled its first digital human technology small language model in the upcoming multiplayer mech game, Mecha BREAK.
Join NVIDIA experts at this year's #VMwareExplore, in Las Vegas. Learn how our collaboration with @VMware is transforming the #generativeAI landscape, offering platforms to accelerate AI adoption for enterprises across industries. https://t.co/xwQFQDNerk
— NVIDIA (@nvidia) August 19, 2024
Developed by Amazing Seasun Games, this innovative on-device model enhances the conversational abilities of game characters, making interactions more intuitive and natural for players. The on-device model, called Nemotron-4 4B Instruct, improves role-play and function-calling capabilities.
This allows game characters to better understand player instructions and respond more accurately.
🐜 ant gameplay time 🐜
Play as a fearless ant and guide your colony through the dangers of the forest as Empire of the Ants goes #RTXOn with DLSS 3 & NVIDIA Reflex!
Learn More: https://t.co/3cNawsTNYX pic.twitter.com/mTy4gdz0Qv
— NVIDIA GeForce (@NVIDIAGeForce) August 21, 2024
Optimized for low memory usage, it offers faster response times and is available as a microservice, simplifying the development and deployment of AI-powered applications. Nemotron-4 4B Instruct is part of a suite of digital human technologies from Nvidia that includes speech, intelligence, and animation powered by generative AI.
It is available for both cloud and on-device deployment, making it versatile for game developers. Mecha BREAK showcases the Nvidia Nemotron-4 4B Instruct NIM running on-device.
Unawake is #RTXON with DLSS 3 & NVIDIA Reflex!
As humanity’s last hope, fight for humanity’s survival in a merciless war between angels and demons in an immersive world featuring hard-hitting first person combat.
Learn More: https://t.co/8APCu3FuHD pic.twitter.com/ApB7m6azY1
— NVIDIA GeForce (@NVIDIAGeForce) August 21, 2024
The game also integrates Nvidia Audio2Face-3D NIM and Whisper, an automatic speech recognition model by OpenAI, for facial animation and speech recognition, powering character voices through the cloud.
This demonstration was first shown at Gamescom, one of the world’s largest gaming expos.
Nvidia’s digital human tech in action
Nvidia ACE and digital human technologies in Mecha BREAK allow players to interact with a mechanic non-playable character (NPC) that can assist them in choosing mechs, customizing appearances, and preparing for battles.
Kris Kwok, CEO of Amazing Seasun Games, said, “We’re excited to showcase the power and potential of ACE NIM microservices in Mecha BREAK, using Audio2Face and Nemotron-4 4B Instruct to dramatically enhance in-game immersion.”
Nvidia ACE and its digital human technologies continue to expand in the gaming industry. Global game publisher and developer Perfect World Games is advancing its digital human technology demo, Legends, with new AI-powered vision capabilities. In this demo, the character Yun Ni can identify people and objects in the real world using the computer’s camera powered by ChatGPT-4, adding an augmented reality layer to the gameplay experience.
These advancements enhance immersion and accessibility for PC games. Mecha Break is scheduled for release in 2025 and will be available on PC, Xbox Series X / S, and PlayStation 5. The game will feature a variety of mechs across different categories such as assault, melee, and sniper for online matches.
Players will be able to battle both on the ground and in the air, reminiscent of the Armored Core series, but focused solely on multiplayer gameplay. A demonstration showcased the AI NPCs responding to player inquiries about missions and mechs, including the ability to change a mech’s paint job by voice command. However, early feedback suggests that the NPCs’ dialogue delivery was somewhat flat and uninspiring, with occasional delays in responses.
It remains to be seen whether these aspects will improve as the game approaches its release date next year.
