If you've ever dreamed of being able to chat with your favorite video game characters as if they were real people, you might be closer than you think to making that dream come true. A leaked internal video from Sony's PlayStation group has revealed that the Japanese company is secretly working on an AI-enhanced version of the protagonist of Horizon forbidden west. Aloy, the red-haired heroine that millions of players have come to love, could soon transform from a “pre-written” character into an entity capable of maintaining a fluid and reactive conversation in real time.
The footage, illustrated by the voice of an executive from Sony’s Advanced Technology Group, shows something that until recently would have been considered science fiction: Aloy answering spoken questions with a natural tone, appropriate facial expressions, and a contextual understanding of what she’s being asked, all while the game continues to unfold.
When Artificial Intelligence Meets PlayStation
You stop during a game session to ask a question to the character you're controlling: I don't know, a curiosity about the game's plot, perhaps, or a tip on how to proceed. Cool? Sharwin Raghoebardajal, director of software engineering at Sony Interactive Entertainment, demonstrated exactly this scenario in a demonstration video.
The technology shown combines different systems of artificial intelligence: OpenAI Whisper to convert voice into text, GPT-4 e llama 3 to manage conversations and make decisions. But the most interesting aspect concerns Sony's proprietary technologies: a system called Emotional Voice Synthesis (EVS) to generate Aloy's expressive voice, and the technology Mockingbird to synchronize audio with facial animations.
The result is surprisingly fluid, with Aloy answering spontaneous questions both in a test environment and during actual gameplay. Horizon forbidden west. What strikes me most is that, according to Raghoebardajal, parts of this technology have been tested directly on “low-resource” PS5 consoles (suggesting that implementation may be closer than we think).
Aloy and friends: the race for artificial intelligence in videogames
Sony isn't alone in exploring this territory. Nvidia presented its technology several times in 2024 Ace for AI-powered NPCs, going so far as to create a playable demo called “Covert Protocol” in collaboration with Inworld AI.
The Microsoft is moving quickly in this direction, collaborating with Inworld AI itself to bring AI characters to Xbox consoles. The Redmond company has even developed a proprietary model called Muse AI, designed to generate gameplay and game environments for prototyping and ideation.
It is clear that there is a real race between the giants of the sector, and this is not surprising: Whoever can implement this technology effectively could completely redefine the gaming experience for years to come.
An exciting or worrying future?
The implications of this technology are huge, and not all of them are positive. On the one hand, the idea of video game characters capable of dynamic conversations and contextual responses opens up incredible narrative possibilities. Imagine a video game where every character can have unique conversations and remember your past interactions, or an adventure where the protagonist can actually help you when you're stuck.
On the other hand, there is one thing to think about: what will be the impact of these technologies on voice actors and game developers?
This is just a taste of what is possible
Raghoebardajal said in the video, but the phrase has different connotations depending on who reads it. For some developers, it could sound like a threat to their creative work, especially at a time when layoffs are hitting the video game industry hard.
Un recent survey of the Game Developers Conference (GDC) 2024 revealed that almost half (49%) of the 3.000 interviewees already uses generative AI tools in the workplace, with 31% saying they use them personally. The trend is clear and unstoppable.
These experiments by Sony with AI-enhanced PlayStation characters are set to spark further discussion about the role of AI in video game creation. And while the debate rages on, technology continues to advance, leading us toward a future where the line between programmed characters and autonomous entities becomes increasingly blurred.
