Thanks to ChatGPT and Microsoft’s efforts to integrate the natural language processing tool into Bing, you can’t escape the mention of AI in tech news over the past few weeks. But while some might wring their hands thinking this is the genesis of the rise of machines, others have been quick to point out the dumb answers these ‘artificial intelligence’ tools spit out.
And as it stands, I remain unconvinced that the likes of journalists and people with specialist knowledge will be easily replaced by smart algorithms and chatbots that aren’t quite as idiotic as before. But as I chewed over this idea and the potential for ChatGPT to be more than a flash in the pan, I had an idea: AI-powered chatbots and advanced natural language processing could really be a boon for virtual reality games and experiences.
Hot off the back of my PSVR 2 review, I’ve been considering whether VR could really break out of the niche it’s in; even though that niche is somewhat large. While Horizon Call of the Mountain has impressed so far, and I think the PSVR 2 has a lot of potential, there are still precious few VR experiences that offer an experience that’s more than 'oh I can look all around me and get a sense of depth,’ and ‘oh I’ve got vertigo from dangling off a virtual cliff.’
As good as the PSVR 2 is at communicating feedback, I’ve already had the experience of climbing structures in VR and enjoyed kinetic battles in the likes of SuperHot. With all the tech the PSVR 2 has and the power of the PS5 at its disposal, I’d been keen to see a VR game that really does something completely new. It's here I think AI and the likes of ChatGPT could help.
AI and VR could offer engaging virtual experiences
Most VR games are very much on the rails affairs or have limited ways to make your experience feel different from that of others. That’s understandable given I don’t quite imagine open-world games would work well with current VR headsets — the PSVR 2 is comfortable, but I can't see myself wearing it for extended gaming sessions.
While I’ll never turn my nose up at being able to pinpoint arrow shots into the eyes of a prowling Watcher or throw an unloaded gun to shatter an approaching enemy, I’d like a fresher way to interact with a virtual world.
Now the PSVR 2 has eye tracking that can seemingly have NPCs in some games react to your gaze — this wasn't obvious in my testing, but it’s early days — so what if that was combined with a form of AI-powered chatbot? You could have a game where NPCs can use natural language processing and the reading of one's gaze to have unique conversations with the player.
I understand the basics of a neural network and how AI systems can learn, but I’m no expert and I can totally understand how these ideas might be an engineering nightmare. But with even the likes of Siri getting better at handling requests said in natural language, there’s more scope for chatbots to better parse what humans are spouting.
Imagine if you will, a VR Mass Effect game where you can wander a ship and can talk to a variety of characters, as is the case in the regular Mass Effect games. Only, rather than select rough answers via a conversation wheel, you answer in your own way, with AI processing figuring out what you mean, and eye and gesture tracking tech used to interpret the relationship between your words, gaze and movements to better figure out how you’re saying something. In some way, you could think of this as the near-ultimate conclusion to text-based adventures, only you're feeding an AI with a lot more variables and natural language.
Prompts or rough interaction prompts could be used to angle the player into saying something the AI would be able to handle (say a popup that notes Tali wants to talk about the Migrant Fleet), given trying to process an entire language, semantics, context within a game’s story and world and so on, might be a bit much for even the most advanced AI tech of today and the near-future.
Plus safeguards would be needed against people trying to push the limits of the AI, say have Liara raise her eyebrows at a certain crude remark. Or have Garrus go back to his calibrations if the player keeps asking him about his favorite sandwich or other such nonsense.
Alternatively, we could use the combination of AI, procedural generation and a headset’s tracking tech, to better customize and evolve a person's experience within a game or virtual environment. For example, a route around an area could be adapted to how fast the player is moving or what they are looking at. That could help make such dives into VR feel unique and fresh rather than an evolution of what’s come before.
Again, I’m aware this could be a wildly ambitious use of ChatGPT and OpenAI-based tech. But the use of AI to further deepen a virtual reality experience, beyond haptics and a better sense of space, could be the key to making VR games really stand out and become more than experiences you only occasionally dip your toes into.
Of course, I’m still quite happy to delve more into Horizon Call of the Mountain and try out Gran Turismo 7 in VR. But I’m hoping that the PSVR 2 will help spearhead a fresh wave of VR games that look at handling virtual reactions a little differently, beyond tracking my attempts to flip the bird at an annoying NPC.
