Camping at the orb’s location is a surefire strategy: Players must pick up orbs to win (imagine if Pac-Man’s specter just loitered near the entrance to every corner of the map). It also makes the game less enjoyable. Players no longer experience an exciting chase. Instead, the AI can create an unpredictable ambush. Trachel and Peyrot say their goal is “not to create superhuman bots—that wouldn’t be interesting and appealing to novice players—but instead to find ways to incorporate machine learning into games.” The game’s AI engine was used in production.”
That might sound dull to players who aspire to better AI. However, the machine learning techniques demonstrated by Trachel and Peyrot are still useful for difficulty adjustment even if the enemies the player faces in the game end up not using it. Julian Togeliusco-founder and research director at Modl.ai, has spent almost 5 years using AI to test the game. Modl.ai uses bots to look for graphical glitches, find holes in world geometry, and uncover situations that keep you from winning.
“You can tell us what kind of error status you are interested in. And then it will basically run. “You submit a job and it runs depending on how much you want to discover,” says Togelius. “And of course, we can group these for you and provide a report, indicating this is where you seem to be having problems, etc.”
Modl.ai’s test bots use machine learning to adapt each game under test, although its current implementation limits those adjustments to specific titles. Togelius says the company is prototyping adding a deep learning feature that will train bot behavior across multiple games. Once used, Modl.ai’s bots will learn to simulate the behavior of real players, which will more effectively spot the problems players would find.
For real Machine Learning, Game Engine needs a revolution
When the going gets tough, machine learning can be both the problem and the solution. However, creating a fair, fun challenge isn’t the only hurdle faced by developers looking to use machine learning in games. The problems run deeper—so deep, in fact, that they can force a rethink of how games are built.
Performance is a barrier. Machine learning requires a lot of training data to get worthwhile results, and that data can only be obtained by playing the game thousands or tens of thousands of times (although bots can offload, a tactic that Trachel and Peyrot used when building their demo). And once the training data is collected, the resulting model can become heavy to execute in real time.
“Yes, performance is clearly an issue, especially with large ML models that process frames for every tick of the game clock,” Trachel and Peyrot said in an email. “In our case, to avoid performance issues, we used a small neural network that only infers at precise moments of the game.” Scaling to the massive open-world environments that modern players expect is another matter entirely.
Togelius says the way modern game engines work exacerbates the problem. Machine learning is “definitely going to slow down because game engines aren’t built for this,” he said. One of the many reasons we don’t find modern AI more interesting in games is because Unreal and Unity and all of their likeness are fundamentally terrible—against AI in many ways.”
Animation is another matter. Most modern game engines expect animation to be strictly defined frame-by-frame. This works well when the animators know for sure how the game characters will behave, but the machine learning-driven AI can behave in a way that the animators don’t. expectation. Designers can solve this problem with a physics-based approach to animationbut this puts even more strain on performance on a game console or computer hardware and comes with its own development challenges.
In short, developers are faced with a monster of their own making. The game engine is built to use a tree of behavior and prescribed actions to create worlds of AI-driven NPCs that work well even on meager hardware. But as machine learning evolves, these classic solutions will need to be revisited.
“If you talk to a machine learning researcher who doesn’t know how to design games, they’ll say, ‘Why don’t you just use the new stuff and get NPCs that are more lifelike and adapt to the way you play’, and so on,” said Togelius. “But you can’t plug it into an existing game. You have to rethink what the game even is.
