What self-driving cars can learn by playing Grand Theft Auto
Spending thousands of hours playing Grand Theft Auto might have questionable benefits for humans, but it could help make computers significantly more intelligent.
Several research groups are now using the hugely popular game, which features fast cars and various nefarious activities, to train algorithms that might enable a self-driving car to navigate a real road.
There’s little chance of a computer learning bad behavior by playing violent computer games. But the stunningly realistic scenery found in Grand Theft Auto and other virtual worlds could help a machine perceive elements of the real world correctly.
A technique known as machine learning is enabling computers to do impressive new things, like identifying faces and recognizing speech as well as a person can. But the approach requires huge quantities of curated data, and it can be challenging and time-consuming to gather enough. The scenery in many games is so fantastically realistic that it can be used to generate data that’s as good as that generated by using real-world imagery.
Some researchers already build 3-D simulations using game engines to generate training data for their algorithms (see “To Get Truly Smart, AI Might Need to Play More Video Games”). However, off-the-shelf computer games, featuring hours of photorealistic imagery, could provide an easier way to gather large quantities of training data.
A team of researchers from Intel Labs and Darmstadt University in Germany has developed a clever way to extract useful training data from Grand Theft Auto.
The researchers created a software layer that sits between the game and a computer’s hardware, automatically classifying different objects in the road scenes shown in the game. This provides the labels that can then be fed to a machine-learning algorithm, allowing it to recognize cars, pedestrians, and other objects shown, either in the game or on a real street. According to a paper posted by the team recently, it would be nearly impossible to have people label all of the scenes with similar detail manually.
