New advances in embodied AI research to better understand and navigate physical and virtual worlds.
To create and interact with immersive new experiences in the metaverse — where people can navigate virtual worlds as well as our physical world with augmented reality — AI systems must learn to move through the complexities of the physical world as people do. AR glasses that show us where we left our keys, for example, require foundational new technologies that help AI understand the layout and dimensions of unfamiliar, ever-changing environments without high-compute resources, like preprovided maps. As humans, for example, we don’t need to learn the precise location or length of our coffee table to be able to walk around it without bumping into its corners (most of the time).
Today, we’re announcing new scientific research that will help AI learn much more flexibly and efficiently to understand the physical world. Collectively, this body of work pushes advancements in visual navigation of embodied AI, a research area focused on training AI systems through interactions in 3D simulations rather than traditional 2D datasets.
Meta AI has been committed to long-term investments in the burgeoning field of embodied AI. Since we introduced Habitat three years ago, we’ve effectively solved the task of point-goal navigation using only an RGB-D camera, GPS, and compass data, and we’ve successfully tested our model with tasks in real-world physical settings using our PyRobot platform. Now, our latest advancements use the power of high-speed simulations to build more flexible, efficient navigation systems.
Models achieving high performance at point-goal navigation often require access to GPS sensors in simulation. These models do not transfer well to physical robots, because GPS data can be noisy, unreliable, or unavailable in indoor spaces.
We’ve developed new methods to improve the way AI tracks its location solely from visual inputs, also known as visual odometry. Our new data-augmentation technique trains simple but highly effective neural models without human data annotations. Robust visual odometry is all you need to push the state of the art from 71.7 percent success to 94 percent success on the Realistic PointNav task without GPS or compass data, under noisy action dynamics.
While our approach does not yet completely solve this dataset, this research provides evidence to support that explicit mapping may not be necessary for navigation, even in realistic settings. Read the paper here..
In another path toward more efficient, map-free learning methods, we have shown how to scale the paradigm of imitation learning from human demonstrations, which — until now — hasn’t been possible, as there hasn’t been a large enough data collection of human demonstrations. To fill this gap, we built Habitat-Web, a new data collection infrastructure for embodied AI, connecting our Habitat simulator running in a web browser to Mechanical Turk and allowing remote users to teleoperate virtual robots safely and at scale. We’ve collected an order of magnitude larger human demo data than existing datasets in simulation and two orders of magnitude larger than existing datasets on real robots.
