Researchers from MetaPlatforms' basic artificial intelligence research team said today that they will release a more advanced version of Habitat, an artificial intelligence simulation environment used to teach robots how to interact with the physical world.
Along with the launch of Habitat 3.0, the company also announced the release of the Habitat Synthetic Scenes Dataset, a three-dimensional artist-authored dataset that can be used to train artificial intelligence navigation agents; and HomeRobot, an affordable robotic assistant software and hardware platform that can be used in simulated and real environments.
FAIR researchers explained in a blog post that the newly released products represent their continued progress in "embodied artificial intelligence." By "embodied AI," they mean artificial intelligence agents capable of sensing and interacting with their environment, securely sharing the environment with human partners, and communicating with and assisting human partners in the digital and physical worlds.
Habitat is a catalog of virtual environments, such as office spaces, homes, and warehouses, that can be used to train and improve the ability of artificial intelligence robots to navigate the real world. The virtual environment is carefully constructed using an infrared capture system that can accurately measure the shape and size of objects such as tables, chairs, and even books. In these environments, researchers can train robots to complete complex, multi-step tasks that require the robot's ability to see and understand its surroundings.
Habitat 3.0 builds on these existing capabilities by supporting both robots and humanoid avatars, enabling humans and robots to collaborate on many different tasks. For example, humans and robots can work together to clean the living room or prepare recipes in the kitchen. FAIR said this opens up new ways to study the collaboration between humans and robots in various real-world tasks. FAIR stated that the human avatars in "Habitat 3.0" are very realistic, with natural gaits and movements, allowing for the most realistic low-level and high-level interactions.
"This coexistence of humans and robots in a simulated environment allows us, for the first time, to learn robot AI strategies in the presence of a humanoid avatar in a home-like environment and to evaluate these strategies alongside real people during everyday tasks," the researchers wrote.
FAIR stated that "Habitat 3.0" will shorten the learning time of robot artificial intelligence agents from months or even years to a few days. It also allows for faster testing of new models in a safe simulation environment without any risk.
The Habitat Synthetic Scenes Dataset, known as HSSD-200, will also help accelerate AI research, as three-dimensional simulations of real-world scenes are critical for training. FAIR explains that HSSD-200 outperforms its previous datasets because the three-dimensional scenes more accurately reflect those of the physical world than before. It consists of 211 high-quality 3D scenes that replicate real-world houses and other environments and contains 18,656 physical-world object models in 466 semantic categories.
According to FAIR, HSSD-200 provides fine-grained semantic classification corresponding to the WordNet ontology, while its asset compression capabilities enable higher-performance embodied AI simulations. Individual objects are crafted by professional 3D artists to accurately match real-world brands of furniture and appliances in appearance and size.
Finally, FAIR introduces a new HomeRobot library, a hardware and software specification for researchers who want to create physical robots to apply the models they trained in Habitat to the physical world.
HomeRobot is based on a user-friendly software stack and affordable hardware components, which means it’s quick and easy to set up and be ready for real-world testing. It is designed for open-vocabulary mobile manipulation research, which is the ability of a robot to pick up objects and place them into designated locations in any unseen environment. To do this, robots must be able to sense and understand new scenarios they encounter.
Holger Mueller of Constellation Research Inc. said Meta's announcement shows the company is making real progress beyond the hype of generative artificial intelligence, with powerful software that can be used to train and test intelligent robots in virtual worlds. He said: "Habitat 3.0 is now focused on human-robot interaction because it is a key milestone that must be perfected if we are to build robots that can function in daily life. The HSSD-200 dataset is very useful because generating physical objects in these environments is expensive and takes a lot of time."
FAIR said there is much more to come from these developments. The next step in its ongoing embodied AI research will focus on how robots can collaborate with humans in dynamic, ever-changing environments that reflect the real world we live in.
The researchers explain: "In the next phase of research, we will use the Habitat 3.0 simulator to train our artificial intelligence models so that these robots can assist human partners and adapt to their preferences. We will use the HSSD-200 in conjunction with Habitat 3.0 to collect data on large-scale human-robot interaction and collaboration to train more powerful models. We will focus on deploying the models learned in the simulations into the physical world to better measure their performance."