According to "Business Insider", Tesla insists on using self-driving technologyPure visual solution, now it has used this set of mature strategies inTraining of the humanoid robot Optimus. Insiders familiar with the matter told Business Insider that Tesla informed employees at the end of June that the company would focus more on advancing the training of the Optimus humanoid robot through purely visual solutions in the future.
Musk changes Optimus training strategy
Previously, Tesla usedMotion capture suits and virtual reality (VR) headsetsto record data from human operators and remotely control robots. Now, Tesla will mainly passRecord videos of employees performing tasksto train the robot to learn how to do e.g.Pick up objectsOr actions like folding a T-shirt.
Tesla said ditching motion capture suits and remote operations would allow the team to scale up data collection more quickly, insiders said.
This adjustment marksA major shift in Tesla's robotics strategy, will align Optimus’s training methods with Elon Musk’s long-standing philosophy:AI can master complex tasks through cameras. For a long time,Tesla uses a similar approach to train its self-driving system.
Break the rules
Remote operation and motion capture are standard practices in the industry when training robots. For example, well-known robotics companiesBoston Dynamicshave usedremote operationto train its Atlas robot. During training, employees don motion capture suits to perform various tasks. This data is then transmitted to the robot. Motion capture suits can also be used to control robots remotely.
It's unclear whether Tesla will prioritize the use of motion capture suits and remote operations again in the future, or whether it will use video data to build on previously collected information for further training.
Robert Griffin, senior research scientist at the Institute for Human and Machine Cognition, told Business Insider that large amounts of teleoperation data enable robots to learn through actual interactions with their environment. Teaching robots to translate video data into real-world actions is no easy task, he said.
"If you just use video data, you don't have this direct interactive experience," he said.
In May of this year, Tesla released a video that appeared to show the Optimus robot completing various tasks based on video footage.
Optimus
Konstantinos Laskaris, Tesla’s director of Optimus hardware development, appears to have acknowledged the new strategy. In May of this year, he posted on LinkedIn: "This may sound unbelievable, but our robots are learning new tasks directly from human videos!"
Musk also said in May,Optimus will eventually be able to learn to perform tasks by watching YouTube videos.
Insiders familiar with the matter said that Tesla briefly stopped recruiting for the Optimus team during the technology transformation period. As of the end of August, Tesla's job page still listed more than 50 Optimus-related positions.
technological transformation
People familiar with the matter said that until the end of June, Tesla was still operating Optimus remotely and training the robot through a motion capture suit. Employees needed to spend time troubleshooting issues with the motion capture suits and Optimus, which limited the amount of data the team could collect.
Since the shift in training, employees have focused on using5 in-house homemade camerasRecord your own actions. The cameras are mounted on helmets and heavy backpacks worn by employees. The camera shoots in all directions to provide the AI model with precise environmental positioning data.
Christian Hubicki, director of the Robotics Laboratory at Florida State University's FAMU-FSU College of Engineering, said that multi-angle shooting may help Tesla collect finer details "such as the position of joints and fingers" and achieve precise spatial positioning of the robot. He noted that the videos could also be used to enhance data sets previously collected through remote operations.
Staff performing training tasks receive specific instructions, particularly regarding hand movements, to ensure that the movements are as close to the human form as possible. One person said some workers could spend months performing the same simple action over and over again.
Jonathan Aitken, a robotics expert at the University of Sheffield, told Business Insider that Tesla will likely need to find a way to teach Optimus to use movements that can be applied to a variety of tasks.
"In an operation of this size, they have to build a common library of moves, otherwise it would take an infinite amount of time to train them all one by one," Aitken said.
He added that Tesla may adopt a strategy similar to that of Physical Intelligence, which feeds robots massive amounts of demonstration data so that they can learn transferable skills and apply them flexibly, rather than just mechanically memorizing how to perform a single task.
More difficult than autonomous driving
The new strategy is consistent with Tesla's approach to training its self-driving system. While other self-driving companies use lidar and radar sensors to train software, Tesla relies primarily on multiple cameras.
Tesla uses data collected from millions of Tesla owners, whose vehicles are equipped with eight to nine cameras. Musk said,Tesla was able to release its assisted driving system in China because the company used public videos of Asian streets to train its AI system.
However, Musk admitted during an earnings call in January that "the training requirements for the Optimus humanoid robot may end up being at least 10 times greater than what is required for the car."
"This is a very Tesla-esque approach to developing robots," Aitken said. "No other company is trying to do this at the same scale. The amount of data they need will be comparable to the amount of data used to train the car."
Alan Fern, an AI and robotics expert at Oregon State University, said training Optimus will be more arduous for Tesla than developing a self-driving car.
"Driving is a single task," Fein said. He said that video learning mainly requires robots to understand what is happening in the video and then have the skills to complete the task. "Some skills can be learned through observation, and some require physical practice in simulators or reality."
As of press time, Tesla has not commented on this.