A[Apple’s research team disclosed progress in robot “brain”, with an average success rate of 91% in single task tests] On November 7, Apple’s research team disclosed progress in robot “brain” research. On November 4, local time, the Apple research team submitted the paper "Perceptive Humanoid Motion from Simulation" on the preprint platform arXiv, disclosing the humanoid robot autonomous control system HALO. The research team also disclosed the data generation, model training and deployment code, covering action generation, action tracking, task orchestration, simulation and real robot operation. According to the paper, the research team conducted 360 single task tests in 6 real environments without pre-scanning or mapping, with an average success rate of 91%. In another real environment, 10 tests were conducted on three combined tasks, and the average success rate was 70%. Among them, the success rate of the combination of "leaning over the obstacle - walking towards the target - pushing the box" is 50%. According to reports, the research team used 250 NVIDIA H100 GPUs and took about 48 hours to synthesize the main training data. The action generator is trained through imitation learning, using 64 B200s;
The action tracker is trained through reinforcement learning, policy distillation and other steps, using 4 H100s. It should be pointed out that the data generated by simulation does not completely get rid of human participation. The team stated that the new physical tasks still require manual writing of prompt words and design of action key frames, and some actions also use human video or motion capture materials. The research team also pointed out that current robots mainly rely on visual perception and it is difficult to directly perceive contact situations; when objects leave the camera's field of view or are occluded, the performance of tasks involving more contacts will decline. Adding tactile perception is one of the directions for subsequent improvements.
— All loaded —
