It's not just the engineers who develop the AI ​​systems that keep Waymo and Zoox vehicles on the road: Every day, thousands of people around the world sift through reams of driving data collected by cars equipped with clunky sensors. Employees have different titles - validators, annotators, annotators. But their mission is the same: to help the AI ​​driver understand what it's seeing.

“Essentially, they help the car understand its position in space and time, and more importantly, help the model understand how to drive safely in various scenarios,” explained Sapien CEO Rowan Stone. Sapien is a data platform that has customers like Zoox.

How AI is trained

This work might include helping AI identify objects captured by road sensors such as cameras or lidar systems: traffic cones, stop signs, or other obstacles, for example.

Stone also pointed to scenarios like police roadblocks or school buses picking up kids — real-life situations Waymo’s robotaxis would need to deal with — and said annotators would provide more guidance on how taxis should respond appropriately.

"Obviously, here again we need to bring in manpower," Stone said. “We need to refine the data set, we need to retrain the model with additional context, apply their corrections, and then we can move forward.”

Niche jobs in big industries

The overall scale of the data annotation industry may be very large. Stone said Sapien has more than 1 million "contributors" around the world.

However, especially in the field of autonomous driving systems, this number is much smaller. Stone estimates there are fewer than 5,000 employees worldwide working on autonomous driving systems. However, as more robotaxis become available, this number is likely to grow.

Omar Zoubi, vice president of TaskUs, explained that the company employs just under 2,000 people in all areas related to autonomous driving, and that number is likely to double by the second quarter of this year. TaskUs is a company that provides data annotation and remote support agent services to companies such as Waymo.

The annotation work itself may not be particularly glamorous: At Sapien, average hourly rates are typically set by customers or autonomous driving operators and can range from $3 to $6 an hour, Stone said. TaskUs does not disclose how much its data annotators earn.

Sapien's CEO said many of the company's employees live in Germany, Japan and Southeast Asia. Overall, Sapien's "employee base" spans approximately 100 countries.

AI takes over some work

Artificial intelligence is also taking on some of the data labeling work, but it has not yet made human work redundant.

Lukas Grapentine, director of solutions engineering at Sapien, explained that the AI ​​“pre-annotates” parts of the raw data set, while humans are tasked with reviewing the AI’s work.

In autonomous driving systems where human lives are at stake, it is particularly important to ensure the accuracy of AI.

"What it comes down to is that you can trust the data," Grapentine said.

The evolution of the future of work

TaskUs’ Zoubi expects the role of data annotators to evolve as automakers and self-driving companies gain access to more data, meaning they will face more complex driving scenarios. AI may be able to handle simpler tasks, but humans must take over interpreting more complex scenarios, he said.

"What I think is going to change here - at least personally - is that it's not just about doing basic data annotation and annotation, it's going to be more about root cause analysis and data optimization to enable self-driving vehicles to operate and navigate in specific situations," Zoubi said.

Stone has a similar prediction—that AI models will get better over time and rely less on humans as robotaxis become more adaptable to new cities and their peculiarities.

"I think the need for humans will decrease," he said, "but I don't think it will go to zero."