IBM has developed a new optical technology that can train AI models at the speed of light while saving huge amounts of energy. The company says that by applying its breakthrough technology to data centers, it can save energy equivalent to the annual energy use of 5,000 U.S. homes when training AI models.
The company explained that while the data center is connected to the outside world via fiber optic cables, internally, they are still using copper wires. These wires connect GPU accelerators, which spend a lot of time idle while waiting for data from other devices, consuming energy and driving up costs.
Commenting on the development, Dario Gil, senior vice president and director of research at IBM, said:
“Because generating AI requires more energy and processing power, data centers must continue to evolve – and co-packaged optics can future-proof these data centers. With this breakthrough, future chips will communicate like fiber optic cables carrying data in and out of data centers, ushering in a new era of faster, more sustainable communications that can handle future AI workloads.”
IBM outlined its new co-packaged optics (CPO) prototype in a technical paper. By significantly increasing data center bandwidth, GPU downtime can be minimized, thus accelerating AI processing.
IBM explains that large language model (LLM) training time can be reduced from three months to three weeks. At the same time, improving energy efficiency will reduce energy usage and lower the costs associated with training LLM.