After four months, the world's largest supercomputing center "Supercluster" officially started operation. Elon Musk publicly announced on social platforms that at 4:20 a.m. on July 22, U.S. time, the “Supercluster” jointly built by xAI, X, and NVIDIA has begun training. It consists of 100,000 H100 blocks and is currently the strongest training cluster in the world.
100,000 Nvidia GPUs are equivalent to Musk building a "Great Wall of Chips" in the AI world. Compared with the 25,000 GPUs used by OpenAI to train GPT4, the number has increased by four times.
It is worth mentioning that the world's largest supercomputing center just started construction in March this year and was scheduled to be completed in 2025. Unexpectedly, it took only four months to complete ahead of schedule.
The official opening of the supercomputing center will sound the clarion call for Musk to counterattack OpenAI.
A hugely expensive supercomputing center
The fundamental reason why Musk built the supercomputing center was to train xAI’s chatbot Grok.
Musk established xAI in July 2023, and its core founding members come from well-known companies such as OpenAI, Google DeepMind, Microsoft Research, and Tesla. In order to allow it to directly compete with ChatGPT, Musk has spent most of his energy in the past two years on xAI.
In order to ensure the implementation of the supercomputing center, Musk publicly stated that he would personally take charge and supervise the development of this supercomputer.
If you want to build a supercomputing center, site selection is undoubtedly the key.
According to "Forbes", Musk's team negotiated with seven or eight cities, and finally settled on Memphis, Tennessee, in March because Memphis will provide sufficient power and rapid construction capabilities.
It is worth mentioning that supercomputing centers require a lot of resources. In addition to basic hardware such as Nvidia chips, the factory may use up to 150 megawatts of power per hour, which is equivalent to the power required by 100,000 households. This may cause a huge burden on the local area.
In addition to power resources, cooling of the supercomputing center is expected to require 1.3 million gallons of water per day, which means that it will need to be pumped from Memphis' main water source every day.
This also sparked dissatisfaction among Memphis citizens and environmental groups, who jointly issued an open letter protesting: "We must consider how an industry that uses such a large amount of energy will further impact communities that are already overwhelmed by pollution and high energy burdens."
To complete the project, xAI committed to improving the city's public infrastructure to support the development of the supercomputing center, including building a new electrical substation and a wastewater treatment facility.
It can be said that Musk paid a sky-high price for the supercomputing center. According to cost estimates, 100,000 NVIDIA H100s will cost US$30,000-40,000 each, and the total value will reach US$4 billion (equivalent to over 29 billion RMB).
Even if Musk is the "world's richest man", the huge capital investment is too much to bear, so he teamed up with Dell and Supermicro to build a cluster and obtained 24,000 H100 resources from Oracle.
Although huge amounts of money have been spent, Musk still unswervingly promotes the construction of supercomputing centers. He summed it up in one sentence on social platforms:
"We have to take the wheel ourselves."
xAI’s valuation exceeds 130 billion
It has to be said that in order for xAI to catch up with OpenAI, Musk has given everything.
With the blessing of the world's richest man, xAI, which was only established in July 2023, has just turned one year old and already has the capital and confidence to challenge OpenAI.
On April 13 this year, xAI launched its first multi-modal large model Grok-1.5V. In addition to text functions, Grok can now handle a variety of visual information, including documents, charts, diagrams, screenshots, photos, and can perform multi-disciplinary reasoning.
With the birth of Grok-1.5V, xAI is very popular in the capital market.
On May 26, xAI announced the completion of US$6 billion in financing, with a valuation of US$24 billion (approximately RMB 173.8 billion). This round of financing has a strong lineup of investors, including well-known Silicon Valley venture capital firms Sequoia Capital and Andreessen Horowitz, as well as an investment fund closely linked to the Saudi royal family.
OpenAI has received a total of US$14.3 billion in financing in the past ten years, and xAI raised US$6 billion in just 10 months, which is equivalent to half of OpenAI’s ten-year financing.
This is also one of the highest single financings in the history of the large model track, surpassing the 4 billion financing received by Anthropic on September 12, 2023, and the US$2 billion financing received on October 28, 2023, as well as the US$1.3 billion financing received by InflectionAI on June 30, 2023. The total financing is second only to OpenAI and Anthropic.
xAI’s ability to obtain the highest financing in the history of the AI track is naturally inseparable from Musk’s halo. In particular, other companies in Musk’s business empire have helped xAI a lot.
According to media reports, when xAI was first established, Tesla sent about 50 software engineers to assist with code review and other tasks. There are even some Tesla engineers who directly joined xAI. At the same time, the video data captured by the large number of cameras and other sensors equipped with Tesla vehicles can provide real-time data input for the xAI model to help it receive training.
X, the well-known social platform under Musk’s name, can also provide xAI with massive amounts of human-generated data. This data is an ideal resource for training AI systems on how to better interact with humans.
The combination of various advantages can create an AI unicorn with a valuation of up to US$24 billion in one year. It can compete with OpenAI in terms of capital, data, chips, etc.
On July 1, Musk’s social platform publicly stated that xAI’s next step is to clear large language models from Internet training data, and plans to launch the next-generation upgraded version of Grok-2 in August.
I believe that next month, we can see Grok-2 fighting GPT-4o.
The war for talent in the AI world
In addition to funds and chips, there is another important factor in the competition between large AI models, and that is talent.
Musk once lamented on social platforms: The war for AI talent is the craziest talent war he has ever seen.
As a "well-known Internet celebrity" around the world, Musk's every move can affect the direction of a track. This influence and popularity also provide convenience for Musk to recruit professional talents.
According to media reports, Musk has poached many scientific researchers from technology giants such as Google and Microsoft. Executives at other companies have complained that xAI offers extremely competitive pay packages that put them at a disadvantage in the hiring market.
At the same time, Musk is one of the main co-founders of OpenAI and served as a director on the board of directors from 2015 to 2018. He not only contributed tens of millions of dollars in funding and provided suggestions on research directions, but also played a key role in recruiting world-class talents.
Although he left OpenAI due to different development concepts, he is undoubtedly very familiar with OpenAI's leadership.
When establishing xAI, Musk poached data scientists Kyle Kosic and Igor Babuschkin from OpenAI with high salaries.