When it comes to who was the biggest winner among the technology giants last month, no one should have any objections to the World Super League’s nomination of Google, right? Relying on the explosive performance of Gemini 3, in just half a month, the stock price has skyrocketed, and it is still punching OpenAI in the arena and kicking Nvidia outside the arena.

Pull back a little, it won’t be a big problem


As for why Lao Huang, who sells shovels, followed suit, the reason is very simple. Google said that Gemini 3 Pro is trained on a self-developed TPU (Tensor Processing Unit). At least literally, there is no mention of Nvidia.

Immediately afterwards, the media and the melon-eating public began to speculate that Google might really end the CUDA moat this time.


So the question is, what is the TPU that seems to have brought an end to Nvidia’s good days?

It can also be seen from the name that it is actually a type of chip, which is a close relative of the GPU, but it is made into an AI special edition.

Although TPU has only recently attracted everyone's attention, it is an old project that has continued from 2015 to the present.

The first generation TPU looks like this


At that time, Google was going through the pain of technological transformation and wanted to replace all traditional search and prediction algorithms with deep learning. As a result, they found that the GPU was not only insufficient, but also consumed a lot of power and could not be used at all.

The problem with the GPU is that it wants to be all-powerful. In order to be able to do everything, we had to build a complex architecture of hard drives, memory, video memory, and cores.

This brings about a big trouble. In the world of chips, the cost of moving data is much higher than the calculation itself. The physical distance between data running from the video memory to the core may be only a few centimeters, but the electrons have to cross mountains and ridges.

How GPU works


As a result, most of the electricity bill was not spent on calculations, but all on the travel expenses for express delivery. Eventually it turns into heat, and you have to ask a fan to blow it away.

This is not a problem when doing graphics rendering, because the picture is highly random, and you can't predict what materials you will need, and you can only retrieve them from the echo memory.

But in AI matrix operations, how each number is calculated, who is calculated, and how many times it is calculated are all fixed.I know that this number has been calculated and I will continue to use it later. The GPU has to save it back and wait for others to retrieve it into the computing unit. Isn't this a waste?


Therefore, as an AI-specific tool maker, TPU was born. It dismantles and suppresses the graphics, control flow, scheduling modules, etc. that are not needed on the GPU.

The core idea is to specifically optimize matrix multiplication, which is most commonly used in AI, and develop a method called "systolic array".

Using this trick, once each data is calculated, it will be transferred between densely arranged computing units, and it is not allowed to return to the storage unit until it is used up. In this way, there is no need to read and write frequently.


In this way, the number of computing operations per cycle of TPU reaches hundreds of thousands, which is nearly ten times that of GPU. The energy efficiency ratio of the first generation TPU v1 is 30 times that of NVIDIA Tesla K80 at the same time, making it extremely cost-effective.

Of course, at the beginning, Google was just experimenting on the edge and not so big. TPU is also only engaged in inference, not training, and has a single function, which is completely incomparable with GPU.

Starting from the second generation, Google began to pile up materials on the memory to increase the capacity and data transmission speed, allowing the TPU to quickly record and modify massive intermediate data (such as gradients and weights) while calculating. Since then, it has lit up the training skill tree.

As the scale of TPUv3 increases, model training speed increases


But for so many years, it has become clear that the cost of using TPU to train inference is lower, and the performance is comparable to that of GPU. Why do the giants still have to grab Nvidia's chips?

In fact, it’s not that everyone is not greedy, but that Google is bad and has been exerting control. All TPUs are only rented and not sold, and are bound to Google Cloud. Large companies cannot move TPU home, which is equivalent to handing over their wealth and life to Google Cloud, which makes them feel uncomfortable.I'm afraid that Nvidia's neck is not stuck, and Google's fast forwarding is so fast that it will be strangled.

Even so, Apple did not resist the temptation of a cheap and big bowl, and more or less rented it.

The popularity this time is so high. On the one hand, Gemini 3 has proved the success of TPU and its quality is assured. On the other hand, it is because of the seventh-generation TPU Ironwood that Google is finally willing to sell it.


According to The Information, Meta is already negotiating a multi-billion dollar contract with Google and is preparing to deploy TPUs in computer rooms starting in 2027. It also plans to rent TPUs from Google as early as next year.

As soon as the news came out, Google's stock price immediately rose 2.1%, while Nvidia's stock price fell 1.8%.

Some Google insiders even said that our grand opening may take away billions of dollars from Nvidia and directly cut off 10% of their annual revenue.


Wall Street also loves TPU very much and feels that this good thing has a bright future for money. Even Broadcom, which is responsible for design and manufacturing, took advantage and had its performance expectations raised.

However, it is not true that TPU will replace GPU.

TPU is an ASIC (Application-Specific Integrated Circuit), also known as an application-specific integrated circuit. In human terms, TPU is not very good at anything except matrix calculations in AI.

This is its advantage and its pain point.

How TPU works


Catching up with the good times when large models are in power, the demand for matrix calculations is ridiculously large, and TPU follows suit. But if there is any more popular AI technology route in the future, if we don’t adopt the current approach, TPU will be unemployed in minutes.

And because TPU is too specialized, once it has no performance advantage in computing, it completely loses value.It is difficult for us to see TPU v4 four years ago.

In contrast, GPUs are different. Take the 3090, which was born five years ago before the wave of large models, as an example. It just relies on its 24G ultra-large video memory and the backward compatibility ecosystem of CUDA that never abandons it. Until now, it is still a value-for-money main card for ordinary people to play AI. It is not a problem to run a Llama 8B small model.

To take a step back, even if the AI ​​rice bowl is no longer good, at worst, it can go back to serving game players and designers, and it will still be a prosperous life.


In addition, the CUDA ecosystem is still Nvidia’s biggest killer move.

This is just like you are used to iOS. Although Android is also very good, if you are asked to migrate all the photos you have saved for ten years, the operating gestures you are used to, and a bunch of apps you bought, you will most likely choose to use it next time.

The same is true for today's AI developers. Their codes are written based on CUDA, and the libraries they call are optimized by NVIDIA. Even when trying to correct errors, they only search for CUDA.


Want to switch to TPU? OK, refactor the code first and then adapt to the new development environment.

Even if PyTorch is integrated, many underlying optimizations and custom operators still have to be re-debugged when switched to TPU. The specially designated JAX language also raises the bar for talent recruitment.

For most small and medium-sized factories that just want to get their models running quickly, instead of struggling to adapt the TPU, or even not being able to get it at all, buying NVIDIA chips directly is the easiest option.


Among other things, Google itself is still purchasing a large amount of Nvidia GPUs. Even if it doesn't use them, so many customers of Google Cloud still have to use them.

Therefore, the launch of TPU sales has indeed taught Nvidia a lesson in cost-effectiveness in the area of ​​large model training. But it is definitely not as good as it is to grab the GPU job.

In the future computing power market, it is more likely that TPU will occupy the specialized needs of leading manufacturers, while GPU will continue to dominate the general market.

But as long as the giants compete, it is possible to drive down the price of computing power. No matter how you look at it, this is a good thing.