NVIDIA is an early leader in cloud computing, primarily because of its proprietary Compute Unified Architecture ("CUDA") platform, which enables general-purpose processing on its GPUs. The company has successfully carved a niche in the rapidly expanding cloud computing artificial intelligence market. Alphabet Inc, Google and Amazon.com Inc have deployed their own proprietary AI chips in some cases but have also been turning to Nvidia. But now, Nvidia is facing competition from Intel and AMD, both of which are also actively exploring this market segment.
In addition, there are many artificial intelligence chip startups that have received large amounts of venture capital and have chosen not to compete with Nvidia, but to find other ways to take some differentiated routes in an attempt to get a piece of the market. In this regard, analyst Robert Castellano made an analysis.
Nvidia’s dominance
Nvidia raised its annual performance forecast and expects third-quarter revenue to reach about $16 billion. This figure significantly exceeded the consensus forecast of $12.61 billion and represented a 170% increase compared to the same period last year.
The chart below shows Nvidia's data center revenue growth, illustrating the demand for its chips and the knock-on effect on Arm. Analysts expect revenue to increase by 21% quarter-on-quarter in the third quarter of fiscal 2024 (third quarter of fiscal 2023). The financial report will be announced on November 21, 2023.
For this quarter, Nvidia expects earnings per share of $3.34, a significant increase of +475.9% compared to the same period last year.
The consensus earnings estimate for the current fiscal year is $10.74, which represents a substantial increase of 221.6% compared to the previous fiscal year. This data shows that people have strong growth expectations for Nvidia's financial performance this fiscal year.
Cloud Super Extender
In recent years, the cloud computing market has made some significant progress in adopting Arm-based processors. Arm claimed that its share of the cloud computing market has increased to 10.1% from 7.2% as of December 31, 2020, mainly due to Amazon's increasing use of its in-house Arm chips. Amazon Web Services (AWS) has deployed its custom Graviton chips in 15% of server instances in 2021, marking a major shift by the cloud computing giant towards Arm architecture.
Google reported cloud computing revenue rose 22% to $8.41 billion, missing forecasts of $8.64 billion. In June, Google's cloud computing business grew 28%.
In 2023, Google released its latest independently developed chip TPUV4, whose performance is significantly improved by 2.1 times compared to the previous generation chip. By integrating 4,096 such chips, supercomputing performance is increased by an impressive 10 times.
Google says that in systems of comparable size, the performance of TPUV4 isNVIDIA1.7 times that of A100,Energy efficiency is also improved by 1.9 times. Similar to its predecessor TPUV3, each TPUV4 consists of two tensor core (TC) units. Each TC unit consists of four 128x128 matrix multiplication units (MXUs), a vector processing unit ("VPU") with 128 channels (each containing 16 ALUs), and 16MiB of vector memory ("VMEM").
In addition to the next-generation TPU, Google will also begin making Nvidia's H100 GPU fully available to developers at the end of 2023 as part of its A3 series of virtual machines.
AmazonAWS
Amazon Web Services' performance has been declining for the past six quarters, but its performance in the third quarter stabilized, maintaining 12% year-over-year growth. The segment's operating income also surged 29% year-on-year to approximately $7 billion.
In May this year, AWS launched EC2P5 virtual machine instances based on NVIDIA H100 GPU. The configuration includes eight Nvidia H100 TensorCore GPUs, each equipped with 640GB of high-bandwidth GPU memory. It also has a third-generation AMD EPYC processor, 2TB of system memory, 30TB of local NVMe storage, an impressive 3200Gbps of total network bandwidth, and support for GPU Direct RDMA. The latter enables direct node-to-node communication without using the CPU, thereby reducing latency and improving scale-out performance.
In addition, Amazon EC2P5 instances can be deployed in second generation ultra-scale clusters (called Amazon EC2 UltraClusters). These clusters include high-performance computing, network resources, and cloud storage. These clusters can accommodate up to 20,000 H100 TensorCore GPUs, enabling users to deploy machine learning models with billions or trillions of parameters.
Microsoft Corporation
Microsoft's cloud computing revenue grew 24%, reaching $31.8 billion in September. Among Microsoft's three major business units, Intelligent Cloud performed the most prominently, with revenue increasing by 19% to $24.3 billion. The segment includes server products and cloud services, with Azure achieving strong growth of 29%, beating Wall Street expectations of 26%.
In March of this year, Microsoft announced in a blog post that it planned a major upgrade to Azure. The upgrade will use tens of thousands of NVIDIA's cutting-edge H100 graphics cards, as well as faster InfiniBand network interconnect technology.
The NDH100v5 instance also uses Intel's latest fourth-generation Intel Xeon Scalable CPU and achieves low-latency networking through Nvidia's Quantum-2CX7 InfiniBand technology. They also feature PCIe Gen5, providing 64 gigabytes per second of bandwidth per GPU, and DDR5 memory for faster data transfer speeds to handle the largest AI training data sets.
successful start-up
Applications such as ChatGPT further solidify Nvidia’s position in the artificial intelligence industry. Its GPU chips have become key to various artificial intelligence applications. As a result, any startup hoping to challenge Nvidia in this space is under intense pressure, as Nvidia has established the dominance and reliability of its technology.
Cerebras
Nvidia's A100 GPU is already quite impressive, with a chip area of 826 square millimeters. By comparison, Cerebras' new WSE-2 chip is huge, measuring 45,225 square millimeters, essentially covering the entire surface of an 8-inch silicon wafer. Since its founding in 2016, Cerebras has successfully raised $730 million in financing. The company is currently valued at $4 billion, according to CBInsights Global Unicorn Club.
Cerebras has partnered with Abu Dhabi’s G42 to build the first of nine artificial intelligence supercomputers in a project that cost more than $100 million. In addition, Cerebras is actively looking for opportunities in the field of generative artificial intelligence. While its CS-2 model is impressively fast at training in a GPT environment, it has yet to gain adoption by major manufacturers in the industry.
SambaNova
Founded in 2017, SambaNova is one of the most well-funded companies in the artificial intelligence chip industry. It has successfully raised $1 billion, with notable backers including SoftBank and Intel. This not only makes SambaNova the most funded AI chip startup, but also makes it one of Nvidia's strongest emerging competitors, with a valuation of up to $5 billion.
SambaNova recently launched its latest fourth-generation SN40L processor. This cutting-edge chip is manufactured using TSMC's advanced 5nm process, has more than 102 billion transistors, and has a computing speed of up to 638 teraflops. It features a unique three-tier memory system that includes on-chip memory, high-bandwidth memory, and high-capacity memory, all designed to efficiently handle the massive data streams associated with AI workloads. SambaNova claims that just eight such chips in a node can support models with up to 50 trillion parameters, nearly three times more than OpenAI's GPT-4LLM report.
Tenstorrent
Tenstorrent is another famous startup in the artificial intelligence chip industry, founded in 2016. The company has raised nearly $335 million in funding to date, with recent investments from major companies including Samsung and Hyundai Motor, and is currently valued at about $1 billion.
Tenstorrent is aiming to challenge Nvidia's dominance in the field of artificial intelligence and develop artificial intelligence CPUs using RISC-V and Chiplet technology. It is worth noting that the company recently reached a production cooperation with Samsung and intends to use Samsung's advanced 4nm process to manufacture chips. This collaboration demonstrates Tenstorrent’s commitment to advancing its technology and competing in the artificial intelligence chip market.
Less successful start-ups
Graphcore
Graphcore has achieved remarkable results in the field of European semiconductor startups, especially in raising funds. The company was founded in 2016 by Nigel Toon and Simon Knowles and focuses on developing intelligent processing units (IPUs), which are different from GPUs (graphics processing units) commonly used in artificial intelligence applications. Graphcore claims that its IPU technology has clear advantages over GPUs in meeting the special requirements of artificial intelligence.
PitchBook data shows that Graphcore has successfully received more than $600 million in investment. However, despite receiving significant funding, the company's revenue remains relatively limited. That situation took a major turn in 2020, when Microsoft decided to stop using Graphcore's chips in its cloud computing centers, causing the company to lose a major customer and creating more serious challenges.
According to the Financial Times, Graphcore’s revenue will plummet 46% to only $2.7 million by 2022. Meanwhile, its pre-tax losses increased 11% to $204.6 million, leaving it with a year-end cash balance of $157 million. Graphcore said it would need additional financing to break even by May of the following year. The company attributed the setback to "unfavorable macroeconomic conditions" and delays in hardware procurement from "key strategic customers," particularly in China.
Currently, Graphcore is adjusting its business strategy to transition its IPU chips from data centers to cloud computing environments. This shift is a strategic adjustment made by the company to adapt to the changing market dynamics and challenges of the semiconductor industry.
GSI Technologies(GSIT)
GSI Technology is a developer of Gemini Associative Processing Units ("APUs"), providing artificial intelligence and high-performance parallel computing solutions for the networking, telecommunications and military markets.
As shown in Table 1, Gemini-I outperforms other types of processors. The Gemini-I chip can perform 2 millionx1-bit operations per 600MHz clock cycle and has a memory bandwidth of 26TB/sec, while the Intel Xeon 8280 can perform 28x2x512-bit operations at 2.7GHz and has a memory bandwidth of 1TB/sec.
In the second quarter of fiscal 2024, the company had a net loss of $4.1 million and net income of $5.7 million, equivalent to $0.16 per diluted share. This performance is in sharp contrast to the net loss of $3.2 million, diluted loss per share of $0.13, and net income of $9 million in the second quarter of fiscal 2023. Additionally, in the first quarter of fiscal 2024, the company's net loss was $5.1 million, or $0.21 per diluted share, on net income of $5.6 million.
Gross profit margin in the second quarter of fiscal 2024 was 54.7%, down from 62.6% in the same period last year and slightly lower than the 54.9% in the first quarter. This data shows a company's financial performance and profit trends over a specific period of time.
Mythic
Mythic is a well-known company specializing in artificial intelligence chip simulation, focusing on computer memory (CIM) technology. However, according to a report by technology website TheRegister, the artificial intelligence chip startup faces major challenges in financing. Although the company initially raised about $160 million in funding, it has experienced financial difficulties over the past year and was on the verge of ceasing operations.
Fortunately, in March 2023, Mythic successfully received an investment of US$13 million, allowing it to continue operations. Dave Rick, CEO of Mythic, said that Nvidia indirectly caused the broader AI chip financing dilemma. That's because investors tend to gravitate toward opportunities with the potential for huge returns, creating a challenging environment for AI chip startups like Mythic to obtain the capital they need to operate.
Rivos
Server chip maker Rivos finds itself embroiled in a legal dispute with Apple, which is accused of illegally recruiting Rivos engineers and misappropriating trade secrets. In August 2023, Rivos unfortunately took the following measures to lay off about 20 people, accounting for about 6% of the company's total employees. Along the way, management revealed to the remaining employees that the company's prospects for new funding were increasingly dim.
NVIDIA Investment
Venture capital investment in chip startups is experiencing an unprecedented decline, largely due to Nvidia's dominance of the artificial intelligence chip market. Data from the United States shows that the transaction volume of chip start-ups fell by 80% compared with the previous year, a staggering decline.
As of the end of August, U.S. chip startups had raised $881.4 million, according to PitchBook data. This compares to $1.79 billion raised in the first three quarters of 2022. As of the end of August, the number of transactions dropped from 23 to 4.
However, Nvidia has been at the center of its own investment universe. Figure 3 shows Pitchbook data as of October 23, 2023, since the launch of ChatGPT, which contains the most important strategic investments. Nvidia is an investor in all but two companies.
Nvidia has been the most active investor, not only among the top investments by size, but also in terms of overall volume.Nearly half of all investments made between November 2022 and October 2023 haveNVIDIAfigure.
NVIDIA's investment strategy seems to be focused mainly on growth-stage companies, with more than 75% of its investments going into this area. Notably, they participated in 8 of the 10 largest funding rounds during this period. Infrastructure/LLM (Likewise Learning Models) is their main area of investment, accounting for nearly half of their total investment. Healthcare/therapeutics is the next most important investment area for Nvidia.
Investor Inspiration
For now, it seems unlikely that any emerging company will become the third largest player in the GPU market alongside industry giants Nvidia and AMD. Even chip giant Intel Corp. has encountered challenges trying to develop high-end GPUs popular with gamers. Intel's next discrete GPU is scheduled to be released in 2025. This situation highlights the strong position of Nvidia and AMD in the GPU market, with competition expected to be very limited in the near future.
It turns out that GPUs are ideal hardware for handling the massive computations required for large language models (LLMs) like GPT-3, which can involve training on a large number of parameters, such as GPT-3’s 175 billion parameters. NVIDIA has strategically strengthened its position in this space by developing and expanding the Cuda software platform. Cuda provides a range of proprietary libraries, compilers, frameworks and development tools to provide artificial intelligence professionals with the tools they need to build models. The most important thing is that Cuda is an exclusive product of NVIDIA GPU. This integration of software and hardware greatly reduces customers' switching costs in the field of artificial intelligence and enhances NVIDIA's competitive advantage.
Even if a chip competitor produces a product that is comparable to Nvidia's GPUs, analysts have reason to believe that code and models already built on CUDA may not be easily transferred to different GPUs. This gives Nvidia an inherent advantage. While alternatives that don't rely on Cuda or Nvidia GPUs are possible, Nvidia faces minimal competition in this space as of 2023. Therefore, anyone engaged in LLM development while waiting for alternatives risks falling behind as Nvidia continues to dominate the field.
When assessing the competitive landscape of the chip industry, it is clear that Nvidia faces multiple competitors and potential threats:
AMD:AMD is a well-funded chipmaker with significant GPU expertise. However, its relative weakness in software may prevent it from competing effectively with Nvidia.
Intel: While Intel hasn’t had much success with AI accelerators or GPUs, its capabilities shouldn’t be underestimated. As a major player in the semiconductor industry, Intel has the resources and capabilities to make significant progress in this area.
On-premises solutions for hyperscalers: Companies like Google, Amazon, Microsoft, and MetaPlatform are all developing their own in-house chips like TPU, Trainium, and Inferentia. While these chips may perform well in specific workloads, they may not be able to outperform Nvidia's GPUs in a wide range of applications.
Cloud Computing Companies: Cloud computing providers need to offer a variety of GPUs and accelerators to meet the needs of enterprise customers running artificial intelligence workloads. While Amazon and Google may use in-house chips for their own AI models, convincing a broad range of enterprise customers to optimize their AI models for these proprietary semiconductors could lead to vendor lock-in, something enterprises typically avoid.
Despite these competitive forces, enterprise customers are expected to continue to demand neutral merchant GPU vendors. Nvidia is likely to maintain its market leadership for the foreseeable future, mainly due to its strong software and hardware integration capabilities, widespread adoption of the CUDA software platform, and the substantial customer switching costs associated with its technology. These factors together constitute Nvidia's competitive advantage and help consolidate its position in the artificial intelligence chip market.