NVIDIA & the Disruption of Intel

Behind Nvidia's journey from a graphics computing company to being the market leader in AI computing and at the cusp of upending Intel.

Credits: TheNounProject

(I was on vacation for last two weeks which is why there was no blog post for a month. I will proactively let you know next time there is a break.)

Nvidia in a way is a classic example of Clayton Christenson's theory of disruption. The theory essentially tries to explain how big companies who are at their peak get disrupted by startups targeting a particular niche. In this case the latter is Nvidia and the former is Intel. There are two high level points on which the theory is based on.

• The business model and the revenue stream of incumbents actually act as a deterrent for them to try out new technologies and opportunities for innovation. If something doesn't match up to their revenue expectations, new or smaller market ideas are never able to reach critical adoption.

• The startups or smaller companies are at an advantage for the very simple reason that they are small and more open to take risks. As such, they can target opportunities which are overlooked by bigger companies and work on smaller revenues. This is until the market grows to become popular and more receptive to the idea at which point it gets market traction and poses direct threat to the incumbent.

Nvidia is not a small company but it started out relatively small and at the time when its first graphic card was released, Intel, then an incumbent was already cashing in $30 billion dollar in annual revenue. Over time Nvidia’s annual revenue has increased to almost $10 billion dollars, half of which is from graphics cards and half from products related to artificial intelligence and accelerated computing. Intel so far hasn’t technically faced disruption from Nvidia except for it taking the majority of share in the GPU market. The former dominates the market for PC and Data centers which is ripe for disruption and Nvidia is probably a top contender. Its worthwhile to understand how it has come so far, how does it want the future to be and make sense of how this disruption is happening.

The Past - Nvidia and Graphics computing

One of the first use cases of graphics card or GPU, as the name suggests, helps accelerate graphics rendering, both for video and image and is a performance intensive process. Without going too much into details as to how exactly its achieved, a graphics card is specifically built to handle the mathematical operations required to process binary data as an image. When Nvidia started in 1993, the most evident market for a graphics card was the gaming market. As explained in this article -

The wave Nvidia's cofounders saw coming was the nascent market for so-called graphics processor units, or GPUs. These chips, typically sold as cards that video gamers plug into a PC's motherboard, provide ultrafast 3-D graphics. Marketed under testosterone-drenched labels like "Titan X" or "GeForce GTX 1080," these cards can cost up to $1,200, and two decades later they still produce more than half of Nvidia's $5 billion in revenues.

Nvidia took advantage of being small and targeted this industry which was completely overlooked by giants like Intel. Over time, the performance and quality of its GPU products improved and they also released a programming kit called CUDA in 2006 to help developers easily utilize the GPU in an easy and efficient way. Intel released GPUs as well, but they were packaged with its CPU processors rather than standalone or discrete GPUs. (Integrated GPUs lack in performance than discrete GPU primarily because the latter has its own RAM which leads to increased performance.) Integrated graphics card did lead to increased performance and was probably a good fit for non professional gamers but for 3D gaming and for people who did not want to compromise on the experience, discrete graphic cards from Nvidia was the primary choice. In time, this also gave Nvidia competitive advantage in a completely new domain, machine learning or AI.

The present - Nvidia and Parallel Processing

The fact that gaming provided an opportunity to Nvidia was the much needed element to breakthrough in the processor market where Intel was much farther ahead in terms of revenue and big buck customers. Deep learning (a technique to implement machine learning) provided the opportunity for Nvidia to go upmarket and establish itself as a big player. To understand why this opportunity opened up for Nvidia, we have to understand the essential benefit provided by a GPU. A GPU is meant to perform computationally intensive things for which gaming was the first big use case. But in 2009, a paper released by professors at Stanford postulated the use of GPUs for deep learning.

In this paper, we exploit the power of modern graphics processors (GPUs) to tractably learn large DBN and sparse coding models. The typical graphics card shipped with current desktops contains over a hundred processing cores, and has a peak memory bandwidth several times higher than modern CPUs. The hardware can work concurrently with thousands of threads, and is able to schedule these threads on the available cores with very little overhead. Such finegrained parallelism makes GPUs increasingly attractive for general-purpose computation that is hard to parallelize on other distributed architectures.

Now machine learning had been a work in progress at the time this paper was released with moderate success. Deep learning was able to produce much better results specifically because they cover more data parameters and more training data in itself. As a result, the data computation and processing required for the same was impractical using general purpose CPUs. As a result of this paper and many other successful experiments, the adoption of discrete GPUs spread like wildfire. Besides deep learning, they became popular for other computationally intensive things as well, for e.g cryptocurrency mining, climate modelling, oil & gas recovery etc. It was further helped by the investment Nvidia had made with CUDA which is the primary way developers could harness the power of GPU to do carry out computationally intensive tasks. With GPUs rising in popularity, CUDA became popular as well and even though alternatives like OpenCL are available, using CUDA with Nvidia’s GPU provides much better performance. Any investment that Nvidia makes into CUDA naturally makes its GPUs more popular plus easy to use and vice versa. Nvidia also improvised this combination around use cases for deep learning. As mentioned in this article-

The company quickly adapted its GPUs for these new work loads, adding new math functions and even dedicated processing elements called Tensor Cores. NVIDIA also developed a series of software libraries under the name cuDNN optimized for CUDA and deep neural networks.

The stage was set for Nvidia to take its GPU computing business to the next level and the accelerated computing part of its business now comprises half of its revenue. It also has the potential to be a game changer in how world looks at computing infrastructure in future.

The Future: Nvidia & AI computing

If you’ve been following the tech space, you must have realized that the application space of Artificial Intelligence or AI as they call it is enormous. From health, automotive, personal computing, there is virtually no space left untouched where AI cannot spread its roots. And Nvidia is probably the company that is primed to take advantage of it the most. It is also probably how the disruption of Intel takes place and the second point in Clayton Christensen’s theory of disruption becomes a reality. I would say that some of it is happening already.

• Nvidia’s data center business is booming and has surpassed its gaming revenue for the first time. Many cloud computing companies are using GPUs to accelerate their processing capabilities as well as machine learning models. Many of them are using Nvidia’s products. As explained in this article -

  Nvidia's new A100 GPU is already shipping to customers around the globe. It will be used by the biggest names in cloud computing, including Alibaba, Amazon, Baidu, Google and Microsoft. The companies operate huge server farms that house the world's data. Netflix, Reddit and most other online services rely on the cloud operators to keep their sites up and running. Nvidia said that Microsoft, with its Azure cloud, will be one of the first companies to use the A100.

• Nvidia is also making a significant investment into edge computing which is about delivering high performance computing closer to the source of data for a particular company. It has released chips specifically to target these use cases; for e.g in case of Walmart.

  Using the NVIDIA EGX platform, Walmart is able to compute in real time more than 1.6 terabytes of data generated a second. It can use AI for a wide variety of tasks, such as automatically alerting associates to restock shelves, retrieve shopping carts or open up new checkout lanes.

• Self or autonomous driving basically turns the car into a mini data processing system with real time driving decision based on hundreds of data inputs. Nvidia has taken a lead since 2015 and has been able to release SoCs and the developer kit; together called as DRIVE for self driving cars. Although they started with mostly driver assist technologies, it has now grown to support higher levels of autonomous driving capabilities. In May of this year, they launched DRIVE AGX Pegasus platform which delivers unparalleled performance and the ability to implement Level 5 self driving capability. As described in this article -

  Finally, there's the next-generation Drive AGX Pegasus platform for Level 5 autonomous robotaxis that combines two Orin SoCs with two Ampere GPUs providing a total of 2,000 TOPS compute at 800W (compared to 320 TOPS for the current Turing-based DRIVE AGX Pegasus). Vehicles powered by this advanced Pegasus platform would need absolutely no human intervention and could help revolutionize the transportation industry.

All this goes to show how Nvidia is clearly taking a lead over Intel in the space of AI computing with its chips and the associated developer platform. Where Nvidia has not been able to gain a significant advantage is in the traditional CPU business where Intel (data center and laptops) and other companies like Qualcomm (mobile) have most of the market-share. And this is where the proposed acquisition of ARM by Nvidia comes into play. To be clear, ARM in itself doesn’t develop any of the processors, they license the instruction set which other companies use to design and then eventually manufacture the chip-sets. This poses a direct competition to Intel which uses its own ISA (x86) to design and manufacture and does not allow others to license. Apple, Qualcomm, Samsung have all been licensing ARM’s ISA to design and develop chips in case of mobile and laptops. This means that there is no clear or direct advantage to Nvidia in acquiring ARM. But ARM could play an important role in future in the space of data center business specifically. ARM designed processors for data centers are generally less power hungry and could be designed as per the needs of the company through licensing. Intel on the other hand has a closed model and the server technology it provides comes as an integrated product and many companies are realizing that its probably not the most efficient way to run your data center. As pointed out in this article

Cloud and service providers are designing their data centers for the best efficiency. They are big hardware spenders and every improvement they are able to make can easily result in big savings. Hyperscalers such as Microsoft and Amazon are already working hard in this direction and many others, even smaller ones, are doing the same.

ARM is generally what is being used by these companies to build custom processors and redesign their data centers for efficiency and cost savings. Jensen Huang understands this and the significance of ARM. As explained by himself in this investor call -

the number of companies out there that are considering building ARM CPUs out of their ARM CPU cores is really exciting. The investments that Simon and the team have made in the last four years, while they were out of the public market, has proven to be incredibly valuable, and now we want to lean hard into that, and make ARM a first-class data center platform, from the chips to the GPUs to the DPUs to the software stack, system stack, to all the application stack on top, we want to make it a full out first-class data center platform.

And in this interview -

My attitude is not to think about the server as a computer, but to think of a CPU server, a GPU server, a storage server, and a programmable switch as computing elements that are inside the computer, and the computer unit is now the data-center. And in that world, networking is all important. And in that world, knowing how to build a computer end-to-end and recognizing the importance of the software stack, which is so complicated from top to bottom, is where we are focused. And I think that in new world where the data-center is the computer is really quite exciting and I think we have we have the essential pieces.

Nvidia is building for a data center platform and it wants to utilize the strength of ARM as well as its partner ecosystem to promote the same. If Nvidia is able to do this, the meaning of data center in itself will change, enabling medium sized and smaller companies to redesign their data centers replacing Intel. This enables NVIDIA to move upmarket in its most traditional sense and dislodge an incumbent proving to be yet another quintessential example of Christensen’s theory. But, data center is a big investment for any company and this makes it slow in terms of new changes. This can give Intel and other competitors time to play catch up. Only time will tell though. For now, Nvidia’s position as a market leader in AI computing seems entrenched and gaining strength with every step.

---

If you enjoyed this article, please consider subscribing to the newsletter as well. You can also read about me and the blog here or follow me on twitter.