# Is software still eating the world or did AI change the menu?

On the front end, software was the main character for all those years. Shopping, working, communicating, moving through the world.

In other words, as Marc Andreessen wrote,

> **Software is eating the world.**

His theory predicted a new era of software companies taking over large swathes of the economy, in fact, it came true.

[Disney bought Pixar](https://mnacommunity.com/insights/disney-and-pixar-merger/).

[Netflix killed Blockbuster.](https://www.theatlantic.com/culture/2013/11/long-live-video-store/354947/)

In 2000, the cost of a customer running a basic Internet application was approximately $150,000 a month.

Then cloud software arrived.

By 2011, running that same application in Amazon’s cloud costs = $1,500 a month.

### *Now,*

> **AI is to Gen Z what Software was to Millennials.**

ChatGPT is a website. Claude is an application. Midjourney takes a sentence and returns an image. You type something into a box, the software replies, and everything happens on the screen.

The screen is the front end. The fun part is the back end.

When you ask ChatGPT a question,

%%[chatgptflow] 

AI needs to perform an enormous number of calculations, many of them at the same time.

***Ladies and gentlemen, please welcome the chips.***

For years, computers mainly depended on CPUs\*. Imagine the general worker of a computer. That’s what a CPU is.\* It is designed to handle many different types of jobs, moving from one instruction to another.

Imagine that you need to solve one difficult calculation. You might give it to one very intelligent mathematician. *But what if you need to solve one million smaller calculations at the same time?*

*CPU out. GPU in.*

GPUs were originally created to produce computer graphics. A video game contains thousands of pixels, shadows, shapes and movements that must be calculated at the same time. GPUs contain thousands of smaller processing cores that can work in parallel.

*It turned out that the same ability was extremely useful for AI.*

The maths behind an AI model can be divided into thousands of smaller operations and processed simultaneously. That is why GPUs can train and run large AI models much faster and more efficiently than relying only on ordinary CPUs. NVIDIA describes modern AI models as layers of linear-algebra equations, with thousands of GPU cores working like tiny calculators in parallel.

*A chip created for gaming accidentally became one of the most valuable tools in the AI race.*

Oops..

GPUs were not the final step.

Once companies understood how much computing power AI would require, some started designing chips specifically for it. Google discovered this problem more than a decade ago while considering speech recognition for hundreds of millions of users.

> Even if each person spoke to Google for only three minutes a day, the feature would have required the company to roughly double the number of computers in its data centres.

The software idea existed. The demand existed. The computers underneath it could not keep up.

Google built a different type of chip called a Tensor Processing Unit, or TPU.

Unlike a CPU, which is designed to perform many types of work, a TPU was built for one specific purpose: performing the large matrix calculations used in machine learning. Google deployed its first TPU internally in 2015. The company originally expected to build fewer than 10,000 of them. It eventually built more than 100,000 to support products including Search, Ads, speech recognition and AlphaGo.

### **The BIG TECH is no longer concerned only about software.**

***The matter has become Chips.***

*Google built TPUs.*

*Amazon built Trainium and Inferentia chips for training and running AI models.*

*Microsoft introduced its own AI accelerator.*

*Meta began developing chips for its recommendation and AI workloads.*

You might ask,

> Why would software companies suddenly become interested in something so physical?

Because software could no longer move faster than the hardware underneath it.

AI changed the bottleneck. Have the data, have the researchers, have the idea for a more capable model. But if you cannot access enough computing power, you cannot train it at the scale you want.

Don't get me wrong.

This does not mean that one AI model lives inside one magical chip. Large models are trained across thousands of chips connected together.

One chip completes part of the calculation. Another chip completes another part. They constantly exchange information as if they are pieces of one enormous machine.

And making thousands of chips work together is not simple. They need extremely fast connections, access to data, memory that can feed information to the processors quickly enough, software that knows how to distribute the work, cooling systems because all of this computation produces heat, a reliable supply of electricity. At this point,

### *We are talking about warehouses filled with racks of computers.*

Data centres.

In Non-Tech Girlie language, Google started by making 256 AI chips work together like one giant machine. Then it looked at that machine and thought:

*"why stop there?"*

Now Google wants up to a million chips to behave like one giant machine. An advanced AI chip also needs high-bandwidth memory, which allows it to access huge amounts of data quickly.

Needs, needs, needs...

This is why saying “there is a chip shortage” does not always mean that the industry has simply run out of small pieces of silicon.

Then, there is power.

Training a large model requires many GPUs to work together for long periods. Those GPUs are normally placed close to one another because they need to communicate at extremely high speeds.

That concentrates an enormous amount of electricity demand inside individual data centres. *The companies may be ready to buy the chips, but is the electricity grid?*

Because, without enough energy, nothing moves.

AI data-centre expansion has already placed pressure on available data-centre capacity, electricity grids and power generation. The problem is the combined demand of thousands of chips, networking equipment, memory and cooling systems operating together.

*And cooling is not optional.*

Every calculation uses energy, and part of that energy becomes heat.

The more chips companies pack into a rack, the harder it becomes to remove that heat. Traditional air cooling is no longer enough for some of the densest AI systems, which is why companies are increasingly using liquid cooling. So the intelligence that feels weightless on your screen depends on water systems, power lines, cables, factories and enormous physical buildings.

AI may be digital. But the race is deeply physical.

A powerful chip without the right software is difficult to use. A brilliant AI model without enough chips is difficult to train and expensive to run.

In a nutshell, software is still eating the world.

*AI just changed what it needs to eat.*