AI chips and memory explained: GPUs, TPUs and HBM

What an AI chip is, how GPUs differ from Google TPUs, and why HBM memory matters. Plain-English explainer with official sources, for November 2026.

An AI chip is a processor built to train or run neural networks, usually a GPU you can buy or rent, or a custom chip such as Google's TPU. The chips are paired with high-bandwidth memory (HBM): DRAM stacked beside the processor on a very wide bus. The chip does the math; HBM feeds it data fast enough to matter. An AI chip is a processor built to train or run neural networks, usually a GPU you can buy or rent, or a custom chip such as Google's TPU. The chips are paired with high-bandwidth memory (HBM): DRAM stacked beside the processor on a very wide bus. The chip does the math; HBM feeds it data fast enough to matter.

Training and running large models is limited as much by memory bandwidth as by raw compute, which is why Nvidia, Google TPUs and PC RAM prices keep showing up in the same stories. This hub is the evergreen page for those questions. New earnings, export-rule and product stories should link here instead of restating the basics.

What is an AI chip?

IBM describes an AI accelerator — also called an AI chip, deep-learning processor or NPU — as hardware built to speed up neural networks, deep learning and machine learning. In practice you will meet a few families:

GPUs (graphics processing units) started as graphics chips. They have thousands of smaller cores that are good at the same matrix math AI models need, which is why they became the default AI workhorse. You can buy one for a PC, rent one in many clouds, or find it inside a data-center server.
ASICs (application-specific integrated circuits) are chips designed for a narrower job. Google's TPU, or Tensor Processing Unit, is the best-known AI ASIC: it is built for tensor operations and, Google says, sits behind services such as Search, YouTube and DeepMind's large language models.
CPUs still run the rest of the server (networking, storage, orchestration). They are not what people usually mean by "AI chip," but every GPU or TPU server still needs them.

Google's own explainer is the cleanest official contrast: CPUs are general-purpose, GPUs are flexible parallel processors, and TPUs are Google's more specialized AI chips.

TPU vs GPU: which one runs your AI?

A GPU is a flexible chip for parallel math that you can buy, rent in many places, or put in a workstation. A TPU is Google's custom machine-learning chip, available in Google's data centers and on Google Cloud, not as a card you install at home. Google says TPUs suit large, matrix-heavy models with big batches, and GPUs suit models that need many custom operations. That is Google's guidance, not a rule that one is "faster" in every test. Our longer walkthrough is TPU vs GPU explained. Google has also been expanding the systems around those chips, including TPU 8tx and 8i Max.

Nvidia remains the name most buyers meet first because its data-center GPUs, software stack and rack systems are what many clouds and AI labs rent. AMD sells competing data-center CPUs and accelerators; other custom chips exist at large cloud providers. This page does not assign market-share percentages we cannot source from a primary document.

What is HBM, and why does it matter?

HBM, or high-bandwidth memory, is DRAM stacked vertically and placed next to the AI chip on a very wide bus. That design trades cost and manufacturing difficulty for a lot more bandwidth than standard DDR or the GDDR memory used on many gaming GPUs. Bandwidth limits how fast a model can read weights and generate tokens; more HBM also leaves room for longer contexts.

JEDEC, the industry standards body, published the HBM4 standard (JESD270-4) on April 16, 2025. Our HBM explainer notes that the standard allows up to 2 TB/s of bandwidth and 64 GB per stack. SK hynix, Samsung and Micron are the suppliers to watch; we track that contest separately in the HBM4 race. For how HBM compares with GDDR7 and GDDR6, see GDDR7 vs GDDR6 vs HBM.

The same tightness in advanced memory is one reason ordinary PC RAM has been expensive. That is a related market, not the same product; start with why RAM is so expensive in 2026 rather than treating DDR5 and HBM as interchangeable.

How is an AI chip made, and who fabricates it?

Design, fabrication and packaging are separate steps. Many AI accelerators are designed by Nvidia, AMD, Google or others, then manufactured at a foundry. TSMC is the foundry most often named for leading-edge AI silicon. In its October 2026 revenue report, TSMC said October revenue was about NT$586.89 billion, up 59.7% from October 2025 and 14.7% from September. That release is a monthly net-revenue print. It does not break the figure out by node or by AI-chip orders, so it is a demand signal, not a direct AI-chip sales number. For what a "2nm" process name actually means, see TSMC N2 and nanosheet transistors.

Packaging matters as much as the process node because HBM has to be assembled beside the compute die. When people talk about a "GPU shortage," they are often talking about packaging and HBM supply as much as about wafer capacity.

Training, inference, and the systems around the chip

Training is how a model learns from data. Inference is how it answers a new prompt. Both can run on the same class of chip, but the system design differs: training jobs are huge, tightly networked runs; inference is the always-on service that has to be cheap enough to offer at scale. That is why you now see deals that split the work — for example Nvidia, Rebellions and SK Telecom describing a plan to use Nvidia GPUs for prefill and Rebellions NPUs for decode, with R200 systems targeted for 2028. A target year is a plan, not a shipment.

The chip also never works alone: it sits in a server, a rack and a data center that needs power, cooling and optical links. Capex and neocloud explainers cover the money side; this hub stays on the hardware.

Sources

FAQ

What is an AI chip in one sentence?

It is a processor, usually a GPU or a custom ASIC such as a TPU, designed to train or run neural networks much faster than a general-purpose CPU.

What is the difference between a GPU and a TPU?

A GPU is a flexible parallel processor you can buy or rent in many places. A TPU is Google's custom AI chip, used in Google's data centers and on Google Cloud. Google says TPUs fit large matrix-heavy workloads, and GPUs fit models with many custom operations.

What is HBM?

High-bandwidth memory is stacked DRAM placed next to an AI chip on a wide bus so the processor can move data faster than it could with ordinary DDR or most GDDR memory. JEDEC's HBM4 standard allows up to 2 TB/s and 64 GB per stack.

Why are AI chips, and sometimes PC RAM, so expensive?

AI accelerators depend on leading-edge manufacturing and on scarce HBM. Demand for that memory has also tightened the wider DRAM market, which is one reason PC RAM prices have risen. Those are related supply chains, not the same product.

Can I use a gaming GPU for AI?

Yes, for many local or small-scale jobs. Data-center GPUs add more HBM, different networking and different software support. How much memory you need depends on the model and the precision you run; see how much VRAM you need for a local LLM.

Does TSMC's October revenue jump mean AI chip demand is still rising?

TSMC reported October 2026 revenue of about NT$586.89 billion, up 59.7% from a year earlier. That is a company-wide monthly revenue figure. The release does not split out AI-chip orders, so it is not, by itself, an AI-chip sales report.

Who makes HBM?

The suppliers we cover in the HBM4 race are SK hynix, Samsung and Micron. They sell memory that is then packaged with accelerators designed by companies such as Nvidia.