Hardware Foundations of AI

AI runs on physical substrates that shape what models can learn and deploy. This sub-article expands Section 1 of Hardware & Software.

General-Purpose Computing (CPUs)

• CPUs execute general instructions and are well suited to symbolic reasoning, search, and expert systems.
• Early AI ran entirely on CPUs.

The Parallel Shift

• Neural networks rely on dense matrix multiplication, which CPUs handle poorly.
• This drove the adoption of GPUs and specialized accelerators.

Memory and Storage

• Training needs enormous memory bandwidth and capacity.
• Models are sharded across accelerators; high-bandwidth interconnects (NVLink, InfiniBand) matter.

Energy and Infrastructure

• Frontier training consumes megawatt-scale power and generates heat.
• Liquid cooling and purpose-built data centers are now standard.
• Inference increasingly happens at the edge (phones, cars, cameras) for latency and privacy.

Why Hardware Matters

Hardware determines the feasible scale of models and the economics of AI. Algorithmic progress often arrives together with new hardware.

Sources

• Hennessy, J. & Patterson, D. Computer Architecture: A Quantitative Approach.
• NVIDIA. CUDA documentation.

Back to Hardware & Software · See GPU Computing.