# GPU Computing for AI

Graphics Processing Units (GPUs) became the workhorse of deep learning because they parallelize dense matrix math. This sub-article expands [Section 1.2 of Hardware & Software](../hardware-software.md).

## Why GPUs
- Thousands of cores compute many matrix operations simultaneously.
- High memory bandwidth feeds the arithmetic units.
- CUDA (2006 onward) made GPUs programmable for general compute.

## The Deep-Learning Catalyst
- AlexNet (2012) was trained on GPUs, dramatically outperforming prior methods.
- Almost all major model training now uses GPU clusters.

## Programming and Frameworks
- CUDA and cuBLAS underpin kernels; PyTorch and TensorFlow schedule GPU work.
- Mixed-precision training (FP16/BF16) speeds up arithmetic.

## Limits
- GPUs consume significant power and produce heat.
- Memory per GPU bounds model size; multi-GPU sharding is required for frontier models.

## Alternatives
- See [Specialized AI Hardware](specialized-ai-hardware.md) for TPUs, NPUs, and neuromorphic chips.

## Sources
- NVIDIA. *CUDA Programming Guide.*
- Krizhevsky, A. et al. (2012). *ImageNet Classification with Deep Convolutional Neural Networks.*

*Back to [Hardware & Software](../hardware-software.md).*