GPUs: The Chips That Help AI Work
Why does AI use teams of computing chips?
How it works
A GPU (graphics processing unit) is a chip. It handles many calculations together. GPUs draw game pictures through steps that happen side by side.
AI (programs that learn patterns from examples) needs repeated math steps. GPUs spread suitable steps across many small working units.
Memory keeps those numbers nearby. Fast connections carry results between chips. Waiting for numbers leaves chips with less useful work.
Adding GPUs helps when tasks split into pieces. Some pieces wait for earlier results. Moving numbers between chips also takes time. More chips can help without making every job equally faster.
Picture the scale
NVIDIA lists 6,912 small calculation units called CUDA cores in its A100 GPU. Picture many checkout lanes handling separate baskets at once. Suitable math jobs can also run side by side, though these tiny units each do much simpler work than a whole computer.
Why should I care?
A game's graphics and an AI reply both depend on chips doing work. Those chips need power and cooling, even when the screen shows only a few words.
Words to know
- GPU
- A chip doing many calculations at once; like a team handling similar jobs.
- Memory
- Storage for numbers needed during work; like materials kept on a desk.
Check yourself
1. Which job suits many GPU workers?
- Similar calculations on many numbers
- A single chain of steps that all depend on earlier results
- Waiting for fuel to reach a generator
Show answer and why
A. Similar calculations on many numbers. Independent similar work can be shared in parallel.
2. Why keep memory near the chip?
- To generate replacement electricity
- To supply needed data quickly
- To remove all cooling needs
Show answer and why
B. To supply needed data quickly. The chip needs numbers available while it works.
3. Why can adding chips bring a smaller speed gain?
- Communication takes no time
- Each chip removes every delay
- The chips may wait for shared results
Show answer and why
C. The chips may wait for shared results. Communication and waiting remain part of the job.