EXPLAINER

History of AI: from room-sized computers to AI factories

From ENIAC's 30 tons of vacuum tubes to buildings full of GPUs: how AI went from a 1956 workshop idea to an industry.

1. The room-sized computer

In 1945, the most powerful computer in the world was ENIAC, built at the University of Pennsylvania for the U.S. Army to calculate artillery firing tables. It weighed 30 tons, filled a room 30 feet by 50 feet, and used around 18,000 vacuum tubes. It consumed about 140 kilowatts of power — roughly what a modern AI rack draws — and it could do about 5,000 additions a second. It was formally dedicated in February 1946.

ENIAC was the first programmable, electronic, general-purpose digital computer. "General-purpose" is the important word: earlier machines were built for one calculation. ENIAC could be reprogrammed, by hand, for another. But every part of it was a problem. Vacuum tubes burned out, so the machine was often down for repair. It had no stored program in the modern sense — programming meant physically rewiring it. And it was the size of a room to do arithmetic a pocket calculator would later do in a fraction of a second.

The machine that replaced it was not a bigger ENIAC. It was a different kind of switch.

2. The transistor, the chip, and the doubling

In December 1947, John Bardeen and Walter Brattain at Bell Labs demonstrated the first transistor: a sliver of germanium with two gold contacts that could amplify a signal without a glowing filament. William Shockley, their group leader, conceived the junction transistor shortly afterwards. The three shared the 1956 Nobel Prize in Physics. A transistor did what a vacuum tube did, with no warm-up, far less power, and no glass to break.

The next step took a decade. In 1958 Jack Kilby at Texas Instruments built the first working integrated circuit — several components made from a single piece of semiconductor. In 1959 Robert Noyce at Fairchild Semiconductor filed the planar version, which could be manufactured at scale. The two are credited as co-inventors of the chip.

In 1965 Gordon Moore, then director of research at Fairchild, observed in Electronics magazine that the number of components on a chip had been doubling every year, and predicted it would keep doing so for at least a decade. He revised the forecast in 1975 to a doubling about every two years — the version that became known as Moore's law. (The "18 months" version often quoted is not Moore's: it comes from Intel executive David House, and it is about chip performance, not transistor density.) The observation held, near enough, for fifty years. It is the reason a computer went from a room to a desk to a pocket — and the reason the same chip that drew video-game frames could later be repurposed to train AI.

3. Naming the field

The question of machine intelligence predates the machines. In October 1950, Alan Turing published "Computing Machinery and Intelligence" in the philosophy journal Mind. He refused to answer "Can machines think?" directly, and replaced it with a game: an interrogator, communicating only in writing, tries to tell a machine from a person. He predicted that in about fifty years, computers with a storage capacity of about a billion units could play the game well enough that an average interrogator would be right no more than 70% of the time after five minutes of questioning. Turing had already described the universal machine — the theoretical model of a general-purpose computer — in 1936.

The field got its name six years later. On August 31, 1955, John McCarthy, Marvin Minsky, Nathaniel Rochester and Claude Shannon submitted a proposal to the Rockefeller Foundation for a two-month summer workshop at Dartmouth College. The proposal introduced the term "artificial intelligence" — McCarthy coined it, partly to escape the then-dominant field of cybernetics — and set out the conjecture that "every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it." The workshop ran in the summer of 1956. At it, Allen Newell and Herbert Simon demonstrated the Logic Theorist, a program that proved theorems. Little of substance came out of the workshop itself, but the field now had a name, a community, and funding.

The first learning machine followed almost immediately. In 1957 Frank Rosenblatt, a psychologist at the Cornell Aeronautical Laboratory, described the perceptron: a network of simple units whose connections changed with experience, so the machine could learn to recognise patterns instead of following fixed rules. The Mark I Perceptron, built in 1957–58, used a 20-by-20 grid of 400 photocells as a retina. It was, in miniature, the ancestor of every neural network in use today.

It was also the first hype cycle. A 1958 Navy press conference produced front-page claims that the perceptron was the forerunner of a machine brain. Then, in 1969, Marvin Minsky and Seymour Papert published Perceptrons, showing rigorously that a single layer of perceptron units could not compute even simple functions like exclusive-or. Funding and confidence drained away.

4. The winters

AI has collapsed twice, and the pattern is the same both times: a burst of optimism, promises that outrun the hardware, and then a funding cut when the results do not arrive.

The first winter ran roughly from 1974 to 1980. In 1973 the Lighthill Report, commissioned by the British government, concluded that AI research had failed to achieve its "grandiose objectives," and Britain cut AI funding to a handful of universities. In the United States, DARPA cut academic AI funding, including about $3 million a year for speech-understanding research at Carnegie Mellon. By 1974, funding for AI projects was hard to find.

The field came back in the 1980s in a narrower form: expert systems, which encoded one human expert's rules for one narrow domain — configuring computers, diagnosing infections — instead of trying to reason generally. For a few years they sold. Then they hit their ceiling: every new edge case needed a human expert to hand-write another rule, and the systems were expensive to maintain and brittle outside their niche.

The second winter ran from the late 1980s into the early 1990s. In 1987 the market for specialised Lisp machines — the expensive hardware built to run AI software — collapsed in about a year, once ordinary desktop computers from Apple and IBM became more powerful than the specialised machines. DARPA cut AI spending again in 1988, and Japan's Fifth Generation Project, which had promised a new generation of intelligent computers, had not met its goals by 1991.

What survived both winters was the work that did not need the hype. In 1986, in the middle of the second winter's approach, David Rumelhart, Geoffrey Hinton and Ronald Williams published "Learning representations by back-propagating errors" in Nature. It showed that a network with several layers, trained by backpropagation — sending the error backwards through the network so each connection could be adjusted — could learn features nobody had programmed into it, including functions like exclusive-or that had defeated the single-layer perceptron. The mathematics was not entirely new — Paul Werbos had described the method in his 1974 thesis — but the 1986 paper demonstrated it, and made it famous. The hardware to use it at scale did not exist yet. That took another 25 years.

5. Data, graphics cards, and the 2012 turning point

Three ingredients arrived separately, and then together.

The first was data. In 2009, Fei-Fei Li's group at Stanford published ImageNet, a database of labelled photographs built on the WordNet hierarchy. The annual competition built on it, the ImageNet Large Scale Visual Recognition Challenge, gave the field a benchmark: sort 1.2 million images into 1,000 categories, and be scored on your five best guesses. For the first time, a network with tens of millions of parameters had enough labelled examples to train on instead of memorising.

The second was hardware that was already parallel for an unrelated reason. A graphics chip exists to draw frames — see what a GPU does: a 1920-by-1080 display has over two million pixels, and the chip shades them all at once. Researchers had been forcing mathematics through graphics pipelines since the early 2000s, disguising arithmetic as textures and shaders — a practice called general-purpose computing on graphics processing units, or GPGPU. In November 2006 NVIDIA announced CUDA alongside the GeForce 8800 GTX, the first graphics processor designed to run general programs as well as render images; the public beta of the CUDA toolkit followed in February 2007, and CUDA 1.0 shipped that June. For the first time, a programmer could write ordinary code for the GPU in C, with no graphics detour.

The third was a network ready to use both. In 2012, Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton at the University of Toronto entered a deep convolutional network into the ImageNet competition. AlexNet had about 60 million parameters and eight learned layers. It was trained on two consumer NVIDIA GTX 580 graphics cards, each with 3 GB of memory, in about five to six days. It won with a top-5 error rate of 15.3%, against 26.2% for the second-place entry — a gap of more than ten percentage points, in a competition where the previous winners had improved by one or two. Within months, Google, Facebook and Baidu had all started deep-learning labs. The modern era of AI dates from that result, not from a theory: it was a demonstration that deep networks, trained on GPUs, worked at scale.

Training a large model takes days, and a failure partway through can waste the whole run — which is why training uses checkpoints.

The network itself was not new. Yann LeCun and colleagues had built LeNet-5, a convolutional network for recognising handwritten digits, in 1998. What changed between 1998 and 2012 was not the idea. It was the data to train on and the hardware to train with.

6. The transformer and the scaling era

The next turning point was an architecture. On June 12, 2017, eight researchers at Google Brain and Google Research posted "Attention Is All You Need" (arXiv:1706.03762). The transformer dispensed with recurrence and convolutions entirely and built the network only from attention mechanisms — letting every word in a sentence look at every other word at once, and work out which ones matter. It was presented at NeurIPS 2017, and it became the foundation of virtually every large language model since.

OpenAI's GPT line shows what scaling that architecture did. GPT-1 arrived in June 2018 with about 117 million parameters. GPT-2 followed in February 2019 with 1.5 billion. GPT-3 arrived in May 2020 with 175 billion — roughly a hundred times GPT-2 — and could learn a task from instructions and a few examples placed in the prompt, instead of needing task-specific training. ChatGPT, built on GPT-3.5, launched on November 30, 2022. GPT-4 followed in March 2023, accepting images as well as text; OpenAI did not disclose its size.

Training a model and running it are different jobs with different power profiles — see training vs inference.

The pattern across all of it is the same: the architecture changed twice (convolutional networks, then transformers), and each time the models got bigger because the hardware and the data allowed it. Nothing in the GPT line required a new idea about intelligence. It required more of everything the 2012 result had already shown to work.

7. From one computer to an AI factory

The scale jump is the part of this history that lives in buildings. ENIAC drew about 140 kilowatts and did 5,000 additions a second. A single modern AI rack draws about 120 kilowatts — and holds 72 GPUs doing matrix arithmetic, not additions. Our explainer on the 120 kW AI rack sets out what that density does to power and cooling. The average data-center rack, for comparison, still runs at about 7.5 kW.

The jump from a rack to a factory is a jump in power, not just in count. A campus of these racks needs tens or hundreds of megawatts — a scale where the question stops being "how fast is the chip" and becomes "where does the power come from, and who pays for the grid connection." What 100 MW of AI capacity actually means explains the numbers behind that question, and our explainer on grid connection explains how a campus gets that power delivered.

Adding more chips does not add speed in proportion — more chips aren't always faster.

That is the arc of this history. In 1945 the computer was the building. In 1965 the computer was a chip. In 2012 the computer was two gaming cards on a desk. Today, the computer is the building again — except now the building is a data center, the chips number in the tens of thousands, and the power draw is measured in megawatts. The idea from Dartmouth did not change. The hardware finally caught up with it.

Questions readers ask

Why did AI take 70 years from Dartmouth to ChatGPT?

Because the idea arrived before the hardware. The perceptron of 1957 and the backpropagation paper of 1986 both described how a network could learn. What they lacked was labelled data at scale — which arrived with ImageNet in 2009 — and cheap parallel hardware — which arrived when NVIDIA made GPUs programmable with CUDA in 2006–07. The 2012 AlexNet result was the first time all three were in place at once.

Was the perceptron a failure?

No — it was a single-layer network, and Minsky and Papert proved in 1969 that a single layer could not compute even simple functions like exclusive-or. That proof discredited neural networks for years, but it said nothing about multi-layer networks. Backpropagation, in 1986, showed how to train those — and the perceptron's core idea, connections that change with experience, is inside every neural network in use today.

Why did graphics cards, built for games, end up running AI?

Because drawing a frame and training a network need the same thing: doing the same arithmetic on millions of numbers at once. A graphics chip shades two million pixels in parallel. A neural network multiplies matrices. Before CUDA, researchers had to disguise their mathematics as graphics to use the hardware. CUDA let them program the GPU directly, and AlexNet in 2012 showed the result.

What actually changed in 2012 and 2017?

In 2012, AlexNet proved deep networks worked at scale — 15.3% error against 26.2% for the next-best system, trained on two consumer graphics cards. In 2017, the transformer gave language models an architecture built only on attention, which could be scaled up far beyond what recurrent networks allowed. GPT-1 to GPT-3 is mostly the same architecture getting bigger: 117 million parameters in 2018, 1.5 billion in 2019, 175 billion in 2020.

Is AI still in a boom, or is another winter coming?

This page does not predict that. What the history shows is the pattern: winters followed promises that outran the hardware. The current boom is built on hardware that demonstrably exists — the chips, the data centers, and the power are real and measurable. Whether the economics hold is a different question, and it is answered by power prices and balance sheets, not by the history of the ideas.

Sources and further reading

Glossary — plain definitions for words used here.