Programming gave computers a way to execute procedures without rebuilding the machine. But programming still required someone to specify the procedure. Machine learning introduced a different relationship between computation and knowledge: instead of describing every rule explicitly, a system could adjust internal parameters in response to examples and an objective. The critical transition was not that machines suddenly became intelligent. It was that learning itself became a computational procedure.
The limitation of instructions
A programmable machine still needs its behaviour specified
A conventional program tells a computer what to do. A programmer decomposes a problem into operations, encodes those operations and gives the machine a procedure it can execute. This works well when the relevant rules can be stated clearly. It becomes harder when a task depends on subtle patterns in images, speech, language or noisy observations.
For these problems, writing down every rule can be more difficult than showing a system many examples of the result. Machine learning introduced a different arrangement: people specify a model and a way to evaluate its output, then use data to adjust the model's parameters. The machine still follows a computational procedure, but not every detail of its eventual behaviour is written directly as a rule.
Mathematics supplies the bridge
Inference turns observations into a problem that can be formulated
Probability and statistics provided a language for reasoning about uncertainty, relationships and patterns in observations. Rather than treating each example as an isolated fact, a learning system can estimate parameters and use evidence to make predictions about cases that remain unseen.
This did not make learning automatic. It made the problem expressible: define what is observed, choose a model that can represent relevant relationships, and specify what counts as a useful result. A procedure was still needed to adjust the model when its predictions failed.
A defining framework
The Learning Loop
The defining difference is the feedback loop. A program primarily applies specified rules. A learning system uses examples and an objective to modify parameters so that its future behaviour changes.
Optimization makes adjustment systematic
The machine needs a way to improve
A model that produces predictions is not yet a learning system. It needs a mechanism for changing itself in response to how well those predictions perform. Optimization supplied a general mathematical framework for doing this.
The basic idea is to define an objective that measures the difference between desired and actual behaviour, then search for parameter values that improve the objective. In simple cases this can be understood as moving toward lower error. In more complex models, gradients provide information about how changes in parameters affect the objective.
This created an important convergence. Statistics supplied ways to formulate relationships from observations. Optimization supplied mechanisms for improving a model. Computation supplied the machinery required to perform those calculations repeatedly.
The perceptron makes learning concrete
A machine can adjust a decision boundary from examples
Frank Rosenblatt's perceptron, introduced in the late 1950s, provided an influential demonstration that a computational system could learn a classification rule by adjusting weights associated with its inputs. Instead of manually specifying every boundary, the system could update its parameters as it encountered examples. The broader development of learning systems is documented in institutional histories of artificial intelligence.
The perceptron was limited. A single-layer perceptron cannot represent every possible classification function, and the limitations of simple networks became an important part of the subsequent history of neural-network research. Minsky and Papert's 1969 analysis of perceptrons helped make those limitations explicit.
But the historical significance of the perceptron was larger than its limitations. It made a new computational idea concrete: a machine could contain adjustable parameters whose values were changed through experience with examples.
Neural networks add layers
Learning can become a problem of representation
A single adjustable layer can learn some relationships, but richer tasks require transformations that cannot be captured by a simple linear decision boundary. Multi-layer neural networks offered a way to build more expressive computational structures by composing transformations.
The difficulty was learning the parameters of those layers. If an error appears at the output, how should responsibility for that error be distributed across the many internal connections that produced it?
This question pushed learning toward a deeper mathematical procedure: calculating how the error changes with respect to parameters throughout the network.
Backpropagation changes the problem
The network can learn internal representations
In 1986, David Rumelhart, Geoffrey Hinton and Ronald Williams described a back-propagation procedure for networks of neuron-like units. The method repeatedly adjusted connection weights to reduce the difference between the network's actual output and the desired output. Their paper, “Learning representations by back-propagating errors,” became a landmark in the development of trainable multi-layer networks.
Its deeper importance was not simply that it reduced error. The procedure allowed internal hidden units to develop useful representations of features in the task domain. The network could therefore learn intermediate structure rather than requiring every useful feature to be specified in advance.
This is a critical step toward modern machine learning. Learning was becoming not merely parameter fitting but representation formation.
Learning changes the role of data
Examples become evidence from which behaviour is derived
In a conventional program, data is usually processed according to rules specified in advance. In machine learning, examples also influence the rules the model comes to apply. They provide evidence about relationships the model should capture, while the learning procedure determines how that evidence changes its parameters.
The distinction connects this essay to the previous one, The World Becomes Data. Digitization and representation make information available to computation; learning uses selected examples to estimate patterns. The presence of more data does not guarantee better learning. Its usefulness depends on relevance, coverage, measurement and the relationship between examples and the task.
Learning did not emerge from a single discovery. It brought together mathematical ideas about inference, models that could represent patterns, optimization procedures that could adjust parameters, and methods for testing whether learned behaviour carried beyond the training examples.
The underlying stack also includes computation, programmable machines and digital information, developed across earlier stages of this series. Here they matter as prerequisites for learning, not as the subject of a second infrastructure history. The next essay examines the systems that made learning practical at scale.
Learning changes where the rules come from
Machine learning changes where part of a system's behaviour is established. Developers still choose the problem, model family, training objective, data, constraints and conditions of use. But the resulting behaviour is shaped by parameter values adjusted through examples, rather than being fully specified as a list of hand-written rules.
This is not the disappearance of human design. It is a redistribution of design work. Some decisions move from writing explicit rules to choosing examples, objectives and procedures that influence what the model learns.
Generalization is the central test
A model is useful when patterns learned from available examples help it handle cases it has not seen. This is generalization. Memorizing training examples is insufficient when the intended task is to predict, classify or make a decision beyond those observations.
Generalization depends on how well the examples represent the conditions where the model will be used. Narrow coverage, incorrect labels, historical bias or changing circumstances can produce failures that a strong training score does not reveal. Learning therefore needs evaluation on evidence that is distinct from the examples used to adjust the model.
The central question is not simply whether a model improves on its training objective, but whether that improvement carries into the situation the model is meant to address.
Evaluation determines what improvement means
Every learning procedure optimizes something, but its objective is only a proxy for the wider purpose of the system. A model that minimizes average error can still fail on rare consequential cases. A classifier can achieve a high overall score while making uneven errors across groups or operating conditions.
Evaluation must therefore test more than whether optimization succeeded. It asks which errors matter, which cases are missing, how performance changes under different conditions and whether the chosen measure reflects the actual purpose. What is measured influences what counts as improvement.
Learning introduces new questions of evidence and responsibility
Because learned behaviour is shaped by examples and objectives, its weaknesses may not be visible by inspecting the program alone. Training data can reflect incomplete coverage, historical inequalities or choices about what to measure. A model may also rely on a relationship that holds in one setting and fails when circumstances change.
These risks do not make learning inherently unreliable. They make evidence, evaluation and responsibility part of the design. People remain accountable for deciding where a model is appropriate, what outcomes it should support and what safeguards are needed when its predictions affect others.
What lies beneath learning?
From learning to scale
The capability exists, but the conditions for scale are a different problem
Machine learning established a new way to produce computational behaviour. Instead of specifying every operation that leads to a result, developers could define a model, an objective and a procedure through which parameters were adjusted from examples.
That conceptual change did not by itself make large-scale learning practical. Training larger models on larger datasets would require specialized processors, coordinated computation, software, storage, networking, energy and physical infrastructure. Those conditions belong to the next essay.
The next transformation was not another definition of learning. It was the construction of a system capable of sustaining learning as an industrial workload.
The argument
Machine learning changed how computational behaviour could be established. Probability and statistics made patterns in observations expressible; optimization supplied a way to adjust parameters; perceptrons and neural networks provided models of increasing expressive power; and backpropagation made systematic adjustment across multiple layers possible.
The result was not a machine free of human design. It was a different allocation of design: people selected the model, examples, objective and evaluation, while the model's parameters were shaped through training. Whether the learned behaviour generalized remained the central test.
These sources support the historical discussion of early learning systems, neural-network limitations, backpropagation and the development of artificial intelligence. Each source is listed once to avoid duplicate citations.
IBM: The History of Artificial Intelligence Nature: Learning Representations by Back-Propagating Errors (1986) Nature Reviews Neuroscience: Backpropagation and the Brain Geoffrey Hinton: Backpropagation Publications