How Machines Learn from Data
A machine learns when its performance changes because of experience represented as data. Training is an optimisation process: the model adjusts internal parameters to reduce error on examples.
Learning objectives
What you will be able to do
- Explain how machines learn from data accurately in your own words.
- Recognise how the concept appears in real AI products and professional work.
- Identify an important limitation, risk or evaluation requirement.
- Apply the concept through a practical activity and knowledge check.
Core explanation
Build the right mental model
Begin with examples and a measurable objective. The model produces an output, a loss function measures the difference between output and desired result, and an optimisation algorithm adjusts parameters. Repeating this process across many examples is called training.
Learning does not mean memorising every example. The goal is generalisation—performing well on new cases drawn from the intended environment. If the training data is unrepresentative, incomplete or historically biased, the model can learn the wrong pattern very efficiently.
After deployment, the real world may change. Customer behaviour, language, network traffic and fraud tactics evolve. Teams therefore monitor quality and decide when new data and retraining are required.
Follow the learning loop
A training example supplies information from which a model can produce a prediction. A loss function gives a numerical measure of mismatch between prediction and target. An optimiser changes parameters in an attempt to reduce that loss across training examples. These roles are distinct: the data is not the model, the model is not the loss, and the optimiser is not the final business objective.
In a simple line model, prediction = slope × input + intercept. Training changes the slope and intercept. At inference time, the chosen values are used to calculate predictions for new inputs. This small model is useful for learning because you can inspect every calculation; the same broad distinction between fitting and prediction also appears in much larger models.
Understand what decreasing loss does and does not show
Squared error takes the difference between a prediction and target and squares it. Mean squared error averages those values over the examples. Squaring prevents positive and negative errors from cancelling, while giving larger errors more weight. The number is meaningful only alongside the target’s units and the definition of the dataset.
Gradient descent uses information about how loss changes with the parameters to choose an update direction. Its learning rate controls the update size. A very small step can make progress slow; an excessively large step can overshoot useful values. You do not need calculus for this activity: compare two candidate settings directly and see which has lower loss on the supplied cases.
Check learning beyond the fitting examples
A reduction in training loss is evidence of a better fit to the training set, not proof of improved performance everywhere. An overly flexible model may fit noise or memorise unusual cases. Check separately on data not used to fit parameters and inspect whether the examples represent the intended environment.
Also examine the target being optimised. Minimising average prediction error may not protect against the rare mistakes that matter most to a user. For a maintenance system, a late warning could be more costly than a modest error in an ordinary reading. Define those consequences before interpreting a lower training-loss number as operational success.
Parameters
Internal numerical values adjusted during training.
Loss function
A calculation expressing how wrong a model’s output is.
Optimisation
The procedure used to adjust parameters and reduce loss.
Generalisation
Useful performance on new, unseen cases.
Calculate loss for two candidate models
Use fictional pairs (input, target): (1, 3), (2, 5), (3, 7). Model A predicts y = 2x, giving 2, 4 and 6. Its errors, prediction minus target, are −1, −1 and −1. Squared errors are 1, 1 and 1, so mean squared error is (1 + 1 + 1) ÷ 3 = 1.
Model B predicts y = 2x + 1, giving 3, 5 and 7. Its mean squared error is 0 on these three examples. We have changed the intercept from zero to one and improved the fit. This manual comparison illustrates parameter choice; it is not a full gradient-descent training run.
For a held-out example (4, 9), A predicts 8 and B predicts 9, so B is better on that example too. Still, four deliberately tidy points do not establish performance on noisy measurements or inputs far outside this range. Keep the claim proportional to the evidence.
Apply your learning
Practical activity
- Spend five minutes calculating predictions for y = x + 1 on the three training pairs.
- Spend five minutes calculating its squared errors and mean squared error.
- Spend five minutes comparing it with Models A and B and identifying which parameter differs.
- Spend five minutes explaining what new evaluation examples you would need before using the model on real measurements.
Compare with an explained example answer
For y = x + 1, predictions are 2, 3 and 4. Errors are −1, −2 and −3, with squared errors 1, 4 and 9. Mean squared error is 14 ÷ 3 ≈ 4.67, higher than A’s 1 and B’s 0.
This comparison shows that both slope and intercept matter. Real evaluation should include independently collected cases from the expected input range, measurement noise and relevant operating conditions. An exact fit to the invented points is not a claim about real equipment.
Write your answers in a learning journal. The value comes from connecting the concept to your own profession.
Knowledge check
Test your understanding
Q1Why square errors instead of averaging signed errors?
Answer: Positive and negative differences can cancel. Squared error counts both directions and penalises larger deviations more strongly.
Q2What changed between Models A and B?
Answer: The intercept changed from zero to one; the slope stayed at two.
Q3Does zero training loss establish generalisation?
Answer: No. It establishes a fit to those examples. Independent evaluation is needed to assess new cases.
If you cannot explain the answer without reading it, revisit the core explanation and example before continuing.
Continue with authoritative sources
Important industry resources
These links lead to official organisations, industry laboratories or established open-source learning projects.