The maths behind predictions

Suppose three students scored 60, 70 and 80. If you have no other information, predicting the average score for everyone is a simple starting point. It will miss individual results, but it gives us a baseline: a basic comparison that a more complicated model should justify improving upon.

We can describe mistakes with subtraction and measure their size with averages. These ideas lead to a loss function, a numerical score that tells a training procedure how poorly a model fits its observations. You do not need advanced calculus to trace our first examples, but you do need to understand what each calculation is measuring.

The mean is the sum divided by the number of observations: (60 + 70 + 80) / 3 = 70. Here a residual is the actual score minus the prediction. Predicting 70 produces residuals of -10, 0 and 10. Their average is zero because positive and negative errors cancel; that does not mean every prediction is correct.

Squaring residuals prevents that cancellation. The population variance of these three numbers is (100 + 0 + 100) / 3, or about 66.67 squared score units. Sample variance uses a different denominator when estimating population spread. Standard deviation takes the square root to return to the original units.

Prepare and run

Use Python in the course environment. The input values are included in this program; no separate data file is needed.

Python · complete runnable example

from statistics import mean, pvariance

scores = [60, 70, 80]
baseline = mean(scores)
errors = [score - baseline for score in scores]
print("Mean:", baseline)
print("Residuals:", errors)
print("Population variance:", round(pvariance(scores), 2))

Expected output

Mean: 70
Residuals: [-10, 0, 10]
Population variance: 66.67

Follow the result

Python's mean computes the baseline, and the list comprehension subtracts it from each score. pvariance treats these three scores as the whole population being described. The example reports spread around the mean, rather than estimating a future model's error on an independent dataset.

Your task: What is the mean absolute error of predicting 70 for these three students?

Worked answer

The absolute errors are 10, 0 and 10. Their average is 20 / 3, about 6.67 score points. Absolute values stop errors cancelling and retain the score's units. This differs from the squared-unit variance printed above.

A feature vector is the list of measurements for one observation; a matrix arranges several vectors into rows. A weighted sum multiplies each measurement by a coefficient and adds the results. Linear regression uses that idea to produce a numerical prediction.

Probability describes uncertainty on a scale from zero to one. A model's reported probability is an estimate, not a promise. Derivatives describe how a value changes as another value changes, and some training algorithms use them to reduce loss step by step. We will explain those ideas when needed instead of treating symbols as a prerequisite you must already know.

A mean is a useful starting prediction, but signed errors can cancel. Absolute errors retain target units, squared errors emphasize large misses and variance describes spread under a stated denominator.

Read further: official supporting documentation.