Linear regression: learn a line to predict house prices

Suppose you know the floor area of a house and want a rough estimate of its sale price. Past sales give you examples: an area as the input and a price as the known outcome. Linear regression learns a relationship that predicts a numerical quantity. With one input feature, that relationship is a straight line.

This is a regression task because the answer is an amount, such as a price, rather than a category such as “sold” or “not sold”. Floor area is the feature; the sale price is the target. We will start with four invented sales whose arithmetic is easy to follow, then add imperfect sales and evaluate separate examples.

Before starting, you should understand training versus prediction and why evaluation needs held-out records. This lesson teaches the model, not a real valuation service: location, condition and many other influences are deliberately absent.

Our teaching line is estimated price = 50 + 2 × floor area. Area is measured in square metres; price is measured in thousands of US dollars. Keeping the price numbers small helps us inspect the calculation. A prediction of 230 therefore means $230,000, not $230.

For a 90 m² house, multiply 2 × 90 = 180, then add 50: the estimate is 230 thousand dollars. The model uses the same settings for every new area until it is fitted again.

The intercept places the line; it is not necessarily a sensible price for a real zero-area house. Zero lies outside our observed range of 50–125 m². Likewise, the slope describes the model’s fitted association. It does not establish what would happen if you physically extended a particular house by a square metre.

Prepare and run

Use Python with NumPy and scikit-learn in the course environment. The invented data is included; no file or download is needed.

Python · complete runnable example

import numpy as np
from sklearn.linear_model import LinearRegression

# Invented sales. Targets are in thousands of US dollars.
X = np.array([[50], [75], [100], [125]])
y = np.array([150, 200, 250, 300])
model = LinearRegression().fit(X, y)
print("Slope:", round(model.coef_[0], 1))
print("Intercept:", round(model.intercept_, 1))
print("At 90 square metres:", round(model.predict([[90]])[0], 1))

Expected output

Slope: 2.0
Intercept: 50.0
At 90 square metres: 230.0

X holds four rows with one area feature in each row. The nested brackets make it a two-dimensional array: rows are houses and columns are features. y holds four prices in the matching row order. LinearRegression() creates the estimator; fit(X, y) learns its settings from the paired inputs and targets.

coef_[0] is the coefficient of the first and only feature; intercept_ is the learned starting value. predict([[90]]) supplies one new row. The final [0] selects its single prediction and round makes the displayed number readable.

All four invented training prices lie on 50 + 2 × area, so this example fits perfectly. At 90 m², the model computes 50 + 180 = 230. The new house’s actual sale price was not passed to predict. A neat training fit is an illustration, not evidence of reliable real-house predictions. See the LinearRegression API for the array shapes and fitted attributes.

Real observations rarely follow one feature perfectly. A residual is the observed target minus its fitted prediction. Positive means the observation is above the line; negative means it is below. A prediction for an unsold house has no known residual yet, because its actual price is still unknown.

If the observed price were 218 instead, its residual would be −12. Both misses have absolute size 12. Simply adding signed residuals can hide errors: +12 and −12 sum to zero even though neither prediction was correct.

Ordinary least squares chooses slope and intercept to minimize the sum of squared training residuals. For each candidate line, it predicts the training prices, subtracts each prediction from its observed price, squares each difference and adds the results.

Squaring keeps positive and negative misses from cancelling. It also gives large misses more weight: a residual of 20 contributes 400, while two residuals of 10 contribute only 200 altogether. This makes unusually large observations influential, so investigate data errors and unusual sales rather than deleting inconvenient records automatically.

The estimator solves for the settings; it does not need you to manually slide through every possibility. In the next example, a fitted line still misses every training sale. “Best for this objective” means its total squared error is minimized for this dataset and model form; it does not mean every individual prediction is exact.

The official least-squares explanation defines this fitting objective. An evaluation measure such as mean absolute error answers a different question: how large are the misses on average in the original price units?

Prepare and run

Use Python with NumPy and scikit-learn in the course environment. The invented data is included; no file or download is needed.

Python · complete runnable example

import numpy as np
from sklearn.linear_model import LinearRegression

# Invented sales, with prices in thousands of US dollars.
X = np.array([[50], [75], [100], [125]])
y = np.array([160, 190, 240, 310])
model = LinearRegression().fit(X, y)
predictions = model.predict(X)
residuals = y - predictions
print("Slope and intercept:", round(model.coef_[0], 1),
      round(model.intercept_, 1))
print("Fitted prices:", np.round(predictions, 1).tolist())
print("Residuals:", np.round(residuals, 1).tolist())
print("Squared residual sum:", round(float(np.sum(residuals ** 2)), 1))

Expected output

Slope and intercept: 2.0 50.0
Fitted prices: [150.0, 200.0, 250.0, 300.0]
Residuals: [10.0, -10.0, -10.0, 10.0]
Squared residual sum: 400.0

This complete program repeats all four areas and supplies different invented prices: 160, 190, 240 and 310 thousand dollars. The fitted settings happen to remain 2 and 50 because of how this teaching data was constructed. Noise does not always leave the same settings; this particular arrangement does.

At 50 m², actual 160 minus fitted 150 gives +10. At 75 m², actual 190 minus fitted 200 gives −10. The remaining residuals are −10 and +10. Squaring gives 100 for each, for a total of 400 in squared thousand-dollar units. The absolute errors are all 10, so their mean is 10 thousand dollars.

With slope 2 and intercept 55 instead, predictions would be 155, 205, 255 and 305. Residuals would be 5, −15, −15 and 5; their squared sum would be 500. Raising the line helps two sales and hurts two more. Comparing the total, rather than a single convenient observation, explains the fitting decision.

For a smaller-number comparison, use these four invented student scores: 1, 2, 3 and 4 hours paired with 50, 60, 70 and 80 points. Their exact line is score = 40 + 10 × hours. Adjust a proposed slope and intercept to see how each prediction and mean absolute error change. This is a calculation lab, not an automatic training algorithm.

Try a deliberate change

Keep slope 10 and move intercept to 45. Every prediction increases by five, so every absolute error becomes five and MAE becomes 5.00. The observed targets stay fixed. The lab reports MAE for readability; ordinary least squares fits by squared errors, not by minimizing this displayed MAE. This is not evidence that studying another hour causes ten extra points. The default calculation remains readable if JavaScript is unavailable.

Prepare and run

Use Python with NumPy and scikit-learn in the course environment. The invented data is included; no file or download is needed.

Python · complete runnable example

import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error

# Invented, separate sales; prices in thousands of US dollars.
X_train = np.array([[50], [75], [100], [125]])
y_train = np.array([150, 200, 250, 300])
X_test = np.array([[90], [110]])
y_test = np.array([242, 264])
model = LinearRegression().fit(X_train, y_train)
predictions = model.predict(X_test)
baseline = np.full(len(y_test), y_train.mean())
print("Test predictions:", np.round(predictions, 1).tolist())
print("Actual test prices:", y_test.tolist())
print("Model MAE:", round(mean_absolute_error(y_test, predictions), 1))
print("Mean baseline MAE:", round(mean_absolute_error(y_test, baseline), 1))

Expected output

Test predictions: [230.0, 270.0]
Actual test prices: [242, 264]
Model MAE: 9.0
Mean baseline MAE: 28.0

We return to the original four clean training sales, and separately include two invented test sales. Their actual prices are used only for evaluation. Neither the fitted line nor the baseline uses y_test to choose its settings.

The line predicts 230 and 270, while the actual values are 242 and 264. The absolute misses are 12 and 6 thousand dollars. Mean absolute error (MAE) is (12 + 6) / 2 = 9: an average miss of $9,000 on these two sales.

The baseline ignores area and predicts the training mean, (150 + 200 + 250 + 300) / 4 = 225, for both houses. Its absolute errors are 17 and 39, averaging 28 thousand dollars. The line performs better on this deliberately small check. Two invented observations do not support a claim about real future housing performance.

np.full creates an array repeating the mean. mean_absolute_error compares matching actual and predicted values, as described in its API reference. Other error measures and their trade-offs belong in model evaluation metrics.

Your task: Run the complete original program below with the query changed from 90 to 110. What price will it predict? If that house sells for $264,000, calculate its signed residual and absolute error. Prices in the program are still in thousands of dollars.

Python · complete program to modify

import numpy as np
from sklearn.linear_model import LinearRegression

# Invented sales. Targets are in thousands of US dollars.
X = np.array([[50], [75], [100], [125]])
y = np.array([150, 200, 250, 300])
model = LinearRegression().fit(X, y)
print("Slope:", round(model.coef_[0], 1))
print("Intercept:", round(model.intercept_, 1))
print("At 90 square metres:", round(model.predict([[90]])[0], 1))

Current output before modification

Slope: 2.0
Intercept: 50.0
At 90 square metres: 230.0

Worked answer

Change the last line to the following Python statement, updating both the query and its printed label:

print("At 110 square metres:", round(model.predict([[110]])[0], 1))

At 110 square metres: 270.0

The prediction is 50 + 2 × 110 = 270 thousand dollars. Its residual is actual minus predicted: 264 − 270 = −6. The negative sign says the sale is below the estimate; the absolute error is 6 thousand dollars. The intercept and slope stay fixed because changing a query does not refit the model.

Area alone cannot describe a house completely. With several features, linear regression adds one weighted contribution per feature plus the intercept. A possible teaching formula is price = 30 + 2 × area − 1 × age, with price in thousands of dollars, area in m² and age in years. These settings are supplied for illustration, not fitted or validated housing evidence.

A 100 m² house aged 10 years gets 30 + 200 − 10 = 220 thousand dollars under this formula. The area coefficient means the model raises its estimate by 2 thousand dollars for another square metre while the included age stays fixed. The age coefficient means another year lowers the estimate by 1 thousand dollars while area stays fixed.

Supply a Python prediction row such as [[100, 10]] only to a model trained with two columns in that same order. Reversing columns changes their meaning. Categorical features need preparation such as one-hot encoding; they are not automatically meaningful numeric quantities.

Closely related features can make individual coefficients unstable even when predictions appear reasonable. Ridge and Lasso add penalties to fitting that can limit coefficient size. They introduce settings to select through validation; they do not turn an unsuitable dataset into trustworthy evidence.

Predicting at 90 m² is interpolation: the query lies within the training range, 50–125 m². Predicting at 300 m² is extrapolation: the line extends beyond the observed inputs and returns 50 + 600 = 650 thousand dollars. The model can calculate that number, but the training examples do not establish that the same relationship continues so far.

Even interpolation is not a guarantee. A well-located 90 m² house and a damaged 90 m² house receive identical estimates here, because the model sees only area. The problem is missing information, not an arithmetic mistake.

Inspect residuals against features and predicted values. A curved pattern can suggest that the straight relationship misses structure; a widening spread can indicate different error sizes at different price levels. These are diagnostic clues, not automatic proofs of a particular cause. A useful model needs a suitable evaluation design and errors acceptable for its intended decision.

Unusual prices can pull a least-squares line, so review missing values and outliers with the domain context. For a larger workflow, see house-price prediction with a held-out baseline. Do not claim that the fitted area coefficient proves a causal effect of extending a house.

Linear regression predicts a quantity by combining an intercept with weighted features. Our house example used floor area to learn price = 50 + 2 × area, with prices in thousands of dollars.

• Read the units: a prediction of 230 means $230,000 in this example.
• Separate fitting from prediction: fit learns coefficients from known examples; predict applies them to new inputs.
• Inspect the gaps: a residual is actual minus predicted. Least squares fits by the total squared training residuals.
• Evaluate separately: held-out MAE describes average absolute misses. Compare with a baseline learned from training data.
• Respect the limits: an extrapolated number, a missing feature or a fitted coefficient needs careful interpretation.

Before moving on: calculate a 110 m² estimate, explain its residual when the actual sale is $264,000 and say why a perfect training fit cannot guarantee the next sale. The next lesson changes the question from predicting an amount to predicting a category with logistic regression.