Build a House Price Prediction Model with Regularized Linear Regression

A real estate underwriting team needs a price estimate for 15,000 homes, using tabular data like square footage, lot size, year built, neighborhood, and condition ratings. The output feeds a loan approval workflow, so the model cannot be a black box - an underwriter has to be able to look at a prediction and understand roughly why the model landed there.

Think of it like a real estate appraiser walking through a house with a checklist. They do not just average what similar houses sold for - they weigh square footage, adjust for a renovated kitchen, discount for a busy road, and combine everything into one number they can defend to a bank. A model that predicts price has to make the same kind of defensible, additive judgment call, except across tens of thousands of houses at once.

The naive approach is ordinary least squares (OLS) linear regression on every available feature. It looks fine on paper: it is interpretable, fast to train, and each coefficient has a clean “dollars per unit” meaning. The problem shows up the moment the feature set includes anything like square_footage, above_ground_sqft, and total_rooms together - these are highly correlated, and OLS responds by inflating some coefficients wildly in one direction and others in the opposite direction to fit training noise. The model looks reasonable on the training set and falls apart on new listings.

We need to solve for three things at once: keep the model additive and interpretable for underwriting, control coefficient variance under correlated features, and generalize to homes the model has not seen. Given a home’s structural, location, and condition features, predict its sale price, optimizing for RMSE on held-out sales, under the constraint that every prediction must be explainable as a sum of feature contributions.

Framing the Problem

This is a regression problem: the target is a continuous variable, sale price in dollars (or its log). The label is well-defined and comes directly from closed transaction records, so there is no ambiguity in what counts as “correct” - unlike a classification problem where the positive class boundary is a judgment call.

Constraints:

• Interpretability is mandatory - coefficients must be inspectable by a non-ML underwriter
• Training set is roughly 15,000 rows with 40-60 raw features after joining tax assessor and MLS data
• Inference is batch, not real-time, so latency is not a binding constraint
• No deep learning - the team’s compliance policy requires a model whose decision logic can be written down in a document

Out of scope for this post: image-based valuation (Zestimate-style computer vision on listing photos) and automated comparable-sale retrieval. We assume features are already extracted into a flat table.

Data and Features

The raw dataset is one row per sold property with columns like square_footage, lot_size, year_built, bedrooms, bathrooms, neighborhood, condition_rating, distance_to_downtown, and sale_date. Categorical columns like neighborhood need encoding, skewed numeric columns like lot_size benefit from a log transform, and everything needs scaling before regularization, since Ridge and Lasso penalize coefficient magnitude and unscaled features get penalized unfairly.

# Feature pipeline: encode categoricals, log-transform skewed numerics, scale everything
import numpy as np
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler, FunctionTransformer

numeric_features = ["square_footage", "year_built", "bedrooms", "bathrooms", "distance_to_downtown"]
skewed_features = ["lot_size"]
categorical_features = ["neighborhood", "condition_rating"]

log_transform = FunctionTransformer(np.log1p, feature_names_out="one-to-one")

preprocessor = ColumnTransformer(transformers=[
    ("num", StandardScaler(), numeric_features),
    ("skewed", Pipeline([("log", log_transform), ("scale", StandardScaler())]), skewed_features),
    ("cat", OneHotEncoder(handle_unknown="ignore", drop="first"), categorical_features),
])

The split must be time-aware, not random. Homes sold in the last three months should sit in the test set, mirroring how the model will actually be used - predicting future prices from past sales, never the reverse.

# Time-based split: never let future sales leak into training
df_sorted = df.sort_values("sale_date")
cutoff = int(len(df_sorted) * 0.85)
train_df = df_sorted.iloc[:cutoff]
test_df = df_sorted.iloc[cutoff:]

The Naive Baseline

Plain OLS on the full feature set is the baseline. It trains in milliseconds and gives every feature an unconstrained coefficient.

# Baseline: unregularized OLS on the full feature set
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error

pipe = Pipeline([("prep", preprocessor), ("model", LinearRegression())])
pipe.fit(train_df.drop(columns=["sale_price"]), train_df["sale_price"])
preds = pipe.predict(test_df.drop(columns=["sale_price"]))
baseline_rmse = mean_squared_error(test_df["sale_price"], preds, squared=False)
print(f"Baseline OLS RMSE: ${baseline_rmse:,.0f}")
# Baseline OLS RMSE: $58,400

The baseline RMSE of roughly $58,400 sounds usable until you inspect the coefficients: square_footage and above_ground_sqft (98% correlated in this dataset) get coefficients of +412 and -387 respectively. Both numbers are individually meaningless even though their sum is stable - which is exactly the interpretability requirement this model is supposed to satisfy, broken.

The Solution

Ridge regression fixes the correlated-feature problem by adding an L2 penalty on coefficient size, shrinking correlated coefficients toward each other instead of letting them cancel out unstably. Think of it like a budget cap on how much total “blame” the model is allowed to assign across all features combined - it forces the model to distribute credit more evenly across correlated features rather than betting everything on one of them fitting noise perfectly.

Lasso, with an L1 penalty, does something related but different: it drives some coefficients to exactly zero, which acts as automatic feature selection. For this problem we want both - shrink the correlated features toward stability, and drop the ones that are genuinely irrelevant. That is ElasticNet.

# Solution: ElasticNet with cross-validated alpha and l1_ratio
from sklearn.linear_model import ElasticNetCV

model = ElasticNetCV(
    l1_ratio=[0.1, 0.3, 0.5, 0.7, 0.9, 1.0],  # mix of L1 (Lasso) and L2 (Ridge)
    alphas=np.logspace(-3, 2, 50),
    cv=5,
    max_iter=10000,
    random_state=42,
)
pipe = Pipeline([("prep", preprocessor), ("model", model)])
pipe.fit(train_df.drop(columns=["sale_price"]), train_df["sale_price"])

print("Chosen alpha:", pipe.named_steps["model"].alpha_)
print("Chosen l1_ratio:", pipe.named_steps["model"].l1_ratio_)

alpha controls the overall penalty strength - too small and you get OLS’s instability back, too large and every coefficient shrinks toward zero and the model underfits. l1_ratio controls the L1/L2 mix. ElasticNetCV sweeps both via cross-validation instead of guessing, which matters here because the right mix depends on how much multicollinearity is actually in the feature set, not on intuition.

Evaluation and Metrics

RMSE is the headline metric because the loan workflow cares disproportionately about large misses. We also track MAE as a secondary, robustness-focused metric, and R² to communicate overall fit to non-technical stakeholders.

# Compute RMSE, MAE, and R^2 on the held-out time-based test set
from sklearn.metrics import mean_absolute_error, r2_score

preds = pipe.predict(test_df.drop(columns=["sale_price"]))
rmse = mean_squared_error(test_df["sale_price"], preds, squared=False)
mae = mean_absolute_error(test_df["sale_price"], preds)
r2 = r2_score(test_df["sale_price"], preds)
print(f"RMSE: ${rmse:,.0f}  MAE: ${mae:,.0f}  R2: {r2:.3f}")
# RMSE: $47,900  MAE: $31,200  R2: 0.881

ElasticNet cuts RMSE by roughly 18% over the OLS baseline and produces coefficients that no longer flip sign between retraining runs on slightly different data samples - the actual requirement, not just a better number.

Common Pitfalls and Failure Modes

Scaling and Productionizing

At 15,000 rows the whole pipeline retrains in under a second, so the production concern is not training time - it’s feature drift. Neighborhood price trends shift over months, so a model trained on last year’s sales silently degrades as the market moves.

Given:
  - Monthly retrain cadence
  - 15,000 historical rows, ~500 new closed sales per month
  - ElasticNetCV over 50 alphas x 6 l1_ratios x 5 folds = 1,500 fits

Training time: ~2s per fit x 1,500 fits / 8 parallel jobs = ~6 minutes per retrain
Monitoring signal: rolling 30-day RMSE on newly closed sales vs the trailing 90-day baseline
Alert threshold: rolling RMSE increase of >15% triggers a retrain outside the normal monthly cadence

The signal that catches silent degradation here is tracking RMSE on a rolling window of the most recently closed sales, compared against the model’s RMSE at training time. A widening gap means the market has moved and the coefficients are stale, even though nothing “broke” in the pipeline.

Alternative Approaches

For this problem, ElasticNet is the right call: the compliance requirement rules out black-box models outright, and 15,000 rows is not enough data for a neural net to reliably beat a well-regularized linear model anyway. If the interpretability constraint were lifted and the dataset were 10x larger, gradient boosting with SHAP explanations would be the natural next step.

Key Takeaways

• Multicollinearity, not just overfitting, is often the real reason OLS coefficients look unstable on correlated tabular features.
• ElasticNet combines Ridge’s coefficient stability with Lasso’s automatic feature selection, tunable via l1_ratio.
• Time-based splits are mandatory for any price or market-based regression problem - random splits leak future information.
• Target leakage through post-outcome fields is the single most common way a price model looks great in testing and fails in production.
• RMSE is the right primary metric when large errors carry outsized real-world cost, as they do in loan underwriting.
• Coefficient stability across retrains is a production metric in its own right for any model that needs to stay explainable over time.
• Cross-validation, not manual tuning against the test set, is how alpha and l1_ratio should be chosen.

The counter-intuitive lesson: the biggest accuracy win here didn’t come from a fancier model, it came from correctly diagnosing that the OLS baseline’s problem was multicollinearity, not a lack of model capacity - regularization is a targeted fix for that specific failure mode, not a generic accuracy booster.

Frequently Asked Questions

Q: Why not just drop the correlated features manually instead of using regularization? A: Manual feature dropping requires knowing in advance which correlated feature is “more real” - and that judgment can change across neighborhoods or property types. ElasticNet lets the data decide per-fold via cross-validation, and it degrades gracefully rather than requiring a human to get the choice right every retrain.

Q: Why not use gradient boosting for better accuracy? A: Gradient boosting would likely lower RMSE further, but it breaks the interpretability requirement - a compliance reviewer cannot easily audit a 500-tree ensemble’s reasoning for a specific denied loan the way they can audit a linear coefficient table.

Q: Why RMSE instead of MAE as the primary metric? A: RMSE penalizes large errors quadratically, which matches the actual cost structure of underwriting: a $150K miss is far more than 5x worse than a $30K miss in terms of loan risk, and MAE would treat them as linearly comparable.

Q: How do you know the model isn’t just memorizing the training neighborhoods? A: The time-based test split and the rolling-RMSE production monitor both check this - if the model only worked on training-period neighborhoods, test RMSE on unseen months and unseen listings would already show it before it reaches production.

Q: What happens when a brand-new neighborhood with no sales history appears? A: The one-hot encoder’s handle_unknown="ignore" setting means the new neighborhood category contributes zero rather than crashing, but that also means the model has no signal for it - in practice this needs a fallback to a broader geographic aggregate (e.g. city-level average) until enough sales accumulate.

Interview Questions

Q: Walk through why OLS coefficients become unstable under multicollinearity, mathematically. Expected depth: Discuss how the normal equations require inverting XᵀX, and how near-collinear columns make XᵀX close to singular, inflating the variance of the coefficient estimates even though predictions on the training set stay accurate.

Q: How would you choose between Ridge, Lasso, and ElasticNet for a given dataset? Expected depth: Ridge when you believe most features are relevant and want shrinkage without elimination; Lasso when you expect true sparsity and want automatic feature selection; ElasticNet when features are both correlated and you want sparsity, since pure Lasso handles correlated groups poorly by picking one arbitrarily.

Q: How would you detect that a deployed price model has gone stale due to market shift? Expected depth: Rolling-window RMSE comparison against training-time RMSE, distribution shift checks on input features (e.g. average square_footage or sale_price per neighborhood over time), and a defined retrain trigger threshold rather than a fixed calendar cadence alone.

Q: This dataset has 15,000 rows and 50+ features after one-hot encoding. Would you worry about overfitting, and how would ElasticNet address it? Expected depth: Discuss the effective degrees of freedom versus sample size, why regularization strength (alpha) directly controls the bias-variance tradeoff, and how cross-validated alpha selection avoids both under- and over-regularizing.

Q: How would this approach change if you needed predictions in real time instead of batch? Expected depth: Linear models are already fast enough for real-time serving (a single dot product), so the constraint shifts to feature computation latency - e.g. distance_to_downtown or neighborhood aggregates that require a lookup or geocoding call would need to be precomputed and cached rather than computed on request.