Lecture 3: Linear Regression
The Universal Pattern Finder - From Simple Lines to Complex Curves
Overview
Linear regression is far more powerful than its name suggests. In this hands-on lecture, we’ll discover how the same algorithm that fits straight lines can capture complex curved relationships in real-world data. Through interactive experiments with automotive engineering and weather forecasting datasets, you’ll master polynomial features, understand the “lifting technique,” and learn when models become too complex.
Learning Objectives
By the end of this lecture, you will:
- Understand linear regression model, parameters, and MSE loss function
- Master polynomial features through hands-on interaction
- Apply regression to real problems (automotive MPG, weather bias correction)
- Visualize the “lifting technique” that transforms curves into hyperplanes
- Recognize overfitting and understand model complexity trade-offs
- Engineer domain-specific features that outperform blind polynomial expansion
Materials
Complete the Environment Setup Guide.
Datasets & Acknowledgments
Real-World Datasets Used
Auto MPG Dataset: 398 vehicles from 1970-1982
UCI Machine Learning Repository, accessed via MLxtendECMWF Weather Data: 5.2M observations from 8,000+ weather stations
European Centre for Medium-Range Weather Forecasts, via ClimeLab (Apache 2.0)
Previous: ← Lecture 2: k-NN | Next: Lecture 4: Gradient Descent and Optimization →