Lecture 2: k-Nearest Neighbors

From Similarity to Learned Compatibility

Overview

This lecture begins with practical decisions: predicting whether you will like a track and finding classmates who may work well with you. We use k-nearest neighbors to see how representation, distance, and evaluation turn proximity into a prediction.

Learning Objectives

By the end of this lecture, you will:

• Formulate a practical decision in terms of examples, features, a target, eligibility rules, and evidence of success
• Apply k-NN classification using a neighborhood, a voting rule, and a justified choice of \(k\)
• Explain how representation and distance encode the model’s proximity assumption
• Translate a human preference into a logical constraint and a loss for distance learning

Materials

TipQuick Access

Read the Lecture Notes · PDF version
Lecture Notebook
Open in Colab
Lecture Folder

Key Topics

1. Problem formulation: connect examples, features, targets, and evidence to the decision
2. k-NN classification: retrieve a neighborhood and aggregate its labels
3. Geometry: understand what each representation and distance assumes
4. Compatibility: use human preferences to learn a better distance
5. Whole-pipeline evaluation: locate failures in data, features, eligibility, distance, aggregation, or evidence

Additional Resources

• Scikit-learn k-NN Documentation

---

Previous: ← Lecture 1: Welcome to ML | Next: Lecture 3: Linear Regression →