Understanding K-Nearest Neighbors (KNN) — A Simple Yet Powerful ML Algorithm

When learning machine learning, some algorithms feel like “magic.” But K-Nearest Neighbors (KNN) is different; it’s one of the most intuitive algorithms to understand.
It doesn’t build a complex model; instead, it relies on proximity and similarity, and the idea is that “data points close to each other are likely to belong to the same group.”

You can follow the series of supervised learning.

What is KNN?

K-Nearest Neighbors (KNN) is a supervised machine learning algorithm that can be used for both classification and regression tasks.

It works by finding the ‘K’ nearest data points (neighbors) to a given input and makes predictions based on:

• The majority class (for classification)

• The average value (for regression)

KNN is often called a lazy learner algorithm because it doesn’t learn from the data during training; instead, it stores the dataset and performs computations only when making predictions.

Intuition Behind KNN

Let’s make it simple:

Imagine you’re trying to classify a new fruit based on its size and sweetness. You already have data about other fruits — apples, strawberries, and ranges

If K = 5, the algorithm looks at the 5 nearest fruits to your new one. If 3 of them are oranges and 2 are strawberries, the majority is orange, so KNN will classify the new fruit as orange. That’s how KNN learns, not by rules, but by similarity.

Understanding “K” & the distance

Understanding the “K” in KNN

The value of K determines how many neighbors the algorithm will look at.

• A small K (like 1 or 2) makes the model sensitive to noise.

• A large K makes it more stable but can oversimplify the classification.

Choosing the right K is essential — often found using cross-validation.

How KNN Measures Distance

To find the “nearest” neighbors, we need a way to measure distance.
Common distance metrics include:

1. Euclidean Distance: The straight-line distance between two points.

2. Manhattan Distance: The sum of absolute differences (like walking on a city grid).

3. **Minkowski Distance**: A generalization that can become Euclidean or Manhattan depending on the value of p.

These distances define decision boundaries, often visualized through Voronoi diagrams, showing how space is divided among classes.

Step-by-Step Conceptual Flow

1. Choose a value for K

2. Compute the distance between the new point and all existing data points

3. Sort distances in ascending order

4. Pick the K nearest neighbors

5. Use majority voting (classification) or average (regression)

6. Assign the label/value to the new data point

Step-by-Step Implementation

Let’s implement KNN using the Iris dataset 🌸

# Step 1: Import Libraries
import pandas as pd
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score

# Step 2: Load Dataset
df = sns.load_dataset('iris')

# Step 3: Divide Features & Labels
X = df.iloc[:, 0:4]  # Features
y = df.iloc[:, -1]   # Target (Species)

# Step 4: Encode Labels (convert categories to numbers)
le = LabelEncoder()
y = le.fit_transform(y)

# Step 5: Split Data into Train & Test Sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=45)

# Step 6: Train Model
clf = KNeighborsClassifier(n_neighbors=5)
clf.fit(X_train, y_train)

# Step 7: Prediction & Accuracy
y_pred = clf.predict(X_test)
print("Accuracy:", accuracy_score(y_test, y_pred))

Visualization

If you want to see how KNN forms its decision boundaries, check lusertech.com → KNN Visualizer.
It beautifully shows how changing the value of K reshapes the classification regions.

Final Thoughts

KNN is one of those algorithms that reminds us:

Simplicity can be powerful.

You don’t always need a deep network to solve a problem; sometimes, it’s about understanding proximity and patterns in neighbors. As we know, algorithms are just one line away in Python, but the real learning happens when you understand how and when to use them.

That’s why I started writing my ML journey to decode not just the code, but the thinking behind it.
Here’s another piece in that journey, KNN, explained for those who want to truly understand Machine Learning, not just run it.