Eigenvalues and Eigenvectors, Explained for Complete Beginners — The Final Blog in the Linear Algebra for ML Series

Take a normal photograph and stretch it sideways in an editing app — pull the left and right edges apart, like taffy.

Watch what happens to different arrows you could draw on that photo. An arrow pointing diagonally across the photo doesn't just get longer — it visibly tilts, leaning more towards the direction you're stretching. But an arrow that was already pointing perfectly horizontally? It just gets longer. It doesn't tilt at all. It was already pointing exactly the way the stretch was happening, so the stretch has nothing to rotate it towards — it can only make it longer.

That one, boring-sounding observation — "some directions don't rotate when you stretch something, they only get longer or shorter" — is the entire idea behind eigenvalues and eigenvectors, the two words in this series that probably sound the most intimidating, and are honestly among the simplest once you see them for what they really are.

This is the eighth and final blog in this Linear Algebra for Machine Learning series — starting all the way back from the very first introduction post on why Machine Learning even needs this math, through vectors, matrices, and every operation since. After this one, we're done building the math foundation, and the next series on this blog moves into actual Machine Learning itself — the training loops, the loss functions, the neural networks, the gradient descent — all the words we've been carefully setting aside for "a future blog," across the last seven posts. That future is now. But it made no sense to explain a car engine before explaining what a wheel does, so thank you for sticking with the foundations this far.

Before we dive in — one honest, important note.

A quick note from me: Looking back, I think some of the earlier blogs in this series leaned a bit too high-level for someone genuinely starting from zero. If you read the earlier posts in this series and hit a point where things stopped making sense, that's not on you — that's feedback I actually want. Please comment below exactly where you got lost. If enough of you tell me the same blogs or same sections felt too advanced, I will genuinely go back and rewrite them at a lower, more beginner-friendly level, the same way I've tried to do with this one. This series is only useful if it actually works for beginners, so please don't be shy about saying so.

Alright. Let's finish this series properly.

By the end of this blog, you will understand:

• What eigenvectors and eigenvalues actually are, using nothing more complicated than the stretched-photo idea above

• How to find both, fully by hand, using small, clean numbers

• Why this idea — "special directions that don't rotate" — turns out to be one of the most useful ideas in the whole of Computer Vision, search engines, and data science

• Real, concrete, low-level examples: how a technique built directly on this idea helped build some of the earliest face recognition systems, and how it's connected to how Google Search itself was originally ranked

Let's get into it, slowly, one small piece at a time.

---

Part 1: A Quick, Honest Recap (So Nobody Feels Lost)

In the last blog, we learned that multiplying a vector by a matrix is called a linear transformation — think of it as an instruction: "stretch this way, squash that way, rotate a bit." We used the picture of a rubber sheet with a grid drawn on it, being pulled, stretched, or squashed, while its straight lines stayed straight.

Here's the one thing you need to carry forward from that blog into this one:

A matrix is an instruction for moving points around in space. A vector is simply one point (or one arrow, pointing from the center outwards) that gets moved by that instruction.

Everything in today's blog is just asking one very specific, very useful question about that instruction: "Are there any arrows that this instruction doesn't bother rotating — arrows it can only stretch or shrink, but never tilt?"

If such an arrow exists for a particular matrix, it's called an eigenvector of that matrix. And the amount that arrow gets stretched or shrunk by — is it doubled? Tripled? Cut in half? — is called the matrix's eigenvalue, for that specific direction.

That's genuinely the entire definition. Let's now make it concrete with real numbers, so it stops feeling abstract.

---

Part 2: Seeing Eigenvectors With the Simplest Possible Example

The Easiest Case First

Let's start with the simplest matrix we can possibly use — one that only stretches things sideways and up-down, without any rotating or slanting at all:

Matrix A =    2    0
              0    3

Think about what this matrix does to any arrow you feed it. It doubles the arrow's left-right (horizontal) amount, and triples its up-down (vertical) amount. Nothing about this matrix mixes horizontal and vertical together — it treats them completely separately.

Now, take the simplest possible horizontal arrow: [1, 0] (meaning "1 step right, 0 steps up"). What happens when this matrix transforms it?

[2  0]   [1]     [2×1 + 0×0]     [2]
[0  3] × [0]  =  [0×1 + 3×0]  =  [0]

The result is [2, 0] — still pointing in exactly the same direction (purely horizontal), just twice as long. It didn't tilt even slightly. That makes [1, 0] an eigenvector of this matrix, and since it got exactly doubled, its eigenvalue is 2.

Now try the simplest vertical arrow, [0, 1]:

[2  0]   [0]     [2×0 + 0×1]     [0]
[0  3] × [1]  =  [0×1 + 3×1]  =  [3]

The result is [0, 3] — still purely vertical, just three times as long. Another eigenvector, this time with an eigenvalue of 3.

Notice something important: if you'd tried a diagonal arrow instead, like [1, 1] (one step right, one step up), you'd get [2, 3] — and that new arrow does not point in the same direction as [1, 1] anymore; it's visibly leaned. So [1, 1] is not an eigenvector of this matrix — only the purely horizontal and purely vertical directions survived without tilting, for this particular matrix.

This matches perfectly with the taffy-stretch photo example from the very start of this blog. This matrix stretches horizontally and vertically by different amounts, and only arrows already pointing exactly along those two directions avoid tilting.

A Slightly Trickier, More Realistic Example

Real matrices in Machine Learning are almost never this neatly "separated" — they usually mix horizontal and vertical together, which means their special, non-tilting directions aren't just the obvious up/down and left/right ones anymore. Let's work through one of these properly, fully by hand, with clean numbers.

Matrix A =    2    1
              1    2

Step 1: Finding the Eigenvalues

To find the eigenvalues, we use a specific equation, built directly from the definition "the arrow only gets stretched, not rotated":

A × v = λ × v

In plain English: "the matrix, applied to our special arrow v, gives exactly the same result as just stretching v by some plain number λ" (that Greek letter, pronounced "lambda," is just the traditional name mathematicians use for "the eigenvalue" — don't let the unfamiliar symbol distract you, it's just a stand-in for a plain number, like x in school algebra).

Through some straightforward algebra (which we won't fully unpack here, since it's more useful to focus on using the result than deriving it), this equation reduces to solving:

determinant(A − λ × Identity matrix) = 0

You already know how to calculate a determinant from Blog 6 — nothing new there. We're just calculating it for a slightly modified version of our matrix, where we've subtracted λ from every number along the diagonal:

A − λI  =    (2−λ)      1
                1      (2−λ)

Setting its determinant to zero:

(2−λ)(2−λ) − (1×1) = 0

Let's expand this out slowly, one step at a time:

(2−λ)(2−λ) = 4 − 4λ + λ²

So:  4 − 4λ + λ² − 1 = 0
     λ² − 4λ + 3 = 0

This is now just an ordinary school-level quadratic equation. Factoring it:

(λ − 1)(λ − 3) = 0

Which gives us two possible eigenvalues:

λ = 1   or   λ = 3

This matrix has two special stretch amounts — 1 and 3 — and correspondingly, two special directions that only get stretched by exactly those amounts, without any tilting. Let's find those directions next.

Step 2: Finding the Eigenvectors, One Eigenvalue at a Time

For λ = 3:

We go back to A − λI, this time plugging in λ = 3:

A − 3I  =    (2−3)      1        =    −1    1
                1      (2−3)             1   −1

We want to find an arrow v (with components we'll call x and y) such that this matrix, applied to v, gives exactly [0, 0]:

−1x + 1y = 0     →     y = x

So any arrow where y equals x works — the simplest choice is x = 1, y = 1, giving us the eigenvector [1, 1].

Let's sanity-check this makes sense: does [1, 1], multiplied by our original matrix A, really just get stretched by exactly 3, without tilting?

[2  1]   [1]     [2×1 + 1×1]     [3]
[1  2] × [1]  =  [1×1 + 2×1]  =  [3]

[3, 3] is exactly [1, 1] stretched by a factor of 3, pointing in the exact same direction. Confirmed.

For λ = 1:

A − 1I  =    (2−1)      1        =    1    1
                1      (2−1)             1    1

1x + 1y = 0     →     y = −x

The simplest choice here is x = 1, y = −1, giving us the eigenvector [1, −1].

Sanity-checking:

[2  1]   [1]      [2×1 + 1×(−1)]     [1]
[1  2] × [−1]  =  [1×1 + 2×(−1)]  =  [−1]

[1, −1], stretched by a factor of exactly 1 — meaning it doesn't grow or shrink at all, it stays exactly the same length, only confirming that this specific direction is untouched in terms of size. Confirmed again.

So, for this matrix, there are exactly two special directions — pointing diagonally as [1, 1] and [1, −1] — that never tilt, no matter how this specific transformation is applied. Every other direction you could pick will visibly lean towards one of these two special directions when transformed.

Python Code — Built Entirely From Scratch

Everything we just did by hand — building the A − λI matrix, calculating its determinant, setting that determinant to zero, solving for λ — can be turned into real, working Python code, without relying on any built-in shortcut function. Let's build it up piece by piece, and I'll explain every single line, because there's genuinely nothing here you can't follow if you take it slowly.

The very first piece we need is something you actually already built back in Blog 7 — a function that calculates the determinant of a matrix of any size, from scratch:

import numpy as np

def det(a):
    if len(a) == 1:
        return a[0][0]
    if len(a) == 2:
        return a[0][0] * a[1][1] - a[0][1] * a[1][0]

    result = 0
    for col in range(len(a)):
        minor = [
            [a[row][c] for c in range(len(a)) if c != col]
            for row in range(1, len(a))
        ]
        result += ((-1) ** col) * a[0][col] * det(np.array(minor))

    return result

Let's go through this one line at a time, slowly, assuming you're seeing it for the first time.

• import numpy as np — we're bringing in NumPy again, purely so we can conveniently work with matrices as np.array objects, the same way we have throughout this whole series.

• def det(a): — this defines a function named det, which takes one input, a — the matrix we want to find the determinant of.

• if len(a) == 1: return a[0][0] — this is our simplest possible case. If the matrix is just a single number sitting inside a single row (a 1x1 matrix), its determinant is simply that one number itself. a[0][0] means "the number in the very first row, very first column" — with a 1x1 matrix, that's the only number there is.

• if len(a) == 2: return a[0][0] * a[1][1] - a[0][1] * a[1][0] — this is the direct 2x2 determinant formula you learned in Blog 6: multiply the top-left and bottom-right numbers together, then subtract the product of the top-right and bottom-left numbers. a[0][0] is row 0, column 0 (top-left). a[1][1] is row 1, column 1 (bottom-right). a[0][1] is top-right, a[1][0] is bottom-left.

• result = 0 — for anything bigger than a 2x2 matrix, we can't use one simple formula directly. Instead, we're going to build up the answer piece by piece, so we start with an empty running total of zero, exactly like we did with the running total in matrix multiplication back in Blog 5.

• for col in range(len(a)): — this walks across every column position in the very first row of the matrix, one at a time. col will take the values 0, 1, 2, and so on, up to however many columns the matrix has.

• minor = [ [a[row][c] for c in range(len(a)) if c != col] for row in range(1, len(a)) ] — this is the densest line in the whole function, so let's unpack it fully. We're building a brand new, smaller matrix called minor, by taking the original matrix a, removing its entire first row, and removing whichever column col we're currently looking at. The inner part, for c in range(len(a)) if c != col, walks through every column index c, but skips it entirely (if c != col) whenever c matches the column we're currently removing. The outer part, for row in range(1, len(a)), starts counting rows from 1 instead of 0 — meaning we're deliberately skipping the very first row of the original matrix. Put together, this single line builds exactly the smaller matrix left over once you cross out one row and one column — precisely the "minor" matrix from the cofactor-expansion idea in Blog 6.

• result += ((-1) ** col) * a[0][col] * det(np.array(minor)) — this is where the function calls itself, on the smaller minor matrix we just built. This is called recursion — think of it like a set of Russian nesting dolls, where each doll contains a smaller doll of the exact same shape, until you reach the smallest one that can't be opened any further (our len(a) == 1 and len(a) == 2 base cases from earlier). The ((-1) ** col) part simply flips the sign back and forth (positive, negative, positive, negative...) as col increases — a specific, fixed detail of how this method is mathematically defined, which you don't need to derive yourself, only trust and reuse. a[0][col] is the actual number, from the very first row, that we're currently "expanding" around. Multiplying all three pieces together, and adding the result onto our running total, is exactly how the full determinant gets built up out of smaller and smaller pieces.

• return result — once we've walked across every column in the first row, and added up every one of those smaller pieces, result holds the complete, correct determinant of the original matrix, and we hand it back to whoever called this function.

Using det() to Actually Find the Eigenvalues, By Search

Here's the genuinely elegant part. We already know, from Part 2 above, that an eigenvalue is exactly the number λ that makes determinant(A − λI) equal to zero. So instead of doing the algebra by hand every single time (factoring quadratics, and worse, for bigger matrices), we can use our det() function as a simple detector — and just try out a large number of candidate λ values, watching closely for the exact points where the determinant crosses from positive to negative (or the other way round). That crossing point is, very closely, an eigenvalue.

def find_eigenvalues_by_scanning(A, low=-10, high=10, step=0.001):
    n = len(A)
    identity = np.eye(n)
    found = []
    prev_sign = None
    lam = low

    while lam <= high:
        test_matrix = A - lam * identity
        value = det(test_matrix)
        current_sign = value > 0

        if prev_sign is not None and current_sign != prev_sign:
            found.append(round(lam, 2))

        prev_sign = current_sign
        lam += step

    return found


A = np.array([[2, 1],
              [1, 2]])

print(find_eigenvalues_by_scanning(A))

[1.0, 3.0]

Let's slowly unpack this one too.

• def find_eigenvalues_by_scanning(A, low=-10, high=10, step=0.001): — this defines our new function, which takes in the matrix A, plus three optional settings: low and high (the range of candidate λ values we're willing to test — here, anywhere from −10 to 10), and step (how finely we walk through that range — here, in tiny increments of 0.001).

• n = len(A) and identity = np.eye(n) — n is simply the size of the matrix (2, in our example). np.eye(n) builds the identity matrix of that same size — remember, from Blog 6, this is the special matrix full of 1s along the diagonal and 0s everywhere else, representing "do nothing." We need this so we can build A − λI for each candidate λ.

• found = [] — an empty list, ready to collect any eigenvalues we discover as we search.

• prev_sign = None — before we've tested any candidate λ at all, we don't yet know if the determinant was positive or negative "last time," so we start this as None (meaning "nothing yet").

• lam = low — we start our search at the smallest candidate value.

• while lam <= high: — we keep testing candidate values of λ, one at a time, until we've covered the entire range from low up to high.

• test_matrix = A - lam * identity — this builds exactly the A − λI matrix from our hand calculation earlier, just using whatever candidate lam we're currently testing.

• value = det(test_matrix) — here's where our earlier det() function gets reused directly — we calculate the determinant of this test matrix, exactly the same function, no changes needed.

• current_sign = value > 0 — this simply records whether the determinant came out positive (True) or negative or zero (False) for this particular candidate λ.

• if prev_sign is not None and current_sign != prev_sign: — this is the actual detective work. If the determinant's sign just flipped compared to the very last candidate value we tried (say, it was positive a moment ago, and it's now negative), that means the determinant must have passed through exactly zero somewhere in between — and a determinant of zero is, precisely, an eigenvalue. We record the current lam as an approximate eigenvalue whenever we spot this flip.

• prev_sign = current_sign and lam += step — we save the current sign for next time, then nudge lam forward by our small step amount, and repeat the whole process again.

• return found — once we've covered the entire range from low to high, we hand back the full list of approximate eigenvalues we discovered along the way.

Running this on our matrix [[2, 1], [1, 2]] gives us back [1.0, 3.0] — exactly matching the two eigenvalues, 1 and 3, that we calculated fully by hand earlier in this blog. We didn't factor a single quadratic equation this time. We simply reused the determinant function you already understand, and let the computer try thousands of candidate numbers until it found where the determinant hit zero.

Finding the Matching Eigenvectors, From Scratch

Once we know an eigenvalue, we still need its matching eigenvector — the actual direction that gets stretched by exactly that amount. For a 2x2 matrix specifically, there's a genuinely simple trick we can turn into code, based directly on what we did by hand earlier: take the first row of A − λI (call its two numbers a and b), and the direction [−b, a] will always be a valid eigenvector for that eigenvalue.

def find_eigenvector_2x2(A, eigenvalue):
    a = A[0][0] - eigenvalue
    b = A[0][1]

    if a == 0 and b == 0:
        a = A[1][0]
        b = A[1][1] - eigenvalue

    return np.array([-b, a])


for lam in [1, 3]:
    v = find_eigenvector_2x2(A, lam)
    print(f"Eigenvalue {lam} -> Eigenvector {v}")

Eigenvalue 1 -> Eigenvector [-1  1]
Eigenvalue 3 -> Eigenvector [-1 -1]

• def find_eigenvector_2x2(A, eigenvalue): — this function takes in our matrix A, plus one specific eigenvalue we already found, and will hand back its matching eigenvector.

• a = A[0][0] - eigenvalue and b = A[0][1] — these two lines build exactly the first row of A − λI, the same subtraction we did by hand earlier in this blog.

• if a == 0 and b == 0: — very occasionally, the first row happens to become entirely zero for a particular matrix and eigenvalue combination, which wouldn't give us any useful direction to work with. In that rare case, we simply fall back to using the second row of A − λI instead, calculated the same way.

• return np.array([-b, a]) — this is the actual trick. If a and b are the two numbers in a row that must satisfy a×x + b×y = 0 (exactly the equation we solved by hand earlier), then plugging in x = −b and y = a always makes that equation true, because a×(−b) + b×a always cancels out to exactly zero, no matter what a and b are. That's genuinely all this line is doing — a small, reliable shortcut for solving that specific kind of equation, for a 2x2 matrix.

Look closely at the output — [-1, 1] for eigenvalue 1, and [-1, -1] for eigenvalue 3. These point in exactly the same directions as our hand-calculated [1, -1] and [1, 1] from earlier (just flipped in sign, which makes no difference — an arrow pointing "up-left" represents the exact same direction and line as one pointing "down-right"; eigenvectors are only ever defined up to this kind of sign flip or rescaling). Our from-scratch code, built on nothing but the determinant function you already understood, has independently rediscovered the exact same answer we worked out by hand.

Doing This in Python (Verifying, Instantly, What We Just Did by Hand and by Scratch)

import numpy as np

A = np.array([[2, 1],
              [1, 2]])

eigenvalues, eigenvectors = np.linalg.eig(A)

print("Eigenvalues:", eigenvalues)
print("Eigenvectors:\n", eigenvectors)

Eigenvalues: [3. 1.]
Eigenvectors:
 [[ 0.707 -0.707]
  [ 0.707  0.707]]

Notice NumPy found exactly the eigenvalues we calculated by hand: 3 and 1. The eigenvectors look a little different from our clean [1, 1] and [1, -1] — that's simply because NumPy prefers to report these special directions in a "unit length" form (meaning the arrow is scaled to have a length of exactly 1), rather than the simplest whole numbers we chose. If you check the direction — the ratio between the two numbers in each column — you'll find it matches our hand-calculated [1, 1] and [1, -1] directions exactly, just represented at a different, standardized length. The underlying special direction is identical either way.

This single function, np.linalg.eig(), does — in a tiny fraction of a second, even for enormous, real-world matrices with thousands of rows and columns — exactly the process we just walked through, twice over: by hand, and then again from scratch in code, solving that quadratic-style equation and finding the matching direction for each solution. In real projects, you'll always use np.linalg.eig() — it's faster and more numerically careful than our scanning approach — but now you genuinely know there's no hidden magic inside it.

---

Part 3: Why This Matters — Real, Concrete Examples

1. Eigenfaces — One of the Earliest Real Face Recognition Techniques (Computer Vision)

Here's a genuinely fascinating, low-level story, and it directly connects everything you've learned across this whole series.

Imagine you collect thousands of photos of different human faces, all cropped and aligned the same way (same size, eyes roughly in the same position). Each photo, as you learned back in Blog 5, is really just a big matrix of pixel brightness numbers. If you gather thousands of these face-matrices together and ask a very specific mathematical question — "across all these different faces, what are the handful of 'directions' along which faces vary from each other the most?" — the answer comes directly from finding the eigenvectors of a matrix built from all those combined faces (specifically, a matrix that captures how the brightness at every pixel position tends to vary together, called a covariance matrix — don't worry about that name for now, just know it's built directly from the face photos themselves).

Each one of these special eigenvector directions, when you actually convert it back into an image, looks like an eerie, ghost-like, blurry face — genuinely, researchers nicknamed these "Eigenfaces." One ghost-face might capture "how wide is this face compared to average." Another might capture "how much facial hair is present." You don't manually decide what each ghost-face represents — the eigenvectors reveal these patterns purely from the numbers themselves, without anyone explicitly programming in the concept of "width" or "facial hair."

Here's the genuinely useful part: instead of storing or comparing a full, enormous grid of thousands of pixel numbers per face, an early face recognition system could instead describe any face using just a handful of numbers — "how much of ghost-face #1 is present in this photo, how much of ghost-face #2, how much of ghost-face #3," and so on. A face that might have started as tens of thousands of pixel numbers could now be described using perhaps just 50 or 100 numbers — a massive compression — while still capturing almost everything important needed to tell one face apart from another. This is directly, historically, one of the earliest practical face recognition techniques ever built, and it comes straight from eigenvectors.

2. Google Search's Original Ranking Algorithm (PageRank)

Here's a fact that should genuinely surprise you, and it has nothing to do with photos at all.

In the very early days of Google, the company needed a way to rank billions of webpages by "importance," so that searching for something returned the most genuinely useful pages first, not just pages that happened to repeat your search words the most times (which older search engines were easily tricked by).

Google's original solution, called PageRank, represented the entire internet as one enormous matrix — imagine a giant table where every row and column represents one webpage, and a number in a particular cell represents "does this page link to that page." Finding the "most important" pages, according to this method, comes down to finding one very special eigenvector of this giant matrix — specifically, the direction that represents a stable, settled-down sense of "how much overall importance flows into each page," once you imagine link-importance being passed around the web, over and over, indefinitely.

The genuinely surprising part: an idea that felt purely mathematical and abstract — "which arrow doesn't rotate when I apply this instruction" — turned out to be exactly the right tool to help build one of the most economically significant pieces of software in human history. That's honestly one of the best real-world reminders of just how far a "boring-sounding" mathematical idea can travel once someone finds the right problem to point it at.

3. Simplifying Enormous, Messy, Real-World Datasets (Data Science, and a Preview of the ML Series Ahead)

Beyond faces and search engines, eigenvectors are the mathematical engine behind a hugely important, very commonly used technique in data science called Principal Component Analysis, or PCA for short. We are deliberately not diving deep into PCA itself in this blog — it fully deserves its own dedicated post, and it's one of the very first topics we'll properly unpack in the upcoming Machine Learning series, now that you have the eigenvector foundation needed to actually understand why it works, rather than just memorizing the steps.

For now, here's the one-sentence preview worth carrying forward: PCA uses eigenvectors to find the small number of "most important directions" hiding inside a large, messy dataset — similar to how Eigenfaces found the most important "directions" hiding inside thousands of face photos — letting data scientists shrink enormous datasets down to their essential patterns, without throwing away what actually matters.

---

Part 4: What This Series Didn't Cover — And Why

Before we close this out, I want to be upfront about two names you may already have heard floating around — Singular Value Decomposition (SVD) and Principal Component Analysis (PCA) — and address directly why a linear algebra series wouldn't be complete without at least acknowledging them.

Here's the honest reason they aren't a full blog of their own in this series, and it isn't because they're unimportant — quite the opposite.

Everything we did in this blog — finding eigenvalues and eigenvectors — only works directly on square matrices (the same number of rows as columns). But most real datasets are nowhere near square. Imagine a dataset of 10,000 customer photos, each one flattened into 50,000 pixel-brightness numbers. That's a matrix with 10,000 rows and 50,000 columns — nowhere close to square, and eigenvalues, exactly as we defined them today, simply don't apply to a matrix shaped like that.

To handle this far more common, rectangular case, mathematicians generalized the entire idea you just learned into something more powerful, called Singular Value Decomposition. Rather than looking for special directions a single matrix stretches without rotating, SVD breaks any matrix — square or not, however oddly shaped — into three separate pieces: one matrix capturing special directions in the "input" space, one capturing special directions in the "output" space, and a simple diagonal matrix of "stretch amounts" (called singular values) connecting the two. If eigen-decomposition is the special case of measuring a transformation against itself, SVD is the general tool for measuring any transformation, between any two shapes of space, at all. In practice, most real-world implementations of PCA don't even calculate eigenvectors of a covariance matrix directly (the way our earlier Eigenfaces story described) — they compute an SVD instead, purely because it tends to be more numerically stable on real, messy, large-scale data.

Principal Component Analysis itself, then, is really the practical technique built on top of this exact idea — using either eigen-decomposition or SVD (in practice, almost always SVD) to find the small number of directions along which a dataset varies the most, and re-describing the entire dataset using just those directions, discarding the rest as comparatively unimportant noise. It's the single most commonly used dimensionality-reduction technique in applied Machine Learning today, sitting quietly behind everything from compressing sensor data, to visualizing high-dimensional datasets in two dimensions, to speeding up training by feeding a model a smaller, more meaningful version of its original features.

I made a deliberate choice not to cram SVD and PCA into this final post. This entire series was built on a promise, right from blog one, that nothing gets explained before its foundation is properly in place — and SVD genuinely deserves that same treatment: real, hand-worked numbers, real code built from scratch, and real low-level examples, the same standard I've tried to hold every post in this series to, rather than a rushed, high-level paragraph bolted onto the end of an already-long final blog. Rather than doing that here, I'll be covering SVD and PCA properly, the exact same hands-on way, at the natural moment they belong — when the upcoming Machine Learning series reaches dimensionality reduction and recommendation systems, where these two ideas genuinely earn their place instead of being introduced for their own sake. Consider this paragraph a marker, not a dodge — I know exactly where these two ideas fit into the bigger picture, and we'll do them justice when we get there.

---

Part 5: The Whole Series, Looking Back

You've genuinely covered a complete, real foundation in linear algebra across this whole series. Let's see the whole journey, laid out in one place, one final time. (Number these to match however you've ordered them on your own blog — what matters here is the order the ideas build on each other, not the exact digit.)

Order	What you learned
Introduction	Why Machine Learning needs linear algebra at all
Vectors	What a vector is — one example, described as numbers
Matrices	What a matrix is — many vectors, organized into a table
Addition, subtraction & element-wise multiplication	Combining, comparing, and masking data
Matrix multiplication, transpose & cross product	Weighing, combining, and reorienting data
Linear transformation, determinant & matrix inverse	What a matrix does to data, and whether it can be undone
Eigenvalues & eigenvectors (this one)	The special directions a transformation only stretches, never rotates

That's the complete toolkit. Genuinely, everything from here — every neural network layer, every training loop, every "the model learned a representation" sentence you'll read in the Machine Learning series ahead — is built directly on top of these ideas, just applied at a much bigger scale, with the numbers chosen automatically by the model itself, rather than by us, by hand, on paper.

---

This Is the End of the Linear Algebra Series — Here's What's Next

This is officially the final blog in the Linear Algebra for Machine Learning series. Thank you, genuinely, for sticking with this all the way from "why does ML even need this math" through to eigenvectors. That's not a small journey, and if you followed it start to finish, you now have a real, working mathematical foundation that a lot of people jump straight past when they start learning Machine Learning — and end up confused later because of it.

Starting with the next blog, this space moves into the actual Machine Learning series — and every single "don't worry, future blog" moment scattered across this series is about to get paid off, properly. Gradient descent. Loss functions. What "training" a model actually means, step by step. What a neural network layer really is, now that you know exactly what a linear transformation is. We're going to build every one of those ideas directly on top of everything you've learned across this series.

And before you go — one more time, genuinely, please don't skip this part:

If any part of the earlier posts in this series felt too high-level, too fast, or assumed things you didn't actually know yet — please say so in the comments below, and be specific about where it happened. This series is meant to work for complete beginners, and the only way I'll know if it actually succeeded is if you tell me honestly. If enough of you point to the same rough spots, I will go back and rewrite those posts properly, at a lower level, the same way I've tried to do starting with this last stretch of blogs. Your comments genuinely decide whether that rewrite happens — so don't hold back.

See you in the Machine Learning series.