{
  "source_file": "0930",
  "source_platform": "Hashnode",
  "author": "Tharun",
  "title_listed": "Eigenvalues and Eigenvectors, Explained for Complete Beginners — The Final Blog in the Linear Algebra for ML Series",
  "keywords": "",
  "domain": "文本处理",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://tharunpandya.hashnode.dev/eigenvalues-and-eigenvectors-explained-for-complete-beginners-the-final-blog-in-the-linear-algebra-for-ml-series",
  "publish_date": "2026-07-24",
  "corpus_stage": "",
  "notes": "",
  "id": "14f06e3d9719",
  "collected_at": "2026-10-02T14:10:45",
  "title": "Eigenvalues and Eigenvectors, Explained for Complete Beginners — The Final Blog in the Linear Algebra for ML Series",
  "body_markdown": "Eigenvalues and Eigenvectors, Explained for Complete Beginners — The Final Blog in the Linear Algebra for ML Series\nUpdated\nAugust 8, 2026\n•\n30\nmin read\n•\nView as Markdown\nT\nTharun\nTake a normal photograph and stretch it sideways in an editing app — pull the left and right edges apart, like taffy.\nWatch what happens to different arrows you could draw on that photo. An arrow pointing diagonally across the photo doesn't just get longer — it visibly\ntilts\n, leaning more towards the direction you're stretching. But an arrow that was already pointing perfectly horizontally? It just gets longer. It doesn't tilt at all. It was already pointing exactly the way the stretch was happening, so the stretch has nothing to rotate it towards — it can only make it longer.\nThat one, boring-sounding observation — \"some directions don't rotate when you stretch something, they only get longer or shorter\" — is the entire idea behind\neigenvalues\nand\neigenvectors\n, the two words in this series that probably sound the most intimidating, and are honestly among the simplest once you see them for what they really are.\nThis is the\neighth and final blog in this Linear Algebra for Machine Learning series\n— starting all the way back from the very first introduction post on why Machine Learning even needs this math, through vectors, matrices, and every operation since. After this one, we're done building the math foundation, and the next series on this blog moves into actual Machine Learning itself — the training loops, the loss functions, the neural networks, the gradient descent — all the words we've been carefully setting aside for \"a future blog,\" across the last seven posts. That future is now. But it made no sense to explain a car engine before explaining what a wheel does, so thank you for sticking with the foundations this far.\nBefore we dive in — one honest, important note.\nA quick note from me:\nLooking back, I think some of the earlier blogs in this series leaned a bit too high-level for someone genuinely starting from zero. If you read the earlier posts in this series and hit a point where things stopped making sense, that's not on you — that's feedback I actually want.\nPlease comment below exactly where you got lost.\nIf enough of you tell me the same blogs or same sections felt too advanced, I will genuinely go back and rewrite them at a lower, more beginner-friendly level, the same way I've tried to do with this one. This series is only useful if it actually works for beginners, so please don't be shy about saying so.\nAlright. Let's finish this series properly.\nBy the end of this blog, you will understand:\nWhat eigenvectors and eigenvalues actually are, using nothing more complicated than the stretched-photo idea above\nHow to find both, fully by hand, using small, clean numbers\nWhy this idea — \"special directions that don't rotate\" — turns out to be one of the most useful ideas in the whole of Computer Vision, search engines, and data science\nReal, concrete, low-level examples: how a technique built directly on this idea helped build some of the earliest face recognition systems, and how it's connected to how Google Search itself was originally ranked\nLet's get into it, slowly, one small piece at a time.\nPart 1: A Quick, Honest Recap (So Nobody Feels Lost)\nIn the last blog, we learned that multiplying a vector by a matrix is called a\nlinear transformation\n— think of it as an instruction: \"stretch this way, squash that way, rotate a bit.\" We used the picture of a rubber sheet with a grid drawn on it, being pulled, stretched, or squashed, while its straight lines stayed straight.\nHere's the one thing you need to carry forward from that blog into this one:\nA matrix is an instruction for moving points around in space. A vector is simply one point (or one arrow, pointing from the center outwards) that gets moved by that instruction.\nEverything in today's blog is just asking one very specific, very useful question about that instruction:\n\"Are there any arrows that this instruction doesn't bother rotating — arrows it can only stretch or shrink, but never tilt?\"\nIf such an arrow exists for a particular matrix, it's called an\neigenvector\nof that matrix. And the amount that arrow gets stretched or shrunk by — is it doubled? Tripled? Cut in half? — is called the matrix's\neigenvalue\n, for that specific direction.\nThat's genuinely the entire definition. Let's now make it concrete with real numbers, so it stops feeling abstract.\nPart 2: Seeing Eigenvectors With the Simplest Possible Example\nThe Easiest Case First\nLet's start with the simplest matrix we can possibly use — one that only stretches things sideways and up-down, without any rotating or slanting at all:\nMatrix A =    2    0\n              0    3\nThink about what this matrix does to any arrow you feed it. It doubles the arrow's left-right (horizontal) amount, and triples its up-down (vertical) amount. Nothing about this matrix mixes horizontal and vertical together — it treats them completely separately.\nNow, take the simplest possible horizontal arrow:\n[1, 0]\n(meaning \"1 step right, 0 steps up\"). What happens when this matrix transforms it?\n[2  0]   [1]     [2×1 + 0×0]     [2]\n[0  3] × [0]  =  [0×1 + 3×0]  =  [0]\nThe result is\n[2, 0]\n— still pointing in exactly the same direction (purely horizontal), just twice as long. It didn't tilt even slightly. That makes\n[1, 0]\nan\neigenvector\nof this matrix, and since it got exactly doubled, its\neigenvalue\nis\n2\n.\nNow try the simplest vertical arrow,\n[0, 1]\n:\n[2  0]   [0]     [2×0 + 0×1]     [0]\n[0  3] × [1]  =  [0×1 + 3×1]  =  [3]\nThe result is\n[0, 3]\n— still purely vertical, just three times as long. Another eigenvector, this time with an eigenvalue of\n3\n.\nNotice something important: if you'd tried a diagonal arrow instead, like\n[1, 1]\n(one step right, one step up), you'd get\n[2, 3]\n— and that new arrow\ndoes not\npoint in the same direction as\n[1, 1]\nanymore; it's visibly leaned. So\n[1, 1]\nis\nnot\nan eigenvector of this matrix — only the purely horizontal and purely vertical directions survived without tilting, for this particular matrix.\nThis matches perfectly with the taffy-stretch photo example from the very start of this blog. This matrix stretches horizontally and vertically by different amounts, and only arrows already pointing exactly along those two directions avoid tilting.\nA Slightly Trickier, More Realistic Example\nReal matrices in Machine Learning are almost never this neatly \"separated\" — they usually mix horizontal and vertical together, which means their special, non-tilting directions aren't just the obvious up/down and left/right ones anymore. Let's work through one of these properly, fully by hand, with clean numbers.\nMatrix A =    2    1\n              1    2\nStep 1: Finding the Eigenvalues\nTo find the eigenvalues, we use a specific equation, built directly from the definition \"the arrow only gets stretched, not rotated\":\nA × v = λ × v\nIn plain English: \"the matrix, applied to our special arrow\nv\n, gives exactly the same result as just stretching\nv\nby some plain number\nλ\n\" (that Greek letter, pronounced \"lambda,\" is just the traditional name mathematicians use for \"the eigenvalue\" — don't let the unfamiliar symbol distract you, it's just a stand-in for a plain number, like\nx\nin school algebra).\nThrough some straightforward algebra (which we won't fully unpack here, since it's more useful to focus on\nusing\nthe result than deriving it), this equation reduces to solving:\ndeterminant(A − λ × Identity matrix) = 0\nYou already know how to calculate a determinant from Blog 6 — nothing new there. We're just calculating it for a slightly modified version of our matrix, where we've subtracted\nλ\nfrom every number along the diagonal:\nA − λI  =    (2−λ)      1\n                1      (2−λ)\nSetting its determinant to zero:\n(2−λ)(2−λ) − (1×1) = 0\nLet's expand this out slowly, one step at a time:\n(2−λ)(2−λ) = 4 − 4λ + λ²\n\nSo:  4 − 4λ + λ² − 1 = 0\n     λ² − 4λ + 3 = 0\nThis is now just an ordinary school-level quadratic equation. Factoring it:\n(λ − 1)(λ − 3) = 0\nWhich gives us two possible eigenvalues:\nλ = 1   or   λ = 3\nThis matrix has\ntwo\nspecial stretch amounts —\n1\nand\n3\n— and correspondingly, two special directions that only get stretched by exactly those amounts, without any tilting. Let's find those directions next.\nStep 2: Finding the Eigenvectors, One Eigenvalue at a Time\nFor λ = 3:\nWe go back to\nA − λI\n, this time plugging in\nλ = 3\n:\nA − 3I  =    (2−3)      1        =    −1    1\n                1      (2−3)             1   −1\nWe want to find an arrow\nv\n(with components we'll call\nx\nand\ny\n) such that this matrix, applied to\nv\n, gives exactly\n[0, 0]\n:\n−1x + 1y = 0     →     y = x\nSo any arrow where\ny\nequals\nx\nworks — the simplest choice is\nx = 1, y = 1\n, giving us the eigenvector\n[1, 1]\n.\nLet's sanity-check this makes sense: does\n[1, 1]\n, multiplied by our original matrix\nA\n, really just get stretched by exactly 3, without tilting?\n[2  1]   [1]     [2×1 + 1×1]     [3]\n[1  2] × [1]  =  [1×1 + 2×1]  =  [3]\n[3, 3]\nis exactly\n[1, 1]\nstretched by a factor of 3, pointing in the exact same direction. Confirmed.\nFor λ = 1:\nA − 1I  =    (2−1)      1        =    1    1\n                1      (2−1)             1    1\n1x + 1y = 0     →     y = −x\nThe simplest choice here is\nx = 1, y = −1\n, giving us the eigenvector\n[1, −1]\n.\nSanity-checking:\n[2  1]   [1]      [2×1 + 1×(−1)]     [1]\n[1  2] × [−1]  =  [1×1 + 2×(−1)]  =  [−1]\n[1, −1]\n, stretched by a factor of exactly 1 — meaning it doesn't grow or shrink at all, it stays exactly the same length, only confirming that this specific direction is untouched in terms of size. Confirmed again.\nSo, for this matrix, there are exactly two special directions — pointing diagonally as\n[1, 1]\nand\n[1, −1]\n— that never tilt, no matter how this specific transformation is applied. Every other direction you could pick will visibly lean towards one of these two special directions when transformed.\nPython Code — Built Entirely From Scratch\nEverything we just did by hand — building the\nA − λI\nmatrix, calculating its determinant, setting that determinant to zero, solving for\nλ\n— can be turned into real, working Python code, without relying on any built-in shortcut function. Let's build it up piece by piece, and I'll explain every single line, because there's genuinely nothing here you can't follow if you take it slowly.\nThe very first piece we need is something you actually already built back in Blog 7 — a function that calculates the determinant of a matrix of any size, from scratch:\nimport numpy as np\n\ndef det(a):\n    if len(a) == 1:\n        return a[0][0]\n    if len(a) == 2:\n        return a[0][0] * a[1][1] - a[0][1] * a[1][0]\n\n    result = 0\n    for col in range(len(a)):\n        minor = [\n            [a[row][c] for c in range(len(a)) if c != col]\n            for row in range(1, len(a))\n        ]\n        result += ((-1) ** col) * a[0][col] * det(np.array(minor))\n\n    return result\nLet's go through this one line at a time, slowly, assuming you're seeing it for the first time.\nimport numpy as np\n— we're bringing in NumPy again, purely so we can conveniently work with matrices as\nnp.array\nobjects, the same way we have throughout this whole series.\ndef det(a):\n— this defines a function named\ndet\n, which takes one input,\na\n— the matrix we want to find the determinant of.\nif len(a) == 1: return a[0][0]\n— this is our simplest possible case. If the matrix is just a single number sitting inside a single row (a 1x1 matrix), its determinant is simply that one number itself.\na[0][0]\nmeans \"the number in the very first row, very first column\" — with a 1x1 matrix, that's the only number there is.\nif len(a) == 2: return a[0][0] * a[1][1] - a[0][1] * a[1][0]\n— this is the direct 2x2 determinant formula you learned in Blog 6: multiply the top-left and bottom-right numbers together, then subtract the product of the top-right and bottom-left numbers.\na[0][0]\nis row 0, column 0 (top-left).\na[1][1]\nis row 1, column 1 (bottom-right).\na[0][1]\nis top-right,\na[1][0]\nis bottom-left.\nresult = 0\n— for anything bigger than a 2x2 matrix, we can't use one simple formula directly. Instead, we're going to build up the answer piece by piece, so we start with an empty running total of zero, exactly like we did with the running total in matrix multiplication back in Blog 5.\nfor col in range(len(a)):\n— this walks across every column position in the very first row of the matrix, one at a time.\ncol\nwill take the values\n0\n,\n1\n,\n2\n, and so on, up to however many columns the matrix has.\nminor = [ [a[row][c] for c in range(len(a)) if c != col] for row in range(1, len(a)) ]\n— this is the densest line in the whole function, so let's unpack it fully. We're building a brand new, smaller matrix called\nminor\n, by taking the original matrix\na\n,\nremoving its entire first row\n, and\nremoving whichever column\ncol\nwe're currently looking at\n. The inner part,\nfor c in range(len(a)) if c != col\n, walks through every column index\nc\n, but skips it entirely (\nif c != col\n) whenever\nc\nmatches the column we're currently removing. The outer part,\nfor row in range(1, len(a))\n, starts counting rows from\n1\ninstead of\n0\n— meaning we're deliberately skipping the very first row of the original matrix. Put together, this single line builds exactly the smaller matrix left over once you cross out one row and one column — precisely the \"minor\" matrix from the cofactor-expansion idea in Blog 6.\nresult += ((-1) ** col) * a[0][col] * det(np.array(minor))\n— this is where the function calls\nitself\n, on the smaller\nminor\nmatrix we just built. This is called\nrecursion\n— think of it like a set of Russian nesting dolls, where each doll contains a smaller doll of the exact same shape, until you reach the smallest one that can't be opened any further (our\nlen(a) == 1\nand\nlen(a) == 2\nbase cases from earlier). The\n((-1) ** col)\npart simply flips the sign back and forth (positive, negative, positive, negative...) as\ncol\nincreases — a specific, fixed detail of how this method is mathematically defined, which you don't need to derive yourself, only trust and reuse.\na[0][col]\nis the actual number, from the very first row, that we're currently \"expanding\" around. Multiplying all three pieces together, and adding the result onto our running total, is exactly how the full determinant gets built up out of smaller and smaller pieces.\nreturn result\n— once we've walked across every column in the first row, and added up every one of those smaller pieces,\nresult\nholds the complete, correct determinant of the original matrix, and we hand it back to whoever called this function.\nUsing\ndet()\nto Actually Find the Eigenvalues, By Search\nHere's the genuinely elegant part. We already know, from Part 2 above, that an eigenvalue is exactly the number\nλ\nthat makes\ndeterminant(A − λI)\nequal to zero. So instead of doing the algebra by hand every single time (factoring quadratics, and worse, for bigger matrices), we can use our\ndet()\nfunction as a simple\ndetector\n— and just try out a large number of candidate\nλ\nvalues, watching closely for the exact points where the determinant crosses from positive to negative (or the other way round). That crossing point is, very closely, an eigenvalue.\ndef find_eigenvalues_by_scanning(A, low=-10, high=10, step=0.001):\n    n = len(A)\n    identity = np.eye(n)\n    found = []\n    prev_sign = None\n    lam = low\n\n    while lam <= high:\n        test_matrix = A - lam * identity\n        value = det(test_matrix)\n        current_sign = value > 0\n\n        if prev_sign is not None and current_sign != prev_sign:\n            found.append(round(lam, 2))\n\n        prev_sign = current_sign\n        lam += step\n\n    return found\n\n\nA = np.array([[2, 1],\n              [1, 2]])\n\nprint(find_eigenvalues_by_scanning(A))\n[1.0, 3.0]\nLet's slowly unpack this one too.\ndef find_eigenvalues_by_scanning(A, low=-10, high=10, step=0.001):\n— this defines our new function, which takes in the matrix\nA\n, plus three optional settings:\nlow\nand\nhigh\n(the range of candidate\nλ\nvalues we're willing to test — here, anywhere from −10 to 10), and\nstep\n(how finely we walk through that range — here, in tiny increments of 0.001).\nn = len(A)\nand\nidentity = np.eye(n)\n—\nn\nis simply the size of the matrix (2, in our example).\nnp.eye(n)\nbuilds the\nidentity matrix\nof that same size — remember, from Blog 6, this is the special matrix full of\n1\ns along the diagonal and\n0\ns everywhere else, representing \"do nothing.\" We need this so we can build\nA − λI\nfor each candidate\nλ\n.\nfound = []\n— an empty list, ready to collect any eigenvalues we discover as we search.\nprev_sign = None\n— before we've tested any candidate\nλ\nat all, we don't yet know if the determinant was positive or negative \"last time,\" so we start this as\nNone\n(meaning \"nothing yet\").\nlam = low\n— we start our search at the smallest candidate value.\nwhile lam <= high:\n— we keep testing candidate values of\nλ\n, one at a time, until we've covered the entire range from\nlow\nup to\nhigh\n.\ntest_matrix = A - lam * identity\n— this builds exactly the\nA − λI\nmatrix from our hand calculation earlier, just using whatever candidate\nlam\nwe're currently testing.\nvalue = det(test_matrix)\n— here's where our earlier\ndet()\nfunction gets reused directly — we calculate the determinant of this test matrix, exactly the same function, no changes needed.\ncurrent_sign = value > 0\n— this simply records whether the determinant came out positive (\nTrue\n) or negative or zero (\nFalse\n) for this particular candidate\nλ\n.\nif prev_sign is not None and current_sign != prev_sign:\n— this is the actual detective work. If the determinant's sign just flipped compared to the very last candidate value we tried (say, it was positive a moment ago, and it's now negative), that means the determinant must have passed through exactly zero somewhere in between — and a determinant of zero is, precisely, an eigenvalue. We record the current\nlam\nas an approximate eigenvalue whenever we spot this flip.\nprev_sign = current_sign\nand\nlam += step\n— we save the current sign for next time, then nudge\nlam\nforward by our small\nstep\namount, and repeat the whole process again.\nreturn found\n— once we've covered the entire range from\nlow\nto\nhigh\n, we hand back the full list of approximate eigenvalues we discovered along the way.\nRunning this on our matrix\n[[2, 1], [1, 2]]\ngives us back\n[1.0, 3.0]\n— exactly matching the two eigenvalues,\n1\nand\n3\n, that we calculated fully by hand earlier in this blog. We didn't factor a single quadratic equation this time. We simply reused the determinant function you already understand, and let the computer try thousands of candidate numbers until it found where the determinant hit zero.\nFinding the Matching Eigenvectors, From Scratch\nOnce we know an eigenvalue, we still need its matching eigenvector — the actual direction that gets stretched by exactly that amount. For a 2x2 matrix specifically, there's a genuinely simple trick we can turn into code, based directly on what we did by hand earlier: take the first row of\nA − λI\n(call its two numbers\na\nand\nb\n), and the direction\n[−b, a]\nwill always be a valid eigenvector for that eigenvalue.\ndef find_eigenvector_2x2(A, eigenvalue):\n    a = A[0][0] - eigenvalue\n    b = A[0][1]\n\n    if a == 0 and b == 0:\n        a = A[1][0]\n        b = A[1][1] - eigenvalue\n\n    return np.array([-b, a])\n\n\nfor lam in [1, 3]:\n    v = find_eigenvector_2x2(A, lam)\n    print(f\"Eigenvalue {lam} -> Eigenvector {v}\")\nEigenvalue 1 -> Eigenvector [-1  1]\nEigenvalue 3 -> Eigenvector [-1 -1]\ndef find_eigenvector_2x2(A, eigenvalue):\n— this function takes in our matrix\nA\n, plus one specific eigenvalue we already found, and will hand back its matching eigenvector.\na = A[0][0] - eigenvalue\nand\nb = A[0][1]\n— these two lines build exactly the first row of\nA − λI\n, the same subtraction we did by hand earlier in this blog.\nif a == 0 and b == 0:\n— very occasionally, the first row happens to become entirely zero for a particular matrix and eigenvalue combination, which wouldn't give us any useful direction to work with. In that rare case, we simply fall back to using the second row of\nA − λI\ninstead, calculated the same way.\nreturn np.array([-b, a])\n— this is the actual trick. If\na\nand\nb\nare the two numbers in a row that must satisfy\na×x + b×y = 0\n(exactly the equation we solved by hand earlier), then plugging in\nx = −b\nand\ny = a\nalways makes that equation true, because\na×(−b) + b×a\nalways cancels out to exactly zero, no matter what\na\nand\nb\nare. That's genuinely all this line is doing — a small, reliable shortcut for solving that specific kind of equation, for a 2x2 matrix.\nLook closely at the output —\n[-1, 1]\nfor eigenvalue\n1\n, and\n[-1, -1]\nfor eigenvalue\n3\n. These point in exactly the same directions as our hand-calculated\n[1, -1]\nand\n[1, 1]\nfrom earlier (just flipped in sign, which makes no difference — an arrow pointing \"up-left\" represents the exact same direction and line as one pointing \"down-right\"; eigenvectors are only ever defined up to this kind of sign flip or rescaling). Our from-scratch code, built on nothing but the determinant function you already understood, has independently rediscovered the exact same answer we worked out by hand.\nDoing This in Python (Verifying, Instantly, What We Just Did by Hand and by Scratch)\nimport numpy as np\n\nA = np.array([[2, 1],\n              [1, 2]])\n\neigenvalues, eigenvectors = np.linalg.eig(A)\n\nprint(\"Eigenvalues:\", eigenvalues)\nprint(\"Eigenvectors:\\n\", eigenvectors)\nEigenvalues: [3. 1.]\nEigenvectors:\n [[ 0.707 -0.707]\n  [ 0.707  0.707]]\nNotice NumPy found exactly the eigenvalues we calculated by hand:\n3\nand\n1\n. The eigenvectors look a little different from our clean\n[1, 1]\nand\n[1, -1]\n— that's simply because NumPy prefers to report these special directions in a \"unit length\" form (meaning the arrow is scaled to have a length of exactly 1), rather than the simplest whole numbers we chose. If you check the\ndirection\n— the ratio between the two numbers in each column — you'll find it matches our hand-calculated\n[1, 1]\nand\n[1, -1]\ndirections exactly, just represented at a different, standardized length. The underlying special direction is identical either way.\nThis single function,\nnp.linalg.eig()\n, does — in a tiny fraction of a second, even for enormous, real-world matrices with thousands of rows and columns — exactly the process we just walked through, twice over: by hand, and then again from scratch in code, solving that quadratic-style equation and finding the matching direction for each solution. In real projects, you'll always use\nnp.linalg.eig()\n— it's faster and more numerically careful than our scanning approach — but now you genuinely know there's no hidden magic inside it.\nPart 3: Why This Matters — Real, Concrete Examples\n1. Eigenfaces — One of the Earliest Real Face Recognition Techniques (Computer Vision)\nHere's a genuinely fascinating, low-level story, and it directly connects everything you've learned across this whole series.\nImagine you collect thousands of photos of different human faces, all cropped and aligned the same way (same size, eyes roughly in the same position). Each photo, as you learned back in Blog 5, is really just a big matrix of pixel brightness numbers. If you gather thousands of these face-matrices together and ask a very specific mathematical question —\n\"across all these different faces, what are the handful of 'directions' along which faces vary from each other the most?\"\n— the answer comes directly from finding the eigenvectors of a matrix built from all those combined faces (specifically, a matrix that captures how the brightness at every pixel position tends to vary together, called a\ncovariance matrix\n— don't worry about that name for now, just know it's built directly from the face photos themselves).\nEach one of these special eigenvector directions, when you actually convert it back into an image, looks like an eerie, ghost-like, blurry face — genuinely, researchers nicknamed these\n\"Eigenfaces.\"\nOne ghost-face might capture \"how wide is this face compared to average.\" Another might capture \"how much facial hair is present.\" You don't manually decide what each ghost-face represents — the eigenvectors reveal these patterns purely from the numbers themselves, without anyone explicitly programming in the concept of \"width\" or \"facial hair.\"\nHere's the genuinely useful part: instead of storing or comparing a full, enormous grid of thousands of pixel numbers per face, an early face recognition system could instead describe any face using just a handful of numbers — \"how much of ghost-face #1 is present in this photo, how much of ghost-face #2, how much of ghost-face #3,\" and so on. A face that might have started as tens of thousands of pixel numbers could now be described using perhaps just 50 or 100 numbers — a massive compression — while still capturing almost everything important needed to tell one face apart from another. This is directly, historically, one of the earliest practical face recognition techniques ever built, and it comes straight from eigenvectors.\n2. Google Search's Original Ranking Algorithm (PageRank)\nHere's a fact that should genuinely surprise you, and it has nothing to do with photos at all.\nIn the very early days of Google, the company needed a way to rank billions of webpages by \"importance,\" so that searching for something returned the most genuinely useful pages first, not just pages that happened to repeat your search words the most times (which older search engines were easily tricked by).\nGoogle's original solution, called\nPageRank\n, represented the entire internet as one enormous matrix — imagine a giant table where every row and column represents one webpage, and a number in a particular cell represents \"does this page link to that page.\" Finding the \"most important\" pages, according to this method, comes down to finding one very special eigenvector of this giant matrix — specifically, the direction that represents a stable, settled-down sense of \"how much overall importance flows into each page,\" once you imagine link-importance being passed around the web, over and over, indefinitely.\nThe genuinely surprising part: an idea that felt purely mathematical and abstract — \"which arrow doesn't rotate when I apply this instruction\" — turned out to be exactly the right tool to help build one of the most economically significant pieces of software in human history. That's honestly one of the best real-world reminders of just how far a \"boring-sounding\" mathematical idea can travel once someone finds the right problem to point it at.\n3. Simplifying Enormous, Messy, Real-World Datasets (Data Science, and a Preview of the ML Series Ahead)\nBeyond faces and search engines, eigenvectors are the mathematical engine behind a hugely important, very commonly used technique in data science called\nPrincipal Component Analysis\n, or\nPCA\nfor short. We are deliberately not diving deep into PCA itself in this blog — it fully deserves its own dedicated post, and it's one of the very first topics we'll properly unpack in the upcoming Machine Learning series, now that you have the eigenvector foundation needed to actually understand\nwhy\nit works, rather than just memorizing the steps.\nFor now, here's the one-sentence preview worth carrying forward: PCA uses eigenvectors to find the small number of \"most important directions\" hiding inside a large, messy dataset — similar to how Eigenfaces found the most important \"directions\" hiding inside thousands of face photos — letting data scientists shrink enormous datasets down to their essential patterns, without throwing away what actually matters.\nPart 4: What This Series Didn't Cover — And Why\nBefore we close this out, I want to be upfront about two names you may already have heard floating around —\nSingular Value Decomposition (SVD)\nand\nPrincipal Component Analysis (PCA)\n— and address directly why a linear algebra series wouldn't be complete without at least acknowledging them.\nHere's the honest reason they aren't a full blog of their own in this series, and it isn't because they're unimportant — quite the opposite.\nEverything we did in this blog — finding eigenvalues and eigenvectors — only works directly on\nsquare matrices\n(the same number of rows as columns). But most real datasets are nowhere near square. Imagine a dataset of 10,000 customer photos, each one flattened into 50,000 pixel-brightness numbers. That's a matrix with 10,000 rows and 50,000 columns — nowhere close to square, and eigenvalues, exactly as we defined them today, simply don't apply to a matrix shaped like that.\nTo handle this far more common, rectangular case, mathematicians generalized the entire idea you just learned into something more powerful, called\nSingular Value Decomposition\n. Rather than looking for special directions a single matrix stretches without rotating, SVD breaks\nany\nmatrix — square or not, however oddly shaped — into three separate pieces: one matrix capturing special directions in the \"input\" space, one capturing special directions in the \"output\" space, and a simple diagonal matrix of \"stretch amounts\" (called\nsingular values\n) connecting the two. If eigen-decomposition is the special case of measuring a transformation against itself, SVD is the general tool for measuring any transformation, between any two shapes of space, at all. In practice, most real-world implementations of PCA don't even calculate eigenvectors of a covariance matrix directly (the way our earlier Eigenfaces story described) — they compute an SVD instead, purely because it tends to be more numerically stable on real, messy, large-scale data.\nPrincipal Component Analysis\nitself, then, is really the practical technique built on top of this exact idea — using either eigen-decomposition or SVD (in practice, almost always SVD) to find the small number of directions along which a dataset varies the most, and re-describing the entire dataset using just those directions, discarding the rest as comparatively unimportant noise. It's the single most commonly used dimensionality-reduction technique in applied Machine Learning today, sitting quietly behind everything from compressing sensor data, to visualizing high-dimensional datasets in two dimensions, to speeding up training by feeding a model a smaller, more meaningful version of its original features.\nI made a deliberate choice not to cram SVD and PCA into this final post. This entire series was built on a promise, right from blog one, that nothing gets explained before its foundation is properly in place — and SVD genuinely deserves that same treatment: real, hand-worked numbers, real code built from scratch, and real low-level examples, the same standard I've tried to hold every post in this series to, rather than a rushed, high-level paragraph bolted onto the end of an already-long final blog. Rather than doing that here, I'll be covering SVD and PCA properly, the exact same hands-on way, at the natural moment they belong — when the upcoming Machine Learning series reaches dimensionality reduction and recommendation systems, where these two ideas genuinely earn their place instead of being introduced for their own sake. Consider this paragraph a marker, not a dodge — I know exactly where these two ideas fit into the bigger picture, and we'll do them justice when we get there.\nPart 5: The Whole Series, Looking Back\nYou've genuinely covered a complete, real foundation in linear algebra across this whole series. Let's see the whole journey, laid out in one place, one final time. (Number these to match however you've ordered them on your own blog — what matters here is the order the ideas build on each other, not the exact digit.)\nOrder\nWhat you learned\nIntroduction\nWhy Machine Learning needs linear algebra at all\nVectors\nWhat a vector is — one example, described as numbers\nMatrices\nWhat a matrix is — many vectors, organized into a table\nAddition, subtraction & element-wise multiplication\nCombining, comparing, and masking data\nMatrix multiplication, transpose & cross product\nWeighing, combining, and reorienting data\nLinear transformation, determinant & matrix inverse\nWhat a matrix does to data, and whether it can be undone\nEigenvalues & eigenvectors (this one)\nThe special directions a transformation only stretches, never rotates\nThat's the complete toolkit. Genuinely, everything from here — every neural network layer, every training loop, every \"the model learned a representation\" sentence you'll read in the Machine Learning series ahead — is built directly on top of these ideas, just applied at a much bigger scale, with the numbers chosen automatically by the model itself, rather than by us, by hand, on paper.\nThis Is the End of the Linear Algebra Series — Here's What's Next\nThis is officially the final blog in the\nLinear Algebra for Machine Learning\nseries. Thank you, genuinely, for sticking with this all the way from \"why does ML even need this math\" through to eigenvectors. That's not a small journey, and if you followed it start to finish, you now have a real, working mathematical foundation that a lot of people jump straight past when they start learning Machine Learning — and end up confused later because of it.\nStarting with the next blog, this space moves into the actual\nMachine Learning series\n— and every single \"don't worry, future blog\" moment scattered across this series is about to get paid off, properly. Gradient descent. Loss functions. What \"training\" a model actually means, step by step. What a neural network layer really is, now that you know exactly what a linear transformation is. We're going to build every one of those ideas directly on top of everything you've learned across this series.\nAnd before you go — one more time, genuinely, please don't skip this part:\nIf any part of the earlier posts in this series felt too high-level, too fast, or assumed things you didn't actually know yet — please say so in the comments below, and be specific about where it happened.\nThis series is meant to work for complete beginners, and the only way I'll know if it actually succeeded is if you tell me honestly. If enough of you point to the same rough spots, I will go back and rewrite those posts properly, at a lower level, the same way I've tried to do starting with this last stretch of blogs. Your comments genuinely decide whether that rewrite happens — so don't hold back.\nSee you in the Machine Learning series.\n#\nai\n#\nmatrix\n#\nmachine-learning\n#\nmaths\n#\nmathematics\n#\nlinear-algebra\n#\npython\n#\nnumpy\n#\ndata-science\n#\ndeeplearning\n#\ncomputer-vision\n#\nbeginners\n10\nviews",
  "metadata": {
    "title": "Eigenvalues and Eigenvectors, Explained for Complete Beginners — The Final Blog in the Linear Algebra for ML Series",
    "description": "Take a normal photograph and stretch it sideways in an editing app — pull the left and right edges apart, like taffy.\nWatch what happens to different arrows you could draw on that photo. An arrow poin",
    "keywords": null,
    "author": null,
    "og:title": "Eigenvalues and Eigenvectors, Explained for Complete Beginners — The Final Blog in the Linear Algebra for ML Series",
    "og:description": "Take a normal photograph and stretch it sideways in an editing app — pull the left and right edges apart, like taffy.\nWatch what happens to different arrows you could draw on that photo. An arrow poin",
    "og:url": "https://tharunpandya.hashnode.dev/eigenvalues-and-eigenvectors-explained-for-complete-beginners-the-final-blog-in-the-linear-algebra-for-ml-series",
    "og:site_name": "Machine Learing From scratch",
    "og:image": "https://tharunpandya.hashnode.dev/api/og/post?og=eyJhdXRob3IiOiJUaGFydW4iLCJpc0RlZmF1bHRNb2RlRGFyayI6ZmFsc2UsInJlYWRUaW1lIjozMCwiY29tbWVudHMiOjAsInJlYWN0aW9ucyI6MSwiZG9tYWluIjoidGhhcnVucGFuZHlhLmhhc2hub2RlLmRldiIsInRpdGxlIjoiRWlnZW52YWx1ZXMlMjBhbmQlMjBFaWdlbnZlY3RvcnMlMkMlMjBFeHBsYWluZWQlMjBmb3IlMjBDb21wbGV0ZSUyMEJlZ2lubmVycyUyMCVFMiU4MCU5NCUyMFRoZSUyMEZpbmFsJTIwQmxvZyUyMGluJTIwdGhlJTIwTGluZWFyJTIwQWxnZWJyYSUyMGZvciUyME1MJTIwU2VyaWVzIiwicGhvdG8iOiJodHRwczovL2xoMy5nb29nbGV1c2VyY29udGVudC5jb20vYS9BQ2c4b2NKQzBBNGJubm5RMnJld1ppOFJuWE5tTkRkU21BMGZZemd6eW5teWdSVE1nX1p1eE5HbnFBPXM5Ni1jIiwibG9nbyI6Imh0dHBzOi8vY2RuLmhhc2hub2RlLmNvbS9yZXMvaGFzaG5vZGUvaW1hZ2UvdXBsb2FkL3YxNTkzNjgwMjgyODk2L2tOQzdFOElSNC5wbmciLCJmYXZpY29uIjoiaHR0cHM6Ly9jZG4uaGFzaG5vZGUuY29tL3Jlcy9oYXNobm9kZS9pbWFnZS91cGxvYWQvdjE1OTM2ODAyODI4OTYva05DN0U4SVI0LnBuZyIsInB1Ymxpc2hlZEF0IjoiMjAyNi0wNy0yNFQwMTowMDoyNi41MTFaIiwidXBkYXRlZEF0IjoiMjAyNi0wOC0wOFQwMDoyNTowOS40NjNaIn0=",
    "og:image:width": "1200",
    "og:image:height": "630",
    "og:type": "article",
    "twitter:card": "summary_large_image",
    "twitter:title": "Eigenvalues and Eigenvectors, Explained for Complete Beginners — The Final Blog in the Linear Algebra for ML Series",
    "twitter:description": "Take a normal photograph and stretch it sideways in an editing app — pull the left and right edges apart, like taffy.\nWatch what happens to different arrows you could draw on that photo. An arrow poin",
    "twitter:image": "https://tharunpandya.hashnode.dev/api/og/post?og=eyJhdXRob3IiOiJUaGFydW4iLCJpc0RlZmF1bHRNb2RlRGFyayI6ZmFsc2UsInJlYWRUaW1lIjozMCwiY29tbWVudHMiOjAsInJlYWN0aW9ucyI6MSwiZG9tYWluIjoidGhhcnVucGFuZHlhLmhhc2hub2RlLmRldiIsInRpdGxlIjoiRWlnZW52YWx1ZXMlMjBhbmQlMjBFaWdlbnZlY3RvcnMlMkMlMjBFeHBsYWluZWQlMjBmb3IlMjBDb21wbGV0ZSUyMEJlZ2lubmVycyUyMCVFMiU4MCU5NCUyMFRoZSUyMEZpbmFsJTIwQmxvZyUyMGluJTIwdGhlJTIwTGluZWFyJTIwQWxnZWJyYSUyMGZvciUyME1MJTIwU2VyaWVzIiwicGhvdG8iOiJodHRwczovL2xoMy5nb29nbGV1c2VyY29udGVudC5jb20vYS9BQ2c4b2NKQzBBNGJubm5RMnJld1ppOFJuWE5tTkRkU21BMGZZemd6eW5teWdSVE1nX1p1eE5HbnFBPXM5Ni1jIiwibG9nbyI6Imh0dHBzOi8vY2RuLmhhc2hub2RlLmNvbS9yZXMvaGFzaG5vZGUvaW1hZ2UvdXBsb2FkL3YxNTkzNjgwMjgyODk2L2tOQzdFOElSNC5wbmciLCJmYXZpY29uIjoiaHR0cHM6Ly9jZG4uaGFzaG5vZGUuY29tL3Jlcy9oYXNobm9kZS9pbWFnZS91cGxvYWQvdjE1OTM2ODAyODI4OTYva05DN0U4SVI0LnBuZyIsInB1Ymxpc2hlZEF0IjoiMjAyNi0wNy0yNFQwMTowMDoyNi41MTFaIiwidXBkYXRlZEF0IjoiMjAyNi0wOC0wOFQwMDoyNTowOS40NjNaIn0=",
    "article:published_time": "2026-07-24T01:00:26.511Z",
    "article:modified_time": "2026-08-08T00:25:09.463Z",
    "article:author": "https://tharunpandya.hashnode.dev/@tharun-pandya"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/14f06e3d9719/code/body-000.txt",
      "chars": 42,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-001.txt",
      "chars": 74,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-002.txt",
      "chars": 74,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-003.txt",
      "chars": 42,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-004.txt",
      "chars": 14,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-005.txt",
      "chars": 41,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-006.txt",
      "chars": 55,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-007.txt",
      "chars": 23,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-008.txt",
      "chars": 72,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-009.txt",
      "chars": 19,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-010.txt",
      "chars": 19,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-011.txt",
      "chars": 94,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-012.txt",
      "chars": 29,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-013.txt",
      "chars": 74,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-014.txt",
      "chars": 93,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-015.txt",
      "chars": 29,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-016.txt",
      "chars": 83,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-017.txt",
      "chars": 407,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-018.txt",
      "chars": 552,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-019.txt",
      "chars": 11,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-020.txt",
      "chars": 304,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-021.txt",
      "chars": 72,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-022.txt",
      "chars": 186,
      "language": "text"
    },
    {
      "file": "attachments/14f06e3d9719/code/body-023.txt",
      "chars": 72,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://tharunpandya.hashnode.dev/eigenvalues-and-eigenvectors-explained-for-complete-beginners-the-final-blog-in-the-linear-algebra-for-ml-series.md",
      "label": "View as Markdown",
      "type": "a"
    },
    {
      "href": "https://hashnode.com/@tharun-pandya",
      "label": "T",
      "type": "a"
    },
    {
      "href": "https://linkedin.com/in/tharun-pandya-kodi",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/ai",
      "label": "# ai",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/matrix",
      "label": "# matrix",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/machine-learning",
      "label": "# machine-learning",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/maths",
      "label": "# maths",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/mathematics",
      "label": "# mathematics",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/linear-algebra",
      "label": "# linear-algebra",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/python",
      "label": "# python",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/numpy",
      "label": "# numpy",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/data-science",
      "label": "# data-science",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/deeplearning",
      "label": "# deeplearning",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/computer-vision",
      "label": "# computer-vision",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/beginners",
      "label": "# beginners",
      "type": "a"
    }
  ],
  "retry_at": "2026-10-03T08:05:01",
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:50:29Z",
  "source_url": "https://tharunpandya.hashnode.dev/eigenvalues-and-eigenvectors-explained-for-complete-beginners-the-final-blog-in-the-linear-algebra-for-ml-series",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 34676,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/14f06e3d9719/pages/source.html",
  "local_html": "attachments/14f06e3d9719/pages/article.html",
  "body_base_path": "attachments/14f06e3d9719/pages/",
  "body_markdown_path": "attachments/14f06e3d9719/pages/article.md"
}