{
  "source_file": "0930",
  "source_platform": "Hashnode",
  "author": "Muhammad Fahad Bashir",
  "title_listed": "PCA Explained Simply: Understanding Dimensionality Reduction",
  "keywords": "",
  "domain": "机器学习",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://ml-diaries-by-fahd.hashnode.dev/pca-explained-simply-understanding-dimensionality-reduction",
  "publish_date": "2026-01-12",
  "corpus_stage": "",
  "notes": "",
  "id": "93c42bac22a8",
  "collected_at": "2026-10-02T14:10:53",
  "title": "PCA Explained Simply: Understanding Dimensionality Reduction",
  "body_markdown": "PCA Explained Simply: Understanding Dimensionality Reduction\nPublished\nJanuary 12, 2026\n•\n6\nmin read\n•\nView as Markdown\nM\nMuhammad Fahad Bashir\nMachine Learning, Artificial Intelligence\nPart of series\nUnsupervised Learning\nIn our previous article on\nunsupervised learning\n, we covered a clustering algorithm. Today we will be covering a common algorithm that comes under the category of\ndimensionality reduction.\nSo, in machine learning, having more data is usually helpful but having\ntoo many features\ncan sometimes create more problems than benefits. High-dimensional data can slow down algorithms, increase memory usage, cause overfitting, and make visualization nearly impossible. This is where\ndimensionality reduction\ncomes in, and one of the most widely used techniques for this purpose is\nPrincipal Component Analysis (PCA)\n.\nPCA is an\nunsupervised learning technique\nthat reduces the number of features in a dataset while preserving as much important information as possible. Instead of selecting existing features, PCA creates\nnew features\n, called\nprincipal components\n, which summarize the original data in a more compact form.\nWhat Is PCA (Principal Component Analysis)?\nPrincipal Component Analysis is a technique that transforms a dataset with many correlated features into a smaller set of\nuncorrelated components\n. These components capture the directions where the data varies the most. In simple terms, PCA helps remove redundancy from data and keeps only what matters the most.\nThink of a dataset with features like\nage\n,\nsalary\n,\nyears of experience\n, and\nmonthly income\n. Many of these features are related to each other. PCA identifies these relationships and combines them into fewer dimensions without losing the overall structure of the data.\nIntuition Behind PCA (Simple Explanation)\nImagine a cloud of data points floating in space, like stars in the sky. If you look at this cloud from the wrong angle, it may look scattered and confusing. But if you rotate your view to the\nright angle\n, you’ll notice clear patterns and structure.\nImagine you are taking a group photo of many people standing close together. From some angles, the photo looks messy and crowded, but from the right angle, you can clearly see everyone. PCA finds these\nbest angles automatically\n.\nIn another way, think of students described by height, arm length, and leg length. These features are strongly related. PCA combines them into a smaller number of components representing overall body size, reducing complexity without losing essential information.\nPCA finds these\nbest viewing angles\nautomatically. These angles are called\nprincipal components\n.\nThe first principal component captures the\nmaximum variance\nin the data (where data is spread out the most). The second principal component captures the next highest variance, but in a direction\northogonal (perpendicular)\nto the first. This process continues until all dimensions are covered.\nWhy Do We Need PCA?\nPCA is mainly used to simplify complex datasets while keeping the most important information intact. It helps reduce noise, remove redundancy, and improve computational efficiency. Many machine learning algorithms perform better when trained on fewer, meaningful features rather than a large number of correlated ones.\nPCA is especially useful when:\nThe dataset has\nmany features\nFeatures are\nhighly correlated\nVisualization is difficult due to high dimensions\nModels are slow or overfitting\nHow PCA Works (Step-by-Step)\nPCA follows a structured mathematical process.\nFirst, the data is\nstandardized\nso all features are on the same scale. This is necessary because PCA is sensitive to feature magnitude.\nNext, PCA computes the\ncovariance matrix\n, which shows how features vary with respect to each other. This helps identify correlated features.\nThen, PCA finds\nprincipal components\n, which are new axes where the data has the highest variance. The first principal component captures the maximum variance, the second captures the remaining variance while being orthogonal to the first, and so on.\nFinally, the data is\nprojected onto the top K principal components\n, reducing dimensionality while preserving most of the information.\nKey Concept: Variance and Information\nThe main goal of PCA is to\nmaximize variance\n. Variance represents information in the data. The more variance a component captures, the more information it retains from the original dataset.\nThe first principal component captures the most variance. The second captures the remaining variance while being uncorrelated with the first, and so on. In practice, just the first few components often capture\n80–95% of the total variance\n, which is why PCA is so effective.\nPCA & Feature Selection\nPCA and feature selection both reduce dimensionality, but they work differently. Feature selection keeps a subset of the original features, preserving their meaning and interpretability. PCA, on the other hand, creates new features by combining existing ones, which improves efficiency but reduces interpretability. PCA is best when performance matters more than understanding individual features.\nReal-World Applications of PCA\nPCA is widely used both as a standalone technique and as a preprocessing step before applying machine learning models.\nIt is commonly used to\nvisualize high-dimensional data\nby reducing it to 2D or 3D.\nIt helps in\ndata compression\n, reducing storage and computation costs.\nIn business analytics, PCA simplifies complex datasets to support\nbetter decision-making\n.\nIn machine learning pipelines, PCA improves performance by reducing noise and overfitting.\nAssumptions and Limitations of PCA\nAlthough PCA is powerful, it comes with certain assumptions and limitations. PCA assumes that features are\nlinearly correlated\n. If features are unrelated, PCA cannot find meaningful components.\nPCA is also\nsensitive to feature scale\n, which is why standardization is mandatory. It is not robust to\noutliers\n, as extreme values can significantly affect variance. Additionally, PCA assumes\nlinear relationships\n, so it cannot capture complex non-linear patterns. Lastly, PCA usually requires datasets without missing values.\nPractical Implementation ( Wine Dataset)\nGoal is to\nReduce\n13 features → 2 principal components\nVisualize high-dimensional data\nUnderstand\nvariance preservation\n# Import libraries\nimport\nmatplotlib.pyplot\nas\nplt\nfrom\nsklearn.datasets\nimport\nload_wine\nfrom\nsklearn.preprocessing\nimport\nStandardScaler\nfrom\nsklearn.decomposition\nimport\nPCA\n# Load dataset\nwine = load_wine()\nX = wine.data\n# only features (unlabeled data)\n# Standardize features\nscaler = StandardScaler()\nX_scaled = scaler.fit_transform(X)\n# Apply PCA (reduce to 2 dimensions)\npca = PCA(n_components=\n2\n)\nX_pca = pca.fit_transform(X_scaled)\n\nprint(\n\"Explained variance ratio:\"\n, pca.explained_variance_ratio_)\nprint(\n\"Total variance captured:\"\n, sum(pca.explained_variance_ratio_))\n\nplt.figure(figsize=(\n8\n,\n6\n))\nplt.scatter(X_pca[:,\n0\n], X_pca[:,\n1\n], s=\n50\n)\nplt.xlabel(\n\"Principal Component 1\"\n)\nplt.ylabel(\n\"Principal Component 2\"\n)\nplt.title(\n\"PCA: Data Reduced from 13D to 2D\"\n)\nplt.grid(\nTrue\n)\nplt.show()\nThis plot shows the original 13-dimensional dataset compressed into\n2 principal components\n, making it easy to visualize. The explained variance (~55%) means these two components capture\nmore than half of the important information\nin the data.\nEven after reduction, the structure and patterns of the data are preserved, which is exactly the goal of PCA.\nFinal Thoughts\nPrincipal Component Analysis is one of the most intuitive and widely used dimensionality reduction techniques in machine learning. It helps transform complex, high-dimensional data into a simpler and more meaningful form without losing important information. While PCA is not suitable for every problem, it is an essential tool to understand for anyone working with real-world data.\nIn the context of unsupervised learning, PCA completes the\ncore 80/20 toolkit\nalong with K-Means and DBSCAN, making it a powerful next step in your learning journey.\nFollow us for more :\nML Diaries by Fahd\n#\nunsupervised-learning\n#\nmachine-learning\n#\nml\n#\nai\n#\nartificial-intelligence\n#\ntutorial\n#\nbeginner\n10\nviews",
  "metadata": {
    "title": "PCA Explained Simply: Understanding Dimensionality Reduction",
    "description": "Principal Component Analysis (PCA) reduces dimensionality, simplifying high-dimensional data while retaining essential information",
    "keywords": null,
    "author": null,
    "og:title": "PCA Explained Simply: Understanding Dimensionality Reduction",
    "og:description": "Principal Component Analysis (PCA) reduces dimensionality, simplifying high-dimensional data while retaining essential information",
    "og:url": "https://ml-diaries-by-fahd.hashnode.dev/pca-explained-simply-understanding-dimensionality-reduction",
    "og:site_name": "ML Diaries by Fahd",
    "og:image": "https://cdn.hashnode.com/res/hashnode/image/upload/v1768242928409/8971971a-5b88-4ee4-87d0-e84af988d1a1.png",
    "og:image:width": "1200",
    "og:image:height": "630",
    "og:type": "article",
    "twitter:card": "summary_large_image",
    "twitter:title": "PCA Explained Simply: Understanding Dimensionality Reduction",
    "twitter:description": "Principal Component Analysis (PCA) reduces dimensionality, simplifying high-dimensional data while retaining essential information",
    "twitter:image": "https://cdn.hashnode.com/res/hashnode/image/upload/v1768242928409/8971971a-5b88-4ee4-87d0-e84af988d1a1.png",
    "article:published_time": "2026-01-12T18:37:07.905Z",
    "article:author": "https://ml-diaries-by-fahd.hashnode.dev/@mfahadbashir"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/93c42bac22a8/code/body-000.txt",
      "chars": 800,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/res/hashnode/image/upload/v1768242928409/8971971a-5b88-4ee4-87d0-e84af988d1a1.png",
      "bytes": 503302,
      "reused": true,
      "local_path": "attachments/93c42bac22a8/images/002.png",
      "src": "https://cdn.hashnode.com/res/hashnode/image/upload/v1768242928409/8971971a-5b88-4ee4-87d0-e84af988d1a1.png",
      "alt": "PCA Explained Simply: Understanding Dimensionality Reduction",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://miro.medium.com/1*aEzC_T8gjhUkqEuBPLFhAQ.png",
      "bytes": 149416,
      "reused": true,
      "local_path": "attachments/93c42bac22a8/images/004.png",
      "src": "https://miro.medium.com/1*aEzC_T8gjhUkqEuBPLFhAQ.png",
      "alt": "Principal Component Analysis (PCA) In Depth | by Fraidoon Omarzai | Medium",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://datahacker.rs/wp-content/uploads/2020/06/Picture13-1-845x1024.jpg",
      "bytes": 54420,
      "reused": true,
      "local_path": "attachments/93c42bac22a8/images/005.jpg",
      "src": "https://datahacker.rs/wp-content/uploads/2020/06/Picture13-1-845x1024.jpg",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://miro.medium.com/1*Ne5pL_8mMrdJiPvFpYeSUA.png",
      "bytes": 243594,
      "reused": true,
      "local_path": "attachments/93c42bac22a8/images/006.png",
      "src": "https://miro.medium.com/1*Ne5pL_8mMrdJiPvFpYeSUA.png",
      "alt": "27. Principal Component Analysis (PCA) & Dimensionality Reduction: | by  Kiran vutukuri | Medium",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/res/hashnode/image/upload/v1768242726575/dd6df732-e8b6-46de-b8da-47be295f518e.png",
      "bytes": 70090,
      "reused": true,
      "local_path": "attachments/93c42bac22a8/images/007.png",
      "src": "https://cdn.hashnode.com/res/hashnode/image/upload/v1768242726575/dd6df732-e8b6-46de-b8da-47be295f518e.png",
      "alt": "This plot shows the original 13-dimensional dataset compressed into 2 principal components, making it easy to visualize. The explained variance (~55%) means these two components capture more than half of the important information in the data. Even after reduction, the structure and patterns of the data are preserved, which is exactly the goal of PCA.",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://ml-diaries-by-fahd.hashnode.dev/pca-explained-simply-understanding-dimensionality-reduction.md",
      "label": "View as Markdown",
      "type": "a"
    },
    {
      "href": "https://hashnode.com/@mfahadbashir",
      "label": "M",
      "type": "a"
    },
    {
      "href": "https://www.linkedin.com/in/muhammad-fahad-bashir-09568216a/",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://ml-diaries-by-fahd.hashnode.dev/series/unsupervisedlearning",
      "label": "Part of series Unsupervised Learning",
      "type": "a"
    },
    {
      "href": "https://datahacker.rs/principal-component-analysis-pca/",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://ml-diaries-by-fahd.hashnode.dev/",
      "label": "ML Diaries by Fahd",
      "type": "a"
    },
    {
      "href": "https://ml-diaries-by-fahd.hashnode.dev/tag/unsupervised-learning",
      "label": "# unsupervised-learning",
      "type": "a"
    },
    {
      "href": "https://ml-diaries-by-fahd.hashnode.dev/tag/machine-learning",
      "label": "# machine-learning",
      "type": "a"
    },
    {
      "href": "https://ml-diaries-by-fahd.hashnode.dev/tag/ml",
      "label": "# ml",
      "type": "a"
    },
    {
      "href": "https://ml-diaries-by-fahd.hashnode.dev/tag/ai",
      "label": "# ai",
      "type": "a"
    },
    {
      "href": "https://ml-diaries-by-fahd.hashnode.dev/tag/artificial-intelligence",
      "label": "# artificial-intelligence",
      "type": "a"
    },
    {
      "href": "https://ml-diaries-by-fahd.hashnode.dev/tag/tutorial",
      "label": "# tutorial",
      "type": "a"
    },
    {
      "href": "https://ml-diaries-by-fahd.hashnode.dev/tag/beginner",
      "label": "# beginner",
      "type": "a"
    }
  ],
  "retry_at": "2026-10-03T08:07:04",
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:50:40Z",
  "source_url": "https://ml-diaries-by-fahd.hashnode.dev/pca-explained-simply-understanding-dimensionality-reduction",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 8212,
  "expected_body_images": 5,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/93c42bac22a8/pages/source.html",
  "local_html": "attachments/93c42bac22a8/pages/article.html",
  "body_base_path": "attachments/93c42bac22a8/pages/",
  "body_markdown_path": "attachments/93c42bac22a8/pages/article.md"
}