{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "Priyesh Dave",
  "title_listed": "Day 15: Visualizing Embeddings with t-SNE and PCA",
  "keywords": "",
  "domain": "文本处理",
  "content_type": "技术教程",
  "is_paid_flag": "否",
  "url": "https://dev.to/priyeshdave6/day-15-visualizing-embeddings-with-t-sne-and-pca-21b8",
  "publish_date": "2026-09-10",
  "corpus_stage": "",
  "notes": "",
  "id": "01dfd541bbd0",
  "collected_at": "2026-10-02T13:49:51",
  "title": "Day 15: Visualizing Embeddings with t-SNE and PCA - DEV Community",
  "body_markdown": "Previously, on Day 14:\nExplained how word embeddings represent words as vectors, the limitations of classic approaches like one-hot encoding, and how FastText uses subword units to create robust embeddings that handle rare, new, or misspelled words.\nWhy Visualize Embeddings?\nWord embeddings are dense vectors that represent words as points in space. Imagine each word having a \"position\" in a space with maybe 100 or 300 directions—far beyond our normal three. Each number in the vector says how much the word lines up with one of these directions.\nWords with similar meanings often end up close together in this space. For example, you might find \"cat\" and \"dog\" in roughly the same region, but far away from \"car.\" By plotting these positions, you get a direct look at what the embedding has learned—what it thinks is similar, what feels distant, and which words cluster together.\nVisualizing embeddings isn't just for curiosity. It can help spot patterns, catch mistakes, diagnose biases, and debug your model.\nThe Challenge of High Dimensionality\nMost people are comfortable thinking in two or three dimensions. Word embeddings use dozens or hundreds. If you try to plot 100 axes, you won’t see anything useful.\nTo make these embeddings visible, we use dimensionality reduction. This means taking high-dimensional data and squeezing it down to two or three dimensions in a way that tries to keep the important relationships between points.\nPrincipal Component Analysis (PCA): The Basics\nPrincipal Component Analysis (PCA) is a classic way to reduce dimensions. It helps find the axes (directions) along which your data varies most.\nPicture a swarm of points floating in 3D. You want to shine a light so that this cloud casts the biggest, flattest shadow onto a wall. PCA picks the best direction for that light—finding the axes with the most spread. In higher dimensions, it does the same, but finds the \"shadow\" in 2D or 3D that keeps as much of the structure as possible.\nFor embeddings, PCA projects all those dimensions down to just 2 or 3. It’s a linear technique, so it captures big movements or trends, but misses subtle, nonlinear patterns.\nt-SNE: Visualizing Similarities\nt-Distributed Stochastic Neighbor Embedding (t-SNE) is another way to reduce dimensions. While PCA focuses on big-picture spread, t-SNE tries to make sure points that were close together in the original space stay close together in 2D.\nImagine t-SNE as reshuffling the points so that neighborhoods stay strong. If \"cat\" and \"dog\" were tight friends in the embedding, they’ll show up beside each other in the plot. t-SNE does this by matching up how similar pairs of points are in the high-dimensional space versus the lower-dimensional plot.\nThe tradeoff: t-SNE makes local groups clear, but you can’t trust the exact distances between distant groups. The pattern inside a cluster is meaningful; the gap between two clusters often isn’t.\nPCA vs. t-SNE: A Quick Comparison\nPCA\n: Fast, linear, good for spotting broad trends and outliers. If you want to compare distances and see big-picture structure, start with PCA.\nt-SNE\n: Finds tight clusters and preserves local relationships. Great for spotting groups, analogies, and patterns you might miss with PCA. But don’t use t-SNE to judge how far apart two separate clusters are.\nMany people use both—PCA for the overview, t-SNE for finding groups inside.\nVisualizing Embeddings in Python: Step by Step\nLet's use a tiny, hardcoded example. Real-world code would load hundreds or thousands of real vectors (say, from Word2Vec or GloVe), but it's easier to see what's happening with something simple.\nHere’s code to shrink five 6-dimensional word embeddings down to 2D using both PCA and t-SNE, then plot and label the results:\nimport\nnumpy\nas\nnp\nimport\nmatplotlib.pyplot\nas\nplt\nfrom\nsklearn.decomposition\nimport\nPCA\nfrom\nsklearn.manifold\nimport\nTSNE\n# Hardcoded 6-dimensional \"embeddings\" for 5 words\nwords\n=\n[\n\"\ncat\n\"\n,\n\"\ndog\n\"\n,\n\"\ncar\n\"\n,\n\"\nbus\n\"\n,\n\"\napple\n\"\n]\nembeddings\n=\nnp\n.\narray\n([\n[\n0.4\n,\n0.2\n,\n0.5\n,\n0.1\n,\n0.4\n,\n0.2\n],\n# cat\n[\n0.5\n,\n0.1\n,\n0.4\n,\n0.2\n,\n0.3\n,\n0.2\n],\n# dog\n[\n0.1\n,\n0.6\n,\n0.2\n,\n0.8\n,\n0.2\n,\n0.3\n],\n# car\n[\n0.2\n,\n0.7\n,\n0.1\n,\n0.9\n,\n0.1\n,\n0.3\n],\n# bus\n[\n0.8\n,\n0.2\n,\n0.9\n,\n0.1\n,\n0.7\n,\n0.2\n],\n# apple\n])\n# PCA to 2D\npca\n=\nPCA\n(\nn_components\n=\n2\n)\nreduced_pca\n=\npca\n.\nfit_transform\n(\nembeddings\n)\n# t-SNE to 2D (small perplexity for small dataset)\ntsne\n=\nTSNE\n(\nn_components\n=\n2\n,\nrandom_state\n=\n42\n,\nperplexity\n=\n2\n)\nreduced_tsne\n=\ntsne\n.\nfit_transform\n(\nembeddings\n)\n# Plotting\ndef\nplot_embeddings\n(\nX\n,\ntitle\n):\nplt\n.\nfigure\n(\nfigsize\n=\n(\n5\n,\n4\n))\nfor\ni\n,\nword\nin\nenumerate\n(\nwords\n):\nx\n,\ny\n=\nX\n[\ni\n]\nplt\n.\nscatter\n(\nx\n,\ny\n)\nplt\n.\ntext\n(\nx\n+\n0.01\n,\ny\n+\n0.01\n,\nword\n,\nfontsize\n=\n12\n)\nplt\n.\ntitle\n(\ntitle\n)\nplt\n.\naxis\n(\n'\noff\n'\n)\nplt\n.\nshow\n()\nplot_embeddings\n(\nreduced_pca\n,\n\"\nPCA Visualization\n\"\n)\nplot_embeddings\n(\nreduced_tsne\n,\n\"\nt-SNE Visualization\n\"\n)\nEnter fullscreen mode\nExit fullscreen mode\nThis plots each word, showing how PCA and t-SNE arrange them. Even with just five points, you’ll see \"cat\" and \"dog\" land near each other, and \"car\" and \"bus\" group up. \"Apple,\" less related, stays apart. The two methods usually agree on clusters, but t-SNE's groupings look sharper.\nIf you use a larger dataset, t-SNE’s patterns will move around a bit between runs (unless you fix the random seed). For deeper experiments, try swapping in real pretrained embeddings—libraries like Gensim make this easy.\nVisualizations like this are a first step to understanding your model’s internal map of words. For more precision, try probing embeddings with arithmetic or nearest-neighbor searches. But as a tool for sanity checks, bias hunting, or demoing what word embeddings capture, PCA and t-SNE are invaluable.\nKey Takeaways\nWord embeddings can be visualized to reveal similarities, clusters, and patterns.\nPCA reduces dimensions by capturing broad trends and is linear and fast.\nt-SNE preserves local relationships and reveals tight clusters but may distort global distances.\nVisualization helps debug, interpret, and check for biases in embeddings.\nPython code using sklearn makes it easy to plot word embeddings with PCA and t-SNE.\nTry It Yourself\nPick five words—some related, some unrelated (e.g., 'king', 'queen', 'man', 'woman', 'apple'). Use the provided code to plot their embeddings with both PCA and t-SNE. Examine the plots and write one sentence describing what you notice about how the words are grouped or separated.\nFurther Resources\n🎥\n4.9. Embedding Techniques Visual Comparison: Meaning Map with PCA and t‑SNE\n📄\nIntroduction to t‑SNE: Nonlinear Dimensionality Reduction and Data Visualization\nComing up on Day 16:\nCosine Similarity and Semantic Search Basics",
  "metadata": {
    "title": "Day 15: Visualizing Embeddings with t-SNE and PCA - DEV Community",
    "description": "Previously, on Day 14: Explained how word embeddings represent words as vectors, the limitations of... Tagged with ai, nlp, genai, 100daysofcode.",
    "keywords": "ai, nlp, genai, 100daysofcode, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/priyeshdave6/day-15-visualizing-embeddings-with-t-sne-and-pca-21b8",
    "og:title": "Day 15: Visualizing Embeddings with t-SNE and PCA",
    "og:description": "Previously, on Day 14: Explained how word embeddings represent words as vectors, the limitations of...",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ff1z5z6g9ixodm311r1ip.png",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@",
    "twitter:title": "Day 15: Visualizing Embeddings with t-SNE and PCA",
    "twitter:description": "Previously, on Day 14: Explained how word embeddings represent words as vectors, the limitations of...",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ff1z5z6g9ixodm311r1ip.png"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/01dfd541bbd0/code/body-000.txt",
      "chars": 1074,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [
    {
      "url": "https://www.youtube.com/watch?v=xpmVLDrCvc0",
      "src": "https://www.youtube.com/watch?v=xpmVLDrCvc0",
      "status": "private_or_members_only",
      "kind": "single_video",
      "error": "ERROR: [youtube] xpmVLDrCvc0: This video is available to this channel's members on level: Profesyonel Destek (or any higher level). Join this channel to get access to members-only content and other exclusive perks.\n",
      "title": "",
      "retried_at": "2026-10-20T18:09:25Z"
    }
  ],
  "links": [
    {
      "href": "https://www.youtube.com/watch?v=xpmVLDrCvc0",
      "label": "4.9. Embedding Techniques Visual Comparison: Meaning Map with PCA and t‑SNE",
      "type": "a"
    },
    {
      "href": "https://www.datacamp.com/tutorial/introduction-t-sne",
      "label": "Introduction to t‑SNE: Nonlinear Dimensionality Reduction and Data Visualization",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:48:09Z",
  "source_url": "https://dev.to/priyeshdave6/day-15-visualizing-embeddings-with-t-sne-and-pca-21b8",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 6722,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "partial",
  "missing_reason": "private_or_members_only",
  "raw_html": "attachments/01dfd541bbd0/pages/source.html",
  "local_html": "attachments/01dfd541bbd0/pages/article.html",
  "body_base_path": "attachments/01dfd541bbd0/pages/",
  "body_markdown_path": "attachments/01dfd541bbd0/pages/article.md"
}