{
  "source_file": "0930",
  "source_platform": "个人博客",
  "author": "Andrej Karpathy",
  "title_listed": "Visualizing Top Tweeps with t-SNE, in Javascript",
  "keywords": "t-SNE,降维,可视化",
  "domain": "数据处理",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://karpathy.github.io/2014/07/02/visualizing-top-tweeps-with-t-sne-in-Javascript/",
  "publish_date": "2014-07",
  "corpus_stage": "数据处理",
  "notes": "t-SNE降维可视化实战",
  "id": "804388b2e97f",
  "collected_at": "2026-10-02T14:13:56",
  "title": "Visualizing Top Tweeps with t-SNE, in Javascript",
  "body_markdown": "I was recently looking into various ways of embedding unlabeled, high-dimensional data in 2 dimensions for visualization. A wide variety of methods have been proposed for this task.\nThis Review paper\nfrom 2009 contains nice references to many of them (PCA, Kernel PCA, Isomap, LLE, Autoencoders, etc.). If you have Matlab available, the\nDimensionality Reduction Toolbox\nhas a nice implementation of many of these methods. Scikit Learn also has a brief section on\nManifold Learning\nalong with the implementation.\nAmong these algorithms, t-SNE comes across as one that has a pleasing, intuitive formulation, simple gradient and nice properties. Here is a\nGoogle Tech Talks video\nof Laurens van der Maaten (the author) explaining the method. I set out to re-implement t-SNE from scratch since doing so is the best way of learning something that I know of, and what better language to do this in than - Javascript! :)\nLong story short, I’ve implemented t-SNE in JS, released it as\ntsnejs on Github\n, and created a small demo that uses the library to visualize the top twitter accounts based on what they talk about. In this post, I thought it might be fun to document a small 1-day project like this, from beginning to end. This also gives me an opportunity to describe some of my projects toolkit, which others might find useful.\nFinal demo\nFirst, take a look at the\nfinal demo\n. To create this demo I found the top 500 most followed accounts on Twitter, downloaded 200 of their tweets and then measured differences in what they tweet about. These differences are then fed to t-SNE to produce a 2-dimensional visualization, where nearby people tweet similar things. Fun!\nFetching top tweeps\nWe first have to identify the top 500 tweeps. I googled “top twitter accounts” and found http://twitaholic.com/ , which lists them out. However, the accounts are embedded in the webpage and we need to extract them in structured format. For this, I love a recent YC startup\nKimono\n; I use it extensively to scrape structured data from websites. It lets you click the elements of interest (the Twitter handles in this case), and extracts them out in JSON. Easy as pie!\nCollecting tweets\nNow we have a list of top 500 tweeps and we’d like to obtain their tweets to get an idea about what they tweet about. My library of choice for this task is\nTweepy\n. Their documentation is quite terrible but if you browse the source code things seem relatively simple. Here’s an example call to get 200 tweets for a given user:\ntweets\n=\ntweepy\n.\nCursor\n(\napi\n.\nuser_timeline\n,\nscreen_name\n=\nuser\n).\nitems\n(\n200\n)\nWe iterate this over all users, extract the tweet text, and dumpt it all into files, one per account. I had to be careful with two annoyances in process:\nTwitter puts severe rate limits on API calls, so this actually took several hours to collect, wrapped up in try catch blocks and\ntime.sleep\ncalls.\nThe returned text is in Unicode, which leads to trouble if you’re going to try to write it to file.\nOne solution for the second annoyance is to use the codecs library:\nimport\ncodecs\ncodecs\n.\nopen\n(\nfilename\n,\n'w'\n,\n'utf-8'\n).\nwrite\n(\ntweet_texts\n)\nOh, and lets also grab and save the Twitter profile pictures, which we’ll use in the visualization. An example for one user might be:\nimport\nurllib\n# yes I know this is deprecated\nuserobj\n=\napi\n.\nget_user\n(\nscreen_name\n=\nuser\n)\nurllib\n.\nurlretrieve\n(\nimgname\n,\nuserobj\n.\nprofile_image_url\n)\n# save image to disk\nI should mention that I write a lot of quick and dirty Python code in\nIPython Notebooks\n, which I very warmly recommend. If you’re writing all your Python in text editors, you’re seriously missing out.\nQuantifying Tweep differences\nWe now have 500 tweeps and their 200 most recent tweets concatenated in 500 files. We’d now like to find who tweets about similar things.\nScikit learn\nis very nice for quick NLP tasks like this. In particular, we load up all the files and create a 500-long array where every element are the 200 concatenated tweets. Then we use the\nTfidfVectorizer\nclass to extract all words and bigrams from the text data, and to turn every user’s language into one tfidf vector. This vector is a fingerprint of the language that each person uses. Here’s how we can simply wire this up:\nfrom\nsklearn.feature_extraction.text\nimport\nCountVectorizer\n,\nTfidfVectorizer\nvectorizer\n=\nTfidfVectorizer\n(\nmin_df\n=\n2\n,\nstop_words\n=\n'english'\n,\n\\\nstrip_accents\n=\n'unicode'\n,\nlowercase\n=\nTrue\n,\nngram_range\n=\n(\n1\n,\n2\n),\n\\\nnorm\n=\n'l2'\n,\nsmooth_idf\n=\nTrue\n,\nsublinear_tf\n=\nFalse\n,\nuse_idf\n=\nTrue\n)\nX\n=\nvectorizer\n.\nfit_transform\n(\nuser_language_array\n)\nD\n=\n-\n(\nX\n*\nX\n.\nT\n).\ntodense\n()\n# Distance matrix: dot product between tfidf vectors\nIn the above,\nuser_language_array\nis the 500-element array that has the concatenated tweets. The\nTfidfVectorizer\nclass looks through all tweets and takes note of all words (unigrams) and word bigrams (i.e. series of two words). It builds a dictionary out of all unigram/bigrams and essentially counts up how often every person uses each one. Here’s an example of some tweet text converted to unigram/bigrams:\nThe tfidf vectors are returned stacked up as rows inside\nX\n, which has size\n500 x 87,342\n. Every one of the 87,342 dimensions corresponds to some unigram or bigram. For example, the 10,000th dimension could correspond to the frequency of usage of the unigram “YOLO”. The vectors are L2 normalized, so the dot product between these vectors is related to the angle between any two vectors. This can be interpreted as the similarity of language. Finally, we dump the matrix and the usernames into a JSON file, and we’re ready to load things up in Javascript!\nThe Visualization parts\nWe now create an .html file and import\njQuery\n(as always), and\nd3js\n, which I like to use for any kind of plotting. We load up the JSON that stores our distances and usernames with jQuery, and use d3js to initialize the SVG element that will hold all the users. For starters, we plot the users at random position but we will soon arrange them so that similar users cluster nearby with t-SNE. Inspect the code on the\ndemo page\nto see the jQuery and d3js parts (Ctrl+U). In the code, we see a few things I like to use:\nI like to use\nGoogle Fonts\nto get prettier-than-default fonts. Here, for example I’m importing Roboto, and then using it in the CSS.\nNext, we see an import of\nsyntaxhighlighter\ncode which dynamically highlights code on your page.\nThen we see\nGoogle tracking JS code\n, which lets me track statistics for the website on Google Analytics.\nI didn’t use\nBootstrap\non this website because it’s very small and simple, but normally I would because this makes your website right away work nicely on mobile.\nt-SNE\nFinally we get to the meat! We need to arrange the users in our d3js plot so that similar users appear nearby. The t-SNE cost function was described in this\n2008 paper by van der Maaten and Hinton\n. Similar to many other methods, we set up two distance metrics in the original and the embedded space and minimize their difference. In t-SNE in particular, the original space distance is based on a Gaussian distribution and the embedded space is based on the heavy-tailed Student-t distribution. The KL-divergence formulation has the nice property that it is asymmetric in how it penalizes distances between the two spaces:\nIf two points are close in the original space, there is a strong attractive force between the the points in the embedding\nConversely, if the two points are far apart in the original space, the algorithm is relatively free to place these points around.\nThus, the algorithm preferentially cares about preserving the local structure of the high-dimensional data. Conveniently, the authors link to multiple implementations of t-SNE on\ntheir website\n, which allows us to see some code for reference as well (if you’re like me, reading code can be much easier than reading text descriptions). We’re ready to write up the Javascript version!\nThe final code can be seen in\ntsne.js file\n, on Github. Note how we’re wrapping all the JS code into a function closure so that we don’t pollute the global namespace. This is a very common trick in Javascript that is essentially used to implement classes. Note also the large number of utility boring code I had to include up top because Javascript is not exactly intended for math :) The core function where all magic happens is\ncostGrad()\n, which computes the cost function and the gradient of the objective. The correct implementation of this function is double checked with\ndebugGrad()\ngradient check. Once the analytic gradient checks out compared to numeric gradient, we’re good to go! We set up a piece of Javascript to call our\nstep()\nfunction repeatedly (\nsetInterval()\ncall), and we plot the solution as it gets computed.\nPhew! Final result, again:\nt-SNE demo\n.\nI hope some of the references were useful. If you use tsnejs to embed some of your data, let me know!\nBonus: Word Embedding t-SNE Visualization\nI created another demo, this time to visualize word vector embeddings. Head\nover here\nto see it. The word embeddings are trained as described in this\nACL 2012 paper\n.\nThe (unsupervised) objective function makes it so that words that are interchangable (i.e. occur in very similar surrounding context) are close in the embedding. This comes across in the visualization!",
  "metadata": {
    "title": "Visualizing Top Tweeps with t-SNE, in Javascript",
    "description": "Musings of a Computer Scientist.",
    "keywords": null,
    "author": null
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/804388b2e97f/code/body-000.txt",
      "chars": 71,
      "language": "text"
    },
    {
      "file": "attachments/804388b2e97f/code/body-001.txt",
      "chars": 69,
      "language": "text"
    },
    {
      "file": "attachments/804388b2e97f/code/body-002.txt",
      "chars": 165,
      "language": "text"
    },
    {
      "file": "attachments/804388b2e97f/code/body-003.txt",
      "chars": 393,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://karpathy.github.io/assets/tsne_preview.jpeg",
      "bytes": 93579,
      "reused": false,
      "local_path": "attachments/804388b2e97f/images/body-25111132e262fba810f6.jpeg",
      "src": "https://karpathy.github.io/assets/tsne_preview.jpeg",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://karpathy.github.io/assets/tsne_sentprepro.jpeg",
      "bytes": 165449,
      "reused": false,
      "local_path": "attachments/804388b2e97f/images/body-7855c355c583a77800fe.jpeg",
      "src": "https://karpathy.github.io/assets/tsne_sentprepro.jpeg",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://karpathy.github.io/assets/tsne_eg.jpeg",
      "bytes": 52376,
      "reused": false,
      "local_path": "attachments/804388b2e97f/images/body-dcc49733cab0c446dba9.jpeg",
      "src": "https://karpathy.github.io/assets/tsne_eg.jpeg",
      "alt": "",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [
    {
      "url": "https://www.youtube.com/watch?v=RJVL80Gg3lA",
      "src": "http://www.youtube.com/watch?v=RJVL80Gg3lA",
      "status": "ok",
      "kind": "single_video",
      "local_path": "attachments/804388b2e97f/videos/repair-RJVL80Gg3lA.mp4",
      "bytes": 83813920,
      "valid": true,
      "height": 720,
      "width": 1280,
      "duration": "3309.673033",
      "reused": false
    }
  ],
  "links": [
    {
      "href": "http://homepage.tudelft.nl/19j49/Matlab_Toolbox_for_Dimensionality_Reduction_files/TR_Dimensiereductie.pdf",
      "label": "This Review paper",
      "type": "a"
    },
    {
      "href": "http://homepage.tudelft.nl/19j49/Matlab_Toolbox_for_Dimensionality_Reduction.html",
      "label": "Dimensionality Reduction Toolbox",
      "type": "a"
    },
    {
      "href": "http://scikit-learn.org/stable/modules/manifold.html",
      "label": "Manifold Learning",
      "type": "a"
    },
    {
      "href": "http://www.youtube.com/watch?v=RJVL80Gg3lA",
      "label": "Google Tech Talks video",
      "type": "a"
    },
    {
      "href": "https://github.com/karpathy/tsnejs",
      "label": "tsnejs on Github",
      "type": "a"
    },
    {
      "href": "http://cs.stanford.edu/people/karpathy/tsnejs/",
      "label": "final demo",
      "type": "a"
    },
    {
      "href": "https://www.kimonolabs.com/",
      "label": "Kimono",
      "type": "a"
    },
    {
      "href": "https://github.com/tweepy/tweepy",
      "label": "Tweepy",
      "type": "a"
    },
    {
      "href": "http://ipython.org/notebook.html",
      "label": "IPython Notebooks",
      "type": "a"
    },
    {
      "href": "http://scikit-learn.org/stable/",
      "label": "Scikit learn",
      "type": "a"
    },
    {
      "href": "http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html",
      "label": "TfidfVectorizer",
      "type": "a"
    },
    {
      "href": "http://jquery.com/",
      "label": "jQuery",
      "type": "a"
    },
    {
      "href": "http://d3js.org/",
      "label": "d3js",
      "type": "a"
    },
    {
      "href": "http://jmlr.csail.mit.edu/papers/volume9/vandermaaten08a/vandermaaten08a.pdf",
      "label": "2008 paper by van der Maaten and Hinton",
      "type": "a"
    },
    {
      "href": "http://homepage.tudelft.nl/19j49/t-SNE.html",
      "label": "their website",
      "type": "a"
    },
    {
      "href": "http://cs.stanford.edu/people/karpathy/tsnejs/wordvecs.html",
      "label": "over here",
      "type": "a"
    },
    {
      "href": "http://www.socher.org/index.php/Main/ImprovingWordRepresentationsViaGlobalContextAndMultipleWordPrototypes",
      "label": "ACL 2012 paper",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:51:46Z",
  "source_url": "https://karpathy.github.io/2014/07/02/visualizing-top-tweeps-with-t-sne-in-Javascript/",
  "papers": [
    {
      "status": "download_error",
      "url": "http://homepage.tudelft.nl/19j49/Matlab_Toolbox_for_Dimensionality_Reduction_files/TR_Dimensiereductie.pdf",
      "reason": "ConnectionError: HTTPConnectionPool(host='homepage.tudelft.nl', port=80): Max retries exceeded with url: /19j49/Matlab_Toolbox_for_Dimensionality_Reduction_files/TR_Dimensiereductie.pdf (Caused by ",
      "retried_at": "2026-10-20T22:25:43Z"
    },
    {
      "status": "ok",
      "url": "http://jmlr.csail.mit.edu/papers/volume9/vandermaaten08a/vandermaaten08a.pdf",
      "bytes": 3716712,
      "reused": false,
      "local_path": "papers/repair/997c41e8af74ff38b0d7.pdf"
    }
  ],
  "actual_video_files": [],
  "body_chars": 9338,
  "expected_body_images": 3,
  "discovery_status": "ok",
  "delivery_status": "partial",
  "missing_reason": "download_error",
  "raw_html": "attachments/804388b2e97f/pages/source.html",
  "local_html": "attachments/804388b2e97f/pages/article.html",
  "body_base_path": "attachments/804388b2e97f/pages/",
  "body_markdown_path": "attachments/804388b2e97f/pages/article.md"
}