{
  "source_file": "0930",
  "source_platform": "个人博客",
  "author": "Christopher Olah / colah's blog",
  "title_listed": "Neural Networks, Manifolds, and Topology",
  "keywords": "神经网络,流形,拓扑",
  "domain": "机器学习",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/",
  "publish_date": "2014-03",
  "corpus_stage": "预训练/SFT",
  "notes": "神经网络、流形与拓扑的经典论述",
  "id": "fa2424330527",
  "collected_at": "2026-10-02T14:27:35",
  "title": "Neural Networks, Manifolds, and Topology -- colah's blog",
  "body_markdown": "Neural Networks, Manifolds, and Topology\nPosted on April  6, 2014\ntopology, neural networks, deep learning, manifold hypothesis\nRecently, there’s been a great deal of excitement and interest in deep neural networks because they’ve achieved breakthrough results in areas such as computer vision.\n1\nHowever, there remain a number of concerns about them. One is that it can be quite challenging to understand\nwhat\na neural network is really doing. If one trains it well, it achieves high quality results, but it is challenging to understand how it is doing so. If the network fails, it is hard to understand what went wrong.\nWhile it is challenging to understand the behavior of deep neural networks in general, it turns out to be much easier to explore low-dimensional deep neural networks – networks that only have a few neurons in each layer. In fact, we can create visualizations to completely understand the behavior and training of such networks. This perspective will allow us to gain deeper intuition about the behavior of neural networks and observe a connection linking neural networks to an area of mathematics called topology.\nA number of interesting things follow from this, including fundamental lower-bounds on the complexity of a neural network capable of classifying certain datasets.\nA Simple Example\nLet’s begin with a very simple dataset, two curves on a plane. The network will learn to classify points as belonging to one or the other.\nThe obvious way to visualize the behavior of a neural network – or any classification algorithm, for that matter – is to simply look at how it classifies every possible data point.\nWe’ll start with the simplest possible class of neural network, one with only an input layer and an output layer. Such a network simply tries to separate the two classes of data by dividing them with a line.\nThat sort of network isn’t very interesting. Modern neural networks generally have multiple layers between their input and output, called “hidden” layers. At the very least, they have one.\nDiagram of a simple network from Wikipedia\nAs before, we can visualize the behavior of this network by looking at what it does to different points in its domain. It separates the data with a more complicated curve than a line.\nWith each layer, the network transforms the data, creating a new\nrepresentation\n.\n2\nWe can look at the data in each of these representations and how the network classifies them. When we get to the final representation, the network will just draw a line through the data (or, in higher dimensions, a hyperplane).\nIn the previous visualization, we looked at the data in its “raw” representation. You can think of that as us looking at the input layer. Now we will look at it after it is transformed by the first layer. You can think of this as us looking at the hidden layer.\nEach dimension corresponds to the firing of a neuron in the layer.\nThe hidden layer learns a representation so that the data is linearly separable\nContinuous Visualization of Layers\nIn the approach outlined in the previous section, we learn to understand networks by looking at the representation corresponding to each layer. This gives us a discrete list of representations.\nThe tricky part is in understanding how we go from one to another. Thankfully, neural network layers have nice properties that make this very easy.\nThere are a variety of different kinds of layers used in neural networks. We will talk about tanh layers for a concrete example. A tanh layer\n\\(\\tanh(Wx+b)\\)\nconsists of:\nA linear transformation by the “weight” matrix\n\\(W\\)\nA translation by the vector\n\\(b\\)\nPoint-wise application of tanh.\nWe can visualize this as a continuous transformation, as follows:\nThe story is much the same for other standard layers, consisting of an affine transformation followed by pointwise application of a monotone activation function.\nWe can apply this technique to understand more complicated networks. For example, the following network classifies two spirals that are slightly entangled, using four hidden layers. Over time, we can see it shift from the “raw” representation to higher level ones it has learned in order to classify the data. While the spirals are originally entangled, by the end they are linearly separable.\nOn the other hand, the following network, also using multiple layers, fails to classify two spirals that are more entangled.\nIt is worth explicitly noting here that these tasks are only somewhat challenging because we are using low-dimensional neural networks. If we were using wider networks, all this would be quite easy.\n(Andrej Karpathy has made a\nnice demo\nbased on ConvnetJS that allows you to interactively explore networks with this sort of visualization of training!)\nTopology of tanh Layers\nEach layer stretches and squishes space, but it never cuts, breaks, or folds it. Intuitively, we can see that it preserves topological properties. For example, a set will be connected afterwards if it was before (and vice versa).\nTransformations like this, which don’t affect topology, are called homeomorphisms. Formally, they are bijections that are continuous functions both ways.\nTheorem\n: Layers with\n\\(N\\)\ninputs and\n\\(N\\)\noutputs are homeomorphisms, if the weight matrix,\n\\(W\\)\n, is non-singular. (Though one needs to be careful about domain and range.)\nProof\n: Let’s consider this step by step:\nLet’s assume\n\\(W\\)\nhas a non-zero determinant. Then it is a bijective linear function with a linear inverse. Linear functions are continuous. So, multiplying by\n\\(W\\)\nis a homeomorphism.\nTranslations are homeomorphisms\ntanh (and sigmoid and softplus but not ReLU) are continuous functions with continuous inverses. They are bijections if we are careful about the domain and range we consider. Applying them pointwise is a homeomorphism\nThus, if\n\\(W\\)\nhas a non-zero determinant, our layer is a homeomorphism. ∎\nThis result continues to hold if we compose arbitrarily many of these layers together.\nTopology and Classification\n\\(A\\)\nis red,\n\\(B\\)\nis blue\nConsider a two dimensional dataset with two classes\n\\(A, B \\subset \\mathbb{R}^2\\)\n:\n\\[A = \\{x | d(x,0) < 1/3\\}\\]\n\\[B = \\{x | 2/3 < d(x,0) < 1\\}\\]\nClaim\n: It is impossible for a neural network to classify this dataset without having a layer that has 3 or more hidden units, regardless of depth.\nAs mentioned previously, classification with a sigmoid unit or a softmax layer is equivalent to trying to find a hyperplane (or in this case a line) that separates\n\\(A\\)\nand\n\\(B\\)\nin the final represenation. With only two hidden units, a network is topologically incapable of separating the data in this way, and doomed to failure on this dataset.\nIn the following visualization, we observe a hidden representation while a network trains, along with the classification line. As we watch, it struggles and flounders trying to learn a way to do this.\nFor this network, hard work isn’t enough.\nIn the end it gets pulled into a rather unproductive local minimum. Although, it’s actually able to achieve\n\\(\\sim 80\\%\\)\nclassification accuracy.\nThis example only had one hidden layer, but it would fail regardless.\nProof\n: Either each layer is a homeomorphism, or the layer’s weight matrix has determinant 0. If it is a homemorphism,\n\\(A\\)\nis still surrounded by\n\\(B\\)\n, and a line can’t separate them. But suppose it has a determinant of 0: then the dataset gets collapsed on some axis. Since we’re dealing with something homeomorphic to the original dataset,\n\\(A\\)\nis surrounded by\n\\(B\\)\n, and collapsing on any axis means we will have some points of\n\\(A\\)\nand\n\\(B\\)\nmix and become impossible to distinguish between. ∎\nIf we add a third hidden unit, the problem becomes trivial. The neural network learns the following representation:\nWith this representation, we can separate the datasets with a hyperplane.\nTo get a better sense of what’s going on, let’s consider an even simpler dataset that’s 1-dimensional:\n\\[A = [-\\frac{1}{3}, \\frac{1}{3}]\\]\n\\[B = [-1, -\\frac{2}{3}] \\cup [\\frac{2}{3}, 1]\\]\nWithout using a layer of two or more hidden units, we can’t classify this dataset. But if we use one with two units, we learn to represent the data as a nice curve that allows us to separate the classes with a line:\nWhat’s happening? One hidden unit learns to fire when\n\\(x > -\\frac{1}{2}\\)\nand one learns to fire when\n\\(x > \\frac{1}{2}\\)\n. When the first one fires, but not the second, we know that we are in A.\nThe Manifold Hypothesis\nIs this relevant to real world data sets, like image data? If you take the manifold hypothesis really seriously, I think it bears consideration.\nThe manifold hypothesis is that natural data forms lower-dimensional manifolds in its embedding space. There are both theoretical\n3\nand experimental\n4\nreasons to believe this to be true. If you believe this, then the task of a classification algorithm is fundamentally to separate a bunch of tangled manifolds.\nIn the previous examples, one class completely surrounded another. However, it doesn’t seem very likely that the dog image manifold is completely surrounded by the cat image manifold. But there are other, more plausible topological situations that could still pose an issue, as we will see in the next section.\nLinks And Homotopy\nAnother interesting dataset to consider is two linked tori,\n\\(A\\)\nand\n\\(B\\)\n.\nMuch like the previous datasets we considered, this dataset can’t be separated without using\n\\(n+1\\)\ndimensions, namely a\n\\(4\\)\nth dimension.\nLinks are studied in knot theory, an area of topology. Sometimes when we see a link, it isn’t immediately obvious whether it’s an unlink (a bunch of things that are tangled together, but can be separated by continuous deformation) or not.\nA relatively simple unlink.\nIf a neural network using layers with only 3 units can classify it, then it is an unlink. (Question: Can all unlinks be classified by a network with only 3 units, theoretically?)\nFrom this knot perspective, our continuous visualization of the representations produced by a neural network isn’t just a nice animation, it’s a procedure for untangling links. In topology, we would call it an\nambient isotopy\nbetween the original link and the separated ones.\nFormally, an ambient isotopy between manifolds\n\\(A\\)\nand\n\\(B\\)\nis a continuous function\n\\(F: [0,1] \\times X \\to Y\\)\nsuch that each\n\\(F_t\\)\nis a homeomorphism from\n\\(X\\)\nto its range,\n\\(F_0\\)\nis the identity function, and\n\\(F_1\\)\nmaps\n\\(A\\)\nto\n\\(B\\)\n. That is,\n\\(F_t\\)\ncontinuously transitions from mapping\n\\(A\\)\nto itself to mapping\n\\(A\\)\nto\n\\(B\\)\n.\nTheorem\n: There is an ambient isotopy between the input and a network layer’s representation if: a)\n\\(W\\)\nisn’t singular, b) we are willing to permute the neurons in the hidden layer, and c) there is more than 1 hidden unit.\nProof\n: Again, we consider each stage of the network individually:\nThe hardest part is the linear transformation. In order for this to be possible, we need\n\\(W\\)\nto have a positive determinant. Our premise is that it isn’t zero, and we can flip the sign if it is negative by switching two of the hidden neurons, and so we can guarantee the determinant is positive. The space of positive determinant matrices is\npath-connected\n, so there exists\n\\(p: [0,1] \\to GL_n(\\mathbb{R})\\)\n5\nsuch that\n\\(p(0) = Id\\)\nand\n\\(p(1) = W\\)\n. We can continually transition from the identity function to the\n\\(W\\)\ntransformation with the function\n\\(x \\to p(t)x\\)\n, multiplying\n\\(x\\)\nat each point in time\n\\(t\\)\nby the continuously transitioning matrix\n\\(p(t)\\)\n.\nWe can continually transition from the identity function to the\n\\(b\\)\ntranslation with the function\n\\(x \\to x + tb\\)\n.\nWe can continually transition from the identity function to the pointwise use of σ with the function:\n\\(x \\to (1-t)x + tσ(x)\\)\n. ∎\nI imagine there is probably interest in programs automatically discovering such ambient isotopies and automatically proving the equivalence of certain links, or that certain links are separable. It would be interesting to know if neural networks can beat whatever the state of the art is there.\n(Apparently determining if knots are trivial is NP. This doesn’t bode well for neural networks.)\nThe sort of links we’ve talked about so far don’t seem likely to turn up in real world data, but there are higher dimensional generalizations. It seems plausible such things could exist in real world data.\nLinks and knots are\n\\(1\\)\n-dimensional manifolds, but we need 4 dimensions to be able to untangle all of them. Similarly, one can need yet higher dimensional space to be able to unknot\n\\(n\\)\n-dimensional manifolds. All\n\\(n\\)\n-dimensional manifolds can be untangled in\n\\(2n+2\\)\ndimensions.\n6\n(I know very little about knot theory and really need to learn more about what’s known regarding dimensionality and links. If we know a manifold can be embedded in n-dimensional space, instead of the dimensionality of the manifold, what limit do we have?)\nThe Easy Way Out\nThe natural thing for a neural net to do, the very easy route, is to try and pull the manifolds apart naively and stretch the parts that are tangled as thin as possible. While this won’t be anywhere close to a genuine solution, it can achieve relatively high classification accuracy and be a tempting local minimum.\nIt would present itself as very high derivatives on the regions it is trying to stretch, and sharp near-discontinuities. We know these things happen.\n7\nContractive penalties, penalizing the derivatives of the layers at data points, are the natural way to fight this.\n8\nSince these sort of local minima are absolutely useless from the perspective of trying to solve topological problems, topological problems may provide a nice motivation to explore fighting these issues.\nOn the other hand, if we only care about achieving good classification results, it seems like we might not care. If a tiny bit of the data manifold is snagged on another manifold, is that a problem for us? It seems like we should be able to get arbitrarily good classification results despite this issue.\n(My intuition is that trying to cheat the problem like this is a bad idea: it’s hard to imagine that it won’t be a dead end. In particular, in an optimization problem where local minima are a big problem, picking an architecture that can’t genuinely solve the problem seems like a recipe for bad performance.)\nBetter Layers for Manipulating Manifolds?\nThe more I think about standard neural network layers – that is, with an affine transformation followed by a point-wise activation function – the more disenchanted I feel. It’s hard to imagine that these are really very good for manipulating manifolds.\nPerhaps it might make sense to have a very different kind of layer that we can use in composition with more traditional ones?\nThe thing that feels natural to me is to learn a vector field with the direction we want to shift the manifold:\nAnd then deform space based on it:\nOne could learn the vector field at fixed points (just take some fixed points from the training set to use as anchors) and interpolate in some manner. The vector field above is of the form:\n\\[F(x) = \\frac{v_0f_0(x) + v_1f_1(x)}{1+f_0(x)+f_1(x)}\\]\nWhere\n\\(v_0\\)\nand\n\\(v_1\\)\nare vectors and\n\\(f_0(x)\\)\nand\n\\(f_1(x)\\)\nare n-dimensional gaussians. This is inspired a bit by\nradial basis functions\n.\nK-Nearest Neighbor Layers\nI’ve also begun to think that linear separability may be a huge, and possibly unreasonable, amount to demand of a neural network. In some ways, it feels like the natural thing to do would be to use\nk-nearest neighbors\n(k-NN). However, k-NN’s success is greatly dependent on the representation it classifies data from, so one needs a good representation before k-NN can work well.\nAs a first experiment, I trained some MNIST networks (two-layer convolutional nets, no dropout) that achieved\n\\(\\sim 1\\%\\)\ntest error. I then dropped the final softmax layer and used the k-NN algorithm. I was able to consistently achieve a reduction in test error of 0.1-0.2%.\nStill, this doesn’t quite feel like the right thing. The network is still trying to do linear classification, but since we use k-NN at test time, it’s able to recover a bit from mistakes it made.\nk-NN is differentiable with respect to the representation it’s acting on, because of the 1/distance weighting. As such, we can train a network directly for k-NN classification. This can be thought of as a kind of “nearest neighbor” layer that acts as an alternative to softmax.\nWe don’t want to feedforward our entire training set for each mini-batch because that would be very computationally expensive. I think a nice approach is to classify each element of the mini-batch based on the classes of other elements of the mini-batch, giving each one a weight of 1/(distance from classification target).\n9\nSadly, even with sophisticated architecture, using k-NN only gets down to 5-4% test error – and using simpler architectures gets worse results. However, I’ve put very little effort into playing with hyper-parameters.\nStill, I really aesthetically like this approach, because it seems like what we’re “asking” the network to do is much more reasonable. We want points of the same manifold to be closer than points of others, as opposed to the manifolds being separable by a hyperplane. This should correspond to inflating the space between manifolds for different categories and contracting the individual manifolds. It feels like simplification.\nConclusion\nTopological properties of data, such as links, may make it impossible to linearly separate classes using low-dimensional networks, regardless of depth. Even in cases where it is technically possible, such as spirals, it can be very challenging to do so.\nTo accurately classify data with neural networks, wide layers are sometimes necessary. Further, traditional neural network layers do not seem to be very good at representing important manipulations of manifolds; even if we were to cleverly set weights by hand, it would be challenging to compactly represent the transformations we want. New layers, specifically motivated by the manifold perspective of machine learning, may be useful supplements.\n(This is a developing research project. It’s posted as an experiment in doing research openly. I would be delighted to have your feedback on these ideas: you can comment inline or at the end. For typos, technical errors, or clarifications you would like to see added, you are encouraged to make a pull request\non github\n.)\nAcknowledgments\nThank you to Yoshua Bengio, Michael Nielsen, Dario Amodei, Eliana Lorch, Jacob Steinhardt, and Tamsyn Waterhouse for their comments and encouragement.\nThis seems to have really kicked off with\nKrizhevsky\net al.\n, (2012)\n, who put together a lot of different pieces to achieve outstanding results. Since then there’s been a lot of other exciting work.\n↩\nThese representations, hopefully, make the data “nicer” for the network to classify. There has been a lot of work exploring representations recently. Perhaps the most fascinating has been in Natural Language Processing: the representations we learn of words, called word embeddings, have interesting properties. See\nMikolov\net al.\n(2013)\n,\nTurian\net al.\n(2010)\n, and,\nRichard Socher’s work\n. To give you a quick flavor, there is a\nvery nice visualization\nassociated with the Turian paper.\n↩\nA lot of the natural transformations you might want to perform on an image, like translating or scaling an object in it, or changing the lighting, would form continuous curves in image space if you performed them continuously.\n↩\nCarlsson\net al.\nfound that local patches of images form a klein bottle.\n↩\n\\(GL_n(\\mathbb{R})\\)\nis the set of invertible\n\\(n \\times n\\)\nmatrices on the reals, formally called the\ngeneral linear group\nof degree\n\\(n\\)\n.\n↩\nThis result is mentioned in\nWikipedia’s subsection on Isotopy versions\n.\n↩\nSee\nSzegedy\net al.\n, where they are able to modify data samples and find slight modifications that cause some of the best image classification neural networks to misclasify the data. It’s quite troubling.\n↩\nContractive penalties were introduced in contractive autoencoders. See\nRifai\net al.\n(2011)\n.\n↩\nI used a slightly less elegant, but roughly equivalent algorithm because it was more practical to implement in Theano: feedforward two different batches at the same time, and classify them based on each other.\n↩\nBuilt by\nOinkina\nwith\nHakyll\nusing\nBootstrap\n,\nMathJax\n,\nDisqus\n,\nMathBox.js\n,\nHighlight.js\n, \n                    and\nFootnotes.js\n.\nEnable JavaScript for footnotes, Disqus comments, and other cool stuff.",
  "metadata": {
    "title": "Neural Networks, Manifolds, and Topology -- colah's blog",
    "description": null,
    "keywords": null,
    "author": null
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [],
  "images": [
    {
      "status": "ok",
      "url": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/simple2_data.png",
      "bytes": 33776,
      "reused": false,
      "local_path": "attachments/fa2424330527/images/body-8751958f5fd9f6ec5ed2.png",
      "src": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/simple2_data.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/simple2_linear.png",
      "bytes": 37064,
      "reused": false,
      "local_path": "attachments/fa2424330527/images/body-0cd080d8361404df155e.png",
      "src": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/simple2_linear.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/example_network.svg",
      "bytes": 22393,
      "reused": false,
      "local_path": "attachments/fa2424330527/images/body-4198d08ac838b065f3cb.svg",
      "src": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/example_network.svg",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/simple2_0.png",
      "bytes": 40718,
      "reused": false,
      "local_path": "attachments/fa2424330527/images/body-24e167306866258e1712.png",
      "src": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/simple2_0.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/simple2_1.png",
      "bytes": 183638,
      "reused": false,
      "local_path": "attachments/fa2424330527/images/body-0a9e094b7bcb29a99c1b.png",
      "src": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/simple2_1.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/1layer.gif",
      "bytes": 1394066,
      "reused": false,
      "local_path": "attachments/fa2424330527/images/body-28cdd3411bd1199eb790.gif",
      "src": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/1layer.gif",
      "alt": "Gradually applying a neural network layer",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/spiral.1-2.2-2-2-2-2-2.gif",
      "bytes": 3009244,
      "reused": false,
      "local_path": "attachments/fa2424330527/images/body-7a90ade18793e879d30f.gif",
      "src": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/spiral.1-2.2-2-2-2-2-2.gif",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/spiral.2.2-2-2-2-2-2-2.gif",
      "bytes": 2645236,
      "reused": false,
      "local_path": "attachments/fa2424330527/images/body-0c37d5246d5848190954.gif",
      "src": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/spiral.2.2-2-2-2-2-2-2.gif",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/topology_base.png",
      "bytes": 179754,
      "reused": false,
      "local_path": "attachments/fa2424330527/images/body-0bf88fd400c4ce5f0bee.png",
      "src": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/topology_base.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/topology_2D-2D_train.gif",
      "bytes": 3259579,
      "reused": false,
      "local_path": "attachments/fa2424330527/images/body-90822f3b362cad550c55.gif",
      "src": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/topology_2D-2D_train.gif",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/topology_3d.png",
      "bytes": 148400,
      "reused": false,
      "local_path": "attachments/fa2424330527/images/body-1c62cdae3893b9e380ef.png",
      "src": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/topology_3d.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/topology_1d.png",
      "bytes": 2846,
      "reused": false,
      "local_path": "attachments/fa2424330527/images/body-d4a325d503ecd614b6e3.png",
      "src": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/topology_1d.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/topology_1D-2D_train.gif",
      "bytes": 174406,
      "reused": false,
      "local_path": "attachments/fa2424330527/images/body-99207c3075535e621af4.gif",
      "src": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/topology_1D-2D_train.gif",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/link.png",
      "bytes": 92099,
      "reused": false,
      "local_path": "attachments/fa2424330527/images/body-6f519bf1b2cb9628c2a6.png",
      "src": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/link.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/unlink-2spiral.png",
      "bytes": 72136,
      "reused": false,
      "local_path": "attachments/fa2424330527/images/body-8c57746da3442c33528d.png",
      "src": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/unlink-2spiral.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/tangle.png",
      "bytes": 41821,
      "reused": false,
      "local_path": "attachments/fa2424330527/images/body-f49f0462528c3deeacf7.png",
      "src": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/tangle.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/grid_vec.png",
      "bytes": 101113,
      "reused": false,
      "local_path": "attachments/fa2424330527/images/body-e2eb33f4099fa09a00e1.png",
      "src": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/grid_vec.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/grid_bubble.png",
      "bytes": 103993,
      "reused": false,
      "local_path": "attachments/fa2424330527/images/body-1f2742281c8f87af4b8b.png",
      "src": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/img/grid_bubble.png",
      "alt": "",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "http://cs.stanford.edu/people/karpathy/convnetjs//demo/classify2d.html",
      "label": "nice demo",
      "type": "a"
    },
    {
      "href": "http://en.wikipedia.org/wiki/Connected_space",
      "label": "path-connected",
      "type": "a"
    },
    {
      "href": "http://en.wikipedia.org/wiki/Radial_basis_function",
      "label": "radial basis functions",
      "type": "a"
    },
    {
      "href": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/knn",
      "label": "k-nearest neighbors",
      "type": "a"
    },
    {
      "href": "https://github.com/colah/NN-Topology-Post",
      "label": "on github",
      "type": "a"
    },
    {
      "href": "http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf",
      "label": "Krizhevsky et al. , (2012)",
      "type": "a"
    },
    {
      "href": "http://research.microsoft.com/pubs/189726/rvecs.pdf",
      "label": "Mikolov et al. (2013)",
      "type": "a"
    },
    {
      "href": "http://www.iro.umontreal.ca/~lisa/pointeurs/turian-wordrepresentations-acl10.pdf",
      "label": "Turian et al. (2010)",
      "type": "a"
    },
    {
      "href": "http://www.socher.org/",
      "label": "Richard Socher’s work",
      "type": "a"
    },
    {
      "href": "http://metaoptimize.s3.amazonaws.com/cw-embeddings-ACL2010/embeddings-mostcommon.EMBEDDING_SIZE=50.png",
      "label": "very nice visualization",
      "type": "a"
    },
    {
      "href": "http://comptop.stanford.edu/u/preprints/mumford.pdf",
      "label": "Carlsson et al.",
      "type": "a"
    },
    {
      "href": "http://en.wikipedia.org/wiki/General_linear_group",
      "label": "general linear group",
      "type": "a"
    },
    {
      "href": "http://en.wikipedia.org/wiki/Whitney_embedding_theorem",
      "label": "Wikipedia’s subsection on Isotopy versions",
      "type": "a"
    },
    {
      "href": "http://cs.nyu.edu/~zaremba/docs/understanding.pdf",
      "label": "Szegedy et al.",
      "type": "a"
    },
    {
      "href": "http://www.iro.umontreal.ca/~lisa/pointeurs/ICML2011_explicit_invariance.pdf",
      "label": "Rifai et al. (2011)",
      "type": "a"
    },
    {
      "href": "https://github.com/oinkina",
      "label": "Oinkina",
      "type": "a"
    },
    {
      "href": "http://jaspervdj.be/hakyll",
      "label": "Hakyll",
      "type": "a"
    },
    {
      "href": "http://getbootstrap.com/",
      "label": "Bootstrap",
      "type": "a"
    },
    {
      "href": "http://www.mathjax.org/",
      "label": "MathJax",
      "type": "a"
    },
    {
      "href": "http://disqus.com/",
      "label": "Disqus",
      "type": "a"
    },
    {
      "href": "https://github.com/unconed/MathBox.js",
      "label": "MathBox.js",
      "type": "a"
    },
    {
      "href": "http://highlightjs.org/",
      "label": "Highlight.js",
      "type": "a"
    },
    {
      "href": "http://ignorethecode.net/blog/2010/04/20/footnotes/",
      "label": "Footnotes.js",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:52:33Z",
  "source_url": "https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/",
  "papers": [
    {
      "status": "ok",
      "url": "http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf",
      "bytes": 1414139,
      "reused": true,
      "local_path": "papers/repair/4d740a2183a85e25a439.pdf"
    },
    {
      "status": "invalid_file_type",
      "url": "http://research.microsoft.com/pubs/189726/rvecs.pdf",
      "content_type": "text/html; charset=UTF-8",
      "retried_at": "2026-10-20T17:55:09Z"
    },
    {
      "status": "invalid_file_type",
      "url": "http://www.iro.umontreal.ca/~lisa/pointeurs/turian-wordrepresentations-acl10.pdf",
      "content_type": "text/html; charset=utf-8",
      "retried_at": "2026-10-20T17:55:03Z"
    },
    {
      "status": "download_error",
      "url": "http://comptop.stanford.edu/u/preprints/mumford.pdf",
      "reason": "ConnectionError: HTTPConnectionPool(host='comptop.stanford.edu', port=80): Max retries exceeded with url: /u/preprints/mumford.pdf (Caused by NameResolutionError(\"HTTPConnection(host='comptop.stanf",
      "retried_at": "2026-10-20T22:24:19Z"
    },
    {
      "status": "http_403",
      "url": "http://cs.nyu.edu/~zaremba/docs/understanding.pdf",
      "retried_at": "2026-10-20T17:55:03Z"
    },
    {
      "status": "invalid_file_type",
      "url": "http://www.iro.umontreal.ca/~lisa/pointeurs/ICML2011_explicit_invariance.pdf",
      "content_type": "text/html; charset=utf-8",
      "retried_at": "2026-10-20T17:55:03Z"
    }
  ],
  "actual_video_files": [],
  "body_chars": 20541,
  "expected_body_images": 18,
  "discovery_status": "ok",
  "delivery_status": "partial",
  "missing_reason": "download_error；http_403；invalid_file_type",
  "raw_html": "attachments/fa2424330527/pages/source.html",
  "local_html": "attachments/fa2424330527/pages/article.html",
  "body_base_path": "attachments/fa2424330527/pages/",
  "body_markdown_path": "attachments/fa2424330527/pages/article.md"
}