{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "Mitansh Gor",
  "title_listed": "GEN-AI 2 : Auto Encoders",
  "keywords": "",
  "domain": "大模型算法",
  "content_type": "技术教程",
  "is_paid_flag": "否",
  "url": "https://dev.to/mitanshgor/gen-ai-3-vae-2hoi",
  "publish_date": "2025-04-20",
  "corpus_stage": "",
  "notes": "",
  "id": "1bfdfedc7d4a",
  "collected_at": "2026-10-02T13:53:24",
  "title": "GEN-AI 2 : Auto Encoders - DEV Community",
  "body_markdown": "📌 What Are Generative Models?\nGenerative Models\nare machine learning algorithms that learn the\nunderlying patterns\nor distribution of a dataset to\ngenerate new data\nthat resembles the original.\nIn human terms:\nThey look at thousands of pictures of cats... and then start\nimagining\ntheir own cats.\n🌟 Popular Examples of Generative Models\nModel\nWhat It Does\nCommon Uses\nGANs\n(Generative Adversarial Networks)\nPit 2 networks (Generator & Discriminator) against each other\nDeepfakes, art generation, face synthesis\nVAEs\n(Variational Autoencoders)\nProbabilistic version of AEs with smooth latent space\nImage reconstruction, data generation\nAutoregressive Models\nPredict next element based on previous ones\nGPT, music generation\nDiffusion Models\nDenoise images gradually to generate high-res outputs\nDALL·E 3, Stable Diffusion\n🛣️ How Do Generative Models Learn?\nImagine you’re an artist learning how to draw cats. You spend hours going through cat books, sketching from photos, and watching cat videos on YouTube (because obviously). Over time, without memorizing each specific cat, you start to get a feel for what a “cat” is — four legs, whiskers, maybe some attitude.\nTheir mission:\nLearn the vibe of the data so well that they can create new things that look like it.\nThe Typical Path:\nInput Data\n:  This is your model’s training time — a massive gallery of examples.\nThink: Thousands of cat pictures (or faces, music, text). No labels needed — just raw, unfiltered examples. The model isn’t told: “This is a cat.” It just sees cats, over and over. And from that, it begins to understand patterns.\nLatent Representation\nThe model tries to compress all the messy, pixel-by-pixel cat data into something smaller and meaningful, kind of like the artist thinking: “Okay, I don’t remember every cat, but most cats have pointy ears, some fur texture, and this cat-ish outline... I can work with that.” In AI terms, this is the\nlatent space\n— a multi-dimensional map where similar things live near each other. Imagine a universe where all cats are clustered in one corner, dogs in another, and so on.\nReconstruction or Generation\n:\nOnce the model has this compressed knowledge, it can do either of these:\nReconstruct what it saw: “Can I redraw the same cat just from memory?”\nGenerate something new: “Can I draw a brand new cat that no one’s ever seen, but still looks legit?”\nAnd here’s the twist — different types of models do this learning differently.\nThis learning can be:\nExplicit\n(learning the actual data distribution like VAEs)\nThese models are like a structured student. They try to understand the actual math behind how your data is spread out, trying to learn the probability distribution. They care about making the latent space smooth, organized, and logical.\nYou can walk around the space, sample random points, and get meaningful results.\nImplicit\n(learning to fool a critic like in GANs)\nThese models are the street-smart artists. They don’t care about formulas — they just want their results to look good and be convincing. GANs do this through a game:\nOne network (the Generator) tries to create fake data\nAnother (the Discriminator) tries to spot the fakes\nThey improve together in a loop until the fakes are too good to tell apart.\nThey never explicitly learn the actual math of the data, just how to mimic it well enough to fool someone.\n🏗️ Autoencoders Architecture (AE): Compress, Learn, Reconstruct\nLet’s say we’re dealing with image data (like handwritten digits from MNIST — each image is 28x28 pixels).\nInput Layer\nYou feed the model a raw input vector, say a flattened version of the original data (e.g., 28x28 image = 784 dims). This is just pixel intensity values (usually between 0 and 1 if normalized).\nEncoder\nThe encoder is a neural network that compresses the input into a lower-dimensional representation — the latent vector z.\nz=FuncEncoder(x)\nFor example, you’re squeezing 784 dimensions down to 32. Hence size of the Z vector = 2.\nWe want the model to learn what matters most. By forcing it to compress, the model has to filter out unimportant noise and capture the essence of the data. This is called the bottleneck. You keep only the information about the image that matters.\nHow can we do that?\nWe pass the input image (28X28) through 3 convolutional layers, one after the other.\nWe use a stride of 2 (for eg), which means each layer reduces the image size by half while increasing the number of feature channels (kind of like zooming out but seeing more detail).\nSo image (ColXRowXfeatureChannel) from 28X28X1 -> 14X14X32 -> 7X7X64 -> 4X4X128 sized image.\nFlatten (4 × 4 × 128) → 2048-dimensional vector\n2048-dimensional vector Input as a Fully Connected Layer. Gives 2-Dimensional Output (Latent Vector - Z) - a compact 2D representation of your image.\nDecoder\nNow the decoder takes that compressed vector z and tries to rebuild the original image. It is a reverse-engineering/mirror image of the encoder.\nx ′ = FuncDecoder(z)\nIt converts a 2D latent back to a 28*28 image.\nHow can we do that?\nInstead of using normal convolutional layers that shrink things down, it uses transposed convolutions (or \"deconvolutions\") to build things back up to the original size. While a regular Conv layer with stride 2 shrinks the image, a Conv2DTranspose with stride 2 does the opposite — it doubles the size of the feature map. You can also use upsampling, which simply stretches the image to a larger size, but unlike transposed convolutions, it doesn't learn anything — it just resizes.\nHere’s how the decoder does that:\nStart with the 2-dimensional latent vector (Z), and pass it through a fully connected (dense) layer to expand it into a larger vector — say, 2048 dimensions.\nReshape that 2048-dimensional vector back into a 4 × 4 × 128 tensor, like rewinding a movie to the middle scene.\nNow we apply transposed convolutional layers (Conv2DTranspose) one after another — each with a stride of 2 — which gradually upsamples the feature maps while reducing the number of channels: 4×4×128 → 7×7×64 → 14×14×32 → 28×28×1\nThe final layer outputs an image that's the same size as the input — in this case, 28×28 pixels with 1 channel (grayscale).\nOptionally, we apply a sigmoid activation at the end so all pixel values are between 0 and 1 — perfect for comparing with the original input during loss calculation!\nAfter the model reconstructs the 28*28 image, it checks the loss of the result by comparing it with the original image.\nLoss Func Check\nWe want the input 𝑥 and the reconstruction 𝑥′ to be as close as possible.\nSo we minimize a loss function that measures the difference between them.\nTwo Common Losses:\nMean Square Error (MSE / L2 loss):  It computes the square of the difference between each element in the predicted output vector and the corresponding element in the ground truth, then takes the average across all elements. This loss is particularly useful when dealing with continuous data and is known for its sensitivity to outliers.\nMSE Loss(J) = (1/n) * Σ(y_true - y_pred)^2\nBinary Cross-Entropy (BCE):  used to measure the difference between two probability distributions. We use a sigmoid activation in the final layer of the decoder so that\n0 < x′< 1\n, which makes it interpretable as a probability — a requirement for BCE to work.\nThe model looks at that loss and works backward( backpropagation). It figures out, layer by layer, how much each connection (weight) contributed to the mistake. Then, using an optimizer like Adam or SGD, it tweaks those weights just a little to do better next time. This process repeats again and again, and over time, the model gets good at understanding what matters in the data, like learning the essence of a handwritten “3” without being told it’s a 3.\n❌ Limitations of Autoencoders\nWhen we try to sample (pick) a random point from the latent space, we often end up with strange or meaningless outputs. That’s because some regions of the space are crowded, while others are empty, so we might accidentally pick a point from a “dead zone” where the model has never seen any training data.\nAutoencoders don’t enforce any structure in the latent space. There's no rule saying, “the points should follow a neat, smooth pattern like a normal distribution.” So when we try to generate new data by sampling from the space, we don’t really know where to pick from — it's like throwing darts in the dark.\nThere are gaps or holes in the latent space — areas that never got used during training. The model has no clue how to decode these zones, so it often produces blurry or totally unrecognizable images.\nEven points close together in space can produce wildly different outputs. For example, if (–1, –1) creates a nice image, (–1.1, –1.1) might give total noise. That’s because the model doesn’t care about making the space smooth or continuous — it's just trying to reconstruct training images.\nIn 2D latent spaces (for example, when visualizing MNIST), this problem isn’t too bad — everything is kind of squished together. But when we move to higher dimensions (like when generating complex stuff — faces, clothes, etc.), the gaps and discontinuities get worse. So even though more dimensions are needed to capture complex patterns, they also make the space harder to sample from safely.",
  "metadata": {
    "title": "GEN-AI 2 : Auto Encoders - DEV Community",
    "description": "📌 What Are Generative Models?   Generative Models are machine learning algorithms that... Tagged with deeplearning, ai, generative, autoencoder.",
    "keywords": "deeplearning, ai, generative, autoencoder, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/mitanshgor/gen-ai-3-vae-2hoi",
    "og:title": "GEN-AI 2 : Auto Encoders",
    "og:description": "📌 What Are Generative Models?   Generative Models are machine learning algorithms that...",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1000,height=500,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F4ydjnm3mt24q4gkigf5e.jpeg",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@",
    "twitter:title": "GEN-AI 2 : Auto Encoders",
    "twitter:description": "📌 What Are Generative Models?   Generative Models are machine learning algorithms that...",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1000,height=500,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F4ydjnm3mt24q4gkigf5e.jpeg"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [],
  "images": [
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fy1khlbnbmjo19tesy4k7.png",
      "bytes": 50424,
      "reused": true,
      "local_path": "attachments/1bfdfedc7d4a/images/002.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fy1khlbnbmjo19tesy4k7.png",
      "alt": "IIII3",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fs3n8chlda40ejddng3m4.png",
      "bytes": 44210,
      "reused": true,
      "local_path": "attachments/1bfdfedc7d4a/images/003.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fs3n8chlda40ejddng3m4.png",
      "alt": "I0",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fthdz4m86qtq3yb6ntfpe.png",
      "bytes": 114362,
      "reused": true,
      "local_path": "attachments/1bfdfedc7d4a/images/004.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fthdz4m86qtq3yb6ntfpe.png",
      "alt": "D3",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Flgdzfxle1iqpvx1ojoq2.gif",
      "bytes": 455063,
      "reused": true,
      "local_path": "attachments/1bfdfedc7d4a/images/005.gif",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Flgdzfxle1iqpvx1ojoq2.gif",
      "alt": " ",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fpw5fwby1a9qv7fq5l69p.png",
      "bytes": 56020,
      "reused": true,
      "local_path": "attachments/1bfdfedc7d4a/images/006.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fpw5fwby1a9qv7fq5l69p.png",
      "alt": "I3",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fwjbkny3e96uvdoqvzy15.png",
      "bytes": 41420,
      "reused": true,
      "local_path": "attachments/1bfdfedc7d4a/images/007.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fwjbkny3e96uvdoqvzy15.png",
      "alt": "D1",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fnz7xaay12u4n6ufii2gv.png",
      "bytes": 76612,
      "reused": true,
      "local_path": "attachments/1bfdfedc7d4a/images/008.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fnz7xaay12u4n6ufii2gv.png",
      "alt": "Stride and Filter",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Femu4wpnr26jtk8e1yjyu.png",
      "bytes": 31916,
      "reused": true,
      "local_path": "attachments/1bfdfedc7d4a/images/009.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Femu4wpnr26jtk8e1yjyu.png",
      "alt": "III4",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F7h4qiuarqgyplohdsput.png",
      "bytes": 60162,
      "reused": true,
      "local_path": "attachments/1bfdfedc7d4a/images/010.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F7h4qiuarqgyplohdsput.png",
      "alt": "D2",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F0sj1rzqcxj0vsh4ixmoh.png",
      "bytes": 17284,
      "reused": true,
      "local_path": "attachments/1bfdfedc7d4a/images/011.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F0sj1rzqcxj0vsh4ixmoh.png",
      "alt": "II0",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fh0d7miukjze3oczbkrsc.png",
      "bytes": 290900,
      "reused": true,
      "local_path": "attachments/1bfdfedc7d4a/images/012.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fh0d7miukjze3oczbkrsc.png",
      "alt": "copypasterMeme",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fy1khlbnbmjo19tesy4k7.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fs3n8chlda40ejddng3m4.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fthdz4m86qtq3yb6ntfpe.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Flgdzfxle1iqpvx1ojoq2.gif",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fpw5fwby1a9qv7fq5l69p.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fwjbkny3e96uvdoqvzy15.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fnz7xaay12u4n6ufii2gv.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Femu4wpnr26jtk8e1yjyu.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F7h4qiuarqgyplohdsput.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F0sj1rzqcxj0vsh4ixmoh.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fh0d7miukjze3oczbkrsc.png",
      "label": "",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:48:49Z",
  "source_url": "https://dev.to/mitanshgor/gen-ai-3-vae-2hoi",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 9265,
  "expected_body_images": 11,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/1bfdfedc7d4a/pages/source.html",
  "local_html": "attachments/1bfdfedc7d4a/pages/article.html",
  "body_base_path": "attachments/1bfdfedc7d4a/pages/",
  "body_markdown_path": "attachments/1bfdfedc7d4a/pages/article.md"
}