{ "source_file": "0930", "source_platform": "个人博客", "author": "Jay Alammar", "title_listed": "The Illustrated Stable Diffusion", "keywords": "Stable Diffusion,扩散,文生图", "domain": "图像处理", "content_type": "图解教程", "is_paid_flag": "否", "url": "https://jalammar.github.io/illustrated-stable-diffusion/", "publish_date": "2022-10", "corpus_stage": "预训练/SFT", "notes": "Stable Diffusion图解", "id": "9fa49ae4db5b", "collected_at": "2026-10-02T14:19:26", "title": "The Illustrated Stable Diffusion – Jay Alammar – Visualizing machine learning one concept at a time.", "body_markdown": "The Illustrated Stable Diffusion\nTranslations:\nChinese\n,\nVietnamese\n.\n(\nV2 Nov 2022\n: Updated images for more precise description of forward diffusion. A few more images in this version)\nAI image generation is the most recent AI capability blowing people’s minds (mine included). The ability to create striking visuals from text descriptions has a magical quality to it and points clearly to a shift in how humans create art. The release of\nStable Diffusion\nis a clear milestone in this development because it made a high-performance model available to the masses (performance in terms of image quality, as well as speed and relatively low resource/memory requirements).\nAfter experimenting with AI image generation, you may start to wonder how it works.\nThis is a gentle introduction to how Stable Diffusion works.\nStable Diffusion is versatile in that it can be used in a number of different ways. Let’s focus at first on image generation from text only (text2img). The image above shows an example text input and the resulting generated image (The actual complete prompt is here). Aside from text to image, another main way of using it is by making it alter images (so inputs are text + image).\nLet’s start to look under the hood because that helps explain the components, how they interact, and what the image generation options/parameters mean.\nThe Components of Stable Diffusion\nStable Diffusion is a system made up of several components and models. It is not one monolithic model.\nAs we look under the hood, the first observation we can make is that there’s a text-understanding component that translates the text information into a numeric representation that captures the ideas in the text.\nWe’re starting with a high-level view and we’ll get into more machine learning details later in this article. However, we can say that this text encoder is a special Transformer language model (technically: the text encoder of a CLIP model). It takes the input text and outputs a list of numbers representing each word/token in the text (a vector per token).\nThat information is then presented to the Image Generator, which is composed of a couple of components itself.\nThe image generator goes through two stages:\n1-\nImage information creator\nThis component is the secret sauce of Stable Diffusion. It’s where a lot of the performance gain over previous models is achieved.\nThis component runs for multiple steps to generate image information. This is the\nsteps\nparameter in Stable Diffusion interfaces and libraries which often defaults to 50 or 100.\nThe image information creator works completely in the\nimage information space\n(or\nlatent\nspace). We’ll talk more about what that means later in the post. This property makes it faster than previous diffusion models that worked in pixel space. In technical terms, this component is made up of a UNet neural network and a scheduling algorithm.\nThe word “diffusion” describes what happens in this component. It is the step by step processing of information that leads to a high-quality image being generated in the end (by the next component, the image decoder).\n2-\nImage Decoder\nThe image decoder paints a picture from the information it got from the information creator. It runs only once at the end of the process to produce the final pixel image.\nWith this we come to see the three main components (each with its own neural network) that make up Stable Diffusion:\nClipText\nfor text encoding.\nInput: text.\nOutput: 77 token embeddings vectors, each in 768 dimensions.\nUNet + Scheduler\nto gradually process/diffuse information in the information (latent) space.\nInput: text embeddings and a starting multi-dimensional array (structured lists of numbers, also called a\ntensor\n) made up of noise.\nOutput: A processed information array\nAutoencoder Decoder\nthat paints the final image using the processed information array.\nInput: The processed information array (dimensions: (4,64,64))\nOutput: The resulting image (dimensions: (3, 512, 512) which are (red/green/blue, width, height))\nWhat is Diffusion Anyway?\nDiffusion is the process that takes place inside the pink “image information creator” component. Having the token embeddings that represent the input text, and a random starting\nimage information array\n(these are also called\nlatents\n), the process produces an information array that the image decoder uses to paint the final image.\nThis process happens in a step-by-step fashion. Each step adds more relevant information. To get an intuition of the process, we can inspect the random latents array, and see that it translates to visual noise. Visual inspection in this case is passing it through the image decoder.\nDiffusion happens in multiple steps, each step operates on an input latents array, and produces another latents array that better resembles the input text and all the visual information the model picked up from all images the model was trained on.\nWe can visualize a set of these latents to see what information gets added at each step.\nThe process is quite breathtaking to look at.\nYour browser does not support the video tag.\nSomething especially fascinating happens between steps 2 and 4 in this case. It’s as if the outline emerges from the noise.\nYour browser does not support the video tag.\nHow diffusion works\nThe central idea of generating images with diffusion models relies on the fact that we have powerful computer vision models. Given a large enough dataset, these models can learn complex operations. Diffusion models approach image generation by framing the problem as following:\nSay we have an image, we generate some noise, and add it to the image.\nThis can now be considered a training example. We can use this same formula to create lots of training examples to train the central component of our image generation model.\nWhile this example shows a few noise amount values from image (amount 0, no noise) to total noise (amount 4, total noise), we can easily control how much noise to add to the image, and so we can spread it over tens of steps, creating tens of training examples per image for all the images in a training dataset.\nWith this dataset, we can train the noise predictor and end up with a great noise predictor that actually creates images when run in a certain configuration. A training step should look familiar if you’ve had ML exposure:\nLet’s now see how this can generate images.\nPainting images by removing noise\nThe trained noise predictor can take a noisy image, and the number of the denoising step, and is able to predict a slice of noise.\nThe sampled noise is predicted so that if we subtract it from the image, we get an image that’s closer to the images the model was trained on (not the exact images themselves, but the\ndistribution\n- the world of pixel arrangements where the sky is usually blue and above the ground, people have two eyes, cats look a certain way – pointy ears and clearly unimpressed).\nIf the training dataset was of aesthetically pleasing images (e.g.,\nLAION Aesthetics\n, which Stable Diffusion was trained on), then the resulting image would tend to be aesthetically pleasing. If the we train it on images of logos, we end up with a logo-generating model.\nThis concludes the description of image generation by diffusion models mostly as described in\nDenoising Diffusion Probabilistic Models\n. Now that you have this intuition of diffusion, you know the main components of not only Stable Diffusion, but also Dall-E 2 and Google’s Imagen.\nNote that the diffusion process we described so far generates images without using any text data. So if we deploy this model, it would generate great looking images, but we’d have no way of controlling if it’s an image of a pyramid or a cat or anything else. In the next sections we’ll describe how text is incorporated in the process in order to control what type of image the model generates.\nSpeed Boost: Diffusion on Compressed (Latent) Data Instead of the Pixel Image\nTo speed up the image generation process, the Stable Diffusion paper runs the diffusion process not on the pixel images themselves, but on a compressed version of the image.\nThe paper\ncalls this “Departure to Latent Space”.\nThis compression (and later decompression/painting) is done via an autoencoder. The autoencoder compresses the image into the latent space using its encoder, then reconstructs it using only the compressed information using the decoder.\nNow the forward diffusion process is done on the compressed latents. The slices of noise are of noise applied to those latents, not to the pixel image. And so the noise predictor is actually trained to predict noise in the compressed representation (the latent space).\nThe forward process (using the autoencoder’s encoder) is how we generate the data to train the noise predictor. Once it’s trained, we can generate images by running the reverse process (using the autoencoder’s decoder).\nThese two flows are what’s shown in Figure 3 of the LDM/Stable Diffusion paper:\nThis figure additionally shows the “conditioning” components, which in this case is the text prompts describing what image the model should generate. So let’s dig into the text components.\nThe Text Encoder: A Transformer Language Model\nA Transformer language model is used as the language understanding component that takes the text prompt and produces token embeddings. The released Stable Diffusion model uses ClipText (A\nGPT-based model\n), while the paper used\nBERT\n.\nThe choice of language model is shown by the Imagen paper to be an important one. Swapping in larger language models had more of an effect on generated image quality than larger image generation components.\nLarger/better language models have a significant effect on the quality of image generation models. Source:\nGoogle Imagen paper by Saharia et. al.\n. Figure A.5.\nThe early Stable Diffusion models just plugged in the pre-trained ClipText model released by OpenAI. It’s possible that future models may switch to the newly released and much larger\nOpenCLIP\nvariants of CLIP (Nov2022 update: True enough,\nStable Diffusion V2 uses OpenClip\n). This new batch includes text models of sizes up to 354M parameters, as opposed to the 63M parameters in ClipText.\nHow CLIP is trained\nCLIP is trained on a dataset of images and their captions. Think of a dataset looking like this, only with 400 million images and their captions:\nA dataset of images and their captions.\nIn actuality, CLIP was trained on images crawled from the web along with their “alt” tags.\nCLIP is a combination of an image encoder and a text encoder. Its training process can be simplified to thinking of taking an image and its caption. We encode them both with the image and text encoders respectively.\nWe then compare the resulting embeddings using cosine similarity. When we begin the training process, the similarity will be low, even if the text describes the image correctly.\nWe update the two models so that the next time we embed them, the resulting embeddings are similar.\nBy repeating this across the dataset and with large batch sizes, we end up with the encoders being able to produce embeddings where an image of a dog and the sentence “a picture of a dog” are similar. Just like in\nword2vec\n, the training process also needs to include\nnegative examples\nof images and captions that don’t match, and the model needs to assign them low similarity scores.\nFeeding Text Information Into The Image Generation Process\nTo make text a part of the image generation process, we have to adjust our noise predictor to use the text as an input.\nOur dataset now includes the encoded text. Since we’re operating in the latent space, both the input images and predicted noise are in the latent space.\nTo get a better sense of how the text tokens are used in the Unet, let’s look deeper inside the Unet.\nLayers of the Unet Noise predictor (without text)\nLet’s first look at a diffusion Unet that does not use text. Its inputs and outputs would look like this:\nInside, we see that:\nThe Unet is a series of layers that work on transforming the latents array\nEach layer operates on the output of the previous layer\nSome of the outputs are fed (via residual connections) into the processing later in the network\nThe timestep is transformed into a time step embedding vector, and that’s what gets used in the layers\nLayers of the Unet Noise predictor WITH text\nLet’s now look how to alter this system to include attention to the text.\nThe main change to the system we need to add support for text inputs (technical term: text conditioning) is to add an attention layer between the ResNet blocks.\nNote that the ResNet block doesn’t directly look at the text. But the attention layers merge those text representations in the latents. And now the next ResNet can utilize that incorporated text information in its processing.\nConclusion\nI hope this gives you a good first intuition about how Stable Diffusion works. Lots of other concepts are involved, but I believe they’re easier to understand once you’re familiar with the building blocks above. The resources below are great next steps that I found useful. Please reach out to me on\nTwitter\nfor any corrections or feedback.\nResources\nI have a\none-minute YouTube short\non using\nDream Studio\nto generate images with Stable Diffusion.\nStable Diffusion with 🧨 Diffusers\nThe Annotated Diffusion Model\nHow does Stable Diffusion work? – Latent Diffusion Models EXPLAINED\n[Video]\nStable Diffusion - What, Why, How?\n[Video]\nHigh-Resolution Image Synthesis with Latent Diffusion Models\n[The Stable Diffusion paper]\nFor a more in-depth look at the algorithms and math, see Lilian Weng’s\nWhat are Diffusion Models?\nWatch the\ngreat Stable Diffusion videos from fast.ai\nAcknowledgements\nThanks to Robin Rombach, Jeremy Howard, Hamel Husain, Dennis Soemers, Yan Sidyakin, Freddie Vargus, Anna Golubeva, and the\nCohere For AI\ncommunity for feedback on earlier versions of this article.\nContribute\nPlease help me make this article better. Possible ways:\nSend any feedback or corrections on\nTwitter\nor as a\nPull Request\nHelp make the article more accessible by suggesting captions and alt-text to the visuals (best as a pull request)\nTranslate it to another language and post it to your blog. Send me the link and I’ll add a link to it here. Translators of previous articles have always mentioned how much deeper they understood the concepts by going through the translation process.\nDiscuss\nIf you’re interested in discussing the overlap of image generation models with language models, feel free to post in the #images-and-words channel in the\nCohere community on Discord\n. There, we discuss areas of overlap, including:\nfine-tuning language models to produce good image generation prompts\nUsing LLMs to split the subject, and style components of an image captioning prompt\nImage-to-prompt (via tools like\nClip Interrogator\n)\nCitation\nIf you found this work helpful for your research, please cite it as following:\n@misc{alammar2022diffusion, \n title={The Illustrated Stable Diffusion},\n author={Alammar, J},\n year={2022},\n url={https://jalammar.github.io/illustrated-stable-diffusion/}\n}\nWritten on October 4, 2022", "metadata": { "title": "The Illustrated Stable Diffusion – Jay Alammar – Visualizing machine learning one concept at a time.", "description": "Translations: Chinese, Vietnamese.\n\n\n(V2 Nov 2022: Updated images for more precise description of forward diffusion. A few more images in this version)\n\nAI image generation is the most recent AI capability blowing people’s minds (mine included). The ability to create striking visuals from text descriptions has a magical quality to it and points clearly to a shift in how humans create art. The release of Stable Diffusion is a clear milestone in this development because it made a high-performance model available to the masses (performance in terms of image quality, as well as speed and relatively low resource/memory requirements).\n\n\n\nAfter experimenting with AI image generation, you may start to wonder how it works.\n\nThis is a gentle introduction to how Stable Diffusion works.\n\n\n \n \n\n\n\nStable Diffusion is versatile in that it can be used in a number of different ways. Let’s focus at first on image generation from text only (text2img). The image above shows an example text input and the resulting generated image (The actual complete prompt is here). Aside from text to image, another main way of using it is by making it alter images (so inputs are text + image).", "keywords": null, "author": "Jay Alammar", "og:description": "Translations: Chinese, Vietnamese.\n\n\n(V2 Nov 2022: Updated images for more precise description of forward diffusion. A few more images in this version)\n\nAI image generation is the most recent AI capability blowing people’s minds (mine included). The ability to create striking visuals from text descriptions has a magical quality to it and points clearly to a shift in how humans create art. The release of Stable Diffusion is a clear milestone in this development because it made a high-performance model available to the masses (performance in terms of image quality, as well as speed and relatively low resource/memory requirements).\n\n\n\nAfter experimenting with AI image generation, you may start to wonder how it works.\n\nThis is a gentle introduction to how Stable Diffusion works.\n\n\n \n \n\n\n\nStable Diffusion is versatile in that it can be used in a number of different ways. Let’s focus at first on image generation from text only (text2img). The image above shows an example text input and the resulting generated image (The actual complete prompt is here). Aside from text to image, another main way of using it is by making it alter images (so inputs are text + image).", "og:title": "The Illustrated Stable Diffusion" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 200, "error": "" }, "code_blocks": [ { "file": "attachments/9fa49ae4db5b/code/body-000.txt", "chars": 178, "language": "text" } ], "images": [ { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-text-to-image.png", "bytes": 832466, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-d4af36d7aa41a0b2177c.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-text-to-image.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-img2img-image-to-image.png", "bytes": 698058, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-0aca85d9a5922b3d121e.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-img2img-image-to-image.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-text-understanding-component-image-generation.png", "bytes": 440027, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-4779307e921b5ebd8866.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-text-understanding-component-image-generation.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/Stable-diffusion-text-info-to-image-generator.png", "bytes": 470679, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-bca7d887eba09831a475.png", "src": "https://jalammar.github.io/images/stable-diffusion/Stable-diffusion-text-info-to-image-generator.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/Stable-diffusion-image-generator-information-creator.png", "bytes": 463210, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-8ed910961179d7a7e02a.png", "src": "https://jalammar.github.io/images/stable-diffusion/Stable-diffusion-image-generator-information-creator.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-cliptext-unet-autoencoder-decoder.png", "bytes": 425629, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-c1615c3160d7f46bc79c.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-cliptext-unet-autoencoder-decoder.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-components-and-tensors.png", "bytes": 512823, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-43a5ab0e9c5013767dd6.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-components-and-tensors.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-diffusion-process.png", "bytes": 876511, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-d1e475ab711a7844bc97.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-diffusion-process.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-latent-space-pixel-space.png", "bytes": 276854, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-e07c6e626e5b6f20f286.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-latent-space-pixel-space.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-unet-steps.png", "bytes": 912968, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-9ff68b1f2e5f45c494b4.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-unet-steps.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-denoising-steps-latents.png", "bytes": 606469, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-5dad21de1770fea816d2.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-denoising-steps-latents.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-forward-diffusion-training-example.png", "bytes": 2388629, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-050770ae4f84b51bc0e7.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-forward-diffusion-training-example.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-forward-diffusion-training-example-2.png", "bytes": 1926515, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-5dbc556704ae2efe4f8b.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-forward-diffusion-training-example-2.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-u-net-noise-training-examples-2.png", "bytes": 768488, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-656d0d097af55de5c768.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-u-net-noise-training-examples-2.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-u-net-noise-training-step.png", "bytes": 1140036, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-78a54d1602e2d82d6c77.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-u-net-noise-training-step.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-denoising-step-1v2.png", "bytes": 411498, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-9fc08c60ac6033573b60.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-denoising-step-1v2.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-denoising-step-2v2.png", "bytes": 1294160, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-05568993d0f57d10a4b3.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-denoising-step-2v2.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-image-generation-v2.png", "bytes": 1829445, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-70c85bc2d7fa511a7147.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-image-generation-v2.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-autoencoder.png", "bytes": 157737, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-dcfbc19403f479d62c37.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-autoencoder.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-latent-forward-process-v2.png", "bytes": 1161115, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-891180a9112d24c99a16.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-latent-forward-process-v2.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-forward-and-reverse-process-v2.png", "bytes": 1424629, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-8e02852b8681b876fee4.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-forward-and-reverse-process-v2.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/article-Figure3-1-1536x762.png", "bytes": 294338, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-9c969df2f73d51b7a47c.png", "src": "https://jalammar.github.io/images/stable-diffusion/article-Figure3-1-1536x762.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/text-language-models-clip-image-generation.png", "bytes": 101615, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-804c9361f4a150b910bd.png", "src": "https://jalammar.github.io/images/stable-diffusion/text-language-models-clip-image-generation.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/images-and-captions-dataset.png", "bytes": 575418, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-f3edc3bfb7f34613d600.png", "src": "https://jalammar.github.io/images/stable-diffusion/images-and-captions-dataset.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/clip-training-step-1.png", "bytes": 70518, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-efe3cf5965c7601c0843.png", "src": "https://jalammar.github.io/images/stable-diffusion/clip-training-step-1.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/clip-training-step-2.png", "bytes": 103049, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-4d33b06bf45df8d1618e.png", "src": "https://jalammar.github.io/images/stable-diffusion/clip-training-step-2.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/clip-training-step-3.png", "bytes": 122155, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-45004e0e3f0f4c7c7ad2.png", "src": "https://jalammar.github.io/images/stable-diffusion/clip-training-step-3.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-unet-inputs-v2.png", "bytes": 706717, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-ade074c2e042066b6855.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-unet-inputs-v2.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-text-dataset-v2.png", "bytes": 434793, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-80748694b46ce04a317e.png", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-text-dataset-v2.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/unet-inputs-outputs-v2.png", "bytes": 107729, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-a62cbf464356b3e51e47.png", "src": "https://jalammar.github.io/images/stable-diffusion/unet-inputs-outputs-v2.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/unit-resnet-steps-v2.png", "bytes": 558422, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-9879ae2b9eef85ba0f6f.png", "src": "https://jalammar.github.io/images/stable-diffusion/unit-resnet-steps-v2.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/unet-with-text-inputs-outputs-v2.png", "bytes": 381352, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-e770ddd5263219016894.png", "src": "https://jalammar.github.io/images/stable-diffusion/unet-with-text-inputs-outputs-v2.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/stable-diffusion/unet-with-text-steps-v2.png", "bytes": 749761, "reused": false, "local_path": "attachments/9fa49ae4db5b/images/body-24c8cc030f0534c3e818.png", "src": "https://jalammar.github.io/images/stable-diffusion/unet-with-text-steps-v2.png", "alt": "", "downloaded": true } ], "audio": [], "videos": [ { "url": "https://www.youtube.com/watch?v=MXmacOUJUaw", "src": "https://www.youtube.com/embed/MXmacOUJUaw", "status": "ok", "kind": "single_video", "local_path": "attachments/9fa49ae4db5b/videos/repair-MXmacOUJUaw.mp4", "bytes": 483643633, "valid": true, "height": 1080, "width": 1920, "duration": "1726.161000", "reused": false }, { "url": "https://jalammar.github.io/images/stable-diffusion/diffusion-steps-all-loop.webm", "src": "https://jalammar.github.io/images/stable-diffusion/diffusion-steps-all-loop.webm", "status": "invalid_or_wrong_resolution", "kind": "direct_video", "bytes": 1382414, "reused": true, "content_type": "video/webm", "valid": true, "height": 540, "width": 540, "duration": "7.905000", "local_path": "attachments/9fa49ae4db5b/videos/repair-d39f47068817fab93e50.webm", "title": "", "retried_at": "2026-10-20T18:10:15Z" }, { "url": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-steps-2-4.webm", "src": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-steps-2-4.webm", "status": "invalid_or_wrong_resolution", "kind": "direct_video", "bytes": 1804256, "reused": true, "content_type": "video/webm", "valid": true, "height": 540, "width": 540, "duration": "6.155000", "local_path": "attachments/9fa49ae4db5b/videos/repair-feefb5a8f8797f003a34.webm", "title": "", "retried_at": "2026-10-20T18:10:15Z" }, { "url": "https://www.youtube.com/watch?v=qL6mKRyjK-0", "src": "https://youtube.com/shorts/qL6mKRyjK-0?feature=share", "status": "ok", "kind": "single_video", "local_path": "attachments/9fa49ae4db5b/videos/repair-qL6mKRyjK-0.mp4", "bytes": 3841110, "valid": true, "height": 1080, "width": 602, "duration": "59.961000", "reused": false }, { "url": "https://www.youtube.com/watch?v=J87hffSMB60", "src": "https://www.youtube.com/watch?v=J87hffSMB60", "status": "ok", "kind": "single_video", "local_path": "attachments/9fa49ae4db5b/videos/repair-J87hffSMB60.mp4", "bytes": 121791926, "valid": true, "height": 1080, "width": 1920, "duration": "795.841000", "reused": false }, { "url": "https://www.youtube.com/watch?v=ltLNYA3lWAQ", "src": "https://www.youtube.com/watch?v=ltLNYA3lWAQ", "status": "ok", "kind": "single_video", "local_path": "attachments/9fa49ae4db5b/videos/repair-ltLNYA3lWAQ.mp4", "bytes": 153019319, "valid": true, "height": 1080, "width": 1920, "duration": "3247.881000", "reused": false }, { "url": "https://www.youtube.com/watch?v=_7rMfsA24Ls", "src": "https://www.youtube.com/watch?v=_7rMfsA24Ls&ab_channel=JeremyHoward", "status": "ok", "kind": "single_video", "local_path": "attachments/9fa49ae4db5b/videos/repair-_7rMfsA24Ls.mp4", "bytes": 390941411, "valid": true, "height": 1080, "width": 1920, "duration": "8115.021000", "reused": false } ], "links": [ { "href": "https://blog.csdn.net/yujianmin1990/article/details/129143157", "label": "Chinese", "type": "a" }, { "href": "https://trituenhantao.io/kien-thuc/minh-hoa-stable-diffusion/", "label": "Vietnamese", "type": "a" }, { "href": "https://stability.ai/blog/stable-diffusion-public-release", "label": "Stable Diffusion", "type": "a" }, { "href": "https://www.youtube.com/embed/MXmacOUJUaw", "label": "", "type": "iframe" }, { "href": "https://jalammar.github.io/images/stable-diffusion/diffusion-steps-all-loop.webm", "label": "", "type": "source" }, { "href": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-steps-2-4.webm", "label": "", "type": "source" }, { "href": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-u-net-noise-training-step.png", "label": "", "type": "a" }, { "href": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-denoising-step-2v2.png", "label": "", "type": "a" }, { "href": "https://laion.ai/blog/laion-aesthetics/", "label": "LAION Aesthetics", "type": "a" }, { "href": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-image-generation-v2.png", "label": "", "type": "a" }, { "href": "https://arxiv.org/abs/2006.11239", "label": "Denoising Diffusion Probabilistic Models", "type": "a" }, { "href": "https://arxiv.org/abs/2112.10752", "label": "The paper", "type": "a" }, { "href": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-latent-forward-process-v2.png", "label": "", "type": "a" }, { "href": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-forward-and-reverse-process-v2.png", "label": "", "type": "a" }, { "href": "https://jalammar.github.io/illustrated-gpt2/", "label": "GPT-based model", "type": "a" }, { "href": "https://jalammar.github.io/illustrated-bert/", "label": "BERT", "type": "a" }, { "href": "https://arxiv.org/abs/2205.11487", "label": "Google Imagen paper by Saharia et. al.", "type": "a" }, { "href": "https://laion.ai/blog/large-openclip/", "label": "OpenCLIP", "type": "a" }, { "href": "https://stability.ai/blog/stable-diffusion-v2-release", "label": "Stable Diffusion V2 uses OpenClip", "type": "a" }, { "href": "https://jalammar.github.io/illustrated-word2vec/", "label": "word2vec", "type": "a" }, { "href": "https://jalammar.github.io/images/stable-diffusion/stable-diffusion-unet-inputs-v2.png", "label": "", "type": "a" }, { "href": "https://twitter.com/JayAlammar", "label": "Twitter", "type": "a" }, { "href": "https://youtube.com/shorts/qL6mKRyjK-0?feature=share", "label": "one-minute YouTube short", "type": "a" }, { "href": "https://beta.dreamstudio.ai/", "label": "Dream Studio", "type": "a" }, { "href": "https://huggingface.co/blog/stable_diffusion", "label": "Stable Diffusion with 🧨 Diffusers", "type": "a" }, { "href": "https://huggingface.co/blog/annotated-diffusion", "label": "The Annotated Diffusion Model", "type": "a" }, { "href": "https://www.youtube.com/watch?v=J87hffSMB60", "label": "How does Stable Diffusion work? – Latent Diffusion Models EXPLAINED", "type": "a" }, { "href": "https://www.youtube.com/watch?v=ltLNYA3lWAQ", "label": "Stable Diffusion - What, Why, How?", "type": "a" }, { "href": "https://ommer-lab.com/research/latent-diffusion-models/", "label": "High-Resolution Image Synthesis with Latent Diffusion Models", "type": "a" }, { "href": "https://lilianweng.github.io/posts/2021-07-11-diffusion-models/", "label": "What are Diffusion Models?", "type": "a" }, { "href": "https://www.youtube.com/watch?v=_7rMfsA24Ls&ab_channel=JeremyHoward", "label": "great Stable Diffusion videos from fast.ai", "type": "a" }, { "href": "https://cohere.for.ai/", "label": "Cohere For AI", "type": "a" }, { "href": "https://github.com/jalammar/jalammar.github.io", "label": "Pull Request", "type": "a" }, { "href": "https://discord.gg/co-mmunity", "label": "Cohere community on Discord", "type": "a" }, { "href": "https://colab.research.google.com/github/pharmapsychotic/clip-interrogator/blob/main/clip_interrogator.ipynb", "label": "Clip Interrogator", "type": "a" } ], "repair_version": 2, "repaired_at": "2026-10-14T10:52:29Z", "source_url": "https://jalammar.github.io/illustrated-stable-diffusion/", "papers": [ { "status": "ok", "url": "https://arxiv.org/pdf/2006.11239", "bytes": 10267274, "reused": true, "local_path": "papers/repair/f8a0cafabed0cfe88ca6.pdf" }, { "status": "ok", "url": "https://arxiv.org/pdf/2112.10752", "bytes": 40842586, "reused": true, "local_path": "papers/repair/84de7ccc1cc12ec4cb29.pdf" }, { "status": "ok", "url": "https://arxiv.org/pdf/2205.11487", "bytes": 11365725, "reused": true, "local_path": "papers/repair/8c48830dcf885dd9be79.pdf" } ], "actual_video_files": [], "body_chars": 15164, "expected_body_images": 33, "discovery_status": "ok", "delivery_status": "partial", "missing_reason": "invalid_or_wrong_resolution", "raw_html": "attachments/9fa49ae4db5b/pages/source.html", "local_html": "attachments/9fa49ae4db5b/pages/article.html", "body_base_path": "attachments/9fa49ae4db5b/pages/", "body_markdown_path": "attachments/9fa49ae4db5b/pages/article.md" }