{
  "source_file": "0930",
  "source_platform": "研究机构",
  "author": "Dive into Deep Learning",
  "title_listed": "Neural Style Transfer（章节）",
  "keywords": "风格迁移,卷积,图像",
  "domain": "图像处理",
  "content_type": "教程/书籍",
  "is_paid_flag": "否",
  "url": "https://www.d2l.ai/chapter_computer-vision/neural-style.html",
  "publish_date": "持续更新",
  "corpus_stage": "图像处理",
  "notes": "神经风格迁移",
  "id": "c9c32763767f",
  "collected_at": "2026-10-02T16:07:39",
  "title": "14.12. Neural Style Transfer — Dive into Deep Learning 1.0.3 documentation",
  "body_markdown": "Table Of Contents\n14.12.\nNeural Style Transfer\n¶\nColab [pytorch]\nOpen the notebook in Colab\nColab [mxnet]\nOpen the notebook in Colab\nColab [jax]\nOpen the notebook in Colab\nColab [tensorflow]\nOpen the notebook in Colab\nSageMaker Studio Lab\nOpen the notebook in SageMaker Studio Lab\nIf you are a photography enthusiast, you may be familiar with the\nfilter. It can change the color style of photos so that landscape photos\nbecome sharper or portrait photos have whitened skins. However, one\nfilter usually only changes one aspect of the photo. To apply an ideal\nstyle to a photo, you probably need to try many different filter\ncombinations. This process is as complex as tuning the hyperparameters\nof a model.\nIn this section, we will leverage layerwise representations of a CNN to\nautomatically apply the style of one image to another image, i.e.,\nstyle transfer\n(\nGatys\net al.\n, 2016\n)\n. This task needs two\ninput images: one is the\ncontent image\nand the other is the\nstyle\nimage\n. We will use neural networks to modify the content image to make\nit close to the style image in style. For example, the content image in\nFig. 14.12.1\nis a landscape photo taken by us in Mount\nRainier National Park in the suburbs of Seattle, while the style image\nis an oil painting with the theme of autumn oak trees. In the output\nsynthesized image, the oil brush strokes of the style image are applied,\nleading to more vivid colors, while preserving the main shape of the\nobjects in the content image.\nFig. 14.12.1\nGiven content and style images, style transfer outputs a synthesized\nimage.\n¶\n14.12.1.\nMethod\n¶\nFig. 14.12.2\nillustrates the CNN-based style\ntransfer method with a simplified example. First, we initialize the\nsynthesized image, for example, into the content image. This synthesized\nimage is the only variable that needs to be updated during the style\ntransfer process, i.e., the model parameters to be updated during\ntraining. Then we choose a pretrained CNN to extract image features and\nfreeze its model parameters during training. This deep CNN uses multiple\nlayers to extract hierarchical features for images. We can choose the\noutput of some of these layers as content features or style features.\nTake\nFig. 14.12.2\nas an example. The pretrained\nneural network here has 3 convolutional layers, where the second layer\noutputs the content features, and the first and third layers output the\nstyle features.\nFig. 14.12.2\nCNN-based style transfer process. Solid lines show the direction of\nforward propagation and dotted lines show backward propagation.\n¶\nNext, we calculate the loss function of style transfer through forward\npropagation (direction of solid arrows), and update the model parameters\n(the synthesized image for output) through backpropagation (direction of\ndashed arrows). The loss function commonly used in style transfer\nconsists of three parts: (i)\ncontent loss\nmakes the synthesized image\nand the content image close in content features; (ii)\nstyle loss\nmakes\nthe synthesized image and style image close in style features; and (iii)\ntotal variation loss\nhelps to reduce the noise in the synthesized\nimage. Finally, when the model training is over, we output the model\nparameters of the style transfer to generate the final synthesized\nimage.\nIn the following, we will explain the technical details of style\ntransfer via a concrete experiment.\n14.12.2.\nReading the Content and Style Images\n¶\nFirst, we read the content and style images. From their printed\ncoordinate axes, we can tell that these images have different sizes.\npytorch\nmxnet\n%\nmatplotlib\ninline\nimport\ntorch\nimport\ntorchvision\nfrom\ntorch\nimport\nnn\nfrom\nd2l\nimport\ntorch\nas\nd2l\nd2l\n.\nset_figsize\n()\ncontent_img\n=\nd2l\n.\nImage\n.\nopen\n(\n'../img/rainier.jpg'\n)\nd2l\n.\nplt\n.\nimshow\n(\ncontent_img\n);\nstyle_img\n=\nd2l\n.\nImage\n.\nopen\n(\n'../img/autumn-oak.jpg'\n)\nd2l\n.\nplt\n.\nimshow\n(\nstyle_img\n);\n%\nmatplotlib\ninline\nfrom\nmxnet\nimport\nautograd\n,\ngluon\n,\nimage\n,\ninit\n,\nnp\n,\nnpx\nfrom\nmxnet.gluon\nimport\nnn\nfrom\nd2l\nimport\nmxnet\nas\nd2l\nnpx\n.\nset_np\n()\nd2l\n.\nset_figsize\n()\ncontent_img\n=\nimage\n.\nimread\n(\n'../img/rainier.jpg'\n)\nd2l\n.\nplt\n.\nimshow\n(\ncontent_img\n.\nasnumpy\n());\n[\n22\n:\n41\n:\n40\n]\n../\nsrc\n/\nstorage\n/\nstorage\n.\ncc\n:\n196\n:\nUsing\nPooled\n(\nNaive\n)\nStorageManager\nfor\nCPU\nstyle_img\n=\nimage\n.\nimread\n(\n'../img/autumn-oak.jpg'\n)\nd2l\n.\nplt\n.\nimshow\n(\nstyle_img\n.\nasnumpy\n());\n14.12.3.\nPreprocessing and Postprocessing\n¶\nBelow, we define two functions for preprocessing and postprocessing\nimages. The\npreprocess\nfunction standardizes each of the three RGB\nchannels of the input image and transforms the results into the CNN\ninput format. The\npostprocess\nfunction restores the pixel values in\nthe output image to their original values before standardization. Since\nthe image printing function requires that each pixel has a floating\npoint value from 0 to 1, we replace any value smaller than 0 or greater\nthan 1 with 0 or 1, respectively.\npytorch\nmxnet\nrgb_mean\n=\ntorch\n.\ntensor\n([\n0.485\n,\n0.456\n,\n0.406\n])\nrgb_std\n=\ntorch\n.\ntensor\n([\n0.229\n,\n0.224\n,\n0.225\n])\ndef\npreprocess\n(\nimg\n,\nimage_shape\n):\ntransforms\n=\ntorchvision\n.\ntransforms\n.\nCompose\n([\ntorchvision\n.\ntransforms\n.\nResize\n(\nimage_shape\n),\ntorchvision\n.\ntransforms\n.\nToTensor\n(),\ntorchvision\n.\ntransforms\n.\nNormalize\n(\nmean\n=\nrgb_mean\n,\nstd\n=\nrgb_std\n)])\nreturn\ntransforms\n(\nimg\n)\n.\nunsqueeze\n(\n0\n)\ndef\npostprocess\n(\nimg\n):\nimg\n=\nimg\n[\n0\n]\n.\nto\n(\nrgb_std\n.\ndevice\n)\nimg\n=\ntorch\n.\nclamp\n(\nimg\n.\npermute\n(\n1\n,\n2\n,\n0\n)\n*\nrgb_std\n+\nrgb_mean\n,\n0\n,\n1\n)\nreturn\ntorchvision\n.\ntransforms\n.\nToPILImage\n()(\nimg\n.\npermute\n(\n2\n,\n0\n,\n1\n))\nrgb_mean\n=\nnp\n.\narray\n([\n0.485\n,\n0.456\n,\n0.406\n])\nrgb_std\n=\nnp\n.\narray\n([\n0.229\n,\n0.224\n,\n0.225\n])\ndef\npreprocess\n(\nimg\n,\nimage_shape\n):\nimg\n=\nimage\n.\nimresize\n(\nimg\n,\n*\nimage_shape\n)\nimg\n=\n(\nimg\n.\nastype\n(\n'float32'\n)\n/\n255\n-\nrgb_mean\n)\n/\nrgb_std\nreturn\nnp\n.\nexpand_dims\n(\nimg\n.\ntranspose\n(\n2\n,\n0\n,\n1\n),\naxis\n=\n0\n)\ndef\npostprocess\n(\nimg\n):\nimg\n=\nimg\n[\n0\n]\n.\nas_in_ctx\n(\nrgb_std\n.\nctx\n)\nreturn\n(\nimg\n.\ntranspose\n(\n1\n,\n2\n,\n0\n)\n*\nrgb_std\n+\nrgb_mean\n)\n.\nclip\n(\n0\n,\n1\n)\n14.12.4.\nExtracting Features\n¶\nWe use the VGG-19 model pretrained on the ImageNet dataset to extract\nimage features\n(\nGatys\net al.\n, 2016\n)\n.\npytorch\nmxnet\npretrained_net\n=\ntorchvision\n.\nmodels\n.\nvgg19\n(\npretrained\n=\nTrue\n)\nDownloading: \"https://download.pytorch.org/models/vgg19-dcbb9e9d.pth\" to /home/ci/.cache/torch/hub/checkpoints/vgg19-dcbb9e9d.pth\n100%|██████████| 548M/548M [00:02<00:00, 213MB/s]\npretrained_net\n=\ngluon\n.\nmodel_zoo\n.\nvision\n.\nvgg19\n(\npretrained\n=\nTrue\n)\nDownloading\n/\nopt\n/\nmxnet\n/\nmodels\n/\nvgg19\n-\nad2f660d\n.\nzipf5c89524\n-\n14\nc4\n-\n49\nc9\n-\n9\nbb9\n-\n460\na4480a2b2\nfrom\nhttps\n:\n//\napache\n-\nmxnet\n.\ns3\n-\naccelerate\n.\ndualstack\n.\namazonaws\n.\ncom\n/\ngluon\n/\nmodels\n/\nvgg19\n-\nad2f660d\n.\nzip\n...\nIn order to extract the content features and style features of the\nimage, we can select the output of certain layers in the VGG network.\nGenerally speaking, the closer to the input layer, the easier to extract\ndetails of the image, and vice versa, the easier to extract the global\ninformation of the image. In order to avoid excessively retaining the\ndetails of the content image in the synthesized image, we choose a VGG\nlayer that is closer to the output as the\ncontent layer\nto output the\ncontent features of the image. We also select the output of different\nVGG layers for extracting local and global style features. These layers\nare also called\nstyle layers\n. As mentioned in\nSection 8.2\n, the\nVGG network uses 5 convolutional blocks. In the experiment, we choose\nthe last convolutional layer of the fourth convolutional block as the\ncontent layer, and the first convolutional layer of each convolutional\nblock as the style layer. The indices of these layers can be obtained by\nprinting the\npretrained_net\ninstance.\npytorch\nmxnet\nstyle_layers\n,\ncontent_layers\n=\n[\n0\n,\n5\n,\n10\n,\n19\n,\n28\n],\n[\n25\n]\nstyle_layers\n,\ncontent_layers\n=\n[\n0\n,\n5\n,\n10\n,\n19\n,\n28\n],\n[\n25\n]\nWhen extracting features using VGG layers, we only need to use all those\nfrom the input layer to the content layer or style layer that is closest\nto the output layer. Let’s construct a new network instance\nnet\n,\nwhich only retains all the VGG layers to be used for feature extraction.\npytorch\nmxnet\nnet\n=\nnn\n.\nSequential\n(\n*\n[\npretrained_net\n.\nfeatures\n[\ni\n]\nfor\ni\nin\nrange\n(\nmax\n(\ncontent_layers\n+\nstyle_layers\n)\n+\n1\n)])\nnet\n=\nnn\n.\nSequential\n()\nfor\ni\nin\nrange\n(\nmax\n(\ncontent_layers\n+\nstyle_layers\n)\n+\n1\n):\nnet\n.\nadd\n(\npretrained_net\n.\nfeatures\n[\ni\n])\nGiven the input\nX\n, if we simply invoke the forward propagation\nnet(X)\n, we can only get the output of the last layer. Since we also\nneed the outputs of intermediate layers, we need to perform\nlayer-by-layer computation and keep the content and style layer outputs.\npytorch\nmxnet\ndef\nextract_features\n(\nX\n,\ncontent_layers\n,\nstyle_layers\n):\ncontents\n=\n[]\nstyles\n=\n[]\nfor\ni\nin\nrange\n(\nlen\n(\nnet\n)):\nX\n=\nnet\n[\ni\n](\nX\n)\nif\ni\nin\nstyle_layers\n:\nstyles\n.\nappend\n(\nX\n)\nif\ni\nin\ncontent_layers\n:\ncontents\n.\nappend\n(\nX\n)\nreturn\ncontents\n,\nstyles\ndef\nextract_features\n(\nX\n,\ncontent_layers\n,\nstyle_layers\n):\ncontents\n=\n[]\nstyles\n=\n[]\nfor\ni\nin\nrange\n(\nlen\n(\nnet\n)):\nX\n=\nnet\n[\ni\n](\nX\n)\nif\ni\nin\nstyle_layers\n:\nstyles\n.\nappend\n(\nX\n)\nif\ni\nin\ncontent_layers\n:\ncontents\n.\nappend\n(\nX\n)\nreturn\ncontents\n,\nstyles\nTwo functions are defined below: the\nget_contents\nfunction extracts\ncontent features from the content image, and the\nget_styles\nfunction\nextracts style features from the style image. Since there is no need to\nupdate the model parameters of the pretrained VGG during training, we\ncan extract the content and the style features even before the training\nstarts. Since the synthesized image is a set of model parameters to be\nupdated for style transfer, we can only extract the content and style\nfeatures of the synthesized image by calling the\nextract_features\nfunction during training.\npytorch\nmxnet\ndef\nget_contents\n(\nimage_shape\n,\ndevice\n):\ncontent_X\n=\npreprocess\n(\ncontent_img\n,\nimage_shape\n)\n.\nto\n(\ndevice\n)\ncontents_Y\n,\n_\n=\nextract_features\n(\ncontent_X\n,\ncontent_layers\n,\nstyle_layers\n)\nreturn\ncontent_X\n,\ncontents_Y\ndef\nget_styles\n(\nimage_shape\n,\ndevice\n):\nstyle_X\n=\npreprocess\n(\nstyle_img\n,\nimage_shape\n)\n.\nto\n(\ndevice\n)\n_\n,\nstyles_Y\n=\nextract_features\n(\nstyle_X\n,\ncontent_layers\n,\nstyle_layers\n)\nreturn\nstyle_X\n,\nstyles_Y\ndef\nget_contents\n(\nimage_shape\n,\ndevice\n):\ncontent_X\n=\npreprocess\n(\ncontent_img\n,\nimage_shape\n)\n.\ncopyto\n(\ndevice\n)\ncontents_Y\n,\n_\n=\nextract_features\n(\ncontent_X\n,\ncontent_layers\n,\nstyle_layers\n)\nreturn\ncontent_X\n,\ncontents_Y\ndef\nget_styles\n(\nimage_shape\n,\ndevice\n):\nstyle_X\n=\npreprocess\n(\nstyle_img\n,\nimage_shape\n)\n.\ncopyto\n(\ndevice\n)\n_\n,\nstyles_Y\n=\nextract_features\n(\nstyle_X\n,\ncontent_layers\n,\nstyle_layers\n)\nreturn\nstyle_X\n,\nstyles_Y\n14.12.5.\nDefining the Loss Function\n¶\nNow we will describe the loss function for style transfer. The loss\nfunction consists of the content loss, style loss, and total variation\nloss.\n14.12.5.1.\nContent Loss\n¶\nSimilar to the loss function in linear regression, the content loss\nmeasures the difference in content features between the synthesized\nimage and the content image via the squared loss function. The two\ninputs of the squared loss function are both outputs of the content\nlayer computed by the\nextract_features\nfunction.\npytorch\nmxnet\ndef\ncontent_loss\n(\nY_hat\n,\nY\n):\n# We detach the target content from the tree used to dynamically compute\n# the gradient: this is a stated value, not a variable. Otherwise the loss\n# will throw an error.\nreturn\ntorch\n.\nsquare\n(\nY_hat\n-\nY\n.\ndetach\n())\n.\nmean\n()\ndef\ncontent_loss\n(\nY_hat\n,\nY\n):\nreturn\nnp\n.\nsquare\n(\nY_hat\n-\nY\n)\n.\nmean\n()\n14.12.5.2.\nStyle Loss\n¶\nStyle loss, similar to content loss, also uses the squared loss function\nto measure the difference in style between the synthesized image and the\nstyle image. To express the style output of any style layer, we first\nuse the\nextract_features\nfunction to compute the style layer output.\nSuppose that the output has 1 example,\n\\(c\\)\nchannels, height\n\\(h\\)\n, and width\n\\(w\\)\n, we can transform this output into matrix\n\\(\\mathbf{X}\\)\nwith\n\\(c\\)\nrows and\n\\(hw\\)\ncolumns. This\nmatrix can be thought of as the concatenation of\n\\(c\\)\nvectors\n\\(\\mathbf{x}_1, \\ldots, \\mathbf{x}_c\\)\n, each of which has a length\nof\n\\(hw\\)\n. Here, vector\n\\(\\mathbf{x}_i\\)\nrepresents the style\nfeature of channel\n\\(i\\)\n.\nIn the\nGram matrix\nof these vectors\n\\(\\mathbf{X}\\mathbf{X}^\\top \\in \\mathbb{R}^{c \\times c}\\)\n, element\n\\(x_{ij}\\)\nin row\n\\(i\\)\nand column\n\\(j\\)\nis the dot product\nof vectors\n\\(\\mathbf{x}_i\\)\nand\n\\(\\mathbf{x}_j\\)\n. It represents\nthe correlation of the style features of channels\n\\(i\\)\nand\n\\(j\\)\n. We use this Gram matrix to represent the style output of any\nstyle layer. Note that when the value of\n\\(hw\\)\nis larger, it likely\nleads to larger values in the Gram matrix. Note also that the height and\nwidth of the Gram matrix are both the number of channels\n\\(c\\)\n. To\nallow style loss not to be affected by these values, the\ngram\nfunction below divides the Gram matrix by the number of its elements,\ni.e.,\n\\(chw\\)\n.\npytorch\nmxnet\ndef\ngram\n(\nX\n):\nnum_channels\n,\nn\n=\nX\n.\nshape\n[\n1\n],\nX\n.\nnumel\n()\n//\nX\n.\nshape\n[\n1\n]\nX\n=\nX\n.\nreshape\n((\nnum_channels\n,\nn\n))\nreturn\ntorch\n.\nmatmul\n(\nX\n,\nX\n.\nT\n)\n/\n(\nnum_channels\n*\nn\n)\ndef\ngram\n(\nX\n):\nnum_channels\n,\nn\n=\nX\n.\nshape\n[\n1\n],\nd2l\n.\nsize\n(\nX\n)\n//\nX\n.\nshape\n[\n1\n]\nX\n=\nX\n.\nreshape\n((\nnum_channels\n,\nn\n))\nreturn\nnp\n.\ndot\n(\nX\n,\nX\n.\nT\n)\n/\n(\nnum_channels\n*\nn\n)\nObviously, the two Gram matrix inputs of the squared loss function for\nstyle loss are based on the style layer outputs for the synthesized\nimage and the style image. It is assumed here that the Gram matrix\ngram_Y\nbased on the style image has been precomputed.\npytorch\nmxnet\ndef\nstyle_loss\n(\nY_hat\n,\ngram_Y\n):\nreturn\ntorch\n.\nsquare\n(\ngram\n(\nY_hat\n)\n-\ngram_Y\n.\ndetach\n())\n.\nmean\n()\ndef\nstyle_loss\n(\nY_hat\n,\ngram_Y\n):\nreturn\nnp\n.\nsquare\n(\ngram\n(\nY_hat\n)\n-\ngram_Y\n)\n.\nmean\n()\n14.12.5.3.\nTotal Variation Loss\n¶\nSometimes, the learned synthesized image has a lot of high-frequency\nnoise, i.e., particularly bright or dark pixels. One common noise\nreduction method is\ntotal variation denoising\n. Denote by\n\\(x_{i, j}\\)\nthe pixel value at coordinate\n\\((i, j)\\)\n. Reducing\ntotal variation loss\n(14.12.1)\n¶\n\\[\\sum_{i, j} \\left|x_{i, j} - x_{i+1, j}\\right| + \\left|x_{i, j} - x_{i, j+1}\\right|\\]\nmakes values of neighboring pixels on the synthesized image closer.\npytorch\nmxnet\ndef\ntv_loss\n(\nY_hat\n):\nreturn\n0.5\n*\n(\ntorch\n.\nabs\n(\nY_hat\n[:,\n:,\n1\n:,\n:]\n-\nY_hat\n[:,\n:,\n:\n-\n1\n,\n:])\n.\nmean\n()\n+\ntorch\n.\nabs\n(\nY_hat\n[:,\n:,\n:,\n1\n:]\n-\nY_hat\n[:,\n:,\n:,\n:\n-\n1\n])\n.\nmean\n())\ndef\ntv_loss\n(\nY_hat\n):\nreturn\n0.5\n*\n(\nnp\n.\nabs\n(\nY_hat\n[:,\n:,\n1\n:,\n:]\n-\nY_hat\n[:,\n:,\n:\n-\n1\n,\n:])\n.\nmean\n()\n+\nnp\n.\nabs\n(\nY_hat\n[:,\n:,\n:,\n1\n:]\n-\nY_hat\n[:,\n:,\n:,\n:\n-\n1\n])\n.\nmean\n())\n14.12.5.4.\nLoss Function\n¶\nThe loss function of style transfer is the weighted sum of content loss,\nstyle loss, and total variation loss. By adjusting these weight\nhyperparameters, we can balance among content retention, style transfer,\nand noise reduction on the synthesized image.\npytorch\nmxnet\ncontent_weight\n,\nstyle_weight\n,\ntv_weight\n=\n1\n,\n1e4\n,\n10\ndef\ncompute_loss\n(\nX\n,\ncontents_Y_hat\n,\nstyles_Y_hat\n,\ncontents_Y\n,\nstyles_Y_gram\n):\n# Calculate the content, style, and total variance losses respectively\ncontents_l\n=\n[\ncontent_loss\n(\nY_hat\n,\nY\n)\n*\ncontent_weight\nfor\nY_hat\n,\nY\nin\nzip\n(\ncontents_Y_hat\n,\ncontents_Y\n)]\nstyles_l\n=\n[\nstyle_loss\n(\nY_hat\n,\nY\n)\n*\nstyle_weight\nfor\nY_hat\n,\nY\nin\nzip\n(\nstyles_Y_hat\n,\nstyles_Y_gram\n)]\ntv_l\n=\ntv_loss\n(\nX\n)\n*\ntv_weight\n# Add up all the losses\nl\n=\nsum\n(\nstyles_l\n+\ncontents_l\n+\n[\ntv_l\n])\nreturn\ncontents_l\n,\nstyles_l\n,\ntv_l\n,\nl\ncontent_weight\n,\nstyle_weight\n,\ntv_weight\n=\n1\n,\n1e4\n,\n10\ndef\ncompute_loss\n(\nX\n,\ncontents_Y_hat\n,\nstyles_Y_hat\n,\ncontents_Y\n,\nstyles_Y_gram\n):\n# Calculate the content, style, and total variance losses respectively\ncontents_l\n=\n[\ncontent_loss\n(\nY_hat\n,\nY\n)\n*\ncontent_weight\nfor\nY_hat\n,\nY\nin\nzip\n(\ncontents_Y_hat\n,\ncontents_Y\n)]\nstyles_l\n=\n[\nstyle_loss\n(\nY_hat\n,\nY\n)\n*\nstyle_weight\nfor\nY_hat\n,\nY\nin\nzip\n(\nstyles_Y_hat\n,\nstyles_Y_gram\n)]\ntv_l\n=\ntv_loss\n(\nX\n)\n*\ntv_weight\n# Add up all the losses\nl\n=\nsum\n(\nstyles_l\n+\ncontents_l\n+\n[\ntv_l\n])\nreturn\ncontents_l\n,\nstyles_l\n,\ntv_l\n,\nl\n14.12.6.\nInitializing the Synthesized Image\n¶\nIn style transfer, the synthesized image is the only variable that needs\nto be updated during training. Thus, we can define a simple model,\nSynthesizedImage\n, and treat the synthesized image as the model\nparameters. In this model, forward propagation just returns the model\nparameters.\npytorch\nmxnet\nclass\nSynthesizedImage\n(\nnn\n.\nModule\n):\ndef\n__init__\n(\nself\n,\nimg_shape\n,\n**\nkwargs\n):\nsuper\n(\nSynthesizedImage\n,\nself\n)\n.\n__init__\n(\n**\nkwargs\n)\nself\n.\nweight\n=\nnn\n.\nParameter\n(\ntorch\n.\nrand\n(\n*\nimg_shape\n))\ndef\nforward\n(\nself\n):\nreturn\nself\n.\nweight\nclass\nSynthesizedImage\n(\nnn\n.\nBlock\n):\ndef\n__init__\n(\nself\n,\nimg_shape\n,\n**\nkwargs\n):\nsuper\n(\nSynthesizedImage\n,\nself\n)\n.\n__init__\n(\n**\nkwargs\n)\nself\n.\nweight\n=\nself\n.\nparams\n.\nget\n(\n'weight'\n,\nshape\n=\nimg_shape\n)\ndef\nforward\n(\nself\n):\nreturn\nself\n.\nweight\n.\ndata\n()\nNext, we define the\nget_inits\nfunction. This function creates a\nsynthesized image model instance and initializes it to the image\nX\n.\nGram matrices for the style image at various style layers,\nstyles_Y_gram\n, are computed prior to training.\npytorch\nmxnet\ndef\nget_inits\n(\nX\n,\ndevice\n,\nlr\n,\nstyles_Y\n):\ngen_img\n=\nSynthesizedImage\n(\nX\n.\nshape\n)\n.\nto\n(\ndevice\n)\ngen_img\n.\nweight\n.\ndata\n.\ncopy_\n(\nX\n.\ndata\n)\ntrainer\n=\ntorch\n.\noptim\n.\nAdam\n(\ngen_img\n.\nparameters\n(),\nlr\n=\nlr\n)\nstyles_Y_gram\n=\n[\ngram\n(\nY\n)\nfor\nY\nin\nstyles_Y\n]\nreturn\ngen_img\n(),\nstyles_Y_gram\n,\ntrainer\ndef\nget_inits\n(\nX\n,\ndevice\n,\nlr\n,\nstyles_Y\n):\ngen_img\n=\nSynthesizedImage\n(\nX\n.\nshape\n)\ngen_img\n.\ninitialize\n(\ninit\n.\nConstant\n(\nX\n),\nctx\n=\ndevice\n,\nforce_reinit\n=\nTrue\n)\ntrainer\n=\ngluon\n.\nTrainer\n(\ngen_img\n.\ncollect_params\n(),\n'adam'\n,\n{\n'learning_rate'\n:\nlr\n})\nstyles_Y_gram\n=\n[\ngram\n(\nY\n)\nfor\nY\nin\nstyles_Y\n]\nreturn\ngen_img\n(),\nstyles_Y_gram\n,\ntrainer\n14.12.7.\nTraining\n¶\nWhen training the model for style transfer, we continuously extract\ncontent features and style features of the synthesized image, and\ncalculate the loss function. Below defines the training loop.\npytorch\nmxnet\ndef\ntrain\n(\nX\n,\ncontents_Y\n,\nstyles_Y\n,\ndevice\n,\nlr\n,\nnum_epochs\n,\nlr_decay_epoch\n):\nX\n,\nstyles_Y_gram\n,\ntrainer\n=\nget_inits\n(\nX\n,\ndevice\n,\nlr\n,\nstyles_Y\n)\nscheduler\n=\ntorch\n.\noptim\n.\nlr_scheduler\n.\nStepLR\n(\ntrainer\n,\nlr_decay_epoch\n,\n0.8\n)\nanimator\n=\nd2l\n.\nAnimator\n(\nxlabel\n=\n'epoch'\n,\nylabel\n=\n'loss'\n,\nxlim\n=\n[\n10\n,\nnum_epochs\n],\nlegend\n=\n[\n'content'\n,\n'style'\n,\n'TV'\n],\nncols\n=\n2\n,\nfigsize\n=\n(\n7\n,\n2.5\n))\nfor\nepoch\nin\nrange\n(\nnum_epochs\n):\ntrainer\n.\nzero_grad\n()\ncontents_Y_hat\n,\nstyles_Y_hat\n=\nextract_features\n(\nX\n,\ncontent_layers\n,\nstyle_layers\n)\ncontents_l\n,\nstyles_l\n,\ntv_l\n,\nl\n=\ncompute_loss\n(\nX\n,\ncontents_Y_hat\n,\nstyles_Y_hat\n,\ncontents_Y\n,\nstyles_Y_gram\n)\nl\n.\nbackward\n()\ntrainer\n.\nstep\n()\nscheduler\n.\nstep\n()\nif\n(\nepoch\n+\n1\n)\n%\n10\n==\n0\n:\nanimator\n.\naxes\n[\n1\n]\n.\nimshow\n(\npostprocess\n(\nX\n))\nanimator\n.\nadd\n(\nepoch\n+\n1\n,\n[\nfloat\n(\nsum\n(\ncontents_l\n)),\nfloat\n(\nsum\n(\nstyles_l\n)),\nfloat\n(\ntv_l\n)])\nreturn\nX\ndef\ntrain\n(\nX\n,\ncontents_Y\n,\nstyles_Y\n,\ndevice\n,\nlr\n,\nnum_epochs\n,\nlr_decay_epoch\n):\nX\n,\nstyles_Y_gram\n,\ntrainer\n=\nget_inits\n(\nX\n,\ndevice\n,\nlr\n,\nstyles_Y\n)\nanimator\n=\nd2l\n.\nAnimator\n(\nxlabel\n=\n'epoch'\n,\nylabel\n=\n'loss'\n,\nxlim\n=\n[\n10\n,\nnum_epochs\n],\nylim\n=\n[\n0\n,\n20\n],\nlegend\n=\n[\n'content'\n,\n'style'\n,\n'TV'\n],\nncols\n=\n2\n,\nfigsize\n=\n(\n7\n,\n2.5\n))\nfor\nepoch\nin\nrange\n(\nnum_epochs\n):\nwith\nautograd\n.\nrecord\n():\ncontents_Y_hat\n,\nstyles_Y_hat\n=\nextract_features\n(\nX\n,\ncontent_layers\n,\nstyle_layers\n)\ncontents_l\n,\nstyles_l\n,\ntv_l\n,\nl\n=\ncompute_loss\n(\nX\n,\ncontents_Y_hat\n,\nstyles_Y_hat\n,\ncontents_Y\n,\nstyles_Y_gram\n)\nl\n.\nbackward\n()\ntrainer\n.\nstep\n(\n1\n)\nif\n(\nepoch\n+\n1\n)\n%\nlr_decay_epoch\n==\n0\n:\ntrainer\n.\nset_learning_rate\n(\ntrainer\n.\nlearning_rate\n*\n0.8\n)\nif\n(\nepoch\n+\n1\n)\n%\n10\n==\n0\n:\nanimator\n.\naxes\n[\n1\n]\n.\nimshow\n(\npostprocess\n(\nX\n)\n.\nasnumpy\n())\nanimator\n.\nadd\n(\nepoch\n+\n1\n,\n[\nfloat\n(\nsum\n(\ncontents_l\n)),\nfloat\n(\nsum\n(\nstyles_l\n)),\nfloat\n(\ntv_l\n)])\nreturn\nX\nNow we start to train the model. We rescale the height and width of the\ncontent and style images to 300 by 450 pixels. We use the content image\nto initialize the synthesized image.\npytorch\nmxnet\ndevice\n,\nimage_shape\n=\nd2l\n.\ntry_gpu\n(),\n(\n300\n,\n450\n)\n# PIL Image (h, w)\nnet\n=\nnet\n.\nto\n(\ndevice\n)\ncontent_X\n,\ncontents_Y\n=\nget_contents\n(\nimage_shape\n,\ndevice\n)\n_\n,\nstyles_Y\n=\nget_styles\n(\nimage_shape\n,\ndevice\n)\noutput\n=\ntrain\n(\ncontent_X\n,\ncontents_Y\n,\nstyles_Y\n,\ndevice\n,\n0.3\n,\n500\n,\n50\n)\ndevice\n,\nimage_shape\n=\nd2l\n.\ntry_gpu\n(),\n(\n450\n,\n300\n)\nnet\n.\ncollect_params\n()\n.\nreset_ctx\n(\ndevice\n)\ncontent_X\n,\ncontents_Y\n=\nget_contents\n(\nimage_shape\n,\ndevice\n)\n_\n,\nstyles_Y\n=\nget_styles\n(\nimage_shape\n,\ndevice\n)\noutput\n=\ntrain\n(\ncontent_X\n,\ncontents_Y\n,\nstyles_Y\n,\ndevice\n,\n0.9\n,\n500\n,\n50\n)\nWe can see that the synthesized image retains the scenery and objects of\nthe content image, and transfers the color of the style image at the\nsame time. For example, the synthesized image has blocks of color like\nthose in the style image. Some of these blocks even have the subtle\ntexture of brush strokes.\n14.12.8.\nSummary\n¶\nThe loss function commonly used in style transfer consists of three\nparts: (i) content loss makes the synthesized image and the content\nimage close in content features; (ii) style loss makes the\nsynthesized image and style image close in style features; and (iii)\ntotal variation loss helps to reduce the noise in the synthesized\nimage.\nWe can use a pretrained CNN to extract image features and minimize\nthe loss function to continuously update the synthesized image as\nmodel parameters during training.\nWe use Gram matrices to represent the style outputs from the style\nlayers.\n14.12.9.\nExercises\n¶\nHow does the output change when you select different content and\nstyle layers?\nAdjust the weight hyperparameters in the loss function. Does the\noutput retain more content or have less noise?\nUse different content and style images. Can you create more\ninteresting synthesized images?\nCan we apply style transfer for text? Hint: you may refer to the\nsurvey paper by\nHu\net al.\n(\n2022\n)\n.\npytorch\nmxnet\nDiscussions\nDiscussions\nTable Of Contents\n14.12. Neural Style Transfer\n14.12.1. Method\n14.12.2. Reading the Content and Style Images\n14.12.3. Preprocessing and Postprocessing\n14.12.4. Extracting Features\n14.12.5. Defining the Loss Function\n14.12.5.1. Content Loss\n14.12.5.2. Style Loss\n14.12.5.3. Total Variation Loss\n14.12.5.4. Loss Function\n14.12.6. Initializing the Synthesized Image\n14.12.7. Training\n14.12.8. Summary\n14.12.9. Exercises\nPrevious\n14.11. Fully Convolutional Networks\nNext\n14.13. Image Classification (CIFAR-10) on Kaggle",
  "metadata": {
    "title": "14.12. Neural Style Transfer — Dive into Deep Learning 1.0.3 documentation",
    "description": null,
    "keywords": null,
    "author": null
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/c9c32763767f/code/body-000.txt",
      "chars": 200,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-001.txt",
      "chars": 79,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-002.txt",
      "chars": 252,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-003.txt",
      "chars": 86,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-004.txt",
      "chars": 87,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-005.txt",
      "chars": 583,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-006.txt",
      "chars": 414,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-007.txt",
      "chars": 59,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-008.txt",
      "chars": 180,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-009.txt",
      "chars": 63,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-010.txt",
      "chars": 184,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-011.txt",
      "chars": 56,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-012.txt",
      "chars": 56,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-013.txt",
      "chars": 128,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-014.txt",
      "chars": 118,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-015.txt",
      "chars": 291,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-016.txt",
      "chars": 291,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-017.txt",
      "chars": 415,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-018.txt",
      "chars": 423,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-019.txt",
      "chars": 262,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-020.txt",
      "chars": 67,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-021.txt",
      "chars": 161,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-022.txt",
      "chars": 157,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-023.txt",
      "chars": 93,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-024.txt",
      "chars": 81,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-025.txt",
      "chars": 177,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-026.txt",
      "chars": 171,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-027.txt",
      "chars": 581,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-028.txt",
      "chars": 581,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-029.txt",
      "chars": 247,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-030.txt",
      "chars": 259,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-031.txt",
      "chars": 282,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-032.txt",
      "chars": 358,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-033.txt",
      "chars": 1043,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-034.txt",
      "chars": 1098,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-035.txt",
      "chars": 263,
      "language": "text"
    },
    {
      "file": "attachments/c9c32763767f/code/body-036.txt",
      "chars": 261,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://www.d2l.ai/_static/logo-with-text.png",
      "bytes": 87175,
      "reused": false,
      "local_path": "attachments/c9c32763767f/images/body-509f124ba61493e64845.png",
      "src": "https://www.d2l.ai/_static/logo-with-text.png",
      "alt": "Dive into Deep Learning",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://www.d2l.ai/_images/style-transfer.svg",
      "bytes": 2936848,
      "reused": false,
      "local_path": "attachments/c9c32763767f/images/body-9585e9b518b90f372cc4.svg",
      "src": "https://www.d2l.ai/_images/style-transfer.svg",
      "alt": "../_images/style-transfer.svg",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://www.d2l.ai/_images/neural-style.svg",
      "bytes": 805932,
      "reused": false,
      "local_path": "attachments/c9c32763767f/images/body-4ef87a3cf31cd0adc102.svg",
      "src": "https://www.d2l.ai/_images/neural-style.svg",
      "alt": "../_images/neural-style.svg",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://www.d2l.ai/_images/output_neural-style_5de8ca_3_0.svg",
      "bytes": 156779,
      "reused": false,
      "local_path": "attachments/c9c32763767f/images/body-5ad0f43eece2664d395d.svg",
      "src": "https://www.d2l.ai/_images/output_neural-style_5de8ca_3_0.svg",
      "alt": "../_images/output_neural-style_5de8ca_3_0.svg",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://www.d2l.ai/_images/output_neural-style_5de8ca_4_0.svg",
      "bytes": 200175,
      "reused": false,
      "local_path": "attachments/c9c32763767f/images/body-825ef45b2e549f8f8766.svg",
      "src": "https://www.d2l.ai/_images/output_neural-style_5de8ca_4_0.svg",
      "alt": "../_images/output_neural-style_5de8ca_4_0.svg",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://www.d2l.ai/_images/output_neural-style_5de8ca_7_1.svg",
      "bytes": 156779,
      "reused": false,
      "local_path": "attachments/c9c32763767f/images/body-12f080b7cc274014a0bc.svg",
      "src": "https://www.d2l.ai/_images/output_neural-style_5de8ca_7_1.svg",
      "alt": "../_images/output_neural-style_5de8ca_7_1.svg",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://www.d2l.ai/_images/output_neural-style_5de8ca_8_0.svg",
      "bytes": 200175,
      "reused": false,
      "local_path": "attachments/c9c32763767f/images/body-7ac48f01e97fe0f78574.svg",
      "src": "https://www.d2l.ai/_images/output_neural-style_5de8ca_8_0.svg",
      "alt": "../_images/output_neural-style_5de8ca_8_0.svg",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://www.d2l.ai/_images/output_neural-style_5de8ca_140_0.svg",
      "bytes": 190219,
      "reused": false,
      "local_path": "attachments/c9c32763767f/images/body-4536399cd34651c4e4e3.svg",
      "src": "https://www.d2l.ai/_images/output_neural-style_5de8ca_140_0.svg",
      "alt": "../_images/output_neural-style_5de8ca_140_0.svg",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://www.d2l.ai/_images/output_neural-style_5de8ca_143_0.svg",
      "bytes": 220250,
      "reused": false,
      "local_path": "attachments/c9c32763767f/images/body-c6a521e6629b6cf4c85e.svg",
      "src": "https://www.d2l.ai/_images/output_neural-style_5de8ca_143_0.svg",
      "alt": "../_images/output_neural-style_5de8ca_143_0.svg",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://www.d2l.ai/index.html",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://colab.research.google.com/github/d2l-ai/d2l-pytorch-colab/blob/master/chapter_computer-vision/neural-style.ipynb",
      "label": "Colab [pytorch]",
      "type": "a"
    },
    {
      "href": "https://colab.research.google.com/github/d2l-ai/d2l-en-colab/blob/master/chapter_computer-vision/neural-style.ipynb",
      "label": "Colab [mxnet]",
      "type": "a"
    },
    {
      "href": "https://colab.research.google.com/github/d2l-ai/d2l-jax-colab/blob/master/chapter_computer-vision/neural-style.ipynb",
      "label": "Colab [jax]",
      "type": "a"
    },
    {
      "href": "https://colab.research.google.com/github/d2l-ai/d2l-tensorflow-colab/blob/master/chapter_computer-vision/neural-style.ipynb",
      "label": "Colab [tensorflow]",
      "type": "a"
    },
    {
      "href": "https://studiolab.sagemaker.aws/import/github/d2l-ai/d2l-pytorch-sagemaker-studio-lab/blob/main/GettingStarted-D2L.ipynb",
      "label": "SageMaker Studio Lab",
      "type": "a"
    },
    {
      "href": "https://www.d2l.ai/chapter_references/zreferences.html",
      "label": "Gatys et al. , 2016",
      "type": "a"
    },
    {
      "href": "https://www.d2l.ai/chapter_convolutional-modern/vgg.html",
      "label": "Section 8.2",
      "type": "a"
    },
    {
      "href": "https://discuss.d2l.ai/t/1476",
      "label": "Discussions",
      "type": "a"
    },
    {
      "href": "https://discuss.d2l.ai/t/378",
      "label": "Discussions",
      "type": "a"
    },
    {
      "href": "https://www.d2l.ai/chapter_computer-vision/fcn.html",
      "label": "Previous 14.11. Fully Convolutional Networks",
      "type": "a"
    },
    {
      "href": "https://www.d2l.ai/chapter_computer-vision/kaggle-cifar10.html",
      "label": "Next 14.13. Image Classification (CIFAR-10) on Kaggle",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:53:47Z",
  "source_url": "https://www.d2l.ai/chapter_computer-vision/neural-style.html",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 22699,
  "expected_body_images": 9,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/c9c32763767f/pages/source.html",
  "local_html": "attachments/c9c32763767f/pages/article.html",
  "body_base_path": "attachments/c9c32763767f/pages/",
  "body_markdown_path": "attachments/c9c32763767f/pages/article.md"
}