{ "source_file": "0930", "source_platform": "Medium", "author": "Dhruv Matani", "title_listed": "Efficient Image Segmentation Using PyTorch: Part 3", "keywords": "", "domain": "图像处理", "content_type": "深度技术解析", "is_paid_flag": "否", "url": "https://towardsdatascience.com/efficient-image-segmentation-using-pytorch-part-3-3534cf04fb89/", "publish_date": "2023-06-27", "corpus_stage": "", "notes": "", "id": "42c45c486bcb", "collected_at": "2026-10-02T10:45:42", "title": "Efficient Image Segmentation Using PyTorch: Part 3 | Towards Data Science", "body_markdown": "![Revisit consent button](../images/body-df29beacd7496220cc94.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](../images/body-2159bb67f3de8ad870eb.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n * Cookie\nBCTempID\n * Duration\n10 minutes\n * Description\nNo description available.\n\n\n * Cookie\n__cf_bm\n * Duration\n1 hour\n * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n * Cookie\nAWSALBCORS\n * Duration\n7 days\n * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n * Cookie\n_cfuvid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n * Cookie\nli_gc\n * Duration\n6 months\n * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n * Cookie\n__hssrc\n * Duration\nsession\n * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n * Cookie\n__hssc\n * Duration\n1 hour\n * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n * Cookie\nwpEmojiSettingsSupports\n * Duration\nsession\n * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n * Cookie\nBCSessionID\n * Duration\n1 year 1 month 4 days\n * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n * Cookie\n_octo\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nlogged_in\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\n__Secure-YEC\n * Duration\npast\n * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n * Cookie\n__eoi\n * Duration\n6 months\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTGCORS\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\nlogin-status-p\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTG\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\ncsrf_token\n * Duration\nsession\n * Description\nNo description available.\n\n\n * Cookie\ntoken_v2\n * Duration\n1 day\n * Description\nDescription is currently not available.\n\n\n * Cookie\nD\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nPHPSESSID\n * Duration\nsession\n * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n * Cookie\nVISITOR_PRIVACY_METADATA\n * Duration\n6 months\n * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n * Cookie\ncookietest\n * Duration\nsession\n * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n * Cookie\n__Host-airtable-session\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\n__Host-airtable-session.sig\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\nm\n * Duration\n1 year 1 month 4 days\n * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n * Cookie\nBIGipServer*\n * Duration\nsession\n * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n * Cookie\n__cfruid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n * Cookie\n_GRECAPTCHA\n * Duration\n6 months\n * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n * Cookie\n__Secure-YNID\n * Duration\n6 months\n * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n * Cookie\ncookieyes-consent\n * Duration\n1 year\n * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n * Cookie\nlidc\n * Duration\n1 day\n * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n * Cookie\nbrw\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nbrwConsent\n * Duration\n5 minutes\n * Description\nDescription is currently not available.\n\n\n * Cookie\nWMF-Uniq\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nloom_anon_comment\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nloom_referral_video\n * Duration\nsession\n * Description\nDescription is currently not available.\n\n\n * Cookie\nVISITOR_INFO1_LIVE\n * Duration\n6 months\n * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n * Cookie\nyt-remote-connected-devices\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n * Duration\nNever Expires\n * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n * Cookie\nyt-remote-device-id\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-name\n * Duration\nsession\n * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\nyt-remote-fast-check-period\n * Duration\nsession\n * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-app\n * Duration\nsession\n * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n * Cookie\nyt-remote-cast-available\n * Duration\nsession\n * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n * Cookie\nyt-remote-cast-installed\n * Duration\nsession\n * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\ncp_session\n * Duration\n3 months\n * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n * Cookie\nloid\n * Duration\n1 year 1 month 4 days\n * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n * Cookie\n__hstc\n * Duration\n6 months\n * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n * Cookie\nhubspotutk\n * Duration\n6 months\n * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n * Cookie\n_ga\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n * Cookie\n_ga_*\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n * Cookie\n__Host-psifi.analyticsTrace\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n__Host-psifi.analyticsTraceV2\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n_gh_sess\n * Duration\nsession\n * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n * Cookie\nYSC\n * Duration\nsession\n * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n * Cookie\najs_anonymous_id\n * Duration\n1 year\n * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n * Cookie\nvuid\n * Duration\n1 year 1 month 4 days\n * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n * Cookie\nAWSALB\n * Duration\n7 days\n * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n * Cookie\nacq\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nacq.sig\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nptc\n * Duration\n2 years\n * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n * Cookie\nmuc_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n * Cookie\nguest_id_marketing\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\nguest_id_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\npersonalization_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n * Cookie\nguest_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n * Cookie\nbcookie\n * Duration\n1 year\n * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n * Cookie\n__Secure-ROLLOUT_TOKEN\n * Duration\n6 months\n * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n * Cookie\nyt.innertube::nextId\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nyt.innertube::requests\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nsession_tracker\n * Duration\nsession\n * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n * Cookie\nedgebucket\n * Duration\nsession\n * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n * Cookie\ndid\n * Duration\n1 year\n * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n# Efficient Image Segmentation Using PyTorch: Part 3\nDepthwise separable convolutions\n[Dhruv Matani](https://towardsdatascience.com/author/dhruvbird/)\nJune 27, 202313 min read\nIn this 4-part series, we'll implement image segmentation step by step from scratch using deep learning techniques in PyTorch. This part will focus on optimizing our CNN baseline model using depthwise separable convolutions to reduce the number of trainable parameters, making the model deployable on mobile and other edge devices.\nCo-authored with [Naresh Singh](https://medium.com/@brocolishbroxoli)\n![Figure 1: Result of running image segmentation using a CNN with depth-wise separable convolutions instead of regular convolutions. From top to bottom, input images, ground truth segmentation masks, and predicted segmentation masks. Source: Author\\(s\\)](../images/000.png)\nFigure 1: Result of running image segmentation using a CNN with depth-wise separable convolutions instead of regular convolutions. From top to bottom, input images, ground truth segmentation masks, and predicted segmentation masks. Source: Author(s)\n* * *\n## Article outline\nIn this article, we will augment the [Convolutional Neural Network (CNN)](https://en.wikipedia.org/wiki/Convolutional_neural_network) we [built earlier](https://medium.com/p/bed68cadd7c7/) to reduce the number of learnable parameters in our network. The task of identifying pet pixels (pixels belonging to cats, dogs, hamsters, etc...) in an input image remains unchanged. Our network of choice will remain [SegNet](https://arxiv.org/abs/1511.00561), and the only change we'll make is to replace our convolutional layers with depth-wise-separable-convolutions (DSC). Before we do this, we will dive into the theory and practice of depth-wise-separable-convolutions, and appreciate the idea behind the technique.\nThroughout this article, we will reference code and results from this [[notebook](https://github.com/dhruvbird/ml-notebooks/blob/main/pets_segmentation/types%20of%20convolutions.ipynb)](https://github.com/dhruvbird/ml-notebooks/blob/main/pets_segmentation/oxford-iiit-pets-segmentation-using-pytorch-segnet-and-depth-wise-separable-convs.ipynb) for model training, and this notebook for a primer on DSC. If you wish to reproduce the results, you'll need a GPU to ensure that the first notebook completes running in a reasonable amount of time. The second notebook can be run on a regular CPU.\n* * *\n## Articles in this series\nThis series is for readers at all experience levels with deep learning. If you want to learn about the practice of deep learning and vision AI along with some solid theory and hands-on experience, you've come to the right place! This is expected to be a 4-part series with the following articles:\n 1. [Concepts and Ideas](https://medium.com/p/89e8297a0923/)\n 2. [A CNN-based model](https://medium.com/p/bed68cadd7c7/)\n 3. **Depthwise separable convolutions (this article)**\n 4. [A Vision Transformer-based model](https://medium.com/p/6c86da083432/)\n\n\n* * *\n## Introduction\nLet's start this discussion with a closer look at the convolutions from the perspective of model size and computation cost. The number of trainable parameters is a good indication of the size of a model and the number of the tensor operations reflects the model complexity or computation cost. Consider that we have a convolution layer with n filters with size dₖ x dₖ. Further assume that this layer processes input with shape _m x h x w,_ where _m_ is the number of input channels, and _h_ and _w_ are height and width dimensions respectively. In this case, the convolution layer will produce an output with shape _n x h x w_ as shown in Figure 2. We are assuming that the convolution uses _stride=1_. Let's go ahead and evaluate this setup in terms of trainable parameters and computation cost.\n![Figure 2: Regular convolutional filters applied to input to produce output. Assume stride=1 and padding=dₖ-2. Source: Efficient Deep Learning Book](../images/001.png)\nFigure 2: Regular convolutional filters applied to input to produce output. Assume stride=1 and padding=dₖ-2. Source: [Efficient Deep Learning Book](https://efficientdlbook.com/)\n**Evaluation Of Trainable Parameters:** We have _n_ filters, each of which has _m x dₖ x dₖ_ learnable parameters. This results in a total of _n x m x dₖ x dₖ_ learnable parameters. Bias terms are ignored to simplify this discussion. Let's look at the PyTorch code below to validate our understanding.\npython\n\n```\nimport torch\nfrom torch import nn\ndef num_parameters(m):\nreturn sum([p.numel() for p in m.parameters()])\ndk, m, n = 3, 16, 32\nprint(f\"Expected number of parameters: {m * dk * dk * n}\")\nconv1 = nn.Conv2d(in_channels=m, out_channels=n, kernel_size=dk, bias=False)\nprint(f\"Actual number of parameters: {num_parameters(conv1)}\")\n```\n\nPrints the following.\ntext\n\n```\nExpected number of parameters: 4608\nActual number of parameters: 4608\n```\n\nNow, let's evaluate the computation costs of convolution.\n**Evaluation Of Computational Cost:** A single convolutional filter of shape _m x dₖ x dₖ_ when run with a _stride=1_ and a _padding=dₖ-2_ on an input with size _h x w_ will apply the convolutional filter _h x w_ times, once for each image section with size _dₖ x dₖ_ amounting to a total of _h x w_ sections. It results in a cost of _m x dₖ x dₖ x h x w_ per filter or output channel. Since we wish to compute n output channels, the total cost will be _m x dₖ x dₖ x h x n_. Let's go ahead and validate this using the torchinfo PyTorch package.\npython\n\n```\nfrom torchinfo import summary\nh, w = 128, 128\nprint(f\"Expected total multiplies: {m * dk * dk * h * w * n}\")\nsummary(conv1, input_size=(1, m, h, w))\n```\n\nWill print the following.\ntext\n\n```\nExpected total multiplies: 75497472\n\n==========================================================================================\nLayer (type:depth-idx) Output Shape Param #\n==========================================================================================\nConv2d [1, 32, 128, 128] 4,608\n==========================================================================================\nTotal params: 4,608\nTrainable params: 4,608\nNon-trainable params: 0\nTotal mult-adds (M): 75.50\n==========================================================================================\nInput size (MB): 1.05\nForward/backward pass size (MB): 4.19\nParams size (MB): 0.02\nEstimated Total Size (MB): 5.26\n==========================================================================================\n```\n\nIf we ignore the implementation details of a convolution layer for a moment, we would realize that, on a high level, a convolution layer just transforms a _m x h x w_ input into a _n x h x w_ output. The transformation is achieved through trainable filters which progressively learn features as they _see_ inputs. The question that follows is: Is it possible to achieve this transformation using fewer learnable parameters and simultaneously ensuring minimum compromise in the learning capabilities of the layer? Depthwise Separable Convolutions were proposed to answer this exact question. Let's understand them in detail and learn how they stack up on our evaluation metrics.\n* * *\n## Depthwise Separable Convolution\nThe concept of Depthwise Separable Convolutions (DSC) was first proposed by Laurent Sifre in their PhD thesis titled [Rigid-Motion Scattering For Image Classification](https://www.di.ens.fr/data/publications/papers/phd_sifre.pdf). Since then, they have been used successfully in various popular deep convolutional networks such as [XceptionNet](https://arxiv.org/abs/1610.02357) and [MobileNet](https://towardsdatascience.com/review-mobilenetv1-depthwise-separable-convolution-light-weight-model-a382df364b69).\nThe main difference between a regular convolution, and a DSC is that a DSC is composed of 2 convolutions as described below:\n 1. A **depthwise grouped convolution** , w[here](https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html) the number of input channels m is equal to the number of output channels such that each output channel is affected only by a single input channel. In PyTorch, this is called a \"grouped\" convolution. You can read more about grouped convolutions in PyTorch here.\n 2. A **pointwise convolution** (filter size=1), which operates like a regular convolution such that each of the n filters operates on all m input channels to produce a single output value.\n\n\n![Figure 3: Depthwise Separable Convolution filters applied to input to produce output. Assume stride=1 and padding=dₖ-2. Source: Efficient Deep Learning Book](../images/002.png)\nFigure 3: Depthwise Separable Convolution filters applied to input to produce output. Assume _stride=1_ and _padding=dₖ-2_. Source: [Efficient Deep Learning Book](https://efficientdlbook.com/)\nLet's perform the same exercise that we did for regular convolutions for DSCs and compute the number of trainable parameters and computations.\n**Evaluation Of Trainable Parameters:** The \"grouped\" convolutions have m filters, each of which has _dₖ x dₖ_ learnable parameters which produces m output channels. This results in a total of _m x dₖ x dₖ_ learnable parameters. The pointwise convolution has n filters of size _m x 1 x 1_ which adds up to _n x m x 1 x 1_ learnable parameters. Let's look at the PyTorch code below to validate our understanding.\npython\n\n```\nclass DepthwiseSeparableConv(nn.Sequential):\n def __init__(self, chin, chout, dk):\n super().__init__(\n # Depthwise convolution\n nn.Conv2d(chin, chin, kernel_size=dk, stride=1, padding=dk-2, bias=False, groups=chin),\n # Pointwise convolution\n nn.Conv2d(chin, chout, kernel_size=1, bias=False),\n )\n\nconv2 = DepthwiseSeparableConv(chin=m, chout=n, dk=dk)\nprint(f\"Expected number of parameters: {m * dk * dk + m * 1 * 1 * n}\")\nprint(f\"Actual number of parameters: {num_parameters(conv2)}\")\n```\n\nWhich will print.\ntext\n\n```\nExpected number of parameters: 656\nActual number of parameters: 656\n```\n\nWe can see that the DSC version has roughly _7x_ less parameters. Next, let's focus our attention on the computation costs for a DSC layer.\n**Evaluation Of Computational Cost:** Let's assume our input has spatial dimensions _m x h x w_. In the grouped convolution segment of DSC, we have **m** filters, each with size _dₖ x dₖ_. A filter is applied to its corresponding input channel resulting in the segment cost of _m x dₖ x dₖ x h x w_. For the pointwise convolution, we apply **n** filters of size _m x 1 x 1_ **** to produce** n** output channels. This results in the segment cost of _n x m x 1 x 1 x h x_ w. We need to add up the costs of the grouped and pointwise operations to compute the total cost. Let's go ahead and validate this using the [torchinfo](https://pypi.org/project/torchinfo/) PyTorch package.\npython\n\n```\nprint(f\"Expected total multiplies: {m * dk * dk * h * w + m * 1 * 1 * h * w * n}\")\ns2 = summary(conv2, input_size=(1, m, h, w))\nprint(f\"Actual multiplies: {s2.total_mult_adds}\")\nprint(s2)\n```\n\nWhich will print.\ntext\n\n```\nExpected total multiplies: 10747904\nActual multiplies: 10747904\n==========================================================================================\nLayer (type:depth-idx) Output Shape Param #\n==========================================================================================\nDepthwiseSeparableConv [1, 32, 128, 128] --\n├─Conv2d: 1-1 [1, 16, 128, 128] 144\n├─Conv2d: 1-2 [1, 32, 128, 128] 512\n==========================================================================================\nTotal params: 656\nTrainable params: 656\nNon-trainable params: 0\nTotal mult-adds (M): 10.75\n==========================================================================================\nInput size (MB): 1.05\nForward/backward pass size (MB): 6.29\nParams size (MB): 0.00\nEstimated Total Size (MB): 7.34\n==========================================================================================\n```\n\nLet's compare the sizes and costs of both the convolutions for a few examples to gain some intuition.\n### Size and Cost comparison for regular and depthwise separable convolutions\nTo compare the size and cost of regular and depthwise separable convolution, we will assume an input size of _128 x 128_ to the network, a kernel size of _3 x 3_ , and a network that progressively halves the spatial dimensions and doubles the number of channel dimensions. We assume a single 2d-conv layer at every step, but in practice, there could be more.\n![Figure 4: Comparing the number of trainable parameters \\(size\\) and multi-adds \\(cost\\) of regular and depthwise separable convolutions. We also show the ratio of the size and cost for the 2 types of convolutions. Source: Author\\(s\\).](../images/003.png)\nFigure 4: Comparing the number of trainable parameters (size) and multi-adds (cost) of regular and depthwise separable convolutions. We also show the ratio of the size and cost for the 2 types of convolutions. Source: Author(s).\nYou can see that on average both the size and computational cost of DSC is about 11% to 12% of the cost of regular convolutions for the configuration mentioned above.\n![Figure 5: Relative size and cost of regular v/s DSC. Source: Author\\(s\\).](../images/004.png)\nFigure 5: Relative size and cost of regular v/s DSC. Source: Author(s).\nNow that we have developed a good understanding of the types of convolutions and their relative costs, you must be wondering if there's any downside of using DSCs. Everything we've seen so far seems to suggest that they are better in every way! Well, we haven't yet considered an important aspect which is the impact they have on the accuracy of our model. Let's dive into it via an experiment below.\n* * *\n## SegNet Using Depthwise Separable Convolutions\n[This notebook](https://github.com/dhruvbird/ml-notebooks/blob/main/pets_segmentation/oxford-iiit-pets-segmentation-using-pytorch-segnet-and-depth-wise-separable-convs.ipynb) contains all the code for this section.\nWe will adapt our SegNet model from the [previous post](https://medium.com/p/bed68cadd7c7/) and replace all the regular convolutional layers with a DSC layer. Once we do this, we notice that the number of parameters in our notebook drops from 15.27M to 1.75M parameters, which is a reduction of 88.5%! This is inline with our earlier estimates of an 11% to 12% reduction in the number of trainable parameters of the network.\nA similar configuration as [before](https://medium.com/p/bed68cadd7c7/) was used during model training and validation. The configuration is specified below.\n 1. The _random horizontal flip_ and _colour jitter_ data augmentations are applied to the training set to prevent overfitting\n 2. The images are resized to 128x128 pixels in a non-aspect preserving resize operation\n 3. No input normalization is applied to the images - instead a [batch normalization layer is used as the first layer of the model](https://towardsdatascience.com/replace-manual-normalization-with-batch-normalization-in-vision-ai-models-e7782e82193c)\n 4. The model is trained for 20 epochs using the Adam optimizer with a LR of 0.001 and no LR scheduler\n 5. The cross-entropy loss function is used to classify a pixel as belonging to a pet, the background, or a pet border\n\n\nThe model achieved a validation accuracy of 86.96% after 20 training epochs. This is less than the 88.28% accuracy achieved by the model using regular convolutions for the same number of training epochs. We have determined experimentally that training for more epochs improves the accuracy of both models, so 20 epochs is definitely not the end of the training cycle. We stop at 20 epochs for the purposes of this article for demonstration purposes.\nWe plotted a gif showing how the model is learning to predict the segmentation masks for 21 images in the validation set.\n![Figure 6: A gif showing how the SegNet model with DSC is learning to predict segmentation masks for 21 images in the validation set. Source: Author\\(s\\)](../images/body-4b1e353cb5d5b3e70ef0.img)\nFigure 6: A gif showing how the SegNet model with DSC is learning to predict segmentation masks for 21 images in the validation set. Source: Author(s)\nNow that we have seen how the model progresses through the training cycle, let's compare the train cycles of models with regular convolutions and DSC.\n### Accuracy Comparisons\nWe found it useful to look at the training cycles of the models using regular convolutions and DSC. The main difference we noticed is in the early phases (epochs) of training, after which both models settled roughly into the same prediction flow. In fact after training both models for 100 epochs, we noticed that the accuracy of the model with DSC is just about 1% less than the model with regular convolutions. This is inline with our observations from just 20 epochs of training.\n![Figure 7: A gif showing the progression of segmentation masks predicted by the SegNet model using regular convolutions v/s DSC. Source: Author\\(s\\).](../images/body-4250e1c19aaebbfe9717.img)\nFigure 7: A gif showing the progression of segmentation masks predicted by the SegNet model using regular convolutions v/s DSC. Source: Author(s).\nYou would have noticed that both models get the predictions roughly right after just 6 training epochs - i.e. one can visually see that the models are predicting something useful. Most of the hard work of training the model is then above ensuring that the borders of the predicted masks are as tight as possible and as close to the actual pets in the image as possible. This means that while one can expect a lesser absolute increase in accuracy in the later training epochs, the impact of this on the quality of predictions is much more. We've noticed that a single digit of accuracy improvement at higher absolute accuracy values (going from 89% to 90%) results in significant qualitative improvements to the predictions.\n### Comparison with a UNet model\nWe ran an experiment that changed a lot of hyperparameters with a focus on improving the overall accuracy to get a sense of how far this setting is from close to optimal. Here's the configuration of that experiment.\n 1. Image size: 128 x 128 - same as the experiments so far\n 2. Train epochs: 100 - current experiments trained for 20 epochs\n 3. Augmentations: A lot more augmentations such as image rotation, channel dropping, random block removal. We used Albumentations instead of torchvision transforms. Albumentations automatically transforms segmentation masks for us\n 4. LR Scheduler: A StepLR scheduler was used with a decay of 0.8x every 25 train epochs\n 5. Loss function: We tried 4 different loss functions: Cross Entropy, Focal, Dice, Weighted Cross Entropy. Dice performed worst whereas the rest were pretty much comparable to each other. In fact the difference in best accuracy between the rest after 100 epochs was in the 4th digit after the decimal (assuming the accuracy is a number between 0.0 and 1.0)\n 6. Convolution type: Regular\n 7. Model type: UNet - current experiments used a SegNet model\n\n\nWe achieved a best validation accuracy of 91.3% for the setting above. We noticed that the image size significantly impacts the best validation accuracy. For example, when we changed the image size to 256 x 256, the best validation accuracy went up to 93.0%. However, training took much longer, and used more memory, which meant that we had to reduce the batch size.\n![Figure 8: Result of training a UNet model for 100 train epochs with the hyperparameters mentioned above. Source: Author\\(s\\).](../images/005.png)\nFigure 8: Result of training a UNet model for 100 train epochs with the hyperparameters mentioned above. Source: Author(s).\nYou can see that the predictions are much smoother and crisper compared to the ones we have been seeing so far.\n* * *\n## Conclusion\nIn part-3 of this series, we learned about depth wise separable convolutions (DSC) as a technique to reduce model size and training/inference cost without a significant loss in validation accuracy. We learned about the size/cost tradeoff to expect between regular and DSC for a specific setting.\nWe showed how to adapt the SegNet model to use DSC in PyTorch. This technique can be applied to any deep CNN. In fact we can selectively replace some of the convolutional layers with DSC - i.e. we don't need to necessarily replace all of them. Choosing which layers to replace will depend on the balance you wish to strike between model size/runtime-cost and prediction accuracy. This decision will depend on your specific use case and deployment setup.\nWhile this article trained models for 20 epochs, we explained that this is insufficient for production workloads, and provided a glimpse into what one can expect if one trains the model for more epochs. In addition, we provided an introduction to some of the hyperparameters that one can tune during model training. While this list is by no means comprehensive, it should allow you to appreciate the complexity and decision making needed to train an image segmentation model for production workloads.\nIn the [next part of this series](https://medium.com/p/6c86da083432/), we'll take a look at Vision Transformers, and how we can use this model architecture to perform image segmentation for the pets segmentation task.\n* * *\n## References and further reading\n * [Efficient Deep Learning Book Chapter 04 - Efficient Architectures](https://github.com/EfficientDL/book/raw/main/book/%5BEDL%5D%20Chapter%204%20-%20Efficient%20Architectures.pdf)\n * [A Basic Introduction to Separable Convolutions](https://towardsdatascience.com/a-basic-introduction-to-separable-convolutions-b99ec3102728)\n\n\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](../images/body-a3403563a948ce1eab7b.jpeg)\nWritten By\n## [Dhruv Matani](https://towardsdatascience.com/author/dhruvbird/)\n[See all from Dhruv Matani](https://towardsdatascience.com/author/dhruvbird/)\n[Cnn](https://towardsdatascience.com/tag/cnn/)[Depthwise Convolution](https://towardsdatascience.com/tag/depthwise-convolution/)[Pytorch](https://towardsdatascience.com/tag/pytorch/)[Pytorch Image Segment](https://towardsdatascience.com/tag/pytorch-image-segment/)[Segnet](https://towardsdatascience.com/tag/segnet/)\nShare This Article\n * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fefficient-image-segmentation-using-pytorch-part-3-3534cf04fb89%2F&title=Efficient%20Image%20Segmentation%20Using%20PyTorch%3A%20Part%203)\n * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fefficient-image-segmentation-using-pytorch-part-3-3534cf04fb89%2F&title=Efficient%20Image%20Segmentation%20Using%20PyTorch%3A%20Part%203)\n * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fefficient-image-segmentation-using-pytorch-part-3-3534cf04fb89%2F&text=Efficient%20Image%20Segmentation%20Using%20PyTorch%3A%20Part%203)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n * [![](../images/006.png)](https://towardsdatascience.com/how-to-understand-the-deep-time-series-classifier-with-integrated-gradients-f4f58af48c05/)\n## [How to Understand the Deep Time Series Classifier with Integrated Gradients](https://towardsdatascience.com/how-to-understand-the-deep-time-series-classifier-with-integrated-gradients-f4f58af48c05/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nTime series classification is a general problem that can be seen in many domains and contexts, e.g. for given time series of a financial...\n[Shuyang Xiang](https://towardsdatascience.com/author/vanillaxiangshuyang/)\nNovember 20, 20214 min read\n * [![](../images/007.jpeg)](https://towardsdatascience.com/lets-talk-about-graph-neural-network-python-libraries-a0b23ec983b0/)\n## [Let's Talk About Graph Neural Network Python Libraries!](https://towardsdatascience.com/lets-talk-about-graph-neural-network-python-libraries-a0b23ec983b0/)\n[Technology](https://towardsdatascience.com/category/technology/)\nAnd get our hands dirty by formulating a Node Classification task using DGL, PyG and Spektral.\n[Rohith Teja](https://towardsdatascience.com/author/rohithtejam/)\nAugust 21, 20215 min read\n * [![](../images/008.jpeg)](https://towardsdatascience.com/extracting-rich-embedding-features-from-coco-pictures-using-pytorch-and-resnext-wsl-vademecum-of-6fdbdbe876a8/)\n## [Extracting rich embedding features from COCO pictures using PyTorch and ResNeXt-WSL](https://towardsdatascience.com/extracting-rich-embedding-features-from-coco-pictures-using-pytorch-and-resnext-wsl-vademecum-of-6fdbdbe876a8/)\n[](https://towardsdatascience.com/efficient-image-segmentation-using-pytorch-part-3-3534cf04fb89/)\nHow to leverage a powerful pre-trained convolution neural network to extract embedding vectors for pictures.\n[Gianmario Spacagna](https://towardsdatascience.com/author/gm-spacagna/)\nDecember 2, 20207 min read\n * [![](../images/009.png)](https://towardsdatascience.com/train-conversational-ai-in-3-lines-of-code-with-nemo-and-lightning-a6088988ae37/)\n## [Train Conversational AI in 3 lines of code with NeMo and Lightning](https://towardsdatascience.com/train-conversational-ai-in-3-lines-of-code-with-nemo-and-lightning-a6088988ae37/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nTrain state-of-the-art speech recognition, NLP and TTS models at scale with NeMo and Lightning\n[Sean Narenthiran](https://towardsdatascience.com/author/seannaren/)\nOctober 28, 20204 min read\n * [![](../images/010.jpeg)](https://towardsdatascience.com/implementing-tabnet-in-pytorch-fc977c383279/)\n## [Implementing TabNet in PyTorch](https://towardsdatascience.com/implementing-tabnet-in-pytorch-fc977c383279/)\n[](https://towardsdatascience.com/efficient-image-segmentation-using-pytorch-part-3-3534cf04fb89/)\nDeep Learning has taken over vision, natural language processing, speech recognition, and many other fields achieving astonishing results...\n[Samrat Thapa](https://towardsdatascience.com/author/samratthapa120/)\nOctober 23, 20207 min read\n * [![](../images/011.jpg)](https://towardsdatascience.com/solving-bottlenecks-on-the-data-input-pipeline-with-pytorch-profiler-and-tensorboard-5dced134dbe9/)\n## [Solving Bottlenecks on the Data Input Pipeline with PyTorch Profiler and TensorBoard](https://towardsdatascience.com/solving-bottlenecks-on-the-data-input-pipeline-with-pytorch-profiler-and-tensorboard-5dced134dbe9/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nPyTorch Model Performance Analysis and Optimization - Part 4\n[Chaim Rand](https://towardsdatascience.com/author/chaimrand/)\nAugust 26, 202311 min read\n * [![](../images/012.jpeg)](https://towardsdatascience.com/optimizing-token-generation-in-pytorch-decoder-models/)\n## [Optimizing Token Generation in PyTorch Decoder Models](https://towardsdatascience.com/optimizing-token-generation-in-pytorch-decoder-models/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nHiding host-device synchronization via CUDA stream interleaving\n[Chaim Rand](https://towardsdatascience.com/author/chaimrand/)\nFebruary 24, 202615 min read\n * [![](../images/013.jpg)](https://towardsdatascience.com/custom-dataset-in-pytorch-part-1-images-2df3152895/)\n## [Custom dataset in Pytorch -Part 1. Images](https://towardsdatascience.com/custom-dataset-in-pytorch-part-1-images-2df3152895/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nPytorch has a great ecosystem to load custom datasets for training machine learning models. This is the first part of the two-part series...\n[Utkarsh Garg](https://towardsdatascience.com/author/ug2409/)\nAugust 18, 20215 min read\n * [![](../images/014.jpeg)](https://towardsdatascience.com/ml-engineering-101-a-thorough-explanation-of-the-error-dataloader-worker-pid-s-xxx-exited-f3a6a983911e/)\n## [ML Engineering 101: A Thorough Explanation of The Error \"DataLoader worker (pid(s) xxx) exited...](https://towardsdatascience.com/ml-engineering-101-a-thorough-explanation-of-the-error-dataloader-worker-pid-s-xxx-exited-f3a6a983911e/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nA deep dive into PyTorch DataLoader with Multiprocessing\n[Mengliu Zhao](https://towardsdatascience.com/author/mengliuz/)\nJune 2, 20246 min read\n\n\n * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n * [X](https://x.com/TDataScience)\n * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n * [Threads](https://www.threads.net/@towardsdatascience)\n * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n#### Recommended Articles \nClose\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/4970/image?etag=1780565225463&width=500&height=500) Productivity Tips, Data Career Insights, and Other Recent Must-Reads ](https://towardsdatascience.com/productivity-tips-data-career-insights-and-other-recent-must-reads-5c47eb7a2fe0/ \"Productivity Tips, Data Career Insights, and Other Recent Must-Reads\")\n\n\n", "metadata": { "title": "Efficient Image Segmentation Using PyTorch: Part 3 | Towards Data Science", "description": "Depthwise separable convolutions", "keywords": null, "author": "Dhruv Matani", "og:locale": "en_US", "og:type": "article", "og:title": "Efficient Image Segmentation Using PyTorch: Part 3", "og:description": "Depthwise separable convolutions", "og:url": "https://towardsdatascience.com/efficient-image-segmentation-using-pytorch-part-3-3534cf04fb89/", "og:site_name": "Towards Data Science", "og:image": "https://assets.insightmediagroup.io/media/wp-content/uploads/2023/06/1Q4ZM8UwFIrJKMbPJp6mlJg.png", "og:image:width": "682", "og:image:height": "376", "og:image:type": "image/png", "twitter:card": "summary_large_image", "twitter:site": "@TDataScience", "twitter:creator": "@TDataScience", "twitter:label1": "Written by", "twitter:data1": "Dhruv Matani", "twitter:label2": "Est. reading time", "twitter:data2": "13 min read", "article:published_time": "2023-06-27T05:12:37.000Z", "article:modified_time": "2023-06-27T05:12:37.000Z", "article:author": "https://towardsdatascience.com/author/dhruvbird/" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 202, "error": "" }, "code_blocks": [ { "language": "txt", "file": "attachments/42c45c486bcb/code/block_000.txt", "chars": 325 }, { "language": "txt", "file": "attachments/42c45c486bcb/code/block_001.txt", "chars": 70 }, { "language": "txt", "file": "attachments/42c45c486bcb/code/block_002.txt", "chars": 149 }, { "language": "txt", "file": "attachments/42c45c486bcb/code/block_003.txt", "chars": 850 }, { "language": "txt", "file": "attachments/42c45c486bcb/code/block_004.txt", "chars": 547 }, { "language": "txt", "file": "attachments/42c45c486bcb/code/block_005.txt", "chars": 68 }, { "language": "txt", "file": "attachments/42c45c486bcb/code/block_006.txt", "chars": 188 }, { "language": "txt", "file": "attachments/42c45c486bcb/code/block_007.txt", "chars": 1012 } ], "images": [ { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T17:59:01Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T17:59:01Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T17:59:01Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/4970/image?etag=1780565225463&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/4970/image?etag=1780565225463&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T17:59:01Z" } ], "audio": [], "videos": [ { "url": "https://www.youtube.com/c/TowardsDataScience", "kind": "channel_reference", "status": "reference_only" } ], "links": [ { "type": "internal", "href": "https://towardsdatascience.com/" }, { "type": "internal", "href": "https://towardsdatascience.com/latest/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/editors-pick/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/the-variable/" }, { "type": "internal", "href": "https://towardsdatascience.com/shipai/" }, { "type": "internal", "href": "https://towardsdatascience.com/submissions/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/dhruvbird/" }, { "type": "internal", "href": "https://towardsdatascience.com/review-mobilenetv1-depthwise-separable-convolution-light-weight-model-a382df364b69" }, { "type": "internal", "href": "https://towardsdatascience.com/replace-manual-normalization-with-batch-normalization-in-vision-ai-models-e7782e82193c" }, { "type": "internal", "href": "https://towardsdatascience.com/a-basic-introduction-to-separable-convolutions-b99ec3102728" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/cnn/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/depthwise-convolution/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/pytorch/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/pytorch-image-segment/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/segnet/" }, { "type": "internal", "href": "https://towardsdatascience.com/how-to-understand-the-deep-time-series-classifier-with-integrated-gradients-f4f58af48c05/" }, { "type": "internal", "href": "https://towardsdatascience.com/category/data-science/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/vanillaxiangshuyang/" }, { "type": "internal", "href": "https://towardsdatascience.com/lets-talk-about-graph-neural-network-python-libraries-a0b23ec983b0/" }, { "type": "internal", "href": "https://towardsdatascience.com/category/technology/" }, { "type": "external", "href": "https://write.insightmediagroup.io" }, { "type": "external", "href": "https://www.linkedin.com/company/towards-data-science/" }, { "type": "external", "href": "https://x.com/TDataScience" }, { "type": "external", "href": "https://medium.com/@brocolishbroxoli" }, { "type": "external", "href": "https://en.wikipedia.org/wiki/Convolutional_neural_network" }, { "type": "external", "href": "https://medium.com/p/bed68cadd7c7/" }, { "type": "external", "href": "https://arxiv.org/abs/1511.00561" }, { "type": "external", "href": "https://github.com/dhruvbird/ml-notebooks/blob/main/pets_segmentation/oxford-iiit-pets-segmentation-using-pytorch-segnet-and-depth-wise-separable-convs.ipynb" }, { "type": "external", "href": "https://medium.com/p/89e8297a0923/" }, { "type": "external", "href": "https://medium.com/p/6c86da083432/" }, { "type": "external", "href": "https://efficientdlbook.com/" }, { "type": "external", "href": "https://www.di.ens.fr/data/publications/papers/phd_sifre.pdf" }, { "type": "external", "href": "https://arxiv.org/abs/1610.02357" }, { "type": "external", "href": "https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html" }, { "type": "external", "href": "https://pypi.org/project/torchinfo/" }, { "type": "external", "href": "https://github.com/EfficientDL/book/raw/main/book/%5BEDL%5D%20Chapter%204%20-%20Efficient%20Architectures.pdf" }, { "type": "external", "href": "https://www.facebook.com/sharer/sharer.php?title=Efficient+Image+Segmentation+Using+PyTorch%3A+Part+3&u=https%3A%2F%2Ftowardsdatascience.com%2Fefficient-image-segmentation-using-pytorch-part-3-3534cf04fb89%2F" }, { "type": "external", "href": "https://www.linkedin.com/shareArticle?mini=true&title=Efficient+Image+Segmentation+Using+PyTorch%3A+Part+3&url=https%3A%2F%2Ftowardsdatascience.com%2Fefficient-image-segmentation-using-pytorch-part-3-3534cf04fb89%2F" }, { "type": "external", "href": "https://x.com/share?text=Efficient+Image+Segmentation+Using+PyTorch%3A+Part+3&url=https%3A%2F%2Ftowardsdatascience.com%2Fefficient-image-segmentation-using-pytorch-part-3-3534cf04fb89%2F" }, { "type": "external", "href": "https://www.youtube.com/c/TowardsDataScience" } ], "repair_version": 2, "repaired_at": "2026-10-20T17:29:30Z", "source_url": "https://towardsdatascience.com/efficient-image-segmentation-using-pytorch-part-3-3534cf04fb89/", "delivery_status": "unverified", "discovery_status": "http_403", "missing_reason": "源页面未重新核验:http_403；已按保留正文补采图片", "repair_error": "", "actual_video_files": [], "expected_saved_body_images": 4, "image_discovery": "saved_markdown", "source_body_markdown": "![Revisit consent button](https://cdn-cookieyes.com/assets/images/revisit.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](https://cdn-cookieyes.com/assets/images/close.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n * Cookie\nBCTempID\n * Duration\n10 minutes\n * Description\nNo description available.\n\n\n * Cookie\n__cf_bm\n * Duration\n1 hour\n * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n * Cookie\nAWSALBCORS\n * Duration\n7 days\n * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n * Cookie\n_cfuvid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n * Cookie\nli_gc\n * Duration\n6 months\n * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n * Cookie\n__hssrc\n * Duration\nsession\n * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n * Cookie\n__hssc\n * Duration\n1 hour\n * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n * Cookie\nwpEmojiSettingsSupports\n * Duration\nsession\n * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n * Cookie\nBCSessionID\n * Duration\n1 year 1 month 4 days\n * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n * Cookie\n_octo\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nlogged_in\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\n__Secure-YEC\n * Duration\npast\n * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n * Cookie\n__eoi\n * Duration\n6 months\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTGCORS\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\nlogin-status-p\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTG\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\ncsrf_token\n * Duration\nsession\n * Description\nNo description available.\n\n\n * Cookie\ntoken_v2\n * Duration\n1 day\n * Description\nDescription is currently not available.\n\n\n * Cookie\nD\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nPHPSESSID\n * Duration\nsession\n * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n * Cookie\nVISITOR_PRIVACY_METADATA\n * Duration\n6 months\n * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n * Cookie\ncookietest\n * Duration\nsession\n * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n * Cookie\n__Host-airtable-session\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\n__Host-airtable-session.sig\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\nm\n * Duration\n1 year 1 month 4 days\n * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n * Cookie\nBIGipServer*\n * Duration\nsession\n * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n * Cookie\n__cfruid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n * Cookie\n_GRECAPTCHA\n * Duration\n6 months\n * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n * Cookie\n__Secure-YNID\n * Duration\n6 months\n * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n * Cookie\ncookieyes-consent\n * Duration\n1 year\n * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n * Cookie\nlidc\n * Duration\n1 day\n * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n * Cookie\nbrw\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nbrwConsent\n * Duration\n5 minutes\n * Description\nDescription is currently not available.\n\n\n * Cookie\nWMF-Uniq\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nloom_anon_comment\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nloom_referral_video\n * Duration\nsession\n * Description\nDescription is currently not available.\n\n\n * Cookie\nVISITOR_INFO1_LIVE\n * Duration\n6 months\n * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n * Cookie\nyt-remote-connected-devices\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n * Duration\nNever Expires\n * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n * Cookie\nyt-remote-device-id\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-name\n * Duration\nsession\n * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\nyt-remote-fast-check-period\n * Duration\nsession\n * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-app\n * Duration\nsession\n * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n * Cookie\nyt-remote-cast-available\n * Duration\nsession\n * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n * Cookie\nyt-remote-cast-installed\n * Duration\nsession\n * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\ncp_session\n * Duration\n3 months\n * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n * Cookie\nloid\n * Duration\n1 year 1 month 4 days\n * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n * Cookie\n__hstc\n * Duration\n6 months\n * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n * Cookie\nhubspotutk\n * Duration\n6 months\n * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n * Cookie\n_ga\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n * Cookie\n_ga_*\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n * Cookie\n__Host-psifi.analyticsTrace\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n__Host-psifi.analyticsTraceV2\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n_gh_sess\n * Duration\nsession\n * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n * Cookie\nYSC\n * Duration\nsession\n * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n * Cookie\najs_anonymous_id\n * Duration\n1 year\n * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n * Cookie\nvuid\n * Duration\n1 year 1 month 4 days\n * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n * Cookie\nAWSALB\n * Duration\n7 days\n * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n * Cookie\nacq\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nacq.sig\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nptc\n * Duration\n2 years\n * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n * Cookie\nmuc_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n * Cookie\nguest_id_marketing\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\nguest_id_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\npersonalization_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n * Cookie\nguest_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n * Cookie\nbcookie\n * Duration\n1 year\n * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n * Cookie\n__Secure-ROLLOUT_TOKEN\n * Duration\n6 months\n * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n * Cookie\nyt.innertube::nextId\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nyt.innertube::requests\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nsession_tracker\n * Duration\nsession\n * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n * Cookie\nedgebucket\n * Duration\nsession\n * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n * Cookie\ndid\n * Duration\n1 year\n * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n# Efficient Image Segmentation Using PyTorch: Part 3\nDepthwise separable convolutions\n[Dhruv Matani](https://towardsdatascience.com/author/dhruvbird/)\nJune 27, 202313 min read\nIn this 4-part series, we'll implement image segmentation step by step from scratch using deep learning techniques in PyTorch. This part will focus on optimizing our CNN baseline model using depthwise separable convolutions to reduce the number of trainable parameters, making the model deployable on mobile and other edge devices.\nCo-authored with [Naresh Singh](https://medium.com/@brocolishbroxoli)\n![Figure 1: Result of running image segmentation using a CNN with depth-wise separable convolutions instead of regular convolutions. From top to bottom, input images, ground truth segmentation masks, and predicted segmentation masks. Source: Author\\(s\\)](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/06/1Q4ZM8UwFIrJKMbPJp6mlJg.png)\nFigure 1: Result of running image segmentation using a CNN with depth-wise separable convolutions instead of regular convolutions. From top to bottom, input images, ground truth segmentation masks, and predicted segmentation masks. Source: Author(s)\n* * *\n## Article outline\nIn this article, we will augment the [Convolutional Neural Network (CNN)](https://en.wikipedia.org/wiki/Convolutional_neural_network) we [built earlier](https://medium.com/p/bed68cadd7c7/) to reduce the number of learnable parameters in our network. The task of identifying pet pixels (pixels belonging to cats, dogs, hamsters, etc...) in an input image remains unchanged. Our network of choice will remain [SegNet](https://arxiv.org/abs/1511.00561), and the only change we'll make is to replace our convolutional layers with depth-wise-separable-convolutions (DSC). Before we do this, we will dive into the theory and practice of depth-wise-separable-convolutions, and appreciate the idea behind the technique.\nThroughout this article, we will reference code and results from this [[notebook](https://github.com/dhruvbird/ml-notebooks/blob/main/pets_segmentation/types%20of%20convolutions.ipynb)](https://github.com/dhruvbird/ml-notebooks/blob/main/pets_segmentation/oxford-iiit-pets-segmentation-using-pytorch-segnet-and-depth-wise-separable-convs.ipynb) for model training, and this notebook for a primer on DSC. If you wish to reproduce the results, you'll need a GPU to ensure that the first notebook completes running in a reasonable amount of time. The second notebook can be run on a regular CPU.\n* * *\n## Articles in this series\nThis series is for readers at all experience levels with deep learning. If you want to learn about the practice of deep learning and vision AI along with some solid theory and hands-on experience, you've come to the right place! This is expected to be a 4-part series with the following articles:\n 1. [Concepts and Ideas](https://medium.com/p/89e8297a0923/)\n 2. [A CNN-based model](https://medium.com/p/bed68cadd7c7/)\n 3. **Depthwise separable convolutions (this article)**\n 4. [A Vision Transformer-based model](https://medium.com/p/6c86da083432/)\n\n\n* * *\n## Introduction\nLet's start this discussion with a closer look at the convolutions from the perspective of model size and computation cost. The number of trainable parameters is a good indication of the size of a model and the number of the tensor operations reflects the model complexity or computation cost. Consider that we have a convolution layer with n filters with size dₖ x dₖ. Further assume that this layer processes input with shape _m x h x w,_ where _m_ is the number of input channels, and _h_ and _w_ are height and width dimensions respectively. In this case, the convolution layer will produce an output with shape _n x h x w_ as shown in Figure 2. We are assuming that the convolution uses _stride=1_. Let's go ahead and evaluate this setup in terms of trainable parameters and computation cost.\n![Figure 2: Regular convolutional filters applied to input to produce output. Assume stride=1 and padding=dₖ-2. Source: Efficient Deep Learning Book](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/06/0hZMdN-RcJVCG_efD.png)\nFigure 2: Regular convolutional filters applied to input to produce output. Assume stride=1 and padding=dₖ-2. Source: [Efficient Deep Learning Book](https://efficientdlbook.com/)\n**Evaluation Of Trainable Parameters:** We have _n_ filters, each of which has _m x dₖ x dₖ_ learnable parameters. This results in a total of _n x m x dₖ x dₖ_ learnable parameters. Bias terms are ignored to simplify this discussion. Let's look at the PyTorch code below to validate our understanding.\npython\n\n```\nimport torch\nfrom torch import nn\ndef num_parameters(m):\nreturn sum([p.numel() for p in m.parameters()])\ndk, m, n = 3, 16, 32\nprint(f\"Expected number of parameters: {m * dk * dk * n}\")\nconv1 = nn.Conv2d(in_channels=m, out_channels=n, kernel_size=dk, bias=False)\nprint(f\"Actual number of parameters: {num_parameters(conv1)}\")\n```\n\nPrints the following.\ntext\n\n```\nExpected number of parameters: 4608\nActual number of parameters: 4608\n```\n\nNow, let's evaluate the computation costs of convolution.\n**Evaluation Of Computational Cost:** A single convolutional filter of shape _m x dₖ x dₖ_ when run with a _stride=1_ and a _padding=dₖ-2_ on an input with size _h x w_ will apply the convolutional filter _h x w_ times, once for each image section with size _dₖ x dₖ_ amounting to a total of _h x w_ sections. It results in a cost of _m x dₖ x dₖ x h x w_ per filter or output channel. Since we wish to compute n output channels, the total cost will be _m x dₖ x dₖ x h x n_. Let's go ahead and validate this using the torchinfo PyTorch package.\npython\n\n```\nfrom torchinfo import summary\nh, w = 128, 128\nprint(f\"Expected total multiplies: {m * dk * dk * h * w * n}\")\nsummary(conv1, input_size=(1, m, h, w))\n```\n\nWill print the following.\ntext\n\n```\nExpected total multiplies: 75497472\n\n==========================================================================================\nLayer (type:depth-idx) Output Shape Param #\n==========================================================================================\nConv2d [1, 32, 128, 128] 4,608\n==========================================================================================\nTotal params: 4,608\nTrainable params: 4,608\nNon-trainable params: 0\nTotal mult-adds (M): 75.50\n==========================================================================================\nInput size (MB): 1.05\nForward/backward pass size (MB): 4.19\nParams size (MB): 0.02\nEstimated Total Size (MB): 5.26\n==========================================================================================\n```\n\nIf we ignore the implementation details of a convolution layer for a moment, we would realize that, on a high level, a convolution layer just transforms a _m x h x w_ input into a _n x h x w_ output. The transformation is achieved through trainable filters which progressively learn features as they _see_ inputs. The question that follows is: Is it possible to achieve this transformation using fewer learnable parameters and simultaneously ensuring minimum compromise in the learning capabilities of the layer? Depthwise Separable Convolutions were proposed to answer this exact question. Let's understand them in detail and learn how they stack up on our evaluation metrics.\n* * *\n## Depthwise Separable Convolution\nThe concept of Depthwise Separable Convolutions (DSC) was first proposed by Laurent Sifre in their PhD thesis titled [Rigid-Motion Scattering For Image Classification](https://www.di.ens.fr/data/publications/papers/phd_sifre.pdf). Since then, they have been used successfully in various popular deep convolutional networks such as [XceptionNet](https://arxiv.org/abs/1610.02357) and [MobileNet](https://towardsdatascience.com/review-mobilenetv1-depthwise-separable-convolution-light-weight-model-a382df364b69).\nThe main difference between a regular convolution, and a DSC is that a DSC is composed of 2 convolutions as described below:\n 1. A **depthwise grouped convolution** , w[here](https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html) the number of input channels m is equal to the number of output channels such that each output channel is affected only by a single input channel. In PyTorch, this is called a \"grouped\" convolution. You can read more about grouped convolutions in PyTorch here.\n 2. A **pointwise convolution** (filter size=1), which operates like a regular convolution such that each of the n filters operates on all m input channels to produce a single output value.\n\n\n![Figure 3: Depthwise Separable Convolution filters applied to input to produce output. Assume stride=1 and padding=dₖ-2. Source: Efficient Deep Learning Book](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/06/0g1cvC1PNtX5TCuwc.png)\nFigure 3: Depthwise Separable Convolution filters applied to input to produce output. Assume _stride=1_ and _padding=dₖ-2_. Source: [Efficient Deep Learning Book](https://efficientdlbook.com/)\nLet's perform the same exercise that we did for regular convolutions for DSCs and compute the number of trainable parameters and computations.\n**Evaluation Of Trainable Parameters:** The \"grouped\" convolutions have m filters, each of which has _dₖ x dₖ_ learnable parameters which produces m output channels. This results in a total of _m x dₖ x dₖ_ learnable parameters. The pointwise convolution has n filters of size _m x 1 x 1_ which adds up to _n x m x 1 x 1_ learnable parameters. Let's look at the PyTorch code below to validate our understanding.\npython\n\n```\nclass DepthwiseSeparableConv(nn.Sequential):\n def __init__(self, chin, chout, dk):\n super().__init__(\n # Depthwise convolution\n nn.Conv2d(chin, chin, kernel_size=dk, stride=1, padding=dk-2, bias=False, groups=chin),\n # Pointwise convolution\n nn.Conv2d(chin, chout, kernel_size=1, bias=False),\n )\n\nconv2 = DepthwiseSeparableConv(chin=m, chout=n, dk=dk)\nprint(f\"Expected number of parameters: {m * dk * dk + m * 1 * 1 * n}\")\nprint(f\"Actual number of parameters: {num_parameters(conv2)}\")\n```\n\nWhich will print.\ntext\n\n```\nExpected number of parameters: 656\nActual number of parameters: 656\n```\n\nWe can see that the DSC version has roughly _7x_ less parameters. Next, let's focus our attention on the computation costs for a DSC layer.\n**Evaluation Of Computational Cost:** Let's assume our input has spatial dimensions _m x h x w_. In the grouped convolution segment of DSC, we have **m** filters, each with size _dₖ x dₖ_. A filter is applied to its corresponding input channel resulting in the segment cost of _m x dₖ x dₖ x h x w_. For the pointwise convolution, we apply **n** filters of size _m x 1 x 1_ **** to produce** n** output channels. This results in the segment cost of _n x m x 1 x 1 x h x_ w. We need to add up the costs of the grouped and pointwise operations to compute the total cost. Let's go ahead and validate this using the [torchinfo](https://pypi.org/project/torchinfo/) PyTorch package.\npython\n\n```\nprint(f\"Expected total multiplies: {m * dk * dk * h * w + m * 1 * 1 * h * w * n}\")\ns2 = summary(conv2, input_size=(1, m, h, w))\nprint(f\"Actual multiplies: {s2.total_mult_adds}\")\nprint(s2)\n```\n\nWhich will print.\ntext\n\n```\nExpected total multiplies: 10747904\nActual multiplies: 10747904\n==========================================================================================\nLayer (type:depth-idx) Output Shape Param #\n==========================================================================================\nDepthwiseSeparableConv [1, 32, 128, 128] --\n├─Conv2d: 1-1 [1, 16, 128, 128] 144\n├─Conv2d: 1-2 [1, 32, 128, 128] 512\n==========================================================================================\nTotal params: 656\nTrainable params: 656\nNon-trainable params: 0\nTotal mult-adds (M): 10.75\n==========================================================================================\nInput size (MB): 1.05\nForward/backward pass size (MB): 6.29\nParams size (MB): 0.00\nEstimated Total Size (MB): 7.34\n==========================================================================================\n```\n\nLet's compare the sizes and costs of both the convolutions for a few examples to gain some intuition.\n### Size and Cost comparison for regular and depthwise separable convolutions\nTo compare the size and cost of regular and depthwise separable convolution, we will assume an input size of _128 x 128_ to the network, a kernel size of _3 x 3_ , and a network that progressively halves the spatial dimensions and doubles the number of channel dimensions. We assume a single 2d-conv layer at every step, but in practice, there could be more.\n![Figure 4: Comparing the number of trainable parameters \\(size\\) and multi-adds \\(cost\\) of regular and depthwise separable convolutions. We also show the ratio of the size and cost for the 2 types of convolutions. Source: Author\\(s\\).](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/06/0QDXpcqsv7euq_t-F.png)\nFigure 4: Comparing the number of trainable parameters (size) and multi-adds (cost) of regular and depthwise separable convolutions. We also show the ratio of the size and cost for the 2 types of convolutions. Source: Author(s).\nYou can see that on average both the size and computational cost of DSC is about 11% to 12% of the cost of regular convolutions for the configuration mentioned above.\n![Figure 5: Relative size and cost of regular v/s DSC. Source: Author\\(s\\).](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/06/094FsN28xUjEf4-Hd.png)\nFigure 5: Relative size and cost of regular v/s DSC. Source: Author(s).\nNow that we have developed a good understanding of the types of convolutions and their relative costs, you must be wondering if there's any downside of using DSCs. Everything we've seen so far seems to suggest that they are better in every way! Well, we haven't yet considered an important aspect which is the impact they have on the accuracy of our model. Let's dive into it via an experiment below.\n* * *\n## SegNet Using Depthwise Separable Convolutions\n[This notebook](https://github.com/dhruvbird/ml-notebooks/blob/main/pets_segmentation/oxford-iiit-pets-segmentation-using-pytorch-segnet-and-depth-wise-separable-convs.ipynb) contains all the code for this section.\nWe will adapt our SegNet model from the [previous post](https://medium.com/p/bed68cadd7c7/) and replace all the regular convolutional layers with a DSC layer. Once we do this, we notice that the number of parameters in our notebook drops from 15.27M to 1.75M parameters, which is a reduction of 88.5%! This is inline with our earlier estimates of an 11% to 12% reduction in the number of trainable parameters of the network.\nA similar configuration as [before](https://medium.com/p/bed68cadd7c7/) was used during model training and validation. The configuration is specified below.\n 1. The _random horizontal flip_ and _colour jitter_ data augmentations are applied to the training set to prevent overfitting\n 2. The images are resized to 128x128 pixels in a non-aspect preserving resize operation\n 3. No input normalization is applied to the images - instead a [batch normalization layer is used as the first layer of the model](https://towardsdatascience.com/replace-manual-normalization-with-batch-normalization-in-vision-ai-models-e7782e82193c)\n 4. The model is trained for 20 epochs using the Adam optimizer with a LR of 0.001 and no LR scheduler\n 5. The cross-entropy loss function is used to classify a pixel as belonging to a pet, the background, or a pet border\n\n\nThe model achieved a validation accuracy of 86.96% after 20 training epochs. This is less than the 88.28% accuracy achieved by the model using regular convolutions for the same number of training epochs. We have determined experimentally that training for more epochs improves the accuracy of both models, so 20 epochs is definitely not the end of the training cycle. We stop at 20 epochs for the purposes of this article for demonstration purposes.\nWe plotted a gif showing how the model is learning to predict the segmentation masks for 21 images in the validation set.\n![Figure 6: A gif showing how the SegNet model with DSC is learning to predict segmentation masks for 21 images in the validation set. Source: Author\\(s\\)](https://assets.insightmediagroup.io/media/0*E7Qphfwo2gIGpWDc)\nFigure 6: A gif showing how the SegNet model with DSC is learning to predict segmentation masks for 21 images in the validation set. Source: Author(s)\nNow that we have seen how the model progresses through the training cycle, let's compare the train cycles of models with regular convolutions and DSC.\n### Accuracy Comparisons\nWe found it useful to look at the training cycles of the models using regular convolutions and DSC. The main difference we noticed is in the early phases (epochs) of training, after which both models settled roughly into the same prediction flow. In fact after training both models for 100 epochs, we noticed that the accuracy of the model with DSC is just about 1% less than the model with regular convolutions. This is inline with our observations from just 20 epochs of training.\n![Figure 7: A gif showing the progression of segmentation masks predicted by the SegNet model using regular convolutions v/s DSC. Source: Author\\(s\\).](https://assets.insightmediagroup.io/media/0*-FP-stW37_Wx4BaE)\nFigure 7: A gif showing the progression of segmentation masks predicted by the SegNet model using regular convolutions v/s DSC. Source: Author(s).\nYou would have noticed that both models get the predictions roughly right after just 6 training epochs - i.e. one can visually see that the models are predicting something useful. Most of the hard work of training the model is then above ensuring that the borders of the predicted masks are as tight as possible and as close to the actual pets in the image as possible. This means that while one can expect a lesser absolute increase in accuracy in the later training epochs, the impact of this on the quality of predictions is much more. We've noticed that a single digit of accuracy improvement at higher absolute accuracy values (going from 89% to 90%) results in significant qualitative improvements to the predictions.\n### Comparison with a UNet model\nWe ran an experiment that changed a lot of hyperparameters with a focus on improving the overall accuracy to get a sense of how far this setting is from close to optimal. Here's the configuration of that experiment.\n 1. Image size: 128 x 128 - same as the experiments so far\n 2. Train epochs: 100 - current experiments trained for 20 epochs\n 3. Augmentations: A lot more augmentations such as image rotation, channel dropping, random block removal. We used Albumentations instead of torchvision transforms. Albumentations automatically transforms segmentation masks for us\n 4. LR Scheduler: A StepLR scheduler was used with a decay of 0.8x every 25 train epochs\n 5. Loss function: We tried 4 different loss functions: Cross Entropy, Focal, Dice, Weighted Cross Entropy. Dice performed worst whereas the rest were pretty much comparable to each other. In fact the difference in best accuracy between the rest after 100 epochs was in the 4th digit after the decimal (assuming the accuracy is a number between 0.0 and 1.0)\n 6. Convolution type: Regular\n 7. Model type: UNet - current experiments used a SegNet model\n\n\nWe achieved a best validation accuracy of 91.3% for the setting above. We noticed that the image size significantly impacts the best validation accuracy. For example, when we changed the image size to 256 x 256, the best validation accuracy went up to 93.0%. However, training took much longer, and used more memory, which meant that we had to reduce the batch size.\n![Figure 8: Result of training a UNet model for 100 train epochs with the hyperparameters mentioned above. Source: Author\\(s\\).](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/06/0V27KB5cxTKQTS9Fv.png)\nFigure 8: Result of training a UNet model for 100 train epochs with the hyperparameters mentioned above. Source: Author(s).\nYou can see that the predictions are much smoother and crisper compared to the ones we have been seeing so far.\n* * *\n## Conclusion\nIn part-3 of this series, we learned about depth wise separable convolutions (DSC) as a technique to reduce model size and training/inference cost without a significant loss in validation accuracy. We learned about the size/cost tradeoff to expect between regular and DSC for a specific setting.\nWe showed how to adapt the SegNet model to use DSC in PyTorch. This technique can be applied to any deep CNN. In fact we can selectively replace some of the convolutional layers with DSC - i.e. we don't need to necessarily replace all of them. Choosing which layers to replace will depend on the balance you wish to strike between model size/runtime-cost and prediction accuracy. This decision will depend on your specific use case and deployment setup.\nWhile this article trained models for 20 epochs, we explained that this is insufficient for production workloads, and provided a glimpse into what one can expect if one trains the model for more epochs. In addition, we provided an introduction to some of the hyperparameters that one can tune during model training. While this list is by no means comprehensive, it should allow you to appreciate the complexity and decision making needed to train an image segmentation model for production workloads.\nIn the [next part of this series](https://medium.com/p/6c86da083432/), we'll take a look at Vision Transformers, and how we can use this model architecture to perform image segmentation for the pets segmentation task.\n* * *\n## References and further reading\n * [Efficient Deep Learning Book Chapter 04 - Efficient Architectures](https://github.com/EfficientDL/book/raw/main/book/%5BEDL%5D%20Chapter%204%20-%20Efficient%20Architectures.pdf)\n * [A Basic Introduction to Separable Convolutions](https://towardsdatascience.com/a-basic-introduction-to-separable-convolutions-b99ec3102728)\n\n\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](https://assets.insightmediagroup.io/media/1787657076238_edr3rv.jpeg)\nWritten By\n## [Dhruv Matani](https://towardsdatascience.com/author/dhruvbird/)\n[See all from Dhruv Matani](https://towardsdatascience.com/author/dhruvbird/)\n[Cnn](https://towardsdatascience.com/tag/cnn/)[Depthwise Convolution](https://towardsdatascience.com/tag/depthwise-convolution/)[Pytorch](https://towardsdatascience.com/tag/pytorch/)[Pytorch Image Segment](https://towardsdatascience.com/tag/pytorch-image-segment/)[Segnet](https://towardsdatascience.com/tag/segnet/)\nShare This Article\n * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fefficient-image-segmentation-using-pytorch-part-3-3534cf04fb89%2F&title=Efficient%20Image%20Segmentation%20Using%20PyTorch%3A%20Part%203)\n * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fefficient-image-segmentation-using-pytorch-part-3-3534cf04fb89%2F&title=Efficient%20Image%20Segmentation%20Using%20PyTorch%3A%20Part%203)\n * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fefficient-image-segmentation-using-pytorch-part-3-3534cf04fb89%2F&text=Efficient%20Image%20Segmentation%20Using%20PyTorch%3A%20Part%203)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/11/1e6Zu11x0cvH_UCeLfzBfog.png)](https://towardsdatascience.com/how-to-understand-the-deep-time-series-classifier-with-integrated-gradients-f4f58af48c05/)\n## [How to Understand the Deep Time Series Classifier with Integrated Gradients](https://towardsdatascience.com/how-to-understand-the-deep-time-series-classifier-with-integrated-gradients-f4f58af48c05/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nTime series classification is a general problem that can be seen in many domains and contexts, e.g. for given time series of a financial...\n[Shuyang Xiang](https://towardsdatascience.com/author/vanillaxiangshuyang/)\nNovember 20, 20214 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/10/1XiQR9iuI1iomPscAwUdOug-scaled.jpeg)](https://towardsdatascience.com/lets-talk-about-graph-neural-network-python-libraries-a0b23ec983b0/)\n## [Let's Talk About Graph Neural Network Python Libraries!](https://towardsdatascience.com/lets-talk-about-graph-neural-network-python-libraries-a0b23ec983b0/)\n[Technology](https://towardsdatascience.com/category/technology/)\nAnd get our hands dirty by formulating a Node Classification task using DGL, PyG and Spektral.\n[Rohith Teja](https://towardsdatascience.com/author/rohithtejam/)\nAugust 21, 20215 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1PjJKvr45SO2Z2ISOTQFeog-scaled.jpeg)](https://towardsdatascience.com/extracting-rich-embedding-features-from-coco-pictures-using-pytorch-and-resnext-wsl-vademecum-of-6fdbdbe876a8/)\n## [Extracting rich embedding features from COCO pictures using PyTorch and ResNeXt-WSL](https://towardsdatascience.com/extracting-rich-embedding-features-from-coco-pictures-using-pytorch-and-resnext-wsl-vademecum-of-6fdbdbe876a8/)\n[](https://towardsdatascience.com/efficient-image-segmentation-using-pytorch-part-3-3534cf04fb89/)\nHow to leverage a powerful pre-trained convolution neural network to extract embedding vectors for pictures.\n[Gianmario Spacagna](https://towardsdatascience.com/author/gm-spacagna/)\nDecember 2, 20207 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/10/1FISI15V0aGUHZ79ezgowig.png)](https://towardsdatascience.com/train-conversational-ai-in-3-lines-of-code-with-nemo-and-lightning-a6088988ae37/)\n## [Train Conversational AI in 3 lines of code with NeMo and Lightning](https://towardsdatascience.com/train-conversational-ai-in-3-lines-of-code-with-nemo-and-lightning-a6088988ae37/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nTrain state-of-the-art speech recognition, NLP and TTS models at scale with NeMo and Lightning\n[Sean Narenthiran](https://towardsdatascience.com/author/seannaren/)\nOctober 28, 20204 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/10/1VMttO-wDWlgsyTVefMPBXg.jpeg)](https://towardsdatascience.com/implementing-tabnet-in-pytorch-fc977c383279/)\n## [Implementing TabNet in PyTorch](https://towardsdatascience.com/implementing-tabnet-in-pytorch-fc977c383279/)\n[](https://towardsdatascience.com/efficient-image-segmentation-using-pytorch-part-3-3534cf04fb89/)\nDeep Learning has taken over vision, natural language processing, speech recognition, and many other fields achieving astonishing results...\n[Samrat Thapa](https://towardsdatascience.com/author/samratthapa120/)\nOctober 23, 20207 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/08/0DIuMzMxMwFlWEq49-scaled.jpg)](https://towardsdatascience.com/solving-bottlenecks-on-the-data-input-pipeline-with-pytorch-profiler-and-tensorboard-5dced134dbe9/)\n## [Solving Bottlenecks on the Data Input Pipeline with PyTorch Profiler and TensorBoard](https://towardsdatascience.com/solving-bottlenecks-on-the-data-input-pipeline-with-pytorch-profiler-and-tensorboard-5dced134dbe9/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nPyTorch Model Performance Analysis and Optimization - Part 4\n[Chaim Rand](https://towardsdatascience.com/author/chaimrand/)\nAugust 26, 202311 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2026/02/1-1-1.jpeg)](https://towardsdatascience.com/optimizing-token-generation-in-pytorch-decoder-models/)\n## [Optimizing Token Generation in PyTorch Decoder Models](https://towardsdatascience.com/optimizing-token-generation-in-pytorch-decoder-models/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nHiding host-device synchronization via CUDA stream interleaving\n[Chaim Rand](https://towardsdatascience.com/author/chaimrand/)\nFebruary 24, 202615 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/08/0863QDrCbsLoVeoim-scaled.jpg)](https://towardsdatascience.com/custom-dataset-in-pytorch-part-1-images-2df3152895/)\n## [Custom dataset in Pytorch -Part 1. Images](https://towardsdatascience.com/custom-dataset-in-pytorch-part-1-images-2df3152895/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nPytorch has a great ecosystem to load custom datasets for training machine learning models. This is the first part of the two-part series...\n[Utkarsh Garg](https://towardsdatascience.com/author/ug2409/)\nAugust 18, 20215 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1wV2saM1BFsrZN97dvHMReg.jpeg)](https://towardsdatascience.com/ml-engineering-101-a-thorough-explanation-of-the-error-dataloader-worker-pid-s-xxx-exited-f3a6a983911e/)\n## [ML Engineering 101: A Thorough Explanation of The Error \"DataLoader worker (pid(s) xxx) exited...](https://towardsdatascience.com/ml-engineering-101-a-thorough-explanation-of-the-error-dataloader-worker-pid-s-xxx-exited-f3a6a983911e/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nA deep dive into PyTorch DataLoader with Multiprocessing\n[Mengliu Zhao](https://towardsdatascience.com/author/mengliuz/)\nJune 2, 20246 min read\n\n\n * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n * [X](https://x.com/TDataScience)\n * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n * [Threads](https://www.threads.net/@towardsdatascience)\n * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n#### Recommended Articles \nClose\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/4970/image?etag=1780565225463&width=500&height=500) Productivity Tips, Data Career Insights, and Other Recent Must-Reads ](https://towardsdatascience.com/productivity-tips-data-career-insights-and-other-recent-must-reads-5c47eb7a2fe0/ \"Productivity Tips, Data Career Insights, and Other Recent Must-Reads\")\n\n\n", "body_base_path": "attachments/42c45c486bcb/pages/", "body_markdown_path": "attachments/42c45c486bcb/pages/archived-body.md", "local_html": "attachments/42c45c486bcb/pages/archived-body.html", "body_source": "prior_capture" }