{
  "source_file": "0930",
  "source_platform": "Medium",
  "author": "Muhammad Ardi Putra",
  "title_listed": "CBAM Paper Walkthrough: The Double-Attention Mechanism",
  "keywords": "",
  "domain": "图像处理",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://towardsdatascience.com/cbam-paper-walkthrough-the-double-attention-mechanism/",
  "publish_date": "2026-09-20",
  "corpus_stage": "",
  "notes": "",
  "id": "ae8ac3f1fc45",
  "collected_at": "2026-10-02T02:09:44",
  "title": "CBAM Paper Walkthrough: The Double-Attention Mechanism | Towards Data Science",
  "body_markdown": "![Revisit consent button](../images/body-df29beacd7496220cc94.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](../images/body-2159bb67f3de8ad870eb.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n  * Cookie\nBCTempID\n  * Duration\n10 minutes\n  * Description\nNo description available.\n\n\n  * Cookie\n__cf_bm\n  * Duration\n1 hour\n  * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n  * Cookie\nAWSALBCORS\n  * Duration\n7 days\n  * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n  * Cookie\n_cfuvid\n  * Duration\nsession\n  * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n  * Cookie\nli_gc\n  * Duration\n6 months\n  * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n  * Cookie\n__hssrc\n  * Duration\nsession\n  * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n  * Cookie\n__hssc\n  * Duration\n1 hour\n  * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n  * Cookie\nwpEmojiSettingsSupports\n  * Duration\nsession\n  * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n  * Cookie\nBCSessionID\n  * Duration\n1 year 1 month 4 days\n  * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n  * Cookie\n_octo\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nlogged_in\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\n__Secure-YEC\n  * Duration\npast\n  * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n  * Cookie\n__eoi\n  * Duration\n6 months\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nAWSALBTGCORS\n  * Duration\n7 days\n  * Description\nNo description available.\n\n\n  * Cookie\nlogin-status-p\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nAWSALBTG\n  * Duration\n7 days\n  * Description\nNo description available.\n\n\n  * Cookie\ncsrf_token\n  * Duration\nsession\n  * Description\nNo description available.\n\n\n  * Cookie\ntoken_v2\n  * Duration\n1 day\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nD\n  * Duration\n1 year\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nPHPSESSID\n  * Duration\nsession\n  * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n  * Cookie\nVISITOR_PRIVACY_METADATA\n  * Duration\n6 months\n  * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n  * Cookie\ncookietest\n  * Duration\nsession\n  * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n  * Cookie\n__Host-airtable-session\n  * Duration\n1 year\n  * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n  * Cookie\n__Host-airtable-session.sig\n  * Duration\n1 year\n  * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n  * Cookie\nm\n  * Duration\n1 year 1 month 4 days\n  * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n  * Cookie\nBIGipServer*\n  * Duration\nsession\n  * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n  * Cookie\n__cfruid\n  * Duration\nsession\n  * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n  * Cookie\n_GRECAPTCHA\n  * Duration\n6 months\n  * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n  * Cookie\n__Secure-YNID\n  * Duration\n6 months\n  * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n  * Cookie\ncookieyes-consent\n  * Duration\n1 year\n  * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n  * Cookie\nlidc\n  * Duration\n1 day\n  * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n  * Cookie\nbrw\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nbrwConsent\n  * Duration\n5 minutes\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nWMF-Uniq\n  * Duration\n1 year\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nloom_anon_comment\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nloom_referral_video\n  * Duration\nsession\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nVISITOR_INFO1_LIVE\n  * Duration\n6 months\n  * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n  * Cookie\nyt-remote-connected-devices\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n  * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n  * Duration\nNever Expires\n  * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n  * Cookie\nyt-remote-device-id\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n  * Cookie\nyt-remote-session-name\n  * Duration\nsession\n  * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n  * Cookie\nyt-remote-fast-check-period\n  * Duration\nsession\n  * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n  * Cookie\nyt-remote-session-app\n  * Duration\nsession\n  * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n  * Cookie\nyt-remote-cast-available\n  * Duration\nsession\n  * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n  * Cookie\nyt-remote-cast-installed\n  * Duration\nsession\n  * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n  * Cookie\ncp_session\n  * Duration\n3 months\n  * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n  * Cookie\nloid\n  * Duration\n1 year 1 month 4 days\n  * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n  * Cookie\n__hstc\n  * Duration\n6 months\n  * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n  * Cookie\nhubspotutk\n  * Duration\n6 months\n  * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n  * Cookie\n_ga\n  * Duration\n1 year 1 month 4 days\n  * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n  * Cookie\n_ga_*\n  * Duration\n1 year 1 month 4 days\n  * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n  * Cookie\n__Host-psifi.analyticsTrace\n  * Duration\n6 hours\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\n__Host-psifi.analyticsTraceV2\n  * Duration\n6 hours\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\n_gh_sess\n  * Duration\nsession\n  * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n  * Cookie\nYSC\n  * Duration\nsession\n  * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n  * Cookie\najs_anonymous_id\n  * Duration\n1 year\n  * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n  * Cookie\nvuid\n  * Duration\n1 year 1 month 4 days\n  * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n  * Cookie\nAWSALB\n  * Duration\n7 days\n  * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n  * Cookie\nacq\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nacq.sig\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nptc\n  * Duration\n2 years\n  * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n  * Cookie\nmuc_ads\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n  * Cookie\nguest_id_marketing\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n  * Cookie\nguest_id_ads\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n  * Cookie\npersonalization_id\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n  * Cookie\nguest_id\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n  * Cookie\nbcookie\n  * Duration\n1 year\n  * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n  * Cookie\n__Secure-ROLLOUT_TOKEN\n  * Duration\n6 months\n  * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n  * Cookie\nyt.innertube::nextId\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n  * Cookie\nyt.innertube::requests\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n  * Cookie\nsession_tracker\n  * Duration\nsession\n  * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n  * Cookie\nedgebucket\n  * Duration\nsession\n  * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n  * Cookie\ndid\n  * Duration\n1 year\n  * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\n# CBAM Paper Walkthrough: The Double-Attention Mechanism\nUnderstanding and implementing CBAM (Convolutional Block Attention Module) from scratch with PyTorch\n[Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\nSeptember 20, 202623 min read\n![](../images/000.jpg)\nPhoto by [Waldemar Brandt](https://unsplash.com/@waldemarbrandt67w?utm_source=medium&utm_medium=referral) on [Unsplash](https://unsplash.com?utm_source=medium&utm_medium=referral)\n## Introduction\nIn this article, I am going to review and implement the deep learning paper titled “ _CBAM: Convolutional Block Attention Module_ ” by Woo _et al._ [1]. As the name suggests, this is essentially a block we can attach to a CNN-based model to enhance feature quality by performing an attention mechanism. Despite the name _attention_ , it is completely different from the one in the ViT (Vision Transformer) architecture. Keep in mind that CBAM was first released in 2018, while ViT was only introduced in 2020. So, we can simply say that CBAM is the older approach to apply an attention mechanism to image data. Despite being older, we should not worry about its relevance since CBAM is a lot more lightweight as compared to ViT, which makes it suitable to be used as a starting point for deployment on low-power devices.\n### Better Than SENet\nIf we were to talk about the history, CBAM was actually proposed as the improvement of SENet (Squeeze-and-Excitation Network), which was introduced a year before CBAM. If you remember the SENet architecture, it essentially works by performing attention across the channel dimension. By doing so, the channels that seem unimportant would be weighted less than that of the more important ones. — I actually got a separate article talking more thoroughly about SENet, which you can access through the link given in reference number [2].\nInstead of just assigning weights to each channel, CBAM also gives weights to every single pixel in the spatial dimension of the image. So, we can say that CBAM has two attention mechanisms, which the authors refer to as the CAM (_Channel Attention Module_) and the SAM (_Spatial Attention Module_). So, based solely on this theory, CBAM should perform better than SENet.\nLearn this step by step with the interactive [Machine Learning](https://roadmap.sh/machine-learning) roadmap.\n···\n### CBAM Architecture\nLet’s talk more specifically about the CBAM architecture which I display in Figure 1 below. As I’ve mentioned earlier, CBAM consists of CAM and SAM. These two sub-blocks are responsible for creating attention weights, which will then be applied to the original tensor by multiplication. The output tensor of this block (the one referred to as _Refined Features_) has the exact same dimension as the input (_Input Feature_), meaning that we can easily plug CBAM to any backbone model without needing to worry about altering the tensor shapes.\n![](../images/001.png)\nFigure 1. The high-level view of the CBAM architecture [1].\n### Channel Attention Module (CAM)\nNow let’s take a closer look at the channel attention module in Figure 2 below. This component is actually very similar to the SENet block, except that CAM also uses global maxpooling layer in addition to the global average-pooling layer. It is explained in the paper that the two operations capture different kind of information, allowing the tensor produced by CAM to be more informative as compared to that of the SENet block.\n![](../images/002.png)\nFigure 2. The structure of the channel attention module [1].\nRemember that the spatial dimension of the tensor collapses to 1×1 when we apply global pooling operation to it. This essentially means that the input tensor, which has the original shape of _C_ × _H_ × _W_ , now becomes _C_ ×1×1, allowing us to process it further easily with the MLP in the subsequent step. There are two linear layers in this MLP, where the first one is used to shrink the number of features according to the _reduction ratio_ parameter, whereas the second one works by expanding the feature vector back to the original length (i.e., the number of channels _C_). These two linear layers in the MLP are together responsible to learn the importance of each channel. Also, keep in mind that this MLP is shared for the tensor produced by the maxpooling and the average-pooling operations, meaning that these two tensors will be processed by the exact same MLP.\nAs these two tensors have been processed, we then combine them by element-wise summation and pass it through a sigmoid function. Since this function shrinks any number to the range of 0 to 1, we can now perceive the resulting tensor as the _channel attention weight_. The elements that correspond to the more important channels will have the value close to 1, indicating that these channels will be weighted more than the others. According to the paper, this kind of mechanism helps the model to understand _what_ kind of features to attend.\nYou can see the formal mathematical expression of the channel attention module in Figure 3 below, where _F_ is an arbitrary intermediate tensor within a network. One thing you need to keep in mind here is that there should be a ReLU activation function placed between the two linear layers (i.e., _W₀_ and _W₁_) yet is somehow not written in this equation.\n![](../images/body-f074bef21da6000ffd46.png)\nFigure 3. The formal mathematical expression of the channel attention module [1].\n### Spatial Attention Module (SAM)\nThe spatial attention module is conceptually similar to the channel attention module. Take a look at the illustration of this sub-block in Figure 4 below.\n![](../images/body-df04cc4b6206226277c3.png)\nFigure 4. The structure of the spatial attention module [1].\nWhat essentially differentiates SAM from CAM is the axis where the pooling operation is performed. Previously in CAM the pooling is done across the spatial dimension, allowing each channel to have a single value representing that channel. Meanwhile, here in SAM the pooling is done across the channel dimension for each spatial pixel location. Thus, every pixel now contains a single value that represents all channels at once. By doing so, the input tensor that initially has the shape of _C_ × _H_ × _W_ will collapse to 1× _H_ × _W_. Remember that since we use maximum and average-pooling operations, we will thus have two tensors of that size. These two tensors are then concatenated, forming a new tensor of shape 2× _H_ × _W_. This tensor is then processed with a 7×7 convolution layer of a single kernel, which effectively combines the information from the two channels into one. So, at this point the tensor becomes 1× _H_ × _W_ again and is then forwarded to the sigmoid function. Similar to CAM, the tensor produced by this sigmoid acts as the _spatial attention weight_. By using this weight tensor, we can essentially let the model know _where_ it should pay more attention to. Below is what the formal mathematical definition of the spatial attention module looks like.\n![](../images/body-a7af7ba2838f06e3bdcd.png)\nFigure 5. The mathematical definition of the spatial attention module [1].\n### Integrating CBAM to Any Backbone Model\nPreviously I mentioned that the output shape of CBAM is exactly the same as the input, allowing it to be integrated to any backbone model easily. In fact, the authors also provide an illustration regarding how we can do that, which I show you in Figure 6 below. In this example, they illustrate how to plug CBAM into a ResNet building block. Once we have successfully integrated them like this, we can just stack these blocks as usual. Later in the coding part I will demonstrate how to implement CBAM from scratch and how to plug it into a ResNeXt model.\n![](../images/body-39276b0ea976be8b4cc1.png)\nFigure 6. How to integrate CBAM into any backbone model [1].\n···\n## Experimental Results\nThe design of the CBAM architecture itself was not chosen arbitrarily. Instead, it was built based on empirical results on their ablation studies, in which they proved that their final model is indeed the most optimal one.\n### Ablation Study on the Channel Attention Module\nThe first ablation study they conducted was related to the pooling layers in the CAM. The results of this experimental set are shown in Figure 7. We can see in this table that the errors when we use either of the two poolings are significantly lower than the plain backbone ResNet50 model. This essentially indicates that the information extracted by the maximum-pooling and the average-pooling are both important. Furthermore, when we utilize both pooling mechanisms simultaneously, the top-1 error goes even lower to 22.80%, which I believe this proves that the two tensors contain information that are not only important but also complementary (i.e., completing each other). Theoretically speaking, the features produced by maxpooling and average-pooling should indeed be complementary since the former captures the most prominent pixel value within a channel while the latter extracts the general information of the channel. So, this is essentially the reason why in Figure 2 the authors ended up using both pooling operations.\n![](../images/body-85103e77927dfa787932.png)\nFigure 7. Ablation study on the use of average and maximum-pooling operations in the CAM [1].\n### Ablation Study on the Spatial Attention Module\nRegarding the spatial attention module, it is explained in the paper that the authors also used varying configurations as displayed in Figure 8 below. You can see here that the configuration in the last row produces the best result, where it utilizes the two pooling operations followed by a convolution layer with 7×7 kernel. In the case of SAM, both the maximum and the average-pooling operations are technically replaceable by a 1×1 convolution (which will combine information across channel dimension with learnable parameters instead of using a “fixed” average and max operations), yet the classification performance appears to be suboptimal.\n![](../images/body-c4c5b7c98a8cc12b3b50.png)\nFigure 8. Ablation study on the configuration of the SAM [1].\n### Ablation Study on the Placement of CAM and SAM\nThe last ablation study the authors conducted was related to how the CAM and SAM are arranged within the CBAM. It is shown in Figure 9 below that using sequential method, especially CAM followed by SAM, allows the model to perform best with the top-1 error of only 22.66%. You can see in the next row that they tried to swap the sequence of the two modules, but they found that the error increases instead. Furthermore, the classification performance was getting even worse when they tried to parallelize CAM and SAM, even though this approach is still better than the ResNet50 with SE module only.\n![](../images/body-d62381c56e29ece20aed.png)\nFigure 9. Ablation study on how the CAM and SAM are arranged [1].\n### Comparison with Other Models\nIn the subsequent experiment the authors compared the performance of a plain model, the model with SE module, and the model with CBAM on different backbones. You can see in Figure 10 below that the model that uses CBAM almost always performs better than the other two as highlighted in green. Moreover, in ResNeXt50, although the model with SE module is better than the same model with CBAM, the error gap is only 0.01%, which I think is negligible.\nI also found in this figure that the error of ResNet50 with CBAM is lower than the plain ResNet101 as highlighted in orange. Interestingly, it is seen here that the number of params and the GFLOPs of ResNet50 with CBAM are much smaller. These facts show that CBAM enables a shallower network to outperform the deeper one while significantly conserving computational resources, which is ideal for deployment on low-end devices.\n![](../images/body-8620a36027142879d2fd.png)\nFigure 10. How CBAM performs on different backbone CNN models [1][3].\n### Attention Heatmap\nIn addition to the quantitative results explained above, the authors also used Grad-CAM to perform qualitative evaluation. If you’re not yet familiar with Grad-CAM, it is essentially a method we can use to find out the specific area that contributes more to the predicted class. Figure 11 below displays several examples of the attention heatmap produced using Grad-CAM, where the area highlighted in red indicates that it gives more contribution to the predicted class.\n![](../images/body-1207a0fcec4f7c523db8.png)\nFigure 11. Attention heatmap obtained by Grad-CAM [1].\nThe results are pretty interesting. Let’s now take a look at the _Croquet ball_ class. With the plain ResNet50, it looks like the model focuses on both the ball and the person. As the SE module is applied (i.e., channel-wise attention only), the attention map becomes more refined toward the ball. And then, after we replace the SE module with CBAM, the model achieves an even sharper focus on the target object.\nA similar thing can also be observed in the other classes. In _Eskimo dog_ and _Snow leopard_ , for example, we can see that the model only pays attention to the eyes. If I were to say, this is basically not wrong as long as the predicted class is correct. However, if we were to predict something (as a human), it would make more sense to see the entire object whenever possible, right? And so, this is exactly what the attention modules do. You can see that when CBAM is used, the red area in the attention heatmap covers the entire face, indicating the model now take that facial region into consideration to make predictions.\nAdditionally, it is also seen in the figure that by using CBAM we can make the model more confident when making predictions. Take a look at the _School bus_ image in the above figure. You can see here that the bus is not centered at the middle of the image. This basically causes the plain ResNet50 to have a confidence score of only 0.07 in predicting the bus (which I believe this should have been misclassified). Meanwhile, SE module allows the model to correctly classify it with the confidence of 0.92, and then CBAM improves it even further to 0.98.\n···\n## CBAM Implementation\nAs we have understood all the theories behind CBAM, let’s now roll our sleeves and get our hands dirty with some code! As I’ve mentioned earlier, here I am going to implement CBAM and try to integrate it into the ResNeXt backbone. Although I am implementing both from scratch, I’ll focus the discussion primarily on the CBAM module. So if you’re not yet familiar with ResNeXt, I do encourage you read my previous article about that model beforehand, which you can access through the link at reference [4].\nAs usual, the very first thing we need to do is to import the required modules, i.e., the base `torch` module and its `nn` submodule.\npython\n\n```\n# Codeblock 1\nimport torch\nimport torch.nn as nn\n```\n\nNext, in Codeblock 2 below I initialize the configurable variables. The reduction ratio `R` is used to adjust the width of the MLP layer inside the CAM. I set the value for this to 16 as suggested in the paper. Meanwhile, `CARDINALITY`, `NUM_CHANNELS`, and `NUM_BLOCKS` are the ones belong to ResNeXt.\npython\n\n```\n# Codeblock 2\nR            = 16\nCARDINALITY  = 32\nNUM_CHANNELS = [3, 64, 256, 512, 1024, 2048]\nNUM_BLOCKS   = [3, 4, 6, 3]\nNUM_CLASSES  = 1000\n```\n\n···\n### CAM Implementation\nLet’s start with the CAM first. If you go back to Figure 2, you can see that we have two pooling operations. In Codeblock 3 below, the two layers that correspond to them are initialized at lines `#(1)` and `#(2)`. Don’t forget to set the `output_size` parameter to `(1,1)` since we want each channel to be represented as a single number.\nWhat we do next inside the `__init__()` method is initializing the MLP that consists of two linear layers. The first linear layer is responsible to reduce the number of features according to the `R` parameter (`#(3)`), whereas the second one is used to expand it back to the original number of features (`#(5)`). Also, don’t forget to place the ReLU activation function in between (`#(4)`). In fact, the structure of this MLP layer is exactly the same as the one used in SENet. — You can read more about the underlying idea behind this structure in my previous article about that module at reference [2]. — The last thing we do inside the `__init__()` method is to initialize the sigmoid activation function (`#(6)`), which is responsible to rescale the tensor such that the values will always be between 0 and 1, suitable to be used as an attention weight.\npython\n\n```\n# Codeblock 3\nclass CAM(nn.Module):\n    def __init__(self, num_channels, r=16):\n        super().__init__()\n        \n        self.maxpool = nn.AdaptiveMaxPool2d(output_size=(1,1))  #(1)\n        self.avgpool = nn.AdaptiveAvgPool2d(output_size=(1,1))  #(2)\n        \n        self.mlp = nn.Sequential(\n            nn.Linear(in_features=num_channels,\n                      out_features=num_channels//r,   #(3)\n                      bias=False),\n            \n            nn.ReLU(inplace=True),                    #(4)\n            \n            nn.Linear(in_features=num_channels//r,    #(5)\n                      out_features=num_channels, \n                      bias=False)\n        )\n        \n        self.sigmoid = nn.Sigmoid()                   #(6)\n        \n    def forward(self, x):         #(7)\n        original = x\n        print(f'original\\t\\t: {x.size()}\\n')\n        \n        \n        x_max = self.maxpool(x)   #(8)\n        print(f'x after maxpool (x_max)\\t: {x_max.size()}')\n        \n        x_avg = self.avgpool(x)   #(9)\n        print(f'x after avgpool (x_avg)\\t: {x_avg.size()}\\n')\n        \n        \n        x_max = torch.flatten(x_max, start_dim=1)    #(10)\n        print(f'x_max after flatten\\t: {x_max.size()}')\n        \n        x_avg = torch.flatten(x_avg, start_dim=1)    #(11)\n        print(f'x_avg after flatten\\t: {x_avg.size()}\\n')\n        \n        \n        x_max = self.mlp(x_max)   #(12)\n        print(f'x_max after mlp\\t\\t: {x_max.size()}')\n        \n        x_avg = self.mlp(x_avg)   #(13)\n        print(f'x_avg after mlp\\t\\t: {x_avg.size()}\\n')\n        \n        \n        x = x_max + x_avg         #(14)\n        print(f'after sum\\t\\t: {x.size()}')\n        \n        x = self.sigmoid(x)       #(15)\n        print(f'after sigmoid\\t\\t: {x.size()}')\n        \n        x = x[:, :, None, None]   #(16)\n        print(f'after reshape\\t\\t: {x.size()}')\n        \n        x = x * original          #(17)           \n        print(f'after multiply\\t\\t: {x.size()}')\n        \n        return x\n```\n\nNow let’s move on to the `forward()` method where the execution would happen. You can see at line `#(7)` in the above codeblock that we take a single tensor `x` as the input. This input tensor will then be saved in the `original` variable, which is done because we will later multiply it with the resulting channel attention weight tensor (`#(17)`). The `x` tensor itself will be processed by maxpooling and average-pooling operations in parallel (`#(8–9)`). Both `x_max` and `x_avg` are forwarded to the same MLP, which is the reason why this MLP is called “shared” (`#(12–13)`). Then at line `#(14)`, we combine `x_max` and `x_avg` through element-wise summation before forwarding the resulting tensor to the sigmoid function (`#(15)`). There is a little technical thing we do at line `#(16)`, which is used to reintroduce the spatial dimension we previously dropped at lines `#(10)` and `#(11)`. Finally, as the weight tensor is ready, we can then actually weight the original tensor by multiplying them (`#(17)`).\nAt this point we already got our `CAM` class completed. What we are going to do next is to test it with the following code. Here I initialize a `CAM` instance that accepts a 512-channel image and pass a dummy tensor of size 512×28×28 through it, simulating an arbitrary intermediate tensor within a network.\npython\n\n```\n# Codeblock 4\ncam = CAM(num_channels=512, r=16)\nx = torch.randn(1, 512, 28, 28)\n\nout = cam(x)\n```\n\nIf you run the above code, you should get the following output. Notice that previously in Codeblock 3 I wrote lots of print functions, which is the reason why here you can clearly see the detailed flow of the network.\nhttp\n\n```\n# Codeblock 4 Output\noriginal                : torch.Size([1, 512, 28, 28])\n\nx after maxpool (x_max) : torch.Size([1, 512, 1, 1])\nx after avgpool (x_avg) : torch.Size([1, 512, 1, 1])\n\nx_max after flatten     : torch.Size([1, 512])\nx_avg after flatten     : torch.Size([1, 512])\n\nx_max after mlp         : torch.Size([1, 512])    #(1)\nx_avg after mlp         : torch.Size([1, 512])    #(2)\n\nafter sum               : torch.Size([1, 512])\nafter sigmoid           : torch.Size([1, 512])\nafter reshape           : torch.Size([1, 512, 1, 1])    #(3)\nafter multiply          : torch.Size([1, 512, 28, 28])  #(4)\n```\n\nIt is necessary to understand that although the MLP looks like it doesn’t change the tensor dimension at all (`#(1–2)`), you need to know that the feature vector length is internally reduced to 32 by the first linear layer before eventually expanded back to 512 by the second one. Next, it might also be worth noting that the channel attention weight tensor originally has the shape of 512×1×1 (`#(3)`), indicating that every single channel in the original tensor has a single weighting number associated with it. This attention weight is then applied to the original tensor by using a simple multiplication, which technically speaking, this weight tensor is broadcasted along the spatial dimension of the original tensor (`#(4)`). At this point our tensor is now ready to be forwarded to the SAM, which we are going to build very soon.\n···\n### SAM Implementation\nThe implementation of the spatial attention module is displayed in Codeblock 5 below. What we need to initialize inside the `__init__()` method is only a single 7×7 convolution layer (`#(1)`) and a sigmoid activation function (`#(2)`)\npython\n\n```\n# Codeblock 5\nclass SAM(nn.Module):\n    def __init__(self):\n        super().__init__()\n        \n        self.conv = nn.Conv2d(in_channels=2,     #(1)\n                              out_channels=1, \n                              kernel_size=7, \n                              padding=3, \n                              bias=False)\n        self.sigmoid = nn.Sigmoid()              #(2)\n    \n    def forward(self, x):\n        original = x      #(3)\n        print(f'original\\t\\t: {x.size()}\\n')\n        \n        \n        x_max, _ = torch.max(x,  dim=1, keepdim=True)    #(4)\n        print(f'x after maxpool (x_max)\\t: {x_max.size()}')\n        \n        x_avg    = torch.mean(x, dim=1, keepdim=True)    #(5)\n        print(f'x after avgpool (x_avg)\\t: {x_avg.size()}\\n')\n        \n        \n        x = torch.cat([x_max, x_avg], dim=1)             #(6)\n        print(f'after concatenate\\t: {x.size()}')\n        \n        x = self.conv(x)                                 #(7)\n        print(f'after conv\\t\\t: {x.size()}')\n        \n        x = self.sigmoid(x)                              #(8)\n        print(f'after sigmoid\\t\\t: {x.size()}')\n        \n        x = x * original                                 #(9)\n        print(f'after multiply\\t\\t: {x.size()}')\n        \n        return x\n```\n\nIn the `forward()` method, the first thing we do is to store the original input into a separate variable (`#(3)`), which is exactly the same as what we did in the CAM. The pooling mechanism in the SAM is a bit unique since here we want to do that across the channel dimension. This is essentially the reason that I didn’t initialize any pooling layers in the `__init__()` method since `nn.AdaptiveMaxPool2d()` and `nn.AdaptiveAvgPool2d()` operate on spatial dimension, which is irrelevant for this case. Instead, here we use a simple `torch.max()` and `torch.mean()` functions to do the maxpooling (`#(4)`) and average-pooling (`#(5)`) operations, respectively. Just don’t forget to set the `dim` parameter to 1 so that they really do the operations across the channel dimension.\nDespite taking different values, the tensor shape produced by the two poolings are exactly the same, which is the reason that we can just concatenate them as shown at line `#(6)`. Keep in mind that tensor concatenation does not really combine information as it only stacks the two without blending the numbers. Thus, in the subsequent step we apply the convolution layer we initialized earlier to actually do that (`#(7)`). This convolution only consists of a single kernel, which implies that the resulting tensor will have a single channel as well. This idea is conceptually different from the one in the CAM, where in that module we combine the information by element-wise summation. Technically speaking, we can essentially use summation for the SAM too as it will produce the exact same tensor dimension. However, I do believe that the authors might also intended to capture the correlation between neighboring pixels instead of independently giving weight to each pixel, which is the reason why they decided to use convolution over summation.\nAs the two tensors have been combined, the next thing we do is to pass the resulting tensor through the sigmoid activation function to actually obtain the spatial attention weight (`#(8)`). And finally, we will multiply this weight tensor with the original SAM input as shown at line `#(9)`.\nNow let’s run the Codeblock 6 below to test if our spatial attention module works properly.\npython\n\n```\n# Codeblock 6\nsam = SAM()\nx = torch.randn(1, 512, 28, 28)\n\nout = sam(x)\n```\n\nAnd below is what the flow of the SAM looks like. We can see here that as the pooling operations are applied to the input tensor, the channel dimension collapses to 1 (`#(1–2)`). This essentially means that every pixel is now represented as a single number aggregated from all channels in that spatial location. Then at line `#(3)`, the tensor becomes 2×28×28 as we concatenate the two before eventually reducing it again to 1×28×28 using the convolution layer (`#(4)`). After being processed by the sigmoid function, the resulting spatial attention weight is then multiplied with the original tensor, in which the former is broadcasted along the channel dimension of the latter, allowing the final output tensor to have the exact same shape as the input (`#(5)`).\nhttp\n\n```\n# Codeblock 6 Output\noriginal                : torch.Size([1, 512, 28, 28])\n\nx after maxpool (x_max) : torch.Size([1, 1, 28, 28])    #(1)\nx after avgpool (x_avg) : torch.Size([1, 1, 28, 28])    #(2)\n\nafter concatenate       : torch.Size([1, 2, 28, 28])    #(3)\nafter conv              : torch.Size([1, 1, 28, 28])    #(4)\nafter sigmoid           : torch.Size([1, 1, 28, 28])\nafter multiply          : torch.Size([1, 512, 28, 28])  #(5)\n```\n\n···\n#### The Complete CBAM Block\nNow as the CAM and SAM are done, we will now put them together in the `CBAM` class. See the details in Codeblock 7 below. You can see here that this class is very simple as what we need to do is just to initialize the two attention modules and place them sequentially. Keep in mind that we need to pass the `num_channels` parameter every time we want to initialize a `CBAM` instance (`#(1)`) since we will later integrate this module into ResNeXt, in which every single one of its building blocks accepts different number of channels, and so we need to make this CBAM block flexible as well.\npython\n\n```\n# Codeblock 7\nclass CBAM(nn.Module):\n    def __init__(self, num_channels):    #(1)\n        super().__init__()\n        \n        self.cam = CAM(num_channels=num_channels)\n        self.sam = SAM()\n        \n    def forward(self, x):\n        print(f'original\\t\\t: {x.size()}')\n        \n        x = self.cam(x)\n        print(f'after cam\\t\\t: {x.size()}')\n        \n        x = self.sam(x)\n        print(f'after sam\\t\\t: {x.size()}')\n        \n        return x\n```\n\nAgain, just to ensure that this class works properly, let’s pass a dummy tensor through it using the Codeblock 8 below.\npython\n\n```\n# Codeblock 8\ncbam = CBAM(num_channels=512)\nx = torch.randn(1, 512, 28, 28)\n\nout = cbam(x)\n```\n\nhttp\n\n```\n# Codeblock 8 Output\noriginal   : torch.Size([1, 512, 28, 28])\nafter cam  : torch.Size([1, 512, 28, 28])\nafter sam  : torch.Size([1, 512, 28, 28])\n```\n\n···\n### Implementing CBAM on ResNeXt Building Block\nAlright, so at this point our CBAM is ready and in this section I am going to actually demonstrate how we can attach this module to a ResNeXt building block. The code I write in Codeblock 9 onwards are basically the same as the one used when I demonstrated how to integrate SENet on ResNeXt. I do encourage you to read that article [2] and my explanation on the pure ResNeXt backbone [4] because it would be too long if I explain everything here.\nThe only thing I want to emphasize in Codeblock 9 is that the CBAM module itself is initialized at line `#(1)` which is then attached to the flow at line `#(2)`.\npython\n\n```\n# Codeblock 9\nclass Block(nn.Module):\n    def __init__(self, \n                 in_channels,\n                 add_channel=False,\n                 channel_multiplier=2,\n                 downsample=False):\n        super().__init__()\n\n        self.add_channel = add_channel\n        self.channel_multiplier = channel_multiplier\n        self.downsample = downsample\n        \n        \n        if self.add_channel:\n            out_channels = in_channels*self.channel_multiplier\n        else:\n            out_channels = in_channels\n        \n        mid_channels = out_channels//2\n        \n        \n        if self.downsample:\n            stride = 2\n        else:\n            stride = 1\n\n\n        if self.add_channel or self.downsample:\n            self.projection = nn.Conv2d(in_channels=in_channels,\n                                        out_channels=out_channels, \n                                        kernel_size=1, \n                                        stride=stride, \n                                        padding=0, \n                                        bias=False)\n            nn.init.kaiming_normal_(self.projection.weight, nonlinearity='relu')\n            self.bn_proj = nn.BatchNorm2d(num_features=out_channels)\n\n        self.conv0 = nn.Conv2d(in_channels=in_channels,\n                               out_channels=mid_channels,\n                               kernel_size=1, \n                               stride=1, \n                               padding=0, \n                               bias=False)\n        nn.init.kaiming_normal_(self.conv0.weight, nonlinearity='relu')\n        self.bn0 = nn.BatchNorm2d(num_features=mid_channels)\n\n        self.conv1 = nn.Conv2d(in_channels=mid_channels,\n                               out_channels=mid_channels, \n                               kernel_size=3, \n                               stride=stride,\n                               padding=1, \n                               bias=False, \n                               groups=CARDINALITY)\n        nn.init.kaiming_normal_(self.conv1.weight, nonlinearity='relu')\n        self.bn1 = nn.BatchNorm2d(num_features=mid_channels)\n\n        self.conv2 = nn.Conv2d(in_channels=mid_channels,\n                               out_channels=out_channels,\n                               kernel_size=1, \n                               stride=1, \n                               padding=0, \n                               bias=False)\n        nn.init.kaiming_normal_(self.conv2.weight, nonlinearity='relu')\n        self.bn2 = nn.BatchNorm2d(num_features=out_channels)\n        \n        self.relu = nn.ReLU()\n        \n        self.cbam = CBAM(num_channels=out_channels)               #(1)\n        \n    def forward(self, x):\n        print(f'original\\t\\t: {x.size()}')\n        \n        if self.add_channel or self.downsample:\n            residual = self.bn_proj(self.projection(x))\n            print(f'after projection\\t: {residual.size()}')\n        else:\n            residual = x\n            print(f'no projection\\t\\t: {residual.size()}')\n        \n        x = self.conv0(x)\n        x = self.bn0(x)\n        x = self.relu(x)\n        print(f'after conv0-bn0-relu\\t: {x.size()}')\n\n        x = self.conv1(x)\n        x = self.bn1(x)\n        x = self.relu(x)\n        print(f'after conv1-bn1-relu\\t: {x.size()}')\n        \n        x = self.conv2(x)\n        x = self.bn2(x)\n        print(f'after conv2-bn2\\t\\t: {x.size()}')\n        \n        x = self.cbam(x)                                          #(2)\n        print(f'after cbam\\t\\t: {x.size()}')\n        \n        x = x + residual\n        x = self.relu(x)\n        print(f'after summation\\t\\t: {x.size()}')\n        \n        return x\n```\n\nAnd now we can test the `Block` class above by running the Codeblock 10 below. You can see in the following output that the tensor successfully passes through the entire network, including the CBAM block we attached at the end of the main flow (`#(1)`).\npython\n\n```\n# Codeblock 10\nblock = Block(in_channels=512, add_channel=False, downsample=False)\nx = torch.randn(1, 512, 28, 28)\n\nout = block(x)\n```\n\nhttp\n\n```\n# Codeblock 10 Output\noriginal             : torch.Size([1, 512, 28, 28])\nno projection        : torch.Size([1, 512, 28, 28])\nafter conv0-bn0-relu : torch.Size([1, 256, 28, 28])\nafter conv1-bn1-relu : torch.Size([1, 256, 28, 28])\nafter conv2-bn2      : torch.Size([1, 512, 28, 28])\nafter cbam           : torch.Size([1, 512, 28, 28])    #(1)\nafter summation      : torch.Size([1, 512, 28, 28])\n```\n\n···\n### The Final CBAM-ized ResNeXt\nAs the CBAM module has been attached to the main ResNeXt building block, we can just stack these blocks according to the structure given in the ResNeXt paper. The `CBAMResNeXt` class in Codeblock 11 is literally copy-pasted from my SENet article [2] since the only thing we need to do to attach CBAM module to ResNeXt is modifying the `Block` class back in Codeblock 9.\npython\n\n```\n# Codeblock 11\nclass CBAMResNeXt(nn.Module):\n    def __init__(self):\n        super().__init__()\n\n        # conv1 stage\n        self.resnext_conv1 = nn.Conv2d(in_channels=NUM_CHANNELS[0],\n                                       out_channels=NUM_CHANNELS[1],\n                                       kernel_size=7,\n                                       stride=2,\n                                       padding=3, \n                                       bias=False)\n        nn.init.kaiming_normal_(self.resnext_conv1.weight, \n                                nonlinearity='relu')\n        self.resnext_bn1 = nn.BatchNorm2d(num_features=NUM_CHANNELS[1])\n        self.relu = nn.ReLU()\n        self.resnext_maxpool1 = nn.MaxPool2d(kernel_size=3,\n                                             stride=2, \n                                             padding=1)\n\n        # conv2 stage\n        self.resnext_conv2 = nn.ModuleList([\n            Block(in_channels=NUM_CHANNELS[1],\n                  add_channel=True,\n                  channel_multiplier=4,\n                  downsample=False)\n        ])\n        for _ in range(NUM_BLOCKS[0]-1):\n            self.resnext_conv2.append(Block(in_channels=NUM_CHANNELS[2]))\n\n        # conv3 stage\n        self.resnext_conv3 = nn.ModuleList([Block(in_channels=NUM_CHANNELS[2],\n                                                  add_channel=True, \n                                                  downsample=True)])\n        for _ in range(NUM_BLOCKS[1]-1):\n            self.resnext_conv3.append(Block(in_channels=NUM_CHANNELS[3]))\n            \n            \n        # conv4 stage\n        self.resnext_conv4 = nn.ModuleList([Block(in_channels=NUM_CHANNELS[3],\n                                                  add_channel=True, \n                                                  downsample=True)])\n        \n        for _ in range(NUM_BLOCKS[2]-1):\n            self.resnext_conv4.append(Block(in_channels=NUM_CHANNELS[4]))\n            \n            \n        # conv5 stage\n        self.resnext_conv5 = nn.ModuleList([Block(in_channels=NUM_CHANNELS[4],\n                                                  add_channel=True, \n                                                  downsample=True)])\n        \n        for _ in range(NUM_BLOCKS[3]-1):\n            self.resnext_conv5.append(Block(in_channels=NUM_CHANNELS[5]))\n \n       \n        self.avgpool = nn.AdaptiveAvgPool2d(output_size=(1,1))\n\n        self.fc = nn.Linear(in_features=NUM_CHANNELS[5],\n                            out_features=NUM_CLASSES)\n\n    def forward(self, x):\n        print(f'original\\t\\t: {x.size()}')\n        \n        x = self.relu(self.resnext_bn1(self.resnext_conv1(x)))\n        print(f'after resnext_conv1\\t: {x.size()}')\n        \n        x = self.resnext_maxpool1(x)\n        print(f'after resnext_maxpool1\\t: {x.size()}')\n        \n        for i, block in enumerate(self.resnext_conv2):\n            x = block(x)\n            print(f'after resnext_conv2 #{i}\\t: {x.size()}')\n            \n        for i, block in enumerate(self.resnext_conv3):\n            x = block(x)\n            print(f'after resnext_conv3 #{i}\\t: {x.size()}')\n            \n        for i, block in enumerate(self.resnext_conv4):\n            x = block(x)\n            print(f'after resnext_conv4 #{i}\\t: {x.size()}')\n            \n        for i, block in enumerate(self.resnext_conv5):\n            x = block(x)\n            print(f'after resnext_conv5 #{i}\\t: {x.size()}')\n        \n        x = self.avgpool(x)\n        print(f'after avgpool\\t\\t: {x.size()}')\n        \n        x = torch.flatten(x, start_dim=1)\n        print(f'after flatten\\t\\t: {x.size()}')\n        \n        x = self.fc(x)\n        print(f'after fc\\t\\t: {x.size()}')\n        \n        return x\n```\n\nAnd now we can check if the entire network works properly by running the following testing code. Here I initialize a `CBAMResNeXt` instance and pass a dummy RGB image of size 224×224 through it.\npython\n\n```\n# Codeblock 12\ncbamresnext = CBAMResNeXt()\nx = torch.randn(1, 3, 224, 224)\n\nout = cbamresnext(x)\n```\n\nBelow is what the resulting output looks like. Here we can see that the model successfully passes the original image through the entire network up until the classification head. This essentially indicates that our CBAM is properly attached, and thus this model is ready to be trained for image classification. \nhttp\n\n```\n# Codeblock 12 Output\noriginal               : torch.Size([1, 3, 224, 224])\nafter resnext_conv1    : torch.Size([1, 64, 112, 112])\nafter resnext_maxpool1 : torch.Size([1, 64, 56, 56])\nafter resnext_conv2 #0 : torch.Size([1, 256, 56, 56])\nafter resnext_conv2 #1 : torch.Size([1, 256, 56, 56])\nafter resnext_conv2 #2 : torch.Size([1, 256, 56, 56])\nafter resnext_conv3 #0 : torch.Size([1, 512, 28, 28])\nafter resnext_conv3 #1 : torch.Size([1, 512, 28, 28])\nafter resnext_conv3 #2 : torch.Size([1, 512, 28, 28])\nafter resnext_conv3 #3 : torch.Size([1, 512, 28, 28])\nafter resnext_conv4 #0 : torch.Size([1, 1024, 14, 14])\nafter resnext_conv4 #1 : torch.Size([1, 1024, 14, 14])\nafter resnext_conv4 #2 : torch.Size([1, 1024, 14, 14])\nafter resnext_conv4 #3 : torch.Size([1, 1024, 14, 14])\nafter resnext_conv4 #4 : torch.Size([1, 1024, 14, 14])\nafter resnext_conv4 #5 : torch.Size([1, 1024, 14, 14])\nafter resnext_conv5 #0 : torch.Size([1, 2048, 7, 7])\nafter resnext_conv5 #1 : torch.Size([1, 2048, 7, 7])\nafter resnext_conv5 #2 : torch.Size([1, 2048, 7, 7])\nafter avgpool          : torch.Size([1, 2048, 1, 1])\nafter flatten          : torch.Size([1, 2048])\nafter fc               : torch.Size([1, 1000])\n```\n\n···\n## Ending\nAnd well I think that’s pretty much everything about CBAM and how to implement it from scratch. You can also find the code used in this article in my GitHub repository [6]. Please let me know if you find any mistakes in the discussion or in the code. Thanks for reading, I hope you learn something new today. See ya in my next article!\n···\n## References\n[1] Sanghyun Woo _et al._ CBAM: Convolutional Block Attention Module. Arxiv. <https://arxiv.org/abs/1807.06521> [Accessed November 12, 2025].\n[2] Muhammad Ardi Putra. SENet Paper Walkthrough: The Channel-Wise Attention. Towards Data Science. <https://towardsdatascience.com/the-channel-wise-attention/> [Accessed November 12, 2025]. Also available at <https://medium.com/ai-advances/senet-paper-walkthrough-the-channel-wise-attention-8ac72b9cc252>.\n[3] Image originally created by author.\n[4] Muhammad Ardi Putra. ResNeXt Paper Walkthrough: Taking ResNet to the Next Level. Towards Data Science. <https://towardsdatascience.com/taking-resnet-to-the-next-level/> [Accessed November 12, 2025]. Also available at <https://medium.com/ai-advances/taking-resnet-to-the-next-level-resnext-77088c245698>.\n[5] Saining Xie _et al_. Aggregated Residual Transformations for Deep Neural Networks. Arxiv. <https://arxiv.org/abs/1611.05431> [Accessed November 12, 2025].\n[6] MuhammadArdiPutra. CBAM. GitHub. <https://github.com/MuhammadArdiPutra/medium_articles/blob/main/Deep%20Learning%20From%20Scratch/CBAM.ipynb> [Accessed November 12, 2025].\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](../images/body-59e1c17daab1fcb582c5.jpeg)\nWritten By\n## [Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\nDeep learning storyteller\n[See all from Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\n[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Computer Vision](https://towardsdatascience.com/tag/computer-vision/)[Deep Learning](https://towardsdatascience.com/tag/deep-learning/)[Neural Network](https://towardsdatascience.com/tag/neural-network/)[Pytorch](https://towardsdatascience.com/tag/pytorch/)\nShare This Article\n  * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fcbam-paper-walkthrough-the-double-attention-mechanism%2F&title=CBAM%20Paper%20Walkthrough%3A%20The%20Double-Attention%20Mechanism)\n  * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fcbam-paper-walkthrough-the-double-attention-mechanism%2F&title=CBAM%20Paper%20Walkthrough%3A%20The%20Double-Attention%20Mechanism)\n  * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fcbam-paper-walkthrough-the-double-attention-mechanism%2F&text=CBAM%20Paper%20Walkthrough%3A%20The%20Double-Attention%20Mechanism)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n  * [![](../images/003.png)](https://towardsdatascience.com/visualizing-what-batch-normalization-is-and-its-advantages-a49bbcd2fd86/)\n## [Visualizing What Batch Normalization Is and Its Advantages](https://towardsdatascience.com/visualizing-what-batch-normalization-is-and-its-advantages-a49bbcd2fd86/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nOptimizing your neural network training with batch normalization\n[Peng Qian](https://towardsdatascience.com/author/qtalen/)\nFebruary 3, 20244 min read\n  * [![](../images/004.jpg)](https://towardsdatascience.com/what-i-am-doing-to-stay-relevant-as-a-senior-analytics-consultant-in-2026/)\n## [What I Am Doing to Stay Relevant as a Senior Analytics Consultant in 2026](https://towardsdatascience.com/what-i-am-doing-to-stay-relevant-as-a-senior-analytics-consultant-in-2026/)\n[Data analysis](https://towardsdatascience.com/category/data-science/data-analysis/)\nLearn how to work with AI, while strengthening your unique human skills that technology cannot replace\n[Rashi Desai](https://towardsdatascience.com/author/rashidesai/)\nFebruary 7, 20266 min read\n  * [![](../images/005.jpg)](https://towardsdatascience.com/deepmind-alphafold-the-protein-folding-problem-a331d9ced746/)\n## [DeepMind, AlphaFold & the Protein Folding Problem](https://towardsdatascience.com/deepmind-alphafold-the-protein-folding-problem-a331d9ced746/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\n- - - - - - - - - - - - - - - - - - - - - - - - - - - -\n[Prem Oommen](https://towardsdatascience.com/author/thedataoracle/)\nJanuary 25, 20215 min read\n  * [![](../images/006.gif)](https://towardsdatascience.com/ultimate-volleyball-a-3d-volleyball-environment-built-using-unity-ml-agents-c9d3213f3064/)\n## [🏐 Ultimate Volleyball: A multi-agent reinforcement learning environment built using Unity...](https://towardsdatascience.com/ultimate-volleyball-a-3d-volleyball-environment-built-using-unity-ml-agents-c9d3213f3064/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nTrain reinforcement learning agents to play Volleyball\n[Joy Zhang](https://towardsdatascience.com/author/joooyz/)\nAugust 11, 20215 min read\n  * [![](../images/007.jpg)](https://towardsdatascience.com/why-a-data-science-mentor-can-help-you-in-2022-a7b10152a069/)\n## [Why a Data Science Mentor Can Help You in 2022](https://towardsdatascience.com/why-a-data-science-mentor-can-help-you-in-2022-a7b10152a069/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nLessons learned from two years with a data science mentor\n[Rose Day](https://towardsdatascience.com/author/rjday/)\nDecember 10, 20216 min read\n  * [![](../images/008.jpg)](https://towardsdatascience.com/finding-a-mentor-in-data-science-9863de5b3620/)\n## [Finding A Mentor In Data Science](https://towardsdatascience.com/finding-a-mentor-in-data-science-9863de5b3620/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nHow to climb on to the shoulders of a Giant\n[Kurtis Pykes](https://towardsdatascience.com/author/kurtispykes/)\nNovember 2, 20205 min read\n  * [![](../images/009.jpg)](https://towardsdatascience.com/memory-efficient-embeddings-d637cba7f006/)\n## [Memory-Efficient Embeddings](https://towardsdatascience.com/memory-efficient-embeddings-d637cba7f006/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nCreating smaller models with a new kind of embedding layer\n[Robert Kübler](https://towardsdatascience.com/author/dr-robert-kuebler/)\nJanuary 1, 202414 min read\n  * [![](../images/010.jpg)](https://towardsdatascience.com/signal-processing-on-cell-complexes-with-cell-complex-networks-cxns-fc4cebe24e42/)\n## [Signal Processing on Cell Complexes with Cell Complex Networks (CXNs)](https://towardsdatascience.com/signal-processing-on-cell-complexes-with-cell-complex-networks-cxns-fc4cebe24e42/)\n[](https://towardsdatascience.com/cbam-paper-walkthrough-the-double-attention-mechanism/)\nDeep learning supported on simplicial/cell complexes and beyond\n[Mustafa Hajij](https://towardsdatascience.com/author/mhajij/)\nNovember 19, 20218 min read\n  * [![](../images/011.png)](https://towardsdatascience.com/what-can-flatness-teach-us-understanding-generalisation-in-deep-neural-networks-a7d66f69cb5c/)\n## [What can flatness teach us: understanding generalisation in Deep Neural Networks](https://towardsdatascience.com/what-can-flatness-teach-us-understanding-generalisation-in-deep-neural-networks-a7d66f69cb5c/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nThis is the third post in a series summarising work that seeks to provide a theory of generalisation in Deep Neural Networks (DNNs)...\n[Chris Mingard](https://towardsdatascience.com/author/chrismingard/)\nMarch 29, 202110 min read\n\n\n  * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n  * [X](https://x.com/TDataScience)\n  * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n  * [Threads](https://www.threads.net/@towardsdatascience)\n  * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n####  Recommended Articles \nClose\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500) How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ \"How Relevance Models Foreshadowed Transformers for NLP\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n\n\n",
  "metadata": {
    "title": "CBAM Paper Walkthrough: The Double-Attention Mechanism | Towards Data Science",
    "description": "Understanding and implementing CBAM (Convolutional Block Attention Module) from scratch with PyTorch",
    "keywords": null,
    "author": "Muhammad Ardi Putra",
    "og:locale": "en_US",
    "og:type": "article",
    "og:title": "CBAM Paper Walkthrough: The Double-Attention Mechanism",
    "og:description": "Understanding and implementing CBAM (Convolutional Block Attention Module) from scratch with PyTorch",
    "og:url": "https://towardsdatascience.com/cbam-paper-walkthrough-the-double-attention-mechanism/",
    "og:site_name": "Towards Data Science",
    "og:image": "https://assets.insightmediagroup.io/media/1789357494559_enki42.jpg",
    "og:image:width": "960",
    "og:image:height": "640",
    "og:image:type": "image/jpeg",
    "twitter:card": "summary_large_image",
    "twitter:site": "@TDataScience",
    "twitter:creator": "@TDataScience",
    "twitter:label1": "Written by",
    "twitter:data1": "Muhammad Ardi Putra",
    "twitter:label2": "Est. reading time",
    "twitter:data2": "23 min read",
    "article:published_time": "2026-09-20T13:00:01.896Z",
    "article:modified_time": "2026-09-20T13:00:01.896Z",
    "article:author": "https://towardsdatascience.com/author/muhammad_ardi/"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 202,
    "error": ""
  },
  "code_blocks": [
    {
      "language": "txt",
      "file": "attachments/ae8ac3f1fc45/code/block_000.txt",
      "chars": 49
    },
    {
      "language": "txt",
      "file": "attachments/ae8ac3f1fc45/code/block_001.txt",
      "chars": 143
    },
    {
      "language": "txt",
      "file": "attachments/ae8ac3f1fc45/code/block_002.txt",
      "chars": 1993
    },
    {
      "language": "txt",
      "file": "attachments/ae8ac3f1fc45/code/block_003.txt",
      "chars": 94
    },
    {
      "language": "txt",
      "file": "attachments/ae8ac3f1fc45/code/block_004.txt",
      "chars": 606
    },
    {
      "language": "txt",
      "file": "attachments/ae8ac3f1fc45/code/block_005.txt",
      "chars": 1272
    },
    {
      "language": "txt",
      "file": "attachments/ae8ac3f1fc45/code/block_006.txt",
      "chars": 72
    },
    {
      "language": "txt",
      "file": "attachments/ae8ac3f1fc45/code/block_007.txt",
      "chars": 436
    },
    {
      "language": "txt",
      "file": "attachments/ae8ac3f1fc45/code/block_008.txt",
      "chars": 452
    },
    {
      "language": "txt",
      "file": "attachments/ae8ac3f1fc45/code/block_009.txt",
      "chars": 91
    },
    {
      "language": "txt",
      "file": "attachments/ae8ac3f1fc45/code/block_010.txt",
      "chars": 147
    },
    {
      "language": "txt",
      "file": "attachments/ae8ac3f1fc45/code/block_011.txt",
      "chars": 3657
    },
    {
      "language": "txt",
      "file": "attachments/ae8ac3f1fc45/code/block_012.txt",
      "chars": 131
    },
    {
      "language": "txt",
      "file": "attachments/ae8ac3f1fc45/code/block_013.txt",
      "chars": 394
    },
    {
      "language": "txt",
      "file": "attachments/ae8ac3f1fc45/code/block_014.txt",
      "chars": 3711
    },
    {
      "language": "txt",
      "file": "attachments/ae8ac3f1fc45/code/block_015.txt",
      "chars": 97
    },
    {
      "language": "txt",
      "file": "attachments/ae8ac3f1fc45/code/block_016.txt",
      "chars": 1198
    }
  ],
  "images": [
    {
      "status": "http_404",
      "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500",
      "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500",
      "alt": "",
      "downloaded": false,
      "discovery": "saved_markdown",
      "retried_at": "2026-10-20T17:59:52Z"
    },
    {
      "status": "http_404",
      "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500",
      "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500",
      "alt": "",
      "downloaded": false,
      "discovery": "saved_markdown",
      "retried_at": "2026-10-20T17:59:52Z"
    },
    {
      "status": "http_404",
      "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500",
      "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500",
      "alt": "",
      "downloaded": false,
      "discovery": "saved_markdown",
      "retried_at": "2026-10-20T17:59:52Z"
    },
    {
      "status": "http_404",
      "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500",
      "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500",
      "alt": "",
      "downloaded": false,
      "discovery": "saved_markdown",
      "retried_at": "2026-10-20T17:59:52Z"
    }
  ],
  "audio": [],
  "videos": [
    {
      "url": "https://www.youtube.com/c/TowardsDataScience",
      "kind": "channel_reference",
      "status": "reference_only"
    }
  ],
  "links": [
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/latest/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/editors-pick/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/the-variable/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/shipai/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/submissions/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/category/artificial-intelligence/deep-learning/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/author/muhammad_ardi/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/the-channel-wise-attention/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/taking-resnet-to-the-next-level/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/computer-vision/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/deep-learning/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/neural-network/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/pytorch/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/visualizing-what-batch-normalization-is-and-its-advantages-a49bbcd2fd86/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/author/qtalen/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/what-i-am-doing-to-stay-relevant-as-a-senior-analytics-consultant-in-2026/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/category/data-science/data-analysis/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/author/rashidesai/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/deepmind-alphafold-the-protein-folding-problem-a331d9ced746/"
    },
    {
      "type": "external",
      "href": "https://write.insightmediagroup.io"
    },
    {
      "type": "external",
      "href": "https://www.linkedin.com/company/towards-data-science/"
    },
    {
      "type": "external",
      "href": "https://x.com/TDataScience"
    },
    {
      "type": "external",
      "href": "https://unsplash.com/@waldemarbrandt67w"
    },
    {
      "type": "external",
      "href": "https://unsplash.com"
    },
    {
      "type": "external",
      "href": "https://roadmap.sh/machine-learning"
    },
    {
      "type": "external",
      "href": "https://arxiv.org/abs/1807.06521"
    },
    {
      "type": "external",
      "href": "https://medium.com/ai-advances/senet-paper-walkthrough-the-channel-wise-attention-8ac72b9cc252"
    },
    {
      "type": "external",
      "href": "https://medium.com/ai-advances/taking-resnet-to-the-next-level-resnext-77088c245698"
    },
    {
      "type": "external",
      "href": "https://arxiv.org/abs/1611.05431"
    },
    {
      "type": "external",
      "href": "https://github.com/MuhammadArdiPutra/medium_articles/blob/main/Deep%20Learning%20From%20Scratch/CBAM.ipynb"
    },
    {
      "type": "external",
      "href": "https://www.facebook.com/sharer/sharer.php?title=CBAM+Paper+Walkthrough%3A+The+Double-Attention+Mechanism&u=https%3A%2F%2Ftowardsdatascience.com%2Fcbam-paper-walkthrough-the-double-attention-mechanism%2F"
    },
    {
      "type": "external",
      "href": "https://www.linkedin.com/shareArticle?mini=true&title=CBAM+Paper+Walkthrough%3A+The+Double-Attention+Mechanism&url=https%3A%2F%2Ftowardsdatascience.com%2Fcbam-paper-walkthrough-the-double-attention-mechanism%2F"
    },
    {
      "type": "external",
      "href": "https://x.com/share?text=CBAM+Paper+Walkthrough%3A+The+Double-Attention+Mechanism&url=https%3A%2F%2Ftowardsdatascience.com%2Fcbam-paper-walkthrough-the-double-attention-mechanism%2F"
    },
    {
      "type": "external",
      "href": "https://www.youtube.com/c/TowardsDataScience"
    },
    {
      "type": "external",
      "href": "https://www.threads.net/@towardsdatascience"
    },
    {
      "type": "external",
      "href": "https://bsky.app/profile/towardsdatascience.com"
    },
    {
      "type": "external",
      "href": "https://thenewstack.io"
    },
    {
      "type": "external",
      "href": "https://roadmap.sh"
    },
    {
      "type": "external",
      "href": "https://insightmediagroup.io/creator-network"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-20T17:30:11Z",
  "source_url": "https://towardsdatascience.com/cbam-paper-walkthrough-the-double-attention-mechanism/",
  "delivery_status": "unverified",
  "discovery_status": "http_403",
  "missing_reason": "源页面未重新核验:http_403；已按保留正文补采图片",
  "repair_error": "",
  "actual_video_files": [],
  "expected_saved_body_images": 4,
  "image_discovery": "saved_markdown",
  "source_body_markdown": "![Revisit consent button](https://cdn-cookieyes.com/assets/images/revisit.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](https://cdn-cookieyes.com/assets/images/close.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n  * Cookie\nBCTempID\n  * Duration\n10 minutes\n  * Description\nNo description available.\n\n\n  * Cookie\n__cf_bm\n  * Duration\n1 hour\n  * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n  * Cookie\nAWSALBCORS\n  * Duration\n7 days\n  * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n  * Cookie\n_cfuvid\n  * Duration\nsession\n  * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n  * Cookie\nli_gc\n  * Duration\n6 months\n  * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n  * Cookie\n__hssrc\n  * Duration\nsession\n  * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n  * Cookie\n__hssc\n  * Duration\n1 hour\n  * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n  * Cookie\nwpEmojiSettingsSupports\n  * Duration\nsession\n  * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n  * Cookie\nBCSessionID\n  * Duration\n1 year 1 month 4 days\n  * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n  * Cookie\n_octo\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nlogged_in\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\n__Secure-YEC\n  * Duration\npast\n  * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n  * Cookie\n__eoi\n  * Duration\n6 months\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nAWSALBTGCORS\n  * Duration\n7 days\n  * Description\nNo description available.\n\n\n  * Cookie\nlogin-status-p\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nAWSALBTG\n  * Duration\n7 days\n  * Description\nNo description available.\n\n\n  * Cookie\ncsrf_token\n  * Duration\nsession\n  * Description\nNo description available.\n\n\n  * Cookie\ntoken_v2\n  * Duration\n1 day\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nD\n  * Duration\n1 year\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nPHPSESSID\n  * Duration\nsession\n  * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n  * Cookie\nVISITOR_PRIVACY_METADATA\n  * Duration\n6 months\n  * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n  * Cookie\ncookietest\n  * Duration\nsession\n  * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n  * Cookie\n__Host-airtable-session\n  * Duration\n1 year\n  * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n  * Cookie\n__Host-airtable-session.sig\n  * Duration\n1 year\n  * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n  * Cookie\nm\n  * Duration\n1 year 1 month 4 days\n  * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n  * Cookie\nBIGipServer*\n  * Duration\nsession\n  * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n  * Cookie\n__cfruid\n  * Duration\nsession\n  * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n  * Cookie\n_GRECAPTCHA\n  * Duration\n6 months\n  * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n  * Cookie\n__Secure-YNID\n  * Duration\n6 months\n  * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n  * Cookie\ncookieyes-consent\n  * Duration\n1 year\n  * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n  * Cookie\nlidc\n  * Duration\n1 day\n  * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n  * Cookie\nbrw\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nbrwConsent\n  * Duration\n5 minutes\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nWMF-Uniq\n  * Duration\n1 year\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nloom_anon_comment\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nloom_referral_video\n  * Duration\nsession\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nVISITOR_INFO1_LIVE\n  * Duration\n6 months\n  * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n  * Cookie\nyt-remote-connected-devices\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n  * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n  * Duration\nNever Expires\n  * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n  * Cookie\nyt-remote-device-id\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n  * Cookie\nyt-remote-session-name\n  * Duration\nsession\n  * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n  * Cookie\nyt-remote-fast-check-period\n  * Duration\nsession\n  * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n  * Cookie\nyt-remote-session-app\n  * Duration\nsession\n  * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n  * Cookie\nyt-remote-cast-available\n  * Duration\nsession\n  * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n  * Cookie\nyt-remote-cast-installed\n  * Duration\nsession\n  * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n  * Cookie\ncp_session\n  * Duration\n3 months\n  * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n  * Cookie\nloid\n  * Duration\n1 year 1 month 4 days\n  * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n  * Cookie\n__hstc\n  * Duration\n6 months\n  * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n  * Cookie\nhubspotutk\n  * Duration\n6 months\n  * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n  * Cookie\n_ga\n  * Duration\n1 year 1 month 4 days\n  * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n  * Cookie\n_ga_*\n  * Duration\n1 year 1 month 4 days\n  * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n  * Cookie\n__Host-psifi.analyticsTrace\n  * Duration\n6 hours\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\n__Host-psifi.analyticsTraceV2\n  * Duration\n6 hours\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\n_gh_sess\n  * Duration\nsession\n  * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n  * Cookie\nYSC\n  * Duration\nsession\n  * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n  * Cookie\najs_anonymous_id\n  * Duration\n1 year\n  * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n  * Cookie\nvuid\n  * Duration\n1 year 1 month 4 days\n  * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n  * Cookie\nAWSALB\n  * Duration\n7 days\n  * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n  * Cookie\nacq\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nacq.sig\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nptc\n  * Duration\n2 years\n  * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n  * Cookie\nmuc_ads\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n  * Cookie\nguest_id_marketing\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n  * Cookie\nguest_id_ads\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n  * Cookie\npersonalization_id\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n  * Cookie\nguest_id\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n  * Cookie\nbcookie\n  * Duration\n1 year\n  * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n  * Cookie\n__Secure-ROLLOUT_TOKEN\n  * Duration\n6 months\n  * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n  * Cookie\nyt.innertube::nextId\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n  * Cookie\nyt.innertube::requests\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n  * Cookie\nsession_tracker\n  * Duration\nsession\n  * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n  * Cookie\nedgebucket\n  * Duration\nsession\n  * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n  * Cookie\ndid\n  * Duration\n1 year\n  * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\n# CBAM Paper Walkthrough: The Double-Attention Mechanism\nUnderstanding and implementing CBAM (Convolutional Block Attention Module) from scratch with PyTorch\n[Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\nSeptember 20, 202623 min read\n![](https://assets.insightmediagroup.io/media/1789357494559_enki42.jpg)\nPhoto by [Waldemar Brandt](https://unsplash.com/@waldemarbrandt67w?utm_source=medium&utm_medium=referral) on [Unsplash](https://unsplash.com?utm_source=medium&utm_medium=referral)\n## Introduction\nIn this article, I am going to review and implement the deep learning paper titled “ _CBAM: Convolutional Block Attention Module_ ” by Woo _et al._ [1]. As the name suggests, this is essentially a block we can attach to a CNN-based model to enhance feature quality by performing an attention mechanism. Despite the name _attention_ , it is completely different from the one in the ViT (Vision Transformer) architecture. Keep in mind that CBAM was first released in 2018, while ViT was only introduced in 2020. So, we can simply say that CBAM is the older approach to apply an attention mechanism to image data. Despite being older, we should not worry about its relevance since CBAM is a lot more lightweight as compared to ViT, which makes it suitable to be used as a starting point for deployment on low-power devices.\n### Better Than SENet\nIf we were to talk about the history, CBAM was actually proposed as the improvement of SENet (Squeeze-and-Excitation Network), which was introduced a year before CBAM. If you remember the SENet architecture, it essentially works by performing attention across the channel dimension. By doing so, the channels that seem unimportant would be weighted less than that of the more important ones. — I actually got a separate article talking more thoroughly about SENet, which you can access through the link given in reference number [2].\nInstead of just assigning weights to each channel, CBAM also gives weights to every single pixel in the spatial dimension of the image. So, we can say that CBAM has two attention mechanisms, which the authors refer to as the CAM (_Channel Attention Module_) and the SAM (_Spatial Attention Module_). So, based solely on this theory, CBAM should perform better than SENet.\nLearn this step by step with the interactive [Machine Learning](https://roadmap.sh/machine-learning) roadmap.\n···\n### CBAM Architecture\nLet’s talk more specifically about the CBAM architecture which I display in Figure 1 below. As I’ve mentioned earlier, CBAM consists of CAM and SAM. These two sub-blocks are responsible for creating attention weights, which will then be applied to the original tensor by multiplication. The output tensor of this block (the one referred to as _Refined Features_) has the exact same dimension as the input (_Input Feature_), meaning that we can easily plug CBAM to any backbone model without needing to worry about altering the tensor shapes.\n![](https://cdn-images-1.medium.com/max/640/1*2kHjl86aylQwc09A5Z2xww.png)\nFigure 1. The high-level view of the CBAM architecture [1].\n### Channel Attention Module (CAM)\nNow let’s take a closer look at the channel attention module in Figure 2 below. This component is actually very similar to the SENet block, except that CAM also uses global maxpooling layer in addition to the global average-pooling layer. It is explained in the paper that the two operations capture different kind of information, allowing the tensor produced by CAM to be more informative as compared to that of the SENet block.\n![](https://cdn-images-1.medium.com/max/640/1*h2VWRaErSU6ZHr4zFEr-BA.png)\nFigure 2. The structure of the channel attention module [1].\nRemember that the spatial dimension of the tensor collapses to 1×1 when we apply global pooling operation to it. This essentially means that the input tensor, which has the original shape of _C_ × _H_ × _W_ , now becomes _C_ ×1×1, allowing us to process it further easily with the MLP in the subsequent step. There are two linear layers in this MLP, where the first one is used to shrink the number of features according to the _reduction ratio_ parameter, whereas the second one works by expanding the feature vector back to the original length (i.e., the number of channels _C_). These two linear layers in the MLP are together responsible to learn the importance of each channel. Also, keep in mind that this MLP is shared for the tensor produced by the maxpooling and the average-pooling operations, meaning that these two tensors will be processed by the exact same MLP.\nAs these two tensors have been processed, we then combine them by element-wise summation and pass it through a sigmoid function. Since this function shrinks any number to the range of 0 to 1, we can now perceive the resulting tensor as the _channel attention weight_. The elements that correspond to the more important channels will have the value close to 1, indicating that these channels will be weighted more than the others. According to the paper, this kind of mechanism helps the model to understand _what_ kind of features to attend.\nYou can see the formal mathematical expression of the channel attention module in Figure 3 below, where _F_ is an arbitrary intermediate tensor within a network. One thing you need to keep in mind here is that there should be a ReLU activation function placed between the two linear layers (i.e., _W₀_ and _W₁_) yet is somehow not written in this equation.\n![](https://cdn-images-1.medium.com/max/640/1*RKzELdZsCn1SMv2Zw-Qt5g.png)\nFigure 3. The formal mathematical expression of the channel attention module [1].\n### Spatial Attention Module (SAM)\nThe spatial attention module is conceptually similar to the channel attention module. Take a look at the illustration of this sub-block in Figure 4 below.\n![](https://cdn-images-1.medium.com/max/640/1*bjPlUetS6TBIwSGIa_jHVA.png)\nFigure 4. The structure of the spatial attention module [1].\nWhat essentially differentiates SAM from CAM is the axis where the pooling operation is performed. Previously in CAM the pooling is done across the spatial dimension, allowing each channel to have a single value representing that channel. Meanwhile, here in SAM the pooling is done across the channel dimension for each spatial pixel location. Thus, every pixel now contains a single value that represents all channels at once. By doing so, the input tensor that initially has the shape of _C_ × _H_ × _W_ will collapse to 1× _H_ × _W_. Remember that since we use maximum and average-pooling operations, we will thus have two tensors of that size. These two tensors are then concatenated, forming a new tensor of shape 2× _H_ × _W_. This tensor is then processed with a 7×7 convolution layer of a single kernel, which effectively combines the information from the two channels into one. So, at this point the tensor becomes 1× _H_ × _W_ again and is then forwarded to the sigmoid function. Similar to CAM, the tensor produced by this sigmoid acts as the _spatial attention weight_. By using this weight tensor, we can essentially let the model know _where_ it should pay more attention to. Below is what the formal mathematical definition of the spatial attention module looks like.\n![](https://cdn-images-1.medium.com/max/640/1*roRqge5iK8wbt1d89xollw.png)\nFigure 5. The mathematical definition of the spatial attention module [1].\n### Integrating CBAM to Any Backbone Model\nPreviously I mentioned that the output shape of CBAM is exactly the same as the input, allowing it to be integrated to any backbone model easily. In fact, the authors also provide an illustration regarding how we can do that, which I show you in Figure 6 below. In this example, they illustrate how to plug CBAM into a ResNet building block. Once we have successfully integrated them like this, we can just stack these blocks as usual. Later in the coding part I will demonstrate how to implement CBAM from scratch and how to plug it into a ResNeXt model.\n![](https://cdn-images-1.medium.com/max/960/1*CwusNNOLC2mPZM6fnQnweQ.png)\nFigure 6. How to integrate CBAM into any backbone model [1].\n···\n## Experimental Results\nThe design of the CBAM architecture itself was not chosen arbitrarily. Instead, it was built based on empirical results on their ablation studies, in which they proved that their final model is indeed the most optimal one.\n### Ablation Study on the Channel Attention Module\nThe first ablation study they conducted was related to the pooling layers in the CAM. The results of this experimental set are shown in Figure 7. We can see in this table that the errors when we use either of the two poolings are significantly lower than the plain backbone ResNet50 model. This essentially indicates that the information extracted by the maximum-pooling and the average-pooling are both important. Furthermore, when we utilize both pooling mechanisms simultaneously, the top-1 error goes even lower to 22.80%, which I believe this proves that the two tensors contain information that are not only important but also complementary (i.e., completing each other). Theoretically speaking, the features produced by maxpooling and average-pooling should indeed be complementary since the former captures the most prominent pixel value within a channel while the latter extracts the general information of the channel. So, this is essentially the reason why in Figure 2 the authors ended up using both pooling operations.\n![](https://cdn-images-1.medium.com/max/640/1*EjWot8BzImouHAGdetBK5w.png)\nFigure 7. Ablation study on the use of average and maximum-pooling operations in the CAM [1].\n### Ablation Study on the Spatial Attention Module\nRegarding the spatial attention module, it is explained in the paper that the authors also used varying configurations as displayed in Figure 8 below. You can see here that the configuration in the last row produces the best result, where it utilizes the two pooling operations followed by a convolution layer with 7×7 kernel. In the case of SAM, both the maximum and the average-pooling operations are technically replaceable by a 1×1 convolution (which will combine information across channel dimension with learnable parameters instead of using a “fixed” average and max operations), yet the classification performance appears to be suboptimal.\n![](https://cdn-images-1.medium.com/max/640/1*kaM-ZChiY3Q6s7WRpHZSDA.png)\nFigure 8. Ablation study on the configuration of the SAM [1].\n### Ablation Study on the Placement of CAM and SAM\nThe last ablation study the authors conducted was related to how the CAM and SAM are arranged within the CBAM. It is shown in Figure 9 below that using sequential method, especially CAM followed by SAM, allows the model to perform best with the top-1 error of only 22.66%. You can see in the next row that they tried to swap the sequence of the two modules, but they found that the error increases instead. Furthermore, the classification performance was getting even worse when they tried to parallelize CAM and SAM, even though this approach is still better than the ResNet50 with SE module only.\n![](https://cdn-images-1.medium.com/max/640/1*_KlIvsoX1PngzQXWE2GJrQ.png)\nFigure 9. Ablation study on how the CAM and SAM are arranged [1].\n### Comparison with Other Models\nIn the subsequent experiment the authors compared the performance of a plain model, the model with SE module, and the model with CBAM on different backbones. You can see in Figure 10 below that the model that uses CBAM almost always performs better than the other two as highlighted in green. Moreover, in ResNeXt50, although the model with SE module is better than the same model with CBAM, the error gap is only 0.01%, which I think is negligible.\nI also found in this figure that the error of ResNet50 with CBAM is lower than the plain ResNet101 as highlighted in orange. Interestingly, it is seen here that the number of params and the GFLOPs of ResNet50 with CBAM are much smaller. These facts show that CBAM enables a shallower network to outperform the deeper one while significantly conserving computational resources, which is ideal for deployment on low-end devices.\n![](https://cdn-images-1.medium.com/max/640/1*QdyXGAaoe0j9QMqHGeRs2g.png)\nFigure 10. How CBAM performs on different backbone CNN models [1][3].\n### Attention Heatmap\nIn addition to the quantitative results explained above, the authors also used Grad-CAM to perform qualitative evaluation. If you’re not yet familiar with Grad-CAM, it is essentially a method we can use to find out the specific area that contributes more to the predicted class. Figure 11 below displays several examples of the attention heatmap produced using Grad-CAM, where the area highlighted in red indicates that it gives more contribution to the predicted class.\n![](https://cdn-images-1.medium.com/max/960/1*GBroUkvr_SlM263iDQ9Kiw.png)\nFigure 11. Attention heatmap obtained by Grad-CAM [1].\nThe results are pretty interesting. Let’s now take a look at the _Croquet ball_ class. With the plain ResNet50, it looks like the model focuses on both the ball and the person. As the SE module is applied (i.e., channel-wise attention only), the attention map becomes more refined toward the ball. And then, after we replace the SE module with CBAM, the model achieves an even sharper focus on the target object.\nA similar thing can also be observed in the other classes. In _Eskimo dog_ and _Snow leopard_ , for example, we can see that the model only pays attention to the eyes. If I were to say, this is basically not wrong as long as the predicted class is correct. However, if we were to predict something (as a human), it would make more sense to see the entire object whenever possible, right? And so, this is exactly what the attention modules do. You can see that when CBAM is used, the red area in the attention heatmap covers the entire face, indicating the model now take that facial region into consideration to make predictions.\nAdditionally, it is also seen in the figure that by using CBAM we can make the model more confident when making predictions. Take a look at the _School bus_ image in the above figure. You can see here that the bus is not centered at the middle of the image. This basically causes the plain ResNet50 to have a confidence score of only 0.07 in predicting the bus (which I believe this should have been misclassified). Meanwhile, SE module allows the model to correctly classify it with the confidence of 0.92, and then CBAM improves it even further to 0.98.\n···\n## CBAM Implementation\nAs we have understood all the theories behind CBAM, let’s now roll our sleeves and get our hands dirty with some code! As I’ve mentioned earlier, here I am going to implement CBAM and try to integrate it into the ResNeXt backbone. Although I am implementing both from scratch, I’ll focus the discussion primarily on the CBAM module. So if you’re not yet familiar with ResNeXt, I do encourage you read my previous article about that model beforehand, which you can access through the link at reference [4].\nAs usual, the very first thing we need to do is to import the required modules, i.e., the base `torch` module and its `nn` submodule.\npython\n\n```\n# Codeblock 1\nimport torch\nimport torch.nn as nn\n```\n\nNext, in Codeblock 2 below I initialize the configurable variables. The reduction ratio `R` is used to adjust the width of the MLP layer inside the CAM. I set the value for this to 16 as suggested in the paper. Meanwhile, `CARDINALITY`, `NUM_CHANNELS`, and `NUM_BLOCKS` are the ones belong to ResNeXt.\npython\n\n```\n# Codeblock 2\nR            = 16\nCARDINALITY  = 32\nNUM_CHANNELS = [3, 64, 256, 512, 1024, 2048]\nNUM_BLOCKS   = [3, 4, 6, 3]\nNUM_CLASSES  = 1000\n```\n\n···\n### CAM Implementation\nLet’s start with the CAM first. If you go back to Figure 2, you can see that we have two pooling operations. In Codeblock 3 below, the two layers that correspond to them are initialized at lines `#(1)` and `#(2)`. Don’t forget to set the `output_size` parameter to `(1,1)` since we want each channel to be represented as a single number.\nWhat we do next inside the `__init__()` method is initializing the MLP that consists of two linear layers. The first linear layer is responsible to reduce the number of features according to the `R` parameter (`#(3)`), whereas the second one is used to expand it back to the original number of features (`#(5)`). Also, don’t forget to place the ReLU activation function in between (`#(4)`). In fact, the structure of this MLP layer is exactly the same as the one used in SENet. — You can read more about the underlying idea behind this structure in my previous article about that module at reference [2]. — The last thing we do inside the `__init__()` method is to initialize the sigmoid activation function (`#(6)`), which is responsible to rescale the tensor such that the values will always be between 0 and 1, suitable to be used as an attention weight.\npython\n\n```\n# Codeblock 3\nclass CAM(nn.Module):\n    def __init__(self, num_channels, r=16):\n        super().__init__()\n        \n        self.maxpool = nn.AdaptiveMaxPool2d(output_size=(1,1))  #(1)\n        self.avgpool = nn.AdaptiveAvgPool2d(output_size=(1,1))  #(2)\n        \n        self.mlp = nn.Sequential(\n            nn.Linear(in_features=num_channels,\n                      out_features=num_channels//r,   #(3)\n                      bias=False),\n            \n            nn.ReLU(inplace=True),                    #(4)\n            \n            nn.Linear(in_features=num_channels//r,    #(5)\n                      out_features=num_channels, \n                      bias=False)\n        )\n        \n        self.sigmoid = nn.Sigmoid()                   #(6)\n        \n    def forward(self, x):         #(7)\n        original = x\n        print(f'original\\t\\t: {x.size()}\\n')\n        \n        \n        x_max = self.maxpool(x)   #(8)\n        print(f'x after maxpool (x_max)\\t: {x_max.size()}')\n        \n        x_avg = self.avgpool(x)   #(9)\n        print(f'x after avgpool (x_avg)\\t: {x_avg.size()}\\n')\n        \n        \n        x_max = torch.flatten(x_max, start_dim=1)    #(10)\n        print(f'x_max after flatten\\t: {x_max.size()}')\n        \n        x_avg = torch.flatten(x_avg, start_dim=1)    #(11)\n        print(f'x_avg after flatten\\t: {x_avg.size()}\\n')\n        \n        \n        x_max = self.mlp(x_max)   #(12)\n        print(f'x_max after mlp\\t\\t: {x_max.size()}')\n        \n        x_avg = self.mlp(x_avg)   #(13)\n        print(f'x_avg after mlp\\t\\t: {x_avg.size()}\\n')\n        \n        \n        x = x_max + x_avg         #(14)\n        print(f'after sum\\t\\t: {x.size()}')\n        \n        x = self.sigmoid(x)       #(15)\n        print(f'after sigmoid\\t\\t: {x.size()}')\n        \n        x = x[:, :, None, None]   #(16)\n        print(f'after reshape\\t\\t: {x.size()}')\n        \n        x = x * original          #(17)           \n        print(f'after multiply\\t\\t: {x.size()}')\n        \n        return x\n```\n\nNow let’s move on to the `forward()` method where the execution would happen. You can see at line `#(7)` in the above codeblock that we take a single tensor `x` as the input. This input tensor will then be saved in the `original` variable, which is done because we will later multiply it with the resulting channel attention weight tensor (`#(17)`). The `x` tensor itself will be processed by maxpooling and average-pooling operations in parallel (`#(8–9)`). Both `x_max` and `x_avg` are forwarded to the same MLP, which is the reason why this MLP is called “shared” (`#(12–13)`). Then at line `#(14)`, we combine `x_max` and `x_avg` through element-wise summation before forwarding the resulting tensor to the sigmoid function (`#(15)`). There is a little technical thing we do at line `#(16)`, which is used to reintroduce the spatial dimension we previously dropped at lines `#(10)` and `#(11)`. Finally, as the weight tensor is ready, we can then actually weight the original tensor by multiplying them (`#(17)`).\nAt this point we already got our `CAM` class completed. What we are going to do next is to test it with the following code. Here I initialize a `CAM` instance that accepts a 512-channel image and pass a dummy tensor of size 512×28×28 through it, simulating an arbitrary intermediate tensor within a network.\npython\n\n```\n# Codeblock 4\ncam = CAM(num_channels=512, r=16)\nx = torch.randn(1, 512, 28, 28)\n\nout = cam(x)\n```\n\nIf you run the above code, you should get the following output. Notice that previously in Codeblock 3 I wrote lots of print functions, which is the reason why here you can clearly see the detailed flow of the network.\nhttp\n\n```\n# Codeblock 4 Output\noriginal                : torch.Size([1, 512, 28, 28])\n\nx after maxpool (x_max) : torch.Size([1, 512, 1, 1])\nx after avgpool (x_avg) : torch.Size([1, 512, 1, 1])\n\nx_max after flatten     : torch.Size([1, 512])\nx_avg after flatten     : torch.Size([1, 512])\n\nx_max after mlp         : torch.Size([1, 512])    #(1)\nx_avg after mlp         : torch.Size([1, 512])    #(2)\n\nafter sum               : torch.Size([1, 512])\nafter sigmoid           : torch.Size([1, 512])\nafter reshape           : torch.Size([1, 512, 1, 1])    #(3)\nafter multiply          : torch.Size([1, 512, 28, 28])  #(4)\n```\n\nIt is necessary to understand that although the MLP looks like it doesn’t change the tensor dimension at all (`#(1–2)`), you need to know that the feature vector length is internally reduced to 32 by the first linear layer before eventually expanded back to 512 by the second one. Next, it might also be worth noting that the channel attention weight tensor originally has the shape of 512×1×1 (`#(3)`), indicating that every single channel in the original tensor has a single weighting number associated with it. This attention weight is then applied to the original tensor by using a simple multiplication, which technically speaking, this weight tensor is broadcasted along the spatial dimension of the original tensor (`#(4)`). At this point our tensor is now ready to be forwarded to the SAM, which we are going to build very soon.\n···\n### SAM Implementation\nThe implementation of the spatial attention module is displayed in Codeblock 5 below. What we need to initialize inside the `__init__()` method is only a single 7×7 convolution layer (`#(1)`) and a sigmoid activation function (`#(2)`)\npython\n\n```\n# Codeblock 5\nclass SAM(nn.Module):\n    def __init__(self):\n        super().__init__()\n        \n        self.conv = nn.Conv2d(in_channels=2,     #(1)\n                              out_channels=1, \n                              kernel_size=7, \n                              padding=3, \n                              bias=False)\n        self.sigmoid = nn.Sigmoid()              #(2)\n    \n    def forward(self, x):\n        original = x      #(3)\n        print(f'original\\t\\t: {x.size()}\\n')\n        \n        \n        x_max, _ = torch.max(x,  dim=1, keepdim=True)    #(4)\n        print(f'x after maxpool (x_max)\\t: {x_max.size()}')\n        \n        x_avg    = torch.mean(x, dim=1, keepdim=True)    #(5)\n        print(f'x after avgpool (x_avg)\\t: {x_avg.size()}\\n')\n        \n        \n        x = torch.cat([x_max, x_avg], dim=1)             #(6)\n        print(f'after concatenate\\t: {x.size()}')\n        \n        x = self.conv(x)                                 #(7)\n        print(f'after conv\\t\\t: {x.size()}')\n        \n        x = self.sigmoid(x)                              #(8)\n        print(f'after sigmoid\\t\\t: {x.size()}')\n        \n        x = x * original                                 #(9)\n        print(f'after multiply\\t\\t: {x.size()}')\n        \n        return x\n```\n\nIn the `forward()` method, the first thing we do is to store the original input into a separate variable (`#(3)`), which is exactly the same as what we did in the CAM. The pooling mechanism in the SAM is a bit unique since here we want to do that across the channel dimension. This is essentially the reason that I didn’t initialize any pooling layers in the `__init__()` method since `nn.AdaptiveMaxPool2d()` and `nn.AdaptiveAvgPool2d()` operate on spatial dimension, which is irrelevant for this case. Instead, here we use a simple `torch.max()` and `torch.mean()` functions to do the maxpooling (`#(4)`) and average-pooling (`#(5)`) operations, respectively. Just don’t forget to set the `dim` parameter to 1 so that they really do the operations across the channel dimension.\nDespite taking different values, the tensor shape produced by the two poolings are exactly the same, which is the reason that we can just concatenate them as shown at line `#(6)`. Keep in mind that tensor concatenation does not really combine information as it only stacks the two without blending the numbers. Thus, in the subsequent step we apply the convolution layer we initialized earlier to actually do that (`#(7)`). This convolution only consists of a single kernel, which implies that the resulting tensor will have a single channel as well. This idea is conceptually different from the one in the CAM, where in that module we combine the information by element-wise summation. Technically speaking, we can essentially use summation for the SAM too as it will produce the exact same tensor dimension. However, I do believe that the authors might also intended to capture the correlation between neighboring pixels instead of independently giving weight to each pixel, which is the reason why they decided to use convolution over summation.\nAs the two tensors have been combined, the next thing we do is to pass the resulting tensor through the sigmoid activation function to actually obtain the spatial attention weight (`#(8)`). And finally, we will multiply this weight tensor with the original SAM input as shown at line `#(9)`.\nNow let’s run the Codeblock 6 below to test if our spatial attention module works properly.\npython\n\n```\n# Codeblock 6\nsam = SAM()\nx = torch.randn(1, 512, 28, 28)\n\nout = sam(x)\n```\n\nAnd below is what the flow of the SAM looks like. We can see here that as the pooling operations are applied to the input tensor, the channel dimension collapses to 1 (`#(1–2)`). This essentially means that every pixel is now represented as a single number aggregated from all channels in that spatial location. Then at line `#(3)`, the tensor becomes 2×28×28 as we concatenate the two before eventually reducing it again to 1×28×28 using the convolution layer (`#(4)`). After being processed by the sigmoid function, the resulting spatial attention weight is then multiplied with the original tensor, in which the former is broadcasted along the channel dimension of the latter, allowing the final output tensor to have the exact same shape as the input (`#(5)`).\nhttp\n\n```\n# Codeblock 6 Output\noriginal                : torch.Size([1, 512, 28, 28])\n\nx after maxpool (x_max) : torch.Size([1, 1, 28, 28])    #(1)\nx after avgpool (x_avg) : torch.Size([1, 1, 28, 28])    #(2)\n\nafter concatenate       : torch.Size([1, 2, 28, 28])    #(3)\nafter conv              : torch.Size([1, 1, 28, 28])    #(4)\nafter sigmoid           : torch.Size([1, 1, 28, 28])\nafter multiply          : torch.Size([1, 512, 28, 28])  #(5)\n```\n\n···\n#### The Complete CBAM Block\nNow as the CAM and SAM are done, we will now put them together in the `CBAM` class. See the details in Codeblock 7 below. You can see here that this class is very simple as what we need to do is just to initialize the two attention modules and place them sequentially. Keep in mind that we need to pass the `num_channels` parameter every time we want to initialize a `CBAM` instance (`#(1)`) since we will later integrate this module into ResNeXt, in which every single one of its building blocks accepts different number of channels, and so we need to make this CBAM block flexible as well.\npython\n\n```\n# Codeblock 7\nclass CBAM(nn.Module):\n    def __init__(self, num_channels):    #(1)\n        super().__init__()\n        \n        self.cam = CAM(num_channels=num_channels)\n        self.sam = SAM()\n        \n    def forward(self, x):\n        print(f'original\\t\\t: {x.size()}')\n        \n        x = self.cam(x)\n        print(f'after cam\\t\\t: {x.size()}')\n        \n        x = self.sam(x)\n        print(f'after sam\\t\\t: {x.size()}')\n        \n        return x\n```\n\nAgain, just to ensure that this class works properly, let’s pass a dummy tensor through it using the Codeblock 8 below.\npython\n\n```\n# Codeblock 8\ncbam = CBAM(num_channels=512)\nx = torch.randn(1, 512, 28, 28)\n\nout = cbam(x)\n```\n\nhttp\n\n```\n# Codeblock 8 Output\noriginal   : torch.Size([1, 512, 28, 28])\nafter cam  : torch.Size([1, 512, 28, 28])\nafter sam  : torch.Size([1, 512, 28, 28])\n```\n\n···\n### Implementing CBAM on ResNeXt Building Block\nAlright, so at this point our CBAM is ready and in this section I am going to actually demonstrate how we can attach this module to a ResNeXt building block. The code I write in Codeblock 9 onwards are basically the same as the one used when I demonstrated how to integrate SENet on ResNeXt. I do encourage you to read that article [2] and my explanation on the pure ResNeXt backbone [4] because it would be too long if I explain everything here.\nThe only thing I want to emphasize in Codeblock 9 is that the CBAM module itself is initialized at line `#(1)` which is then attached to the flow at line `#(2)`.\npython\n\n```\n# Codeblock 9\nclass Block(nn.Module):\n    def __init__(self, \n                 in_channels,\n                 add_channel=False,\n                 channel_multiplier=2,\n                 downsample=False):\n        super().__init__()\n\n        self.add_channel = add_channel\n        self.channel_multiplier = channel_multiplier\n        self.downsample = downsample\n        \n        \n        if self.add_channel:\n            out_channels = in_channels*self.channel_multiplier\n        else:\n            out_channels = in_channels\n        \n        mid_channels = out_channels//2\n        \n        \n        if self.downsample:\n            stride = 2\n        else:\n            stride = 1\n\n\n        if self.add_channel or self.downsample:\n            self.projection = nn.Conv2d(in_channels=in_channels,\n                                        out_channels=out_channels, \n                                        kernel_size=1, \n                                        stride=stride, \n                                        padding=0, \n                                        bias=False)\n            nn.init.kaiming_normal_(self.projection.weight, nonlinearity='relu')\n            self.bn_proj = nn.BatchNorm2d(num_features=out_channels)\n\n        self.conv0 = nn.Conv2d(in_channels=in_channels,\n                               out_channels=mid_channels,\n                               kernel_size=1, \n                               stride=1, \n                               padding=0, \n                               bias=False)\n        nn.init.kaiming_normal_(self.conv0.weight, nonlinearity='relu')\n        self.bn0 = nn.BatchNorm2d(num_features=mid_channels)\n\n        self.conv1 = nn.Conv2d(in_channels=mid_channels,\n                               out_channels=mid_channels, \n                               kernel_size=3, \n                               stride=stride,\n                               padding=1, \n                               bias=False, \n                               groups=CARDINALITY)\n        nn.init.kaiming_normal_(self.conv1.weight, nonlinearity='relu')\n        self.bn1 = nn.BatchNorm2d(num_features=mid_channels)\n\n        self.conv2 = nn.Conv2d(in_channels=mid_channels,\n                               out_channels=out_channels,\n                               kernel_size=1, \n                               stride=1, \n                               padding=0, \n                               bias=False)\n        nn.init.kaiming_normal_(self.conv2.weight, nonlinearity='relu')\n        self.bn2 = nn.BatchNorm2d(num_features=out_channels)\n        \n        self.relu = nn.ReLU()\n        \n        self.cbam = CBAM(num_channels=out_channels)               #(1)\n        \n    def forward(self, x):\n        print(f'original\\t\\t: {x.size()}')\n        \n        if self.add_channel or self.downsample:\n            residual = self.bn_proj(self.projection(x))\n            print(f'after projection\\t: {residual.size()}')\n        else:\n            residual = x\n            print(f'no projection\\t\\t: {residual.size()}')\n        \n        x = self.conv0(x)\n        x = self.bn0(x)\n        x = self.relu(x)\n        print(f'after conv0-bn0-relu\\t: {x.size()}')\n\n        x = self.conv1(x)\n        x = self.bn1(x)\n        x = self.relu(x)\n        print(f'after conv1-bn1-relu\\t: {x.size()}')\n        \n        x = self.conv2(x)\n        x = self.bn2(x)\n        print(f'after conv2-bn2\\t\\t: {x.size()}')\n        \n        x = self.cbam(x)                                          #(2)\n        print(f'after cbam\\t\\t: {x.size()}')\n        \n        x = x + residual\n        x = self.relu(x)\n        print(f'after summation\\t\\t: {x.size()}')\n        \n        return x\n```\n\nAnd now we can test the `Block` class above by running the Codeblock 10 below. You can see in the following output that the tensor successfully passes through the entire network, including the CBAM block we attached at the end of the main flow (`#(1)`).\npython\n\n```\n# Codeblock 10\nblock = Block(in_channels=512, add_channel=False, downsample=False)\nx = torch.randn(1, 512, 28, 28)\n\nout = block(x)\n```\n\nhttp\n\n```\n# Codeblock 10 Output\noriginal             : torch.Size([1, 512, 28, 28])\nno projection        : torch.Size([1, 512, 28, 28])\nafter conv0-bn0-relu : torch.Size([1, 256, 28, 28])\nafter conv1-bn1-relu : torch.Size([1, 256, 28, 28])\nafter conv2-bn2      : torch.Size([1, 512, 28, 28])\nafter cbam           : torch.Size([1, 512, 28, 28])    #(1)\nafter summation      : torch.Size([1, 512, 28, 28])\n```\n\n···\n### The Final CBAM-ized ResNeXt\nAs the CBAM module has been attached to the main ResNeXt building block, we can just stack these blocks according to the structure given in the ResNeXt paper. The `CBAMResNeXt` class in Codeblock 11 is literally copy-pasted from my SENet article [2] since the only thing we need to do to attach CBAM module to ResNeXt is modifying the `Block` class back in Codeblock 9.\npython\n\n```\n# Codeblock 11\nclass CBAMResNeXt(nn.Module):\n    def __init__(self):\n        super().__init__()\n\n        # conv1 stage\n        self.resnext_conv1 = nn.Conv2d(in_channels=NUM_CHANNELS[0],\n                                       out_channels=NUM_CHANNELS[1],\n                                       kernel_size=7,\n                                       stride=2,\n                                       padding=3, \n                                       bias=False)\n        nn.init.kaiming_normal_(self.resnext_conv1.weight, \n                                nonlinearity='relu')\n        self.resnext_bn1 = nn.BatchNorm2d(num_features=NUM_CHANNELS[1])\n        self.relu = nn.ReLU()\n        self.resnext_maxpool1 = nn.MaxPool2d(kernel_size=3,\n                                             stride=2, \n                                             padding=1)\n\n        # conv2 stage\n        self.resnext_conv2 = nn.ModuleList([\n            Block(in_channels=NUM_CHANNELS[1],\n                  add_channel=True,\n                  channel_multiplier=4,\n                  downsample=False)\n        ])\n        for _ in range(NUM_BLOCKS[0]-1):\n            self.resnext_conv2.append(Block(in_channels=NUM_CHANNELS[2]))\n\n        # conv3 stage\n        self.resnext_conv3 = nn.ModuleList([Block(in_channels=NUM_CHANNELS[2],\n                                                  add_channel=True, \n                                                  downsample=True)])\n        for _ in range(NUM_BLOCKS[1]-1):\n            self.resnext_conv3.append(Block(in_channels=NUM_CHANNELS[3]))\n            \n            \n        # conv4 stage\n        self.resnext_conv4 = nn.ModuleList([Block(in_channels=NUM_CHANNELS[3],\n                                                  add_channel=True, \n                                                  downsample=True)])\n        \n        for _ in range(NUM_BLOCKS[2]-1):\n            self.resnext_conv4.append(Block(in_channels=NUM_CHANNELS[4]))\n            \n            \n        # conv5 stage\n        self.resnext_conv5 = nn.ModuleList([Block(in_channels=NUM_CHANNELS[4],\n                                                  add_channel=True, \n                                                  downsample=True)])\n        \n        for _ in range(NUM_BLOCKS[3]-1):\n            self.resnext_conv5.append(Block(in_channels=NUM_CHANNELS[5]))\n \n       \n        self.avgpool = nn.AdaptiveAvgPool2d(output_size=(1,1))\n\n        self.fc = nn.Linear(in_features=NUM_CHANNELS[5],\n                            out_features=NUM_CLASSES)\n\n    def forward(self, x):\n        print(f'original\\t\\t: {x.size()}')\n        \n        x = self.relu(self.resnext_bn1(self.resnext_conv1(x)))\n        print(f'after resnext_conv1\\t: {x.size()}')\n        \n        x = self.resnext_maxpool1(x)\n        print(f'after resnext_maxpool1\\t: {x.size()}')\n        \n        for i, block in enumerate(self.resnext_conv2):\n            x = block(x)\n            print(f'after resnext_conv2 #{i}\\t: {x.size()}')\n            \n        for i, block in enumerate(self.resnext_conv3):\n            x = block(x)\n            print(f'after resnext_conv3 #{i}\\t: {x.size()}')\n            \n        for i, block in enumerate(self.resnext_conv4):\n            x = block(x)\n            print(f'after resnext_conv4 #{i}\\t: {x.size()}')\n            \n        for i, block in enumerate(self.resnext_conv5):\n            x = block(x)\n            print(f'after resnext_conv5 #{i}\\t: {x.size()}')\n        \n        x = self.avgpool(x)\n        print(f'after avgpool\\t\\t: {x.size()}')\n        \n        x = torch.flatten(x, start_dim=1)\n        print(f'after flatten\\t\\t: {x.size()}')\n        \n        x = self.fc(x)\n        print(f'after fc\\t\\t: {x.size()}')\n        \n        return x\n```\n\nAnd now we can check if the entire network works properly by running the following testing code. Here I initialize a `CBAMResNeXt` instance and pass a dummy RGB image of size 224×224 through it.\npython\n\n```\n# Codeblock 12\ncbamresnext = CBAMResNeXt()\nx = torch.randn(1, 3, 224, 224)\n\nout = cbamresnext(x)\n```\n\nBelow is what the resulting output looks like. Here we can see that the model successfully passes the original image through the entire network up until the classification head. This essentially indicates that our CBAM is properly attached, and thus this model is ready to be trained for image classification. \nhttp\n\n```\n# Codeblock 12 Output\noriginal               : torch.Size([1, 3, 224, 224])\nafter resnext_conv1    : torch.Size([1, 64, 112, 112])\nafter resnext_maxpool1 : torch.Size([1, 64, 56, 56])\nafter resnext_conv2 #0 : torch.Size([1, 256, 56, 56])\nafter resnext_conv2 #1 : torch.Size([1, 256, 56, 56])\nafter resnext_conv2 #2 : torch.Size([1, 256, 56, 56])\nafter resnext_conv3 #0 : torch.Size([1, 512, 28, 28])\nafter resnext_conv3 #1 : torch.Size([1, 512, 28, 28])\nafter resnext_conv3 #2 : torch.Size([1, 512, 28, 28])\nafter resnext_conv3 #3 : torch.Size([1, 512, 28, 28])\nafter resnext_conv4 #0 : torch.Size([1, 1024, 14, 14])\nafter resnext_conv4 #1 : torch.Size([1, 1024, 14, 14])\nafter resnext_conv4 #2 : torch.Size([1, 1024, 14, 14])\nafter resnext_conv4 #3 : torch.Size([1, 1024, 14, 14])\nafter resnext_conv4 #4 : torch.Size([1, 1024, 14, 14])\nafter resnext_conv4 #5 : torch.Size([1, 1024, 14, 14])\nafter resnext_conv5 #0 : torch.Size([1, 2048, 7, 7])\nafter resnext_conv5 #1 : torch.Size([1, 2048, 7, 7])\nafter resnext_conv5 #2 : torch.Size([1, 2048, 7, 7])\nafter avgpool          : torch.Size([1, 2048, 1, 1])\nafter flatten          : torch.Size([1, 2048])\nafter fc               : torch.Size([1, 1000])\n```\n\n···\n## Ending\nAnd well I think that’s pretty much everything about CBAM and how to implement it from scratch. You can also find the code used in this article in my GitHub repository [6]. Please let me know if you find any mistakes in the discussion or in the code. Thanks for reading, I hope you learn something new today. See ya in my next article!\n···\n## References\n[1] Sanghyun Woo _et al._ CBAM: Convolutional Block Attention Module. Arxiv. <https://arxiv.org/abs/1807.06521> [Accessed November 12, 2025].\n[2] Muhammad Ardi Putra. SENet Paper Walkthrough: The Channel-Wise Attention. Towards Data Science. <https://towardsdatascience.com/the-channel-wise-attention/> [Accessed November 12, 2025]. Also available at <https://medium.com/ai-advances/senet-paper-walkthrough-the-channel-wise-attention-8ac72b9cc252>.\n[3] Image originally created by author.\n[4] Muhammad Ardi Putra. ResNeXt Paper Walkthrough: Taking ResNet to the Next Level. Towards Data Science. <https://towardsdatascience.com/taking-resnet-to-the-next-level/> [Accessed November 12, 2025]. Also available at <https://medium.com/ai-advances/taking-resnet-to-the-next-level-resnext-77088c245698>.\n[5] Saining Xie _et al_. Aggregated Residual Transformations for Deep Neural Networks. Arxiv. <https://arxiv.org/abs/1611.05431> [Accessed November 12, 2025].\n[6] MuhammadArdiPutra. CBAM. GitHub. <https://github.com/MuhammadArdiPutra/medium_articles/blob/main/Deep%20Learning%20From%20Scratch/CBAM.ipynb> [Accessed November 12, 2025].\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](https://assets.insightmediagroup.io/media/1787657598287_taxhya.jpeg)\nWritten By\n## [Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\nDeep learning storyteller\n[See all from Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\n[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Computer Vision](https://towardsdatascience.com/tag/computer-vision/)[Deep Learning](https://towardsdatascience.com/tag/deep-learning/)[Neural Network](https://towardsdatascience.com/tag/neural-network/)[Pytorch](https://towardsdatascience.com/tag/pytorch/)\nShare This Article\n  * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fcbam-paper-walkthrough-the-double-attention-mechanism%2F&title=CBAM%20Paper%20Walkthrough%3A%20The%20Double-Attention%20Mechanism)\n  * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fcbam-paper-walkthrough-the-double-attention-mechanism%2F&title=CBAM%20Paper%20Walkthrough%3A%20The%20Double-Attention%20Mechanism)\n  * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fcbam-paper-walkthrough-the-double-attention-mechanism%2F&text=CBAM%20Paper%20Walkthrough%3A%20The%20Double-Attention%20Mechanism)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/02/1dbB9UfwXmtsDXFyhwoWJag.png)](https://towardsdatascience.com/visualizing-what-batch-normalization-is-and-its-advantages-a49bbcd2fd86/)\n## [Visualizing What Batch Normalization Is and Its Advantages](https://towardsdatascience.com/visualizing-what-batch-normalization-is-and-its-advantages-a49bbcd2fd86/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nOptimizing your neural network training with batch normalization\n[Peng Qian](https://towardsdatascience.com/author/qtalen/)\nFebruary 3, 20244 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2026/02/jonathan-chng-HgoKvtKpyHA-unsplash-1-scaled-1.jpg)](https://towardsdatascience.com/what-i-am-doing-to-stay-relevant-as-a-senior-analytics-consultant-in-2026/)\n## [What I Am Doing to Stay Relevant as a Senior Analytics Consultant in 2026](https://towardsdatascience.com/what-i-am-doing-to-stay-relevant-as-a-senior-analytics-consultant-in-2026/)\n[Data analysis](https://towardsdatascience.com/category/data-science/data-analysis/)\nLearn how to work with AI, while strengthening your unique human skills that technology cannot replace\n[Rashi Desai](https://towardsdatascience.com/author/rashidesai/)\nFebruary 7, 20266 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/01/0R-kug-C1N8rq9rJG-scaled.jpg)](https://towardsdatascience.com/deepmind-alphafold-the-protein-folding-problem-a331d9ced746/)\n## [DeepMind, AlphaFold & the Protein Folding Problem](https://towardsdatascience.com/deepmind-alphafold-the-protein-folding-problem-a331d9ced746/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\n- - - - - - - - - - - - - - - - - - - - - - - - - - - -\n[Prem Oommen](https://towardsdatascience.com/author/thedataoracle/)\nJanuary 25, 20215 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/08/1M2-mrtHSU71_6CbdY3wfWg.gif)](https://towardsdatascience.com/ultimate-volleyball-a-3d-volleyball-environment-built-using-unity-ml-agents-c9d3213f3064/)\n## [🏐 Ultimate Volleyball: A multi-agent reinforcement learning environment built using Unity...](https://towardsdatascience.com/ultimate-volleyball-a-3d-volleyball-environment-built-using-unity-ml-agents-c9d3213f3064/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nTrain reinforcement learning agents to play Volleyball\n[Joy Zhang](https://towardsdatascience.com/author/joooyz/)\nAugust 11, 20215 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/12/0Z3GyZRiphI-yD-wD-scaled.jpg)](https://towardsdatascience.com/why-a-data-science-mentor-can-help-you-in-2022-a7b10152a069/)\n## [Why a Data Science Mentor Can Help You in 2022](https://towardsdatascience.com/why-a-data-science-mentor-can-help-you-in-2022-a7b10152a069/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nLessons learned from two years with a data science mentor\n[Rose Day](https://towardsdatascience.com/author/rjday/)\nDecember 10, 20216 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/11/0FMatbR_wg3EsSrFE-scaled.jpg)](https://towardsdatascience.com/finding-a-mentor-in-data-science-9863de5b3620/)\n## [Finding A Mentor In Data Science](https://towardsdatascience.com/finding-a-mentor-in-data-science-9863de5b3620/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nHow to climb on to the shoulders of a Giant\n[Kurtis Pykes](https://towardsdatascience.com/author/kurtispykes/)\nNovember 2, 20205 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/01/0lzGS70ck_IDbvqXR-scaled.jpg)](https://towardsdatascience.com/memory-efficient-embeddings-d637cba7f006/)\n## [Memory-Efficient Embeddings](https://towardsdatascience.com/memory-efficient-embeddings-d637cba7f006/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nCreating smaller models with a new kind of embedding layer\n[Robert Kübler](https://towardsdatascience.com/author/dr-robert-kuebler/)\nJanuary 1, 202414 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/11/0NFhC_2gOt-n4A1bi-scaled.jpg)](https://towardsdatascience.com/signal-processing-on-cell-complexes-with-cell-complex-networks-cxns-fc4cebe24e42/)\n## [Signal Processing on Cell Complexes with Cell Complex Networks (CXNs)](https://towardsdatascience.com/signal-processing-on-cell-complexes-with-cell-complex-networks-cxns-fc4cebe24e42/)\n[](https://towardsdatascience.com/cbam-paper-walkthrough-the-double-attention-mechanism/)\nDeep learning supported on simplicial/cell complexes and beyond\n[Mustafa Hajij](https://towardsdatascience.com/author/mhajij/)\nNovember 19, 20218 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/05ztp0bJqdMEIL1eF.png)](https://towardsdatascience.com/what-can-flatness-teach-us-understanding-generalisation-in-deep-neural-networks-a7d66f69cb5c/)\n## [What can flatness teach us: understanding generalisation in Deep Neural Networks](https://towardsdatascience.com/what-can-flatness-teach-us-understanding-generalisation-in-deep-neural-networks-a7d66f69cb5c/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nThis is the third post in a series summarising work that seeks to provide a theory of generalisation in Deep Neural Networks (DNNs)...\n[Chris Mingard](https://towardsdatascience.com/author/chrismingard/)\nMarch 29, 202110 min read\n\n\n  * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n  * [X](https://x.com/TDataScience)\n  * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n  * [Threads](https://www.threads.net/@towardsdatascience)\n  * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n####  Recommended Articles \nClose\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500) How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ \"How Relevance Models Foreshadowed Transformers for NLP\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n\n\n",
  "body_base_path": "attachments/ae8ac3f1fc45/pages/",
  "body_markdown_path": "attachments/ae8ac3f1fc45/pages/archived-body.md",
  "local_html": "attachments/ae8ac3f1fc45/pages/archived-body.html",
  "body_source": "prior_capture"
}