{ "source_file": "0930", "source_platform": "Medium", "author": "Muhammad Ardi Putra", "title_listed": "CSPNet Paper Walkthrough: Just Better, No Tradeoffs", "keywords": "", "domain": "图像处理", "content_type": "深度技术解析", "is_paid_flag": "否", "url": "https://towardsdatascience.com/cspnet-paper-walkthrough-just-better-no-tradeoff/", "publish_date": "2026-05-03", "corpus_stage": "", "notes": "", "id": "0d18c97807dc", "collected_at": "2026-10-02T02:21:29", "title": "CSPNet Paper Walkthrough: Just Better, No Tradeoffs | Towards Data Science", "body_markdown": "![Revisit consent button](../images/body-df29beacd7496220cc94.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](../images/body-2159bb67f3de8ad870eb.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n * Cookie\nBCTempID\n * Duration\n10 minutes\n * Description\nNo description available.\n\n\n * Cookie\n__cf_bm\n * Duration\n1 hour\n * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n * Cookie\nAWSALBCORS\n * Duration\n7 days\n * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n * Cookie\n_cfuvid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n * Cookie\nli_gc\n * Duration\n6 months\n * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n * Cookie\n__hssrc\n * Duration\nsession\n * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n * Cookie\n__hssc\n * Duration\n1 hour\n * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n * Cookie\nwpEmojiSettingsSupports\n * Duration\nsession\n * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n * Cookie\nBCSessionID\n * Duration\n1 year 1 month 4 days\n * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n * Cookie\n_octo\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nlogged_in\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\n__Secure-YEC\n * Duration\npast\n * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n * Cookie\n__eoi\n * Duration\n6 months\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTGCORS\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\nlogin-status-p\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTG\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\ncsrf_token\n * Duration\nsession\n * Description\nNo description available.\n\n\n * Cookie\ntoken_v2\n * Duration\n1 day\n * Description\nDescription is currently not available.\n\n\n * Cookie\nD\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nPHPSESSID\n * Duration\nsession\n * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n * Cookie\nVISITOR_PRIVACY_METADATA\n * Duration\n6 months\n * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n * Cookie\ncookietest\n * Duration\nsession\n * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n * Cookie\n__Host-airtable-session\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\n__Host-airtable-session.sig\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\nm\n * Duration\n1 year 1 month 4 days\n * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n * Cookie\nBIGipServer*\n * Duration\nsession\n * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n * Cookie\n__cfruid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n * Cookie\n_GRECAPTCHA\n * Duration\n6 months\n * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n * Cookie\n__Secure-YNID\n * Duration\n6 months\n * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n * Cookie\ncookieyes-consent\n * Duration\n1 year\n * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n * Cookie\nlidc\n * Duration\n1 day\n * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n * Cookie\nbrw\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nbrwConsent\n * Duration\n5 minutes\n * Description\nDescription is currently not available.\n\n\n * Cookie\nWMF-Uniq\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nloom_anon_comment\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nloom_referral_video\n * Duration\nsession\n * Description\nDescription is currently not available.\n\n\n * Cookie\nVISITOR_INFO1_LIVE\n * Duration\n6 months\n * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n * Cookie\nyt-remote-connected-devices\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n * Duration\nNever Expires\n * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n * Cookie\nyt-remote-device-id\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-name\n * Duration\nsession\n * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\nyt-remote-fast-check-period\n * Duration\nsession\n * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-app\n * Duration\nsession\n * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n * Cookie\nyt-remote-cast-available\n * Duration\nsession\n * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n * Cookie\nyt-remote-cast-installed\n * Duration\nsession\n * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\ncp_session\n * Duration\n3 months\n * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n * Cookie\nloid\n * Duration\n1 year 1 month 4 days\n * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n * Cookie\n__hstc\n * Duration\n6 months\n * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n * Cookie\nhubspotutk\n * Duration\n6 months\n * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n * Cookie\n_ga\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n * Cookie\n_ga_*\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n * Cookie\n__Host-psifi.analyticsTrace\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n__Host-psifi.analyticsTraceV2\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n_gh_sess\n * Duration\nsession\n * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n * Cookie\nYSC\n * Duration\nsession\n * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n * Cookie\najs_anonymous_id\n * Duration\n1 year\n * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n * Cookie\nvuid\n * Duration\n1 year 1 month 4 days\n * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n * Cookie\nAWSALB\n * Duration\n7 days\n * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n * Cookie\nacq\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nacq.sig\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nptc\n * Duration\n2 years\n * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n * Cookie\nmuc_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n * Cookie\nguest_id_marketing\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\nguest_id_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\npersonalization_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n * Cookie\nguest_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n * Cookie\nbcookie\n * Duration\n1 year\n * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n * Cookie\n__Secure-ROLLOUT_TOKEN\n * Duration\n6 months\n * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n * Cookie\nyt.innertube::nextId\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nyt.innertube::requests\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nsession_tracker\n * Duration\nsession\n * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n * Cookie\nedgebucket\n * Duration\nsession\n * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n * Cookie\ndid\n * Duration\n1 year\n * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\n# CSPNet Paper Walkthrough: Just Better, No Tradeoffs\nA review of the Cross-Stage Partial Network paper — and a from-scratch PyTorch implementation\n[Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\nMay 3, 202623 min read\n![](../images/000.jpg)\nPhoto by [Mihály Köles](https://unsplash.com/@mihaly_koles?utm_source=medium&utm_medium=referral) on Unsplash\nHow do you make your CNN-based model more lightweight? Just take the smaller version of that model, right? Like with ResNet, for instance, if ResNet-152 feels too heavy, why not just use ResNet-101? Or in the case of DenseNet, why not go with DenseNet-121 rather than DenseNet-169? — Yes, that’s true, but you would have to sacrifice some accuracy for that. Basically, if you want a lighter model then you should expect your accuracy to drop as well. \nNow, what if I told you about a model that’s more lightweight than its base but can still compete on accuracy? Meet CSPNet (Cross Stage Partial Network). You’ll be surprised that it can effectively reduce computational complexity while maintaining high accuracy — no tradeoff! In this article we are going to talk about the CSPNet architecture, including how it works and how to implement it from scratch.\n···\n## A Brief History of CSPNet\nCSPNet was first introduced in a paper titled “ _CSPNet: A New Backbone That Can Enhance Learning Capability of CNN_ ” written by Wang _et al._ back in November 2019 [1]. CSPNet was originally proposed to address the limitations of DenseNet. Despite already being computationally cheaper than ResNet, the authors thought that the computation of DenseNet itself is still considered expensive. Take a look at the main building block of a DenseNet in Figure 1 below to understand why.\nLearn this step by step with the interactive [Machine Learning](https://roadmap.sh/machine-learning) roadmap.\n![](../images/001.png)\nFigure 1. The main building block of a DenseNet model [2].\nIn a DenseNet building block — called _dense_ block — every convolution layer takes information from all previous layers, causing it to have a lot of redundant gradient information that makes training inefficient. We can think of it like a student taught by 5 different teachers for the same material. It’s actually good since the student can get multiple perspectives about that specific topic. However, at some point it becomes redundant and thus inefficient. In the case of DenseNet, we can see the deeper layers as students and all the tensors from shallower layers as teachers. In the example above, if we assume _H₄_ as our student, then the _x₀_ , _x₁_ , _x₂_ , and _x₃_ tensors act as the teachers. Here you can just imagine how that student would get overwhelmed by all that information!\nBefore we get into CSPNet, I actually have a whole separate article specifically talking about DenseNet (reference [3]), which I highly recommend you read if you want the full picture of how this architecture works.\n### Objectives\nThe objective of CSPNet is to enable a network to have cheaper computational complexity and better gradient combination. The reason for the latter is that most gradient information in DenseNet consists of duplicates of each other. It is important to note that CSPNet is not a standalone network. Instead, it is a new paradigm we apply to DenseNet.\nNow let’s take a look at Figure 2 below to see how CSPNet achieves its objectives. You can see the illustration on the left that the number of feature maps gradually increases as we get deeper into the network. If you have read my previous article about DenseNet, this is essentially something we can control through the _growth rate_ parameter, i.e., the number of feature maps produced by each convolution layer within a dense block. In fact, this increase in the number of feature maps is what the authors see as a computational bottleneck.\n![](../images/body-589390aa42b2500aabd9.png)\nFigure 2. Left: the original DenseNet building block (same as Figure 1). Right: The CSPNet version of the DenseNet building block (called CSPDenseNet) [1].\nBy applying the Cross Stage Partial mechanism, we can basically make the computation of a DenseNet cheaper. If we take a look at the illustration on the right, we can see that we have an additional branch coming out from _x₀_ that goes directly to the so-called _Partial Transition Layer_. There are at least two advantages we get with this mechanism, which are in accordance with the objectives I mentioned earlier. First, we can save lots of computations since the number of feature maps processed by the dense block is only half of the original one. And second, the gradient information becomes more diverse since we got an additional path with unprocessed feature maps that avoids the redundant gradient information. So in short, the idea of CSPNet eliminates the computational redundancy of DenseNet (through the skip-path) while at the same time still preserves its feature-reuse property (through the dense block).\n···\n## The Detailed CSPNet Architecture\nSpeaking of the details, the original feature map is first divided into two parts in channel-wise manner, where each of them will be processed in different paths. Suppose we got 64 input channels, the first 32 feature maps (_part 1_) will skip through all computations, whereas the remaining 32 (_part 2_) will be processed by a dense block. Although this splitting step is pretty easy, the merging step is actually not quite trivial. You can see in Figure 3 below that we got several different mechanisms to do so.\n![](../images/body-4ee173ccde6a609db6b1.png)\nFigure 3. Several different ways to perform feature combination in CSPNet [1].\nIn the structure referred to as _fusion first_ (c), we concatenate the _part 1_ tensor with the _part 2_ tensor that has been processed by the dense block prior to passing them through the transition layer. So, option (c) is actually pretty straightforward to implement because the spatial dimension of the two tensors is exactly the same, allowing us to concatenate them easily.\nIn my previous article [3], I mentioned that the transition layer of a DenseNet is used to reduce both the spatial dimension and the number of channels. In fact, this property requires us to rethink how to implement the _fusion last_ (d) structure. This is essentially because the transition layer will cause the _part 2_ tensor to have a smaller spatial dimension than the _part 1_ tensor. So technically speaking, we need to either apply something like a pooling with a stride of 2 to the _part 1_ branch or simply omitting the downsampling operation in the transition layer. By doing this, the spatial dimension of the two tensors will be the same, and thus they are now concatenable.\nInstead of just using a single transition layer placed either before or after feature combination, the authors also proposed another method which they refer to as _CSPDenseNet_ (b). We can think of this as a combination of (c) and (d), where we got two transition layers placed before and after the tensor concatenation process. In this particular case, the first transition layer (the one placed in the _part 2_ branch) will perform channel reduction by _cross-channel pooling_ , i.e., a pooling layer that operates across channel dimension. Meanwhile, the second transition layer will perform both spatial downsampling and channel count reduction. So basically, in this approach we reduce the number of channels twice — well, at least that’s what I understand from the paper about the two transition layers, as the detailed processes within these layers are not explicitly discussed.\n### Experimental Results\nTalking about the experimental results regarding these feature combination mechanisms, it is explained in the paper that _fusion last_ (d) is better than _fusion first_ (c), where the former can significantly reduce computational complexity while only suffers from a very slight drop in accuracy. Variant (c) actually also reduces computational complexity, yet the degradation in accuracy is also significant. Authors found that variant (b) obtained an even better result than the two. Figure 4 below displays several experimental results showing how the three feature combination mechanisms performed compared to the base model. However, instead of using DenseNet, they somehow decided to use PeleeNet to compare these structures.\n![](../images/body-209a358dbcfc042a5c31.png)\nFigure 4. Performance comparison of the base PeleeNet (corresponds to (a) in Figure 3), CSPPeleeNet (b), PeleeNet with fusion first method (c), and PeleeNet with fusion last method (d) [1].\nBased on the above figure, we can see that the _CSP fusion last_ (green) indeed performs better compared to the _CSP fusion first_ (red). This is based on the fact that its accuracy only degrades by 0.1% from its base model while having 21% smaller computational complexity. Meanwhile, even though _CSP fusion first_ successfully reduces computational complexity by 26%, but the accuracy drop is pretty significant as it performs 1.5% worse than the base PeleeNet. And the most impressive structure is the _CSPPeleeNet_ variant (blue), i.e., the one that utilizes two transition layers. Here we can clearly see that although the computational complexity is reduced by 13%, the accuracy of the model actually improves by 0.2% — again, no tradeoff!\nNot only that, but the authors also tried to implement CSPNet on other backbone models. The results in Figure 5 below shows that the CSPNet structure successfully reduces the computational complexity of DenseNet -201-Elastic and ResNeXt-50 by 19% and 22%, respectively. It is interesting to see that the accuracy of the ResNeXt model improves despite the reduction in model complexity, which is in accordance with the result obtained by CSPPeleeNet in Figure 4.\n![](../images/body-c0f1461f43ffef414fb9.png)\nFigure 5. Performance improvement of DenseNet-201-Elastic and ResNeXt-50 after implementing the CSPNet mechanism [1].\n### The Mathematical Expression of CSPDenseNet\nFor those who love math, here I got you some notations that you might find interesting to know. Figures 6 and 7 below display the mathematical expressions of DenseNet and CSPDenseNet blocks during the forward propagation phase.\nIn the DenseNet block, _x₁_ corresponds to the tensor produced by the first conv layer _w₁_ based on the input tensor _x₀_. Next, we concatenate the original tensor _x₀_ with _x₁_ and use them as the input for the _w₂_ layer (or to be more precise, _w_ is actually the weights of the conv layer, not the conv layer itself). We keep producing more feature maps and concatenating the existing ones as we get deeper into the network. In this way, we can basically say that the outputs of all previous layers become the input of the current layer.\n![](../images/body-beee74a750dfb448a8b8.png)\nFigure 6. The mathematical representation of forward propagation within a DenseNet block [1].\nThe case is different for CSPDenseNet. You can see in the notation below that we got _x₀’_ and _x₀’’_ , which we previously refer to as the _part 1_ and _part 2_. The _x₀’’_ tensor undergoes processing like the one in DenseNet block until we got _xₖ_. Next, the output of this dense block is then forwarded to the first transition layer, which is denoted as _wᴛ_. The resulting tensor _xᴛ_ is then concatenated with the _part 1_ tensor _x₀’_ before eventually being passed through the second transition layer _wᴜ_ to obtain the final output tensor _xᴜ_.\n![](../images/body-c8d85a5093f98fbc101b.png)\nFigure 7. The mathematical expression of the forward propagation in CSPDenseNet block [1].\n···\n## CSPDenseNet Implementation\nNow let’s get even deeper into the CSPNet architecture by implementing it from scratch. Although we can basically apply the CSPNet structure to any backbone, here I am going to do that on the DenseNet model to match it with the illustrations and equations I showed you earlier. Figure 8 below displays what the complete DenseNet architecture looks like. Just remember that every single dense block in this architecture originally follows the _DenseNet_ structure in Figure 3a, and our objective here is to replace all these dense blocks with _CSPDenseNet_ block illustrated in Figure 3b.\n![](../images/body-ee91dbedfab1eba3e6ec.png)\nFigure 8. The complete DenseNet architecture [2].\nThe first thing we do is to import the required modules and initialize the configurable parameters as shown in Codeblock 1. The `GROWTH` variable is the _growth rate_ parameter, which denotes the number of feature maps produced by each bottleneck within the dense block. Next, `CHANNEL_POOLING` is the parameter we use to adjust the behavior of the channel-pooling mechanism in our first transition layer. Here I set this parameter to 0.8, meaning that we will shrink the number of channels to 80% of its original channel count. The `COMPRESSION` parameter works similarly to the `CHANNEL_POOLING` variable, yet this one operates in the second transition layer. Finally, here we define the `REPEATS` list, which is used to set the number of _bottleneck_ blocks we will initialize within the dense block of each stage.\npython\n\n```\n# Codeblock 1\nimport torch\nimport torch.nn as nn\n\nGROWTH = 12\nCHANNEL_POOLING = 0.8\nCOMPRESSION = 0.5\nREPEATS = [6, 12, 24, 16]\n```\n\n### Bottleneck Block Implementation\nBelow is the implementation of the bottleneck block to be placed within the dense block. This `Bottleneck` class is exactly the same as the one I used in my DenseNet article [3]. I directly copy-pasted the code from there since we don’t need to modify this part at all. Just keep in mind that a bottleneck block comprises a 1×1 convolution followed by a 3×3 convolution.\npython\n\n```\n# Codeblock 2\nclass Bottleneck(nn.Module):\n def __init__(self, in_channels):\n super().__init__()\n \n self.relu = nn.ReLU()\n self.dropout = nn.Dropout(p=0.2)\n \n self.bn0 = nn.BatchNorm2d(num_features=in_channels)\n self.conv0 = nn.Conv2d(in_channels=in_channels, \n out_channels=GROWTH*4, \n kernel_size=1, \n padding=0, \n bias=False)\n \n self.bn1 = nn.BatchNorm2d(num_features=GROWTH*4)\n self.conv1 = nn.Conv2d(in_channels=GROWTH*4, \n out_channels=GROWTH, \n kernel_size=3, \n padding=1, \n bias=False)\n \n def forward(self, x):\n print(f'original\\t: {x.size()}')\n \n out = self.dropout(self.conv0(self.relu(self.bn0(x))))\n print(f'after conv0\\t: {out.size()}')\n \n out = self.dropout(self.conv1(self.relu(self.bn1(out))))\n print(f'after conv1\\t: {out.size()}')\n \n concatenated = torch.cat((out, x), dim=1) \n print(f'after concat\\t: {concatenated.size()}')\n \n return concatenated\n```\n\nThe following testing code simulates the first bottleneck block within the dense block. Remember that the very first conv layer in the architecture (the one with 7×7 kernel) produces 64 feature maps, but since in the case of CSPNet we only want to process half of them (the _part 2_ tensor), hence here we will test it with a tensor of 32 feature maps.\npython\n\n```\n# Codeblock 3\nbottleneck = Bottleneck(in_channels=32)\n\nx = torch.randn(1, 32, 56, 56)\nx = bottleneck(x)\n```\n\ntext\n\n```\n# Codeblock 3 Output\noriginal : torch.Size([1, 32, 56, 56])\nafter conv0 : torch.Size([1, 48, 56, 56])\nafter conv1 : torch.Size([1, 12, 56, 56])\nafter concat : torch.Size([1, 44, 56, 56])\n```\n\nYou can see in the resulting output above that the number of feature maps becomes 44 at the end of the process, where this number is obtained by adding the input channel count and the growth rate, i.e., 32 + 12 = 44. Again, you can just check out my DenseNet article [3] if you want to get a better understanding about this calculation.\n### Dense Block Implementation\nNow to create a sequence of bottleneck blocks easily, we can just wrap it inside the `DenseBlock` class in Codeblock 4 below. Later on, we can just specify the number of bottleneck blocks to be stacked through the `repeats `parameter. Again, this class is also copy-pasted from my DenseNet article, so I am not going to explain it any further.\npython\n\n```\n# Codeblock 4\nclass DenseBlock(nn.Module):\n def __init__(self, in_channels, repeats):\n super().__init__()\n self.bottlenecks = nn.ModuleList()\n \n for i in range(repeats):\n current_in_channels = in_channels + i * GROWTH\n self.bottlenecks.append(Bottleneck(in_channels=current_in_channels))\n \n def forward(self, x):\n print(f'original\\t\\t\\t: {x.size()}')\n \n for i, bottleneck in enumerate(self.bottlenecks):\n x = bottleneck(x)\n print(f'after bottleneck #{i}\\t\\t: {x.size()}')\n \n return x\n```\n\nIn order to check if our `DenseBlock` class works properly, we will test it using the Codeblock 5 below. Here I am trying to simulate the _part 2_ tensor processed by the first dense block, which contains a sequence of 6 bottleneck blocks.\npython\n\n```\n# Codeblock 5\ndense_block = DenseBlock(in_channels=32, repeats=6)\nx = torch.randn(1, 32, 56, 56)\n\nx = dense_block(x)\n```\n\nAnd below is what the output looks like. Here we can clearly see that each bottleneck block successfully increases the feature maps by 12.\ntext\n\n```\n# Codeblock 5 Output\noriginal : torch.Size([1, 32, 56, 56])\nafter bottleneck #0 : torch.Size([1, 44, 56, 56])\nafter bottleneck #1 : torch.Size([1, 56, 56, 56])\nafter bottleneck #2 : torch.Size([1, 68, 56, 56])\nafter bottleneck #3 : torch.Size([1, 80, 56, 56])\nafter bottleneck #4 : torch.Size([1, 92, 56, 56])\nafter bottleneck #5 : torch.Size([1, 104, 56, 56])\n```\n\n### First Transition\nRemember that the _CSPDenseNet_ variant in Figure 3b uses two transition layers. In this section we are going to discuss the first transition layer, i.e., the one used to process the tensor in the _part 2_ branch. Here we will not perform spatial downsampling, which is the reason why you don’t see any pooling layer within the `__init__()` method in Codeblock 6 below. Instead, here we will only perform cross-channel pooling, which can be perceived as a standard pooling operation yet is done across the channel dimension. To implement it, we can simply use a 1×1 convolution (`#(2)`) and specify the number of output channels we want (`#(1)`). We can think of it like this: in a spatial downsampling process, we can basically do that by using either pooling or a strided convolution layer, which in the latter case it will aggregate the pixel values with specific weightings from the local neighborhood. In the case of cross-channel pooling, since we don’t have a specific PyTorch layer for that, we can simply replace it with a pointwise convolution layer, which by doing so we can basically aggregate pixel values across the channel dimension.\npython\n\n```\n# Codeblock 6\nclass FirstTransition(nn.Module):\n def __init__(self, in_channels, out_channels):\n super().__init__()\n \n self.bn = nn.BatchNorm2d(num_features=in_channels)\n self.relu = nn.ReLU()\n self.conv = nn.Conv2d(in_channels=in_channels, \n out_channels=out_channels, #(1)\n kernel_size=1, #(2)\n padding=0,\n bias=False)\n self.dropout = nn.Dropout(p=0.2)\n \n def forward(self, x):\n print(f'original\\t\\t: {x.size()}')\n \n out = self.dropout(self.conv(self.relu(self.bn(x))))\n print(f'after first_transition\\t: {out.size()}')\n \n return out\n```\n\nThe result given in the Codeblock 5 Output shows that the _part 2_ tensor will have the shape of 104×56×56 after being processed by the dense block. Thus, in the testing code below I will use this tensor shape to simulate the first transition layer within that stage. To adjust the number of output channels, we can simply multiply the input channel count with the `CHANNEL_POOLING` variable we initialized earlier as shown at line `#(1)` in Codeblock 7 below.\npython\n\n```\n# Codeblock 7\nfirst_transition = FirstTransition(in_channels=104, \n out_channels=int(104*CHANNEL_POOLING)) #(1)\n\nx = torch.randn(1, 104, 56, 56)\nx = first_transition(x)\n```\n\nNow as the code above is run, we can see that the number of feature maps shrinks from 104 to 83 (80% of the original).\ntext\n\n```\n# Codeblock 7 Output\noriginal\t\t : torch.Size([1, 104, 56, 56])\nafter first_transition : torch.Size([1, 83, 56, 56])\n```\n\n### Second Transition\nThe structure of the second transition layer is quite a bit the same as the first one, except that here we also have an average pooling layer with a stride of 2 to reduce the spatial dimension by half (`#(1)`).\npython\n\n```\n# Codeblock 8\nclass SecondTransition(nn.Module):\n def __init__(self, in_channels, out_channels):\n super().__init__()\n \n self.bn = nn.BatchNorm2d(num_features=in_channels)\n self.relu = nn.ReLU()\n self.conv = nn.Conv2d(in_channels=in_channels, \n out_channels=out_channels, \n kernel_size=1, \n padding=0,\n bias=False)\n self.dropout = nn.Dropout(p=0.2)\n self.pool = nn.AvgPool2d(kernel_size=2, stride=2) #(1)\n \n def forward(self, x):\n print(f'original\\t\\t: {x.size()}')\n\n out = self.pool(self.dropout(self.conv(self.relu(self.bn(x)))))\n print(f'after second_transition\\t: {out.size()}')\n \n return out\n```\n\nRemember that the tensor coming into the second transition layer is a concatenation of the _part 1_ and the _part 2_ tensors. This is essentially the reason why in the testing code below I set this layer to accept 32 + 83 = 115 feature maps. Similar to the first transition layer, here we multiply this number of feature maps with the `COMPRESSION` variable (`#(1)`) to reduce the number of channels even further.\npython\n\n```\n# Codeblock 9\nsecond_transition = SecondTransition(in_channels=115, \n out_channels=int(115*COMPRESSION)) #(1)\n\nx = torch.randn(1, 115, 56, 56)\nx = second_transition(x)\n```\n\nIn the resulting output below we can see that the spatial dimension halves thanks to the average pooling layer. At the same time, the number of feature maps also decreases from 115 to 57 since we set the `COMPRESSION` parameter to 0.5.\ntext\n\n```\n# Codeblock 9 Output\noriginal : torch.Size([1, 115, 56, 56])\nafter second_transition : torch.Size([1, 57, 28, 28])\n```\n\n### The CSPDenseNet Model\nWith all the components ready, we can now build the entire CSPDenseNet architecture, which I break down in Codeblocks 10a, 10b, and 10c below. Let’s now focus on the Codeblock 10a first, where I initialize all the layers according to the structure given in Figure 8. Here you can see at line `#(1)` that we initialize a 7×7 convolution layer, which acts as the input layer of the network. This layer is then followed by a maxpooling layer (`#(2)`). These two layers use the stride of 2, meaning that the spatial dimensions of the input tensor will be reduced to one-fourth of its original size.\npython\n\n```\n# Codeblock 10a\nclass CSPDenseNet(nn.Module):\n def __init__(self):\n super().__init__()\n \n self.first_conv = nn.Conv2d(in_channels=3, #(1)\n out_channels=64, \n kernel_size=7, \n stride=2, \n padding=3, \n bias=False)\n self.first_pool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1) #(2)\n channel_count = 64\n \n \n \n ##### Stage 0\n self.dense_block_0 = DenseBlock(in_channels=channel_count//2, \n repeats=REPEATS[0])\n \n self.first_transition_0 = FirstTransition(in_channels=(channel_count//2)+(REPEATS[0]*GROWTH), \n out_channels=int(((channel_count//2)+(REPEATS[0]*GROWTH))*CHANNEL_POOLING))\n \n channel_count = (channel_count - (channel_count//2)) + int(((channel_count//2)+(REPEATS[0]*GROWTH))*CHANNEL_POOLING)\n \n self.second_transition_0 = SecondTransition(in_channels=channel_count, \n out_channels=int(channel_count*COMPRESSION))\n \n channel_count = int(channel_count*COMPRESSION)\n #####\n \n \n ##### Stage 1\n self.dense_block_1 = DenseBlock(in_channels=channel_count//2, \n repeats=REPEATS[1])\n \n self.first_transition_1 = FirstTransition(in_channels=(channel_count//2)+(REPEATS[1]*GROWTH), \n out_channels=int(((channel_count//2)+(REPEATS[1]*GROWTH))*CHANNEL_POOLING))\n \n channel_count = (channel_count - (channel_count//2)) + int(((channel_count//2)+(REPEATS[1]*GROWTH))*CHANNEL_POOLING)\n \n self.second_transition_1 = SecondTransition(in_channels=channel_count, \n out_channels=int(channel_count*COMPRESSION))\n \n channel_count = int(channel_count*COMPRESSION)\n #####\n \n \n ##### Stage 2\n self.dense_block_2 = DenseBlock(in_channels=channel_count//2, \n repeats=REPEATS[2])\n \n self.first_transition_2 = FirstTransition(in_channels=(channel_count//2)+(REPEATS[2]*GROWTH), \n out_channels=int(((channel_count//2)+(REPEATS[2]*GROWTH))*CHANNEL_POOLING))\n \n channel_count = (channel_count - (channel_count//2)) + int(((channel_count//2)+(REPEATS[2]*GROWTH))*CHANNEL_POOLING)\n \n self.second_transition_2 = SecondTransition(in_channels=channel_count, \n out_channels=int(channel_count*COMPRESSION))\n \n channel_count = int(channel_count*COMPRESSION)\n #####\n \n \n ##### Stage 3\n self.dense_block_3 = DenseBlock(in_channels=channel_count//2, \n repeats=REPEATS[3])\n \n self.first_transition_3 = FirstTransition(in_channels=(channel_count//2)+(REPEATS[3]*GROWTH), \n out_channels=int(((channel_count//2)+(REPEATS[3]*GROWTH))*CHANNEL_POOLING))\n \n channel_count = (channel_count - (channel_count//2)) + int(((channel_count//2)+(REPEATS[3]*GROWTH))*CHANNEL_POOLING)\n #####\n \n \n self.avgpool = nn.AdaptiveAvgPool2d(output_size=(1,1)) #(3)\n self.fc = nn.Linear(in_features=channel_count, out_features=1000) #(4)\n```\n\nStill with the above codeblock, here I group the layers I initialize based on the stage they belong to. Let’s now focus on the part I refer to as `Stage 0`. Here you can see that we got a dense block (`dense_block_0`) and the first transition layer (`first_transition_0`). These two components are responsible to process the _part 2_ tensor. Next, we initialize the second transition layer (`second_transition_0`), which is used to process the concatenation result of the _part 1_ and _part 2_ tensors. Since the channel count is dynamic depending on the `GROWTH`, `CHANNEL_POOLING`, `COMPRESSION`, and `REPEATS` variables, we need to keep track of the channel count after each step so that the model can adaptively adjust itself according to these variables. We do the same thing for all the remaining stages, except in `Stage 3` we don’t initialize the second transition layer since at that point we won’t reduce the channels and the spatial dimension any further. Instead, we will directly pass the concatenated `part 1` and `part 2` tensors to the average pooling (`#(3)`) and the classification (`#(4)`) layers. And that ends our discussion about the Codeblock 10a above.\nBefore we get into the `forward()` method, there is another function we need to create: `split_channels()`. As the name suggests, this function, which is written in Codeblock 10b below, is used to split a tensor into _part 1_ and _part 2_. The if-else statement here is used to check if the number of channels is odd or even. In fact, it would be very easy if the channel count is an even number as we can just divide them into two (`#(4)`). But if the channel count is odd, we need to manually determine the size of each part as seen at line `#(1)` and `#(2)` before eventually splitting them (`#(3)`).\npython\n\n```\n# Codeblock 10b\n def split_channels(self, x):\n\n channel_count = x.size(1)\n\n if channel_count%2 != 0:\n split_size_2 = channel_count // 2 #(1)\n split_size_1 = channel_count - split_size_2 #(2)\n return torch.split(x, [split_size_1, split_size_2], dim=1) #(3)\n\n else:\n return torch.split(x, channel_count // 2, dim=1) #(4)\n```\n\nAs we have finished defining the `__init__()` and the `split_channel()` methods, we can now implement the `forward()` method in Codeblock 10c below. Generally speaking, what we do here is simply forward the tensor sequentially. But now let’s pay attention to the part I refer to as `Stage 0`. Here you can see that after the tensor is passed through the `first_pool` layer (`#(1)`), we then split it into two using the `split_channels()` function we declared earlier (`#(2)`). From there, we now obtain the `part1` and `part2` tensors. We will leave the `part1` tensor as is all the way to the end of the stage. Meanwhile, for the `part2` tensor, we will process it with the dense block (`#(3)`) and the first transition layer (`#(4)`). Next, we concatenate the resulting tensor with the `part1` tensor to create the skip-connection (`#(5)`). And then, we finally pass it through the second transition layer (`#(6)`). The same steps are repeated for all stages until we eventually reach the output layer to make classification. Just remember that the `Stage 3` is quite different because here we don’t have the second transition layer.\npython\n\n```\n# Codeblock 10c\n def forward(self, x):\n print(f'original\\t\\t\\t: {x.size()}')\n \n x = self.first_conv(x)\n print(f'after first_conv\\t\\t: {x.size()}')\n \n x = self.first_pool(x) #(1)\n print(f'after first_pool\\t\\t: {x.size()}\\n')\n \n \n \n ##### Stage 0\n part1, part2 = self.split_channels(x) #(2)\n print(f'part1\\t\\t\\t\\t: {part1.size()}')\n print(f'part2\\t\\t\\t\\t: {part2.size()}')\n \n part2 = self.dense_block_0(part2) #(3)\n print(f'part2 after dense block 0\\t: {part2.size()}')\n \n part2 = self.first_transition_0(part2) #(4)\n print(f'part2 after first trans 0\\t: {part2.size()}')\n \n x = torch.cat((part1, part2), dim=1) #(5)\n print(f'after concatenate\\t\\t: {x.size()}')\n \n x = self.second_transition_0(x) #(6)\n print(f'after second transition 0\\t: {x.size()}\\n')\n \n \n \n ##### Stage 1\n part1, part2 = self.split_channels(x)\n print(f'part1\\t\\t\\t\\t: {part1.size()}')\n print(f'part2\\t\\t\\t\\t: {part2.size()}')\n \n part2 = self.dense_block_1(part2)\n print(f'part2 after dense block 1\\t: {part2.size()}')\n \n part2 = self.first_transition_1(part2)\n print(f'part2 after first trans 1\\t: {part2.size()}')\n \n x = torch.cat((part1, part2), dim=1)\n print(f'after concatenate\\t\\t: {x.size()}')\n \n x = self.second_transition_1(x)\n print(f'after second transition 1\\t: {x.size()}\\n')\n \n \n \n ##### Stage 2\n part1, part2 = self.split_channels(x)\n print(f'part1\\t\\t\\t\\t: {part1.size()}')\n print(f'part2\\t\\t\\t\\t: {part2.size()}')\n \n part2 = self.dense_block_2(part2)\n print(f'part2 after dense block 2\\t: {part2.size()}')\n \n part2 = self.first_transition_2(part2)\n print(f'part2 after first trans 2\\t: {part2.size()}')\n \n x = torch.cat((part1, part2), dim=1)\n print(f'after concatenate\\t\\t: {x.size()}')\n \n x = self.second_transition_2(x)\n print(f'after second transition 2\\t: {x.size()}\\n')\n \n \n \n ##### Stage 3\n part1, part2 = self.split_channels(x)\n print(f'part1\\t\\t\\t\\t: {part1.size()}')\n print(f'part2\\t\\t\\t\\t: {part2.size()}')\n \n part2 = self.dense_block_3(part2)\n print(f'part2 after dense block 2\\t: {part2.size()}')\n \n part2 = self.first_transition_3(part2)\n print(f'part2 after first trans 2\\t: {part2.size()}')\n \n x = torch.cat((part1, part2), dim=1)\n print(f'after concatenate\\t\\t: {x.size()}\\n')\n \n \n \n x = self.avgpool(x)\n print(f'after avgpool\\t\\t\\t: {x.size()}')\n \n x = torch.flatten(x, start_dim=1)\n print(f'after flatten\\t\\t\\t: {x.size()}')\n \n x = self.fc(x)\n print(f'after fc\\t\\t\\t: {x.size()}')\n \n return x\n```\n\nNow let’s test the CSPDenseNet class we just created by running the Codeblock 11 below. Here I use a dummy tensor of shape 3×224×224 to simulate a 224×224 RGB image passed through the network.\npython\n\n```\n# Codeblock 11\ncspdensenet = CSPDenseNet()\n\nx = torch.randn(1, 3, 224, 224)\nx = cspdensenet(x)\n```\n\nAnd below is what the output looks like. Here you can see that every time a tensor gets into a network, our `split_channels()` method correctly divides the tensor into two (`#(1–2)`). Then, the bottleneck block within each stage also correctly adds the number of channels of the _part 2_ tensor by 12 before eventually being passed through the first transition layer. The first transition layer itself successfully reduces the number of channels by 20% as seen at line `#(3)`, simulating the cross-channel pooling mechanism. Afterwards, the resulting tensor is then concatenated with the tensor from _part 1_ (`#(4)`) and passed through the second transition layer (`#(5)`) to further reduce the number of channels and halve the spatial dimension. We do the same thing for all stages until eventually we got the 1000-class prediction.\ntext\n\n```\n# Codeblock 11 Output\noriginal : torch.Size([1, 3, 224, 224])\nafter first_conv : torch.Size([1, 64, 112, 112])\nafter first_pool : torch.Size([1, 64, 56, 56])\n\npart1 : torch.Size([1, 32, 56, 56]) #(1)\npart2 : torch.Size([1, 32, 56, 56]) #(2)\nafter bottleneck #0 : torch.Size([1, 44, 56, 56])\nafter bottleneck #1 : torch.Size([1, 56, 56, 56])\nafter bottleneck #2 : torch.Size([1, 68, 56, 56])\nafter bottleneck #3 : torch.Size([1, 80, 56, 56])\nafter bottleneck #4 : torch.Size([1, 92, 56, 56])\nafter bottleneck #5 : torch.Size([1, 104, 56, 56])\npart2 after dense block 0 : torch.Size([1, 104, 56, 56])\npart2 after first trans 0 : torch.Size([1, 83, 56, 56]) #(3)\nafter concatenate : torch.Size([1, 115, 56, 56]) #(4)\nafter second transition 0 : torch.Size([1, 57, 28, 28]) #(5)\n\npart1 : torch.Size([1, 29, 28, 28])\npart2 : torch.Size([1, 28, 28, 28])\nafter bottleneck #0 : torch.Size([1, 40, 28, 28])\nafter bottleneck #1 : torch.Size([1, 52, 28, 28])\nafter bottleneck #2 : torch.Size([1, 64, 28, 28])\nafter bottleneck #3 : torch.Size([1, 76, 28, 28])\nafter bottleneck #4 : torch.Size([1, 88, 28, 28])\nafter bottleneck #5 : torch.Size([1, 100, 28, 28])\nafter bottleneck #6 : torch.Size([1, 112, 28, 28])\nafter bottleneck #7 : torch.Size([1, 124, 28, 28])\nafter bottleneck #8 : torch.Size([1, 136, 28, 28])\nafter bottleneck #9 : torch.Size([1, 148, 28, 28])\nafter bottleneck #10 : torch.Size([1, 160, 28, 28])\nafter bottleneck #11 : torch.Size([1, 172, 28, 28])\npart2 after dense block 1 : torch.Size([1, 172, 28, 28])\npart2 after first trans 1 : torch.Size([1, 137, 28, 28])\nafter concatenate : torch.Size([1, 166, 28, 28])\nafter second transition 1 : torch.Size([1, 83, 14, 14])\n\npart1 : torch.Size([1, 42, 14, 14])\npart2 : torch.Size([1, 41, 14, 14])\nafter bottleneck #0 : torch.Size([1, 53, 14, 14])\nafter bottleneck #1 : torch.Size([1, 65, 14, 14])\nafter bottleneck #2 : torch.Size([1, 77, 14, 14])\nafter bottleneck #3 : torch.Size([1, 89, 14, 14])\nafter bottleneck #4 : torch.Size([1, 101, 14, 14])\nafter bottleneck #5 : torch.Size([1, 113, 14, 14])\nafter bottleneck #6 : torch.Size([1, 125, 14, 14])\nafter bottleneck #7 : torch.Size([1, 137, 14, 14])\nafter bottleneck #8 : torch.Size([1, 149, 14, 14])\nafter bottleneck #9 : torch.Size([1, 161, 14, 14])\nafter bottleneck #10 : torch.Size([1, 173, 14, 14])\nafter bottleneck #11 : torch.Size([1, 185, 14, 14])\nafter bottleneck #12 : torch.Size([1, 197, 14, 14])\nafter bottleneck #13 : torch.Size([1, 209, 14, 14])\nafter bottleneck #14 : torch.Size([1, 221, 14, 14])\nafter bottleneck #15 : torch.Size([1, 233, 14, 14])\nafter bottleneck #16 : torch.Size([1, 245, 14, 14])\nafter bottleneck #17 : torch.Size([1, 257, 14, 14])\nafter bottleneck #18 : torch.Size([1, 269, 14, 14])\nafter bottleneck #19 : torch.Size([1, 281, 14, 14])\nafter bottleneck #20 : torch.Size([1, 293, 14, 14])\nafter bottleneck #21 : torch.Size([1, 305, 14, 14])\nafter bottleneck #22 : torch.Size([1, 317, 14, 14])\nafter bottleneck #23 : torch.Size([1, 329, 14, 14])\npart2 after dense block 2 : torch.Size([1, 329, 14, 14])\npart2 after first trans 2 : torch.Size([1, 263, 14, 14])\nafter concatenate : torch.Size([1, 305, 14, 14])\nafter second transition 2 : torch.Size([1, 152, 7, 7])\n\npart1 : torch.Size([1, 76, 7, 7])\npart2 : torch.Size([1, 76, 7, 7])\nafter bottleneck #0 : torch.Size([1, 88, 7, 7])\nafter bottleneck #1 : torch.Size([1, 100, 7, 7])\nafter bottleneck #2 : torch.Size([1, 112, 7, 7])\nafter bottleneck #3 : torch.Size([1, 124, 7, 7])\nafter bottleneck #4 : torch.Size([1, 136, 7, 7])\nafter bottleneck #5 : torch.Size([1, 148, 7, 7])\nafter bottleneck #6 : torch.Size([1, 160, 7, 7])\nafter bottleneck #7 : torch.Size([1, 172, 7, 7])\nafter bottleneck #8 : torch.Size([1, 184, 7, 7])\nafter bottleneck #9 : torch.Size([1, 196, 7, 7])\nafter bottleneck #10 : torch.Size([1, 208, 7, 7])\nafter bottleneck #11 : torch.Size([1, 220, 7, 7])\nafter bottleneck #12 : torch.Size([1, 232, 7, 7])\nafter bottleneck #13 : torch.Size([1, 244, 7, 7])\nafter bottleneck #14 : torch.Size([1, 256, 7, 7])\nafter bottleneck #15 : torch.Size([1, 268, 7, 7])\npart2 after dense block 2 : torch.Size([1, 268, 7, 7])\npart2 after first trans 2 : torch.Size([1, 214, 7, 7])\nafter concatenate : torch.Size([1, 290, 7, 7])\n\nafter avgpool : torch.Size([1, 290, 1, 1])\nafter flatten : torch.Size([1, 290])\nafter fc : torch.Size([1, 1000])\n```\n\n···\n## Ending\nAnd that’s it! We have successfully learned CSPNet and implemented it on DenseNet backbone. As I’ve mentioned earlier, we can actually use the idea of CSPNet to improve the performance of any other backbone models such as ResNet or ResNeXt. So here I challenge you to implement CSPNet on these models from scratch.\nTo be honest I cannot confirm that my implementation is 100% correct since the official GitHub repo [4] of the paper does not provide the PyTorch implementation — but that’s at least everything I understand from the manuscript. Please let me know if you find any mistake in the code or in my explanations. Thanks for reading, and see you again in my next article. Bye!\nBtw you can also find the code used in this article on my GitHub repo [5].\n···\n## References\n[1] Chien-Yao Wang _et al._ CSPnet: A New Backbone That Can Enhance Learning Capability of CNN. Arxiv. [Accessed October 1, 2025].\n[2] Gao Huang _et al._ Densely Connected Convolutional Networks. Arxiv. [Accessed September 18, 2025].\n[3] [Muhammad Ardi](https://medium.com/u/9801a58700ac). DenseNet Paper Walkthrough: All Connected. Towards Data Science. [Accessed April 26, 2026].\n[4] WongKinYiu. CrossStagePartialNetworks. GitHub. [Accessed October 1, 2025].\n[5] MuhammadArdiPutra. CSPNet. GitHub. [https://github.com/MuhammadArdiPutra/medium_articles/blob/main/DenseNet.ipynb](https://github.com/MuhammadArdiPutra/medium_articles/blob/main/Deep%20Learning%20From%20Scratch/CSPNet.ipynb) [Accessed October 1, 2025].\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](../images/body-59e1c17daab1fcb582c5.jpeg)\nWritten By\n## [Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\nDeep learning storyteller\n[See all from Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\n[Deep Dives](https://towardsdatascience.com/tag/deep-dives/)[Artificial Intelligence](https://towardsdatascience.com/tag/artificial-intelligence/)[Cnn](https://towardsdatascience.com/tag/cnn/)[Machine Learning](https://towardsdatascience.com/tag/machine-learning/)[Neural Network](https://towardsdatascience.com/tag/neural-network/)\nShare This Article\n * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fcspnet-paper-walkthrough-just-better-no-tradeoff%2F&title=CSPNet%20Paper%20Walkthrough%3A%20Just%20Better%2C%20No%20Tradeoffs)\n * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fcspnet-paper-walkthrough-just-better-no-tradeoff%2F&title=CSPNet%20Paper%20Walkthrough%3A%20Just%20Better%2C%20No%20Tradeoffs)\n * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fcspnet-paper-walkthrough-just-better-no-tradeoff%2F&text=CSPNet%20Paper%20Walkthrough%3A%20Just%20Better%2C%20No%20Tradeoffs)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n * [![](../images/002.jpeg)](https://towardsdatascience.com/how-to-create-a-celebrity-look-alike-demo-and-run-inference-on-an-npu-1e4e9de660cc/)\n## [How to create a celebrity-look-alike demo and run inference on an NPU](https://towardsdatascience.com/how-to-create-a-celebrity-look-alike-demo-and-run-inference-on-an-npu-1e4e9de660cc/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nFull detailed explanation on how to create, transform, and infer a celebrity-look-alike demo with TensorFlow on an i.MX 8M Plus NPU...\n[Jan Werth](https://towardsdatascience.com/author/janwerth/)\nDecember 9, 202015 min read\n * [![](../images/003.jpeg)](https://towardsdatascience.com/nlp-gaining-insights-from-text-reviews-94ef955c58c0/)\n## [NLP: Gaining insights from text reviews](https://towardsdatascience.com/nlp-gaining-insights-from-text-reviews-94ef955c58c0/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nEvery time we express ourselves, either verbally or in written text, these expressions carry a lot of information. What subjects we talk...\n[Fredrik Olsson](https://towardsdatascience.com/author/fredrik-olsson/)\nDecember 17, 202011 min read\n * [![](../images/004.jpeg)](https://towardsdatascience.com/how-to-use-large-language-models-llm-in-your-own-domains-b4dff2d08464/)\n## [How to Use Large Language Models (LLM) in Your Own Domains](https://towardsdatascience.com/how-to-use-large-language-models-llm-in-your-own-domains-b4dff2d08464/)\n[ChatGPT](https://towardsdatascience.com/category/artificial-intelligence/chatgpt/)\nCanonical and research-proven techniques to adapt large language models to domain specific tasks and the intuition of their effectiveness.\n[Eileen Pangu](https://towardsdatascience.com/author/eileen-code4fun/)\nMarch 20, 20239 min read\n * [![](../images/005.jpg)](https://towardsdatascience.com/the-machine-learning-lessons-ive-learned-this-month-4/)\n## [The Machine Learning Lessons I’ve Learned This Month](https://towardsdatascience.com/the-machine-learning-lessons-ive-learned-this-month-4/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nFebruary 2026: exchange with others, documentation, and MLOps\n[Pascal Janetzky](https://towardsdatascience.com/author/pascaljanetzky/)\nMarch 2, 20265 min read\n * [![](../images/006.jpg)](https://towardsdatascience.com/stacked-autoencoders-f0a4391ae282/)\n## [Stacked Autoencoders.](https://towardsdatascience.com/stacked-autoencoders-f0a4391ae282/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nExtract important features from data using deep learning.\n[Rajas Bakshi](https://towardsdatascience.com/author/rajas-bakshi/)\nJune 28, 20216 min read\n * [![](../images/007.png)](https://towardsdatascience.com/the-map-of-meaning-how-embedding-models-understand-human-language/)\n## [The Map of Meaning: How Embedding Models “Understand” Human Language](https://towardsdatascience.com/the-map-of-meaning-how-embedding-models-understand-human-language/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nA Non-Technical Deep Dive into How AI Uses Digital Fingerprints to Organize the World’s Information.\n[Gustavo Santos](https://towardsdatascience.com/author/gustavorsantos/)\nMarch 31, 202611 min read\n * [![](../images/008.png)](https://towardsdatascience.com/automatic-text-summarization-simplified-3b7c10c4093a/)\n## [Automatic Text Summarization : Simplified](https://towardsdatascience.com/automatic-text-summarization-simplified-3b7c10c4093a/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nTake a peek into the world of Automatic Text Summarization\n[Prakhar Ganesh](https://towardsdatascience.com/author/prakhargannu/)\nJune 13, 20195 min read\n * [![](../images/009.png)](https://towardsdatascience.com/fluorescent-neuronal-cells-dataset-part-iii-287c2a4f1a22/)\n## [Fluorescent Neuronal Cells dataset - part III](https://towardsdatascience.com/fluorescent-neuronal-cells-dataset-part-iii-287c2a4f1a22/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nBenchmark metrics for evaluation\n[Luca Clissa](https://towardsdatascience.com/author/luca.clissa/)\nDecember 13, 20225 min read\n * [![](../images/010.png)](https://towardsdatascience.com/numpy-cheatsheet-for-essential-functions-python-2e7d8618d688/)\n## [Cheatsheet for NumPy: Essential and Lesser-Known Functions](https://towardsdatascience.com/numpy-cheatsheet-for-essential-functions-python-2e7d8618d688/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nLearn various essential functions of NumPy which you might be unaware of (with code)\n[Shantnu & Kartik](https://towardsdatascience.com/author/beginningofthefuture/)\nSeptember 3, 20208 min read\n\n\n * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n * [X](https://x.com/TDataScience)\n * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n * [Threads](https://www.threads.net/@towardsdatascience)\n * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n", "metadata": { "title": "CSPNet Paper Walkthrough: Just Better, No Tradeoffs | Towards Data Science", "description": "A review of the Cross-Stage Partial Network paper  —  and a from-scratch PyTorch implementation", "keywords": null, "author": "Muhammad Ardi Putra", "og:locale": "en_US", "og:type": "article", "og:title": "CSPNet Paper Walkthrough: Just Better, No Tradeoffs", "og:description": "A review of the Cross-Stage Partial Network paper  —  and a from-scratch PyTorch implementation", "og:url": "https://towardsdatascience.com/cspnet-paper-walkthrough-just-better-no-tradeoff/", "og:site_name": "Towards Data Science", "og:image": "https://assets.insightmediagroup.io/media/wp-content/uploads/2026/05/0_bDlIU5aTyIODYvdP.jpg", "og:image:width": "960", "og:image:height": "625", "og:image:type": "image/jpeg", "twitter:card": "summary_large_image", "twitter:site": "@TDataScience", "twitter:creator": "@TDataScience", "twitter:label1": "Written by", "twitter:data1": "Muhammad Ardi Putra", "twitter:label2": "Est. reading time", "twitter:data2": "23 min read", "article:published_time": "2026-05-03T09:00:07.000Z", "article:modified_time": "2026-05-03T09:00:07.000Z", "article:author": "https://towardsdatascience.com/author/muhammad_ardi/" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 202, "error": "" }, "code_blocks": [ { "language": "txt", "file": "attachments/0d18c97807dc/code/block_000.txt", "chars": 149 }, { "language": "txt", "file": "attachments/0d18c97807dc/code/block_001.txt", "chars": 1307 }, { "language": "txt", "file": "attachments/0d18c97807dc/code/block_002.txt", "chars": 104 }, { "language": "txt", "file": "attachments/0d18c97807dc/code/block_003.txt", "chars": 193 }, { "language": "txt", "file": "attachments/0d18c97807dc/code/block_004.txt", "chars": 608 }, { "language": "txt", "file": "attachments/0d18c97807dc/code/block_005.txt", "chars": 117 }, { "language": "txt", "file": "attachments/0d18c97807dc/code/block_006.txt", "chars": 379 }, { "language": "txt", "file": "attachments/0d18c97807dc/code/block_007.txt", "chars": 764 }, { "language": "txt", "file": "attachments/0d18c97807dc/code/block_008.txt", "chars": 203 }, { "language": "txt", "file": "attachments/0d18c97807dc/code/block_009.txt", "chars": 124 }, { "language": "txt", "file": "attachments/0d18c97807dc/code/block_010.txt", "chars": 811 }, { "language": "txt", "file": "attachments/0d18c97807dc/code/block_011.txt", "chars": 205 }, { "language": "txt", "file": "attachments/0d18c97807dc/code/block_012.txt", "chars": 130 }, { "language": "txt", "file": "attachments/0d18c97807dc/code/block_013.txt", "chars": 3704 }, { "language": "txt", "file": "attachments/0d18c97807dc/code/block_014.txt", "chars": 411 }, { "language": "txt", "file": "attachments/0d18c97807dc/code/block_015.txt", "chars": 3081 }, { "language": "txt", "file": "attachments/0d18c97807dc/code/block_016.txt", "chars": 95 }, { "language": "txt", "file": "attachments/0d18c97807dc/code/block_017.txt", "chars": 4938 } ], "images": [ { "status": "ok", "url": "https://cdn-cookieyes.com/assets/images/revisit.svg", "bytes": 2339, "reused": false, "content_type": "image/svg+xml", "local_path": "attachments/0d18c97807dc/images/body-df29beacd7496220cc94.svg", "src": "https://cdn-cookieyes.com/assets/images/revisit.svg", "alt": "Revisit consent button", "downloaded": true, "discovery": "saved_markdown" }, { "status": "ok", "url": "https://cdn-cookieyes.com/assets/images/close.svg", "bytes": 1345, "reused": false, "content_type": "image/svg+xml", "local_path": "attachments/0d18c97807dc/images/body-2159bb67f3de8ad870eb.svg", "src": "https://cdn-cookieyes.com/assets/images/close.svg", "alt": "", "downloaded": true, "discovery": "saved_markdown" }, { "status": "ok", "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2026/05/0_bDlIU5aTyIODYvdP.jpg", "bytes": 204737, "reused": true, "local_path": "attachments/0d18c97807dc/images/000.jpg", "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2026/05/0_bDlIU5aTyIODYvdP.jpg", "alt": "", "downloaded": true, "discovery": "saved_markdown" }, { "status": "ok", "url": "https://assets.insightmediagroup.io/media/max/800/1*r3K1uSN4xpLydqdb_D6UAw.png", "bytes": 67598, "reused": true, "local_path": "attachments/0d18c97807dc/images/001.png", "src": "https://assets.insightmediagroup.io/media/max/800/1*r3K1uSN4xpLydqdb_D6UAw.png", "alt": "", "downloaded": true, "discovery": "saved_markdown" }, { "status": "ok", "url": "https://assets.insightmediagroup.io/media/max/2560/1*6CqTSMukla_HQmdqTCCHaQ.png", "bytes": 190531, "reused": false, "content_type": "image/png", "local_path": "attachments/0d18c97807dc/images/body-589390aa42b2500aabd9.png", "src": "https://assets.insightmediagroup.io/media/max/2560/1*6CqTSMukla_HQmdqTCCHaQ.png", "alt": "", "downloaded": true, "discovery": "saved_markdown" }, { "status": "ok", "url": "https://assets.insightmediagroup.io/media/max/800/1*na__RIorqsVORTSPwIU9Lg.png", "bytes": 90958, "reused": false, "content_type": "image/png", "local_path": "attachments/0d18c97807dc/images/body-4ee173ccde6a609db6b1.png", "src": "https://assets.insightmediagroup.io/media/max/800/1*na__RIorqsVORTSPwIU9Lg.png", "alt": "", "downloaded": true, "discovery": "saved_markdown" }, { "status": "ok", "url": "https://assets.insightmediagroup.io/media/max/800/1*L4zpUsYGFdX2FxCvlK00LA.png", "bytes": 119522, "reused": false, "content_type": "image/png", "local_path": "attachments/0d18c97807dc/images/body-209a358dbcfc042a5c31.png", "src": "https://assets.insightmediagroup.io/media/max/800/1*L4zpUsYGFdX2FxCvlK00LA.png", "alt": "", "downloaded": true, "discovery": "saved_markdown" }, { "status": "ok", "url": "https://assets.insightmediagroup.io/media/max/800/1*GbC5tOTX9HTdsRiwTAzvog.png", "bytes": 194184, "reused": false, "content_type": "image/png", "local_path": "attachments/0d18c97807dc/images/body-c0f1461f43ffef414fb9.png", "src": "https://assets.insightmediagroup.io/media/max/800/1*GbC5tOTX9HTdsRiwTAzvog.png", "alt": "", "downloaded": true, "discovery": "saved_markdown" }, { "status": "ok", "url": "https://assets.insightmediagroup.io/media/max/800/1*MWyqiAvgRayU8tOok3aScg.png", "bytes": 19750, "reused": false, "content_type": "image/png", "local_path": "attachments/0d18c97807dc/images/body-beee74a750dfb448a8b8.png", "src": "https://assets.insightmediagroup.io/media/max/800/1*MWyqiAvgRayU8tOok3aScg.png", "alt": "", "downloaded": true, "discovery": "saved_markdown" }, { "status": "ok", "url": "https://assets.insightmediagroup.io/media/max/800/1*Yh9M2MluucSRM_G-Gi96ow.png", "bytes": 23170, "reused": false, "content_type": "image/png", "local_path": "attachments/0d18c97807dc/images/body-c8d85a5093f98fbc101b.png", "src": "https://assets.insightmediagroup.io/media/max/800/1*Yh9M2MluucSRM_G-Gi96ow.png", "alt": "", "downloaded": true, "discovery": "saved_markdown" }, { "status": "ok", "url": "https://assets.insightmediagroup.io/media/max/1200/1*IjJnHGQpVZUAorEupAjjbg.png", "bytes": 191932, "reused": false, "content_type": "image/png", "local_path": "attachments/0d18c97807dc/images/body-ee91dbedfab1eba3e6ec.png", "src": "https://assets.insightmediagroup.io/media/max/1200/1*IjJnHGQpVZUAorEupAjjbg.png", "alt": "", "downloaded": true, "discovery": "saved_markdown" }, { "status": "ok", "url": "https://assets.insightmediagroup.io/media/1787657598287_taxhya.jpeg", "bytes": 183845, "reused": false, "content_type": "image/jpeg", "local_path": "attachments/0d18c97807dc/images/body-59e1c17daab1fcb582c5.jpeg", "src": "https://assets.insightmediagroup.io/media/1787657598287_taxhya.jpeg", "alt": "", "downloaded": true, "discovery": "saved_markdown" }, { "status": "ok", "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2020/12/1ob_s3G7Kff-svStb3bsolA.jpeg", "bytes": 282512, "reused": true, "local_path": "attachments/0d18c97807dc/images/002.jpeg", "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2020/12/1ob_s3G7Kff-svStb3bsolA.jpeg", "alt": "", "downloaded": true, "discovery": "saved_markdown" }, { "status": "ok", "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2020/12/13WhyN7c9c_3urpV0cndGiQ.jpeg", "bytes": 674733, "reused": true, "local_path": "attachments/0d18c97807dc/images/003.jpeg", "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2020/12/13WhyN7c9c_3urpV0cndGiQ.jpeg", "alt": "", "downloaded": true, "discovery": "saved_markdown" }, { "status": "ok", "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2023/03/1W8Iwg-wftbeMhwRNc0noUQ-scaled.jpeg", "bytes": 285918, "reused": true, "local_path": "attachments/0d18c97807dc/images/004.jpeg", "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2023/03/1W8Iwg-wftbeMhwRNc0noUQ-scaled.jpeg", "alt": "", "downloaded": true, "discovery": "saved_markdown" }, { "status": "ok", "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/07/el-youbi-akram-GjXyGXbLyeE-unsplash-scaled-1.jpg", "bytes": 226608, "reused": true, "local_path": "attachments/0d18c97807dc/images/005.jpg", "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/07/el-youbi-akram-GjXyGXbLyeE-unsplash-scaled-1.jpg", "alt": "", "downloaded": true, "discovery": "saved_markdown" }, { "status": "ok", "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/06/0IMJzSuq7Le49kZbd.jpg", "bytes": 54375, "reused": true, "local_path": "attachments/0d18c97807dc/images/006.jpg", "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/06/0IMJzSuq7Le49kZbd.jpg", "alt": "", "downloaded": true, "discovery": "saved_markdown" }, { "status": "ok", "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2026/03/blog2.png", "bytes": 329670, "reused": true, "local_path": "attachments/0d18c97807dc/images/007.png", "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2026/03/blog2.png", "alt": "", "downloaded": true, "discovery": "saved_markdown" }, { "status": "ok", "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2019/06/1r_ZHEI90PzPnyaHZXI_org.png", "bytes": 35423, "reused": true, "local_path": "attachments/0d18c97807dc/images/008.png", "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2019/06/1r_ZHEI90PzPnyaHZXI_org.png", "alt": "", "downloaded": true, "discovery": "saved_markdown" }, { "status": "ok", "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2022/12/1m_Pg1RFBKAoCjDl6G409Vw.png", "bytes": 717490, "reused": true, "local_path": "attachments/0d18c97807dc/images/009.png", "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2022/12/1m_Pg1RFBKAoCjDl6G409Vw.png", "alt": "", "downloaded": true, "discovery": "saved_markdown" }, { "status": "ok", "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2020/09/1bx9qfZE5gb2GBmHJk9xnhA.png", "bytes": 658110, "reused": true, "local_path": "attachments/0d18c97807dc/images/010.png", "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2020/09/1bx9qfZE5gb2GBmHJk9xnhA.png", "alt": "", "downloaded": true, "discovery": "saved_markdown" } ], "audio": [], "videos": [ { "url": "https://www.youtube.com/c/TowardsDataScience", "kind": "channel_reference", "status": "reference_only" } ], "links": [ { "type": "internal", "href": "https://towardsdatascience.com/" }, { "type": "internal", "href": "https://towardsdatascience.com/latest/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/editors-pick/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/the-variable/" }, { "type": "internal", "href": "https://towardsdatascience.com/shipai/" }, { "type": "internal", "href": "https://towardsdatascience.com/submissions/" }, { "type": "internal", "href": "https://towardsdatascience.com/category/artificial-intelligence/deep-learning/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/muhammad_ardi/" }, { "type": "internal", "href": "https://towardsdatascience.com/densenet-paper-walkthrough-all-connected/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/deep-dives/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/artificial-intelligence/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/cnn/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/machine-learning/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/neural-network/" }, { "type": "internal", "href": "https://towardsdatascience.com/how-to-create-a-celebrity-look-alike-demo-and-run-inference-on-an-npu-1e4e9de660cc/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/janwerth/" }, { "type": "internal", "href": "https://towardsdatascience.com/nlp-gaining-insights-from-text-reviews-94ef955c58c0/" }, { "type": "internal", "href": "https://towardsdatascience.com/category/artificial-intelligence/machine-learning/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/fredrik-olsson/" }, { "type": "internal", "href": "https://towardsdatascience.com/how-to-use-large-language-models-llm-in-your-own-domains-b4dff2d08464/" }, { "type": "external", "href": "https://write.insightmediagroup.io" }, { "type": "external", "href": "https://www.linkedin.com/company/towards-data-science/" }, { "type": "external", "href": "https://x.com/TDataScience" }, { "type": "external", "href": "https://unsplash.com/@mihaly_koles" }, { "type": "external", "href": "https://roadmap.sh/machine-learning" }, { "type": "external", "href": "https://arxiv.org/abs/1911.11929" }, { "type": "external", "href": "https://arxiv.org/abs/1608.06993" }, { "type": "external", "href": "https://medium.com/u/9801a58700ac" }, { "type": "external", "href": "https://github.com/WongKinYiu/CrossStagePartialNetworks" }, { "type": "external", "href": "https://github.com/MuhammadArdiPutra/medium_articles/blob/main/Deep%20Learning%20From%20Scratch/CSPNet.ipynb" }, { "type": "external", "href": "https://www.facebook.com/sharer/sharer.php?title=CSPNet+Paper+Walkthrough%3A+Just+Better%2C+No+Tradeoffs&u=https%3A%2F%2Ftowardsdatascience.com%2Fcspnet-paper-walkthrough-just-better-no-tradeoff%2F" }, { "type": "external", "href": "https://www.linkedin.com/shareArticle?mini=true&title=CSPNet+Paper+Walkthrough%3A+Just+Better%2C+No+Tradeoffs&url=https%3A%2F%2Ftowardsdatascience.com%2Fcspnet-paper-walkthrough-just-better-no-tradeoff%2F" }, { "type": "external", "href": "https://x.com/share?text=CSPNet+Paper+Walkthrough%3A+Just+Better%2C+No+Tradeoffs&url=https%3A%2F%2Ftowardsdatascience.com%2Fcspnet-paper-walkthrough-just-better-no-tradeoff%2F" }, { "type": "external", "href": "https://www.youtube.com/c/TowardsDataScience" }, { "type": "external", "href": "https://www.threads.net/@towardsdatascience" }, { "type": "external", "href": "https://bsky.app/profile/towardsdatascience.com" }, { "type": "external", "href": "https://thenewstack.io" }, { "type": "external", "href": "https://roadmap.sh" }, { "type": "external", "href": "https://insightmediagroup.io/creator-network" } ], "repair_version": 2, "repaired_at": "2026-10-20T17:29:27Z", "source_url": "https://towardsdatascience.com/cspnet-paper-walkthrough-just-better-no-tradeoff/", "delivery_status": "unverified", "discovery_status": "http_403", "missing_reason": "http_403", "repair_error": "", "actual_video_files": [], "expected_saved_body_images": 21, "image_discovery": "saved_markdown", "source_body_markdown": "![Revisit consent button](https://cdn-cookieyes.com/assets/images/revisit.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](https://cdn-cookieyes.com/assets/images/close.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n * Cookie\nBCTempID\n * Duration\n10 minutes\n * Description\nNo description available.\n\n\n * Cookie\n__cf_bm\n * Duration\n1 hour\n * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n * Cookie\nAWSALBCORS\n * Duration\n7 days\n * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n * Cookie\n_cfuvid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n * Cookie\nli_gc\n * Duration\n6 months\n * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n * Cookie\n__hssrc\n * Duration\nsession\n * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n * Cookie\n__hssc\n * Duration\n1 hour\n * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n * Cookie\nwpEmojiSettingsSupports\n * Duration\nsession\n * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n * Cookie\nBCSessionID\n * Duration\n1 year 1 month 4 days\n * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n * Cookie\n_octo\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nlogged_in\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\n__Secure-YEC\n * Duration\npast\n * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n * Cookie\n__eoi\n * Duration\n6 months\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTGCORS\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\nlogin-status-p\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTG\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\ncsrf_token\n * Duration\nsession\n * Description\nNo description available.\n\n\n * Cookie\ntoken_v2\n * Duration\n1 day\n * Description\nDescription is currently not available.\n\n\n * Cookie\nD\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nPHPSESSID\n * Duration\nsession\n * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n * Cookie\nVISITOR_PRIVACY_METADATA\n * Duration\n6 months\n * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n * Cookie\ncookietest\n * Duration\nsession\n * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n * Cookie\n__Host-airtable-session\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\n__Host-airtable-session.sig\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\nm\n * Duration\n1 year 1 month 4 days\n * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n * Cookie\nBIGipServer*\n * Duration\nsession\n * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n * Cookie\n__cfruid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n * Cookie\n_GRECAPTCHA\n * Duration\n6 months\n * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n * Cookie\n__Secure-YNID\n * Duration\n6 months\n * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n * Cookie\ncookieyes-consent\n * Duration\n1 year\n * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n * Cookie\nlidc\n * Duration\n1 day\n * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n * Cookie\nbrw\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nbrwConsent\n * Duration\n5 minutes\n * Description\nDescription is currently not available.\n\n\n * Cookie\nWMF-Uniq\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nloom_anon_comment\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nloom_referral_video\n * Duration\nsession\n * Description\nDescription is currently not available.\n\n\n * Cookie\nVISITOR_INFO1_LIVE\n * Duration\n6 months\n * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n * Cookie\nyt-remote-connected-devices\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n * Duration\nNever Expires\n * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n * Cookie\nyt-remote-device-id\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-name\n * Duration\nsession\n * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\nyt-remote-fast-check-period\n * Duration\nsession\n * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-app\n * Duration\nsession\n * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n * Cookie\nyt-remote-cast-available\n * Duration\nsession\n * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n * Cookie\nyt-remote-cast-installed\n * Duration\nsession\n * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\ncp_session\n * Duration\n3 months\n * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n * Cookie\nloid\n * Duration\n1 year 1 month 4 days\n * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n * Cookie\n__hstc\n * Duration\n6 months\n * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n * Cookie\nhubspotutk\n * Duration\n6 months\n * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n * Cookie\n_ga\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n * Cookie\n_ga_*\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n * Cookie\n__Host-psifi.analyticsTrace\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n__Host-psifi.analyticsTraceV2\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n_gh_sess\n * Duration\nsession\n * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n * Cookie\nYSC\n * Duration\nsession\n * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n * Cookie\najs_anonymous_id\n * Duration\n1 year\n * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n * Cookie\nvuid\n * Duration\n1 year 1 month 4 days\n * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n * Cookie\nAWSALB\n * Duration\n7 days\n * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n * Cookie\nacq\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nacq.sig\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nptc\n * Duration\n2 years\n * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n * Cookie\nmuc_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n * Cookie\nguest_id_marketing\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\nguest_id_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\npersonalization_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n * Cookie\nguest_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n * Cookie\nbcookie\n * Duration\n1 year\n * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n * Cookie\n__Secure-ROLLOUT_TOKEN\n * Duration\n6 months\n * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n * Cookie\nyt.innertube::nextId\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nyt.innertube::requests\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nsession_tracker\n * Duration\nsession\n * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n * Cookie\nedgebucket\n * Duration\nsession\n * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n * Cookie\ndid\n * Duration\n1 year\n * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\n# CSPNet Paper Walkthrough: Just Better, No Tradeoffs\nA review of the Cross-Stage Partial Network paper — and a from-scratch PyTorch implementation\n[Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\nMay 3, 202623 min read\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2026/05/0_bDlIU5aTyIODYvdP.jpg)\nPhoto by [Mihály Köles](https://unsplash.com/@mihaly_koles?utm_source=medium&utm_medium=referral) on Unsplash\nHow do you make your CNN-based model more lightweight? Just take the smaller version of that model, right? Like with ResNet, for instance, if ResNet-152 feels too heavy, why not just use ResNet-101? Or in the case of DenseNet, why not go with DenseNet-121 rather than DenseNet-169? — Yes, that’s true, but you would have to sacrifice some accuracy for that. Basically, if you want a lighter model then you should expect your accuracy to drop as well. \nNow, what if I told you about a model that’s more lightweight than its base but can still compete on accuracy? Meet CSPNet (Cross Stage Partial Network). You’ll be surprised that it can effectively reduce computational complexity while maintaining high accuracy — no tradeoff! In this article we are going to talk about the CSPNet architecture, including how it works and how to implement it from scratch.\n···\n## A Brief History of CSPNet\nCSPNet was first introduced in a paper titled “ _CSPNet: A New Backbone That Can Enhance Learning Capability of CNN_ ” written by Wang _et al._ back in November 2019 [1]. CSPNet was originally proposed to address the limitations of DenseNet. Despite already being computationally cheaper than ResNet, the authors thought that the computation of DenseNet itself is still considered expensive. Take a look at the main building block of a DenseNet in Figure 1 below to understand why.\nLearn this step by step with the interactive [Machine Learning](https://roadmap.sh/machine-learning) roadmap.\n![](https://assets.insightmediagroup.io/media/max/800/1*r3K1uSN4xpLydqdb_D6UAw.png)\nFigure 1. The main building block of a DenseNet model [2].\nIn a DenseNet building block — called _dense_ block — every convolution layer takes information from all previous layers, causing it to have a lot of redundant gradient information that makes training inefficient. We can think of it like a student taught by 5 different teachers for the same material. It’s actually good since the student can get multiple perspectives about that specific topic. However, at some point it becomes redundant and thus inefficient. In the case of DenseNet, we can see the deeper layers as students and all the tensors from shallower layers as teachers. In the example above, if we assume _H₄_ as our student, then the _x₀_ , _x₁_ , _x₂_ , and _x₃_ tensors act as the teachers. Here you can just imagine how that student would get overwhelmed by all that information!\nBefore we get into CSPNet, I actually have a whole separate article specifically talking about DenseNet (reference [3]), which I highly recommend you read if you want the full picture of how this architecture works.\n### Objectives\nThe objective of CSPNet is to enable a network to have cheaper computational complexity and better gradient combination. The reason for the latter is that most gradient information in DenseNet consists of duplicates of each other. It is important to note that CSPNet is not a standalone network. Instead, it is a new paradigm we apply to DenseNet.\nNow let’s take a look at Figure 2 below to see how CSPNet achieves its objectives. You can see the illustration on the left that the number of feature maps gradually increases as we get deeper into the network. If you have read my previous article about DenseNet, this is essentially something we can control through the _growth rate_ parameter, i.e., the number of feature maps produced by each convolution layer within a dense block. In fact, this increase in the number of feature maps is what the authors see as a computational bottleneck.\n![](https://assets.insightmediagroup.io/media/max/2560/1*6CqTSMukla_HQmdqTCCHaQ.png)\nFigure 2. Left: the original DenseNet building block (same as Figure 1). Right: The CSPNet version of the DenseNet building block (called CSPDenseNet) [1].\nBy applying the Cross Stage Partial mechanism, we can basically make the computation of a DenseNet cheaper. If we take a look at the illustration on the right, we can see that we have an additional branch coming out from _x₀_ that goes directly to the so-called _Partial Transition Layer_. There are at least two advantages we get with this mechanism, which are in accordance with the objectives I mentioned earlier. First, we can save lots of computations since the number of feature maps processed by the dense block is only half of the original one. And second, the gradient information becomes more diverse since we got an additional path with unprocessed feature maps that avoids the redundant gradient information. So in short, the idea of CSPNet eliminates the computational redundancy of DenseNet (through the skip-path) while at the same time still preserves its feature-reuse property (through the dense block).\n···\n## The Detailed CSPNet Architecture\nSpeaking of the details, the original feature map is first divided into two parts in channel-wise manner, where each of them will be processed in different paths. Suppose we got 64 input channels, the first 32 feature maps (_part 1_) will skip through all computations, whereas the remaining 32 (_part 2_) will be processed by a dense block. Although this splitting step is pretty easy, the merging step is actually not quite trivial. You can see in Figure 3 below that we got several different mechanisms to do so.\n![](https://assets.insightmediagroup.io/media/max/800/1*na__RIorqsVORTSPwIU9Lg.png)\nFigure 3. Several different ways to perform feature combination in CSPNet [1].\nIn the structure referred to as _fusion first_ (c), we concatenate the _part 1_ tensor with the _part 2_ tensor that has been processed by the dense block prior to passing them through the transition layer. So, option (c) is actually pretty straightforward to implement because the spatial dimension of the two tensors is exactly the same, allowing us to concatenate them easily.\nIn my previous article [3], I mentioned that the transition layer of a DenseNet is used to reduce both the spatial dimension and the number of channels. In fact, this property requires us to rethink how to implement the _fusion last_ (d) structure. This is essentially because the transition layer will cause the _part 2_ tensor to have a smaller spatial dimension than the _part 1_ tensor. So technically speaking, we need to either apply something like a pooling with a stride of 2 to the _part 1_ branch or simply omitting the downsampling operation in the transition layer. By doing this, the spatial dimension of the two tensors will be the same, and thus they are now concatenable.\nInstead of just using a single transition layer placed either before or after feature combination, the authors also proposed another method which they refer to as _CSPDenseNet_ (b). We can think of this as a combination of (c) and (d), where we got two transition layers placed before and after the tensor concatenation process. In this particular case, the first transition layer (the one placed in the _part 2_ branch) will perform channel reduction by _cross-channel pooling_ , i.e., a pooling layer that operates across channel dimension. Meanwhile, the second transition layer will perform both spatial downsampling and channel count reduction. So basically, in this approach we reduce the number of channels twice — well, at least that’s what I understand from the paper about the two transition layers, as the detailed processes within these layers are not explicitly discussed.\n### Experimental Results\nTalking about the experimental results regarding these feature combination mechanisms, it is explained in the paper that _fusion last_ (d) is better than _fusion first_ (c), where the former can significantly reduce computational complexity while only suffers from a very slight drop in accuracy. Variant (c) actually also reduces computational complexity, yet the degradation in accuracy is also significant. Authors found that variant (b) obtained an even better result than the two. Figure 4 below displays several experimental results showing how the three feature combination mechanisms performed compared to the base model. However, instead of using DenseNet, they somehow decided to use PeleeNet to compare these structures.\n![](https://assets.insightmediagroup.io/media/max/800/1*L4zpUsYGFdX2FxCvlK00LA.png)\nFigure 4. Performance comparison of the base PeleeNet (corresponds to (a) in Figure 3), CSPPeleeNet (b), PeleeNet with fusion first method (c), and PeleeNet with fusion last method (d) [1].\nBased on the above figure, we can see that the _CSP fusion last_ (green) indeed performs better compared to the _CSP fusion first_ (red). This is based on the fact that its accuracy only degrades by 0.1% from its base model while having 21% smaller computational complexity. Meanwhile, even though _CSP fusion first_ successfully reduces computational complexity by 26%, but the accuracy drop is pretty significant as it performs 1.5% worse than the base PeleeNet. And the most impressive structure is the _CSPPeleeNet_ variant (blue), i.e., the one that utilizes two transition layers. Here we can clearly see that although the computational complexity is reduced by 13%, the accuracy of the model actually improves by 0.2% — again, no tradeoff!\nNot only that, but the authors also tried to implement CSPNet on other backbone models. The results in Figure 5 below shows that the CSPNet structure successfully reduces the computational complexity of DenseNet -201-Elastic and ResNeXt-50 by 19% and 22%, respectively. It is interesting to see that the accuracy of the ResNeXt model improves despite the reduction in model complexity, which is in accordance with the result obtained by CSPPeleeNet in Figure 4.\n![](https://assets.insightmediagroup.io/media/max/800/1*GbC5tOTX9HTdsRiwTAzvog.png)\nFigure 5. Performance improvement of DenseNet-201-Elastic and ResNeXt-50 after implementing the CSPNet mechanism [1].\n### The Mathematical Expression of CSPDenseNet\nFor those who love math, here I got you some notations that you might find interesting to know. Figures 6 and 7 below display the mathematical expressions of DenseNet and CSPDenseNet blocks during the forward propagation phase.\nIn the DenseNet block, _x₁_ corresponds to the tensor produced by the first conv layer _w₁_ based on the input tensor _x₀_. Next, we concatenate the original tensor _x₀_ with _x₁_ and use them as the input for the _w₂_ layer (or to be more precise, _w_ is actually the weights of the conv layer, not the conv layer itself). We keep producing more feature maps and concatenating the existing ones as we get deeper into the network. In this way, we can basically say that the outputs of all previous layers become the input of the current layer.\n![](https://assets.insightmediagroup.io/media/max/800/1*MWyqiAvgRayU8tOok3aScg.png)\nFigure 6. The mathematical representation of forward propagation within a DenseNet block [1].\nThe case is different for CSPDenseNet. You can see in the notation below that we got _x₀’_ and _x₀’’_ , which we previously refer to as the _part 1_ and _part 2_. The _x₀’’_ tensor undergoes processing like the one in DenseNet block until we got _xₖ_. Next, the output of this dense block is then forwarded to the first transition layer, which is denoted as _wᴛ_. The resulting tensor _xᴛ_ is then concatenated with the _part 1_ tensor _x₀’_ before eventually being passed through the second transition layer _wᴜ_ to obtain the final output tensor _xᴜ_.\n![](https://assets.insightmediagroup.io/media/max/800/1*Yh9M2MluucSRM_G-Gi96ow.png)\nFigure 7. The mathematical expression of the forward propagation in CSPDenseNet block [1].\n···\n## CSPDenseNet Implementation\nNow let’s get even deeper into the CSPNet architecture by implementing it from scratch. Although we can basically apply the CSPNet structure to any backbone, here I am going to do that on the DenseNet model to match it with the illustrations and equations I showed you earlier. Figure 8 below displays what the complete DenseNet architecture looks like. Just remember that every single dense block in this architecture originally follows the _DenseNet_ structure in Figure 3a, and our objective here is to replace all these dense blocks with _CSPDenseNet_ block illustrated in Figure 3b.\n![](https://assets.insightmediagroup.io/media/max/1200/1*IjJnHGQpVZUAorEupAjjbg.png)\nFigure 8. The complete DenseNet architecture [2].\nThe first thing we do is to import the required modules and initialize the configurable parameters as shown in Codeblock 1. The `GROWTH` variable is the _growth rate_ parameter, which denotes the number of feature maps produced by each bottleneck within the dense block. Next, `CHANNEL_POOLING` is the parameter we use to adjust the behavior of the channel-pooling mechanism in our first transition layer. Here I set this parameter to 0.8, meaning that we will shrink the number of channels to 80% of its original channel count. The `COMPRESSION` parameter works similarly to the `CHANNEL_POOLING` variable, yet this one operates in the second transition layer. Finally, here we define the `REPEATS` list, which is used to set the number of _bottleneck_ blocks we will initialize within the dense block of each stage.\npython\n\n```\n# Codeblock 1\nimport torch\nimport torch.nn as nn\n\nGROWTH = 12\nCHANNEL_POOLING = 0.8\nCOMPRESSION = 0.5\nREPEATS = [6, 12, 24, 16]\n```\n\n### Bottleneck Block Implementation\nBelow is the implementation of the bottleneck block to be placed within the dense block. This `Bottleneck` class is exactly the same as the one I used in my DenseNet article [3]. I directly copy-pasted the code from there since we don’t need to modify this part at all. Just keep in mind that a bottleneck block comprises a 1×1 convolution followed by a 3×3 convolution.\npython\n\n```\n# Codeblock 2\nclass Bottleneck(nn.Module):\n def __init__(self, in_channels):\n super().__init__()\n \n self.relu = nn.ReLU()\n self.dropout = nn.Dropout(p=0.2)\n \n self.bn0 = nn.BatchNorm2d(num_features=in_channels)\n self.conv0 = nn.Conv2d(in_channels=in_channels, \n out_channels=GROWTH*4, \n kernel_size=1, \n padding=0, \n bias=False)\n \n self.bn1 = nn.BatchNorm2d(num_features=GROWTH*4)\n self.conv1 = nn.Conv2d(in_channels=GROWTH*4, \n out_channels=GROWTH, \n kernel_size=3, \n padding=1, \n bias=False)\n \n def forward(self, x):\n print(f'original\\t: {x.size()}')\n \n out = self.dropout(self.conv0(self.relu(self.bn0(x))))\n print(f'after conv0\\t: {out.size()}')\n \n out = self.dropout(self.conv1(self.relu(self.bn1(out))))\n print(f'after conv1\\t: {out.size()}')\n \n concatenated = torch.cat((out, x), dim=1) \n print(f'after concat\\t: {concatenated.size()}')\n \n return concatenated\n```\n\nThe following testing code simulates the first bottleneck block within the dense block. Remember that the very first conv layer in the architecture (the one with 7×7 kernel) produces 64 feature maps, but since in the case of CSPNet we only want to process half of them (the _part 2_ tensor), hence here we will test it with a tensor of 32 feature maps.\npython\n\n```\n# Codeblock 3\nbottleneck = Bottleneck(in_channels=32)\n\nx = torch.randn(1, 32, 56, 56)\nx = bottleneck(x)\n```\n\ntext\n\n```\n# Codeblock 3 Output\noriginal : torch.Size([1, 32, 56, 56])\nafter conv0 : torch.Size([1, 48, 56, 56])\nafter conv1 : torch.Size([1, 12, 56, 56])\nafter concat : torch.Size([1, 44, 56, 56])\n```\n\nYou can see in the resulting output above that the number of feature maps becomes 44 at the end of the process, where this number is obtained by adding the input channel count and the growth rate, i.e., 32 + 12 = 44. Again, you can just check out my DenseNet article [3] if you want to get a better understanding about this calculation.\n### Dense Block Implementation\nNow to create a sequence of bottleneck blocks easily, we can just wrap it inside the `DenseBlock` class in Codeblock 4 below. Later on, we can just specify the number of bottleneck blocks to be stacked through the `repeats `parameter. Again, this class is also copy-pasted from my DenseNet article, so I am not going to explain it any further.\npython\n\n```\n# Codeblock 4\nclass DenseBlock(nn.Module):\n def __init__(self, in_channels, repeats):\n super().__init__()\n self.bottlenecks = nn.ModuleList()\n \n for i in range(repeats):\n current_in_channels = in_channels + i * GROWTH\n self.bottlenecks.append(Bottleneck(in_channels=current_in_channels))\n \n def forward(self, x):\n print(f'original\\t\\t\\t: {x.size()}')\n \n for i, bottleneck in enumerate(self.bottlenecks):\n x = bottleneck(x)\n print(f'after bottleneck #{i}\\t\\t: {x.size()}')\n \n return x\n```\n\nIn order to check if our `DenseBlock` class works properly, we will test it using the Codeblock 5 below. Here I am trying to simulate the _part 2_ tensor processed by the first dense block, which contains a sequence of 6 bottleneck blocks.\npython\n\n```\n# Codeblock 5\ndense_block = DenseBlock(in_channels=32, repeats=6)\nx = torch.randn(1, 32, 56, 56)\n\nx = dense_block(x)\n```\n\nAnd below is what the output looks like. Here we can clearly see that each bottleneck block successfully increases the feature maps by 12.\ntext\n\n```\n# Codeblock 5 Output\noriginal : torch.Size([1, 32, 56, 56])\nafter bottleneck #0 : torch.Size([1, 44, 56, 56])\nafter bottleneck #1 : torch.Size([1, 56, 56, 56])\nafter bottleneck #2 : torch.Size([1, 68, 56, 56])\nafter bottleneck #3 : torch.Size([1, 80, 56, 56])\nafter bottleneck #4 : torch.Size([1, 92, 56, 56])\nafter bottleneck #5 : torch.Size([1, 104, 56, 56])\n```\n\n### First Transition\nRemember that the _CSPDenseNet_ variant in Figure 3b uses two transition layers. In this section we are going to discuss the first transition layer, i.e., the one used to process the tensor in the _part 2_ branch. Here we will not perform spatial downsampling, which is the reason why you don’t see any pooling layer within the `__init__()` method in Codeblock 6 below. Instead, here we will only perform cross-channel pooling, which can be perceived as a standard pooling operation yet is done across the channel dimension. To implement it, we can simply use a 1×1 convolution (`#(2)`) and specify the number of output channels we want (`#(1)`). We can think of it like this: in a spatial downsampling process, we can basically do that by using either pooling or a strided convolution layer, which in the latter case it will aggregate the pixel values with specific weightings from the local neighborhood. In the case of cross-channel pooling, since we don’t have a specific PyTorch layer for that, we can simply replace it with a pointwise convolution layer, which by doing so we can basically aggregate pixel values across the channel dimension.\npython\n\n```\n# Codeblock 6\nclass FirstTransition(nn.Module):\n def __init__(self, in_channels, out_channels):\n super().__init__()\n \n self.bn = nn.BatchNorm2d(num_features=in_channels)\n self.relu = nn.ReLU()\n self.conv = nn.Conv2d(in_channels=in_channels, \n out_channels=out_channels, #(1)\n kernel_size=1, #(2)\n padding=0,\n bias=False)\n self.dropout = nn.Dropout(p=0.2)\n \n def forward(self, x):\n print(f'original\\t\\t: {x.size()}')\n \n out = self.dropout(self.conv(self.relu(self.bn(x))))\n print(f'after first_transition\\t: {out.size()}')\n \n return out\n```\n\nThe result given in the Codeblock 5 Output shows that the _part 2_ tensor will have the shape of 104×56×56 after being processed by the dense block. Thus, in the testing code below I will use this tensor shape to simulate the first transition layer within that stage. To adjust the number of output channels, we can simply multiply the input channel count with the `CHANNEL_POOLING` variable we initialized earlier as shown at line `#(1)` in Codeblock 7 below.\npython\n\n```\n# Codeblock 7\nfirst_transition = FirstTransition(in_channels=104, \n out_channels=int(104*CHANNEL_POOLING)) #(1)\n\nx = torch.randn(1, 104, 56, 56)\nx = first_transition(x)\n```\n\nNow as the code above is run, we can see that the number of feature maps shrinks from 104 to 83 (80% of the original).\ntext\n\n```\n# Codeblock 7 Output\noriginal\t\t : torch.Size([1, 104, 56, 56])\nafter first_transition : torch.Size([1, 83, 56, 56])\n```\n\n### Second Transition\nThe structure of the second transition layer is quite a bit the same as the first one, except that here we also have an average pooling layer with a stride of 2 to reduce the spatial dimension by half (`#(1)`).\npython\n\n```\n# Codeblock 8\nclass SecondTransition(nn.Module):\n def __init__(self, in_channels, out_channels):\n super().__init__()\n \n self.bn = nn.BatchNorm2d(num_features=in_channels)\n self.relu = nn.ReLU()\n self.conv = nn.Conv2d(in_channels=in_channels, \n out_channels=out_channels, \n kernel_size=1, \n padding=0,\n bias=False)\n self.dropout = nn.Dropout(p=0.2)\n self.pool = nn.AvgPool2d(kernel_size=2, stride=2) #(1)\n \n def forward(self, x):\n print(f'original\\t\\t: {x.size()}')\n\n out = self.pool(self.dropout(self.conv(self.relu(self.bn(x)))))\n print(f'after second_transition\\t: {out.size()}')\n \n return out\n```\n\nRemember that the tensor coming into the second transition layer is a concatenation of the _part 1_ and the _part 2_ tensors. This is essentially the reason why in the testing code below I set this layer to accept 32 + 83 = 115 feature maps. Similar to the first transition layer, here we multiply this number of feature maps with the `COMPRESSION` variable (`#(1)`) to reduce the number of channels even further.\npython\n\n```\n# Codeblock 9\nsecond_transition = SecondTransition(in_channels=115, \n out_channels=int(115*COMPRESSION)) #(1)\n\nx = torch.randn(1, 115, 56, 56)\nx = second_transition(x)\n```\n\nIn the resulting output below we can see that the spatial dimension halves thanks to the average pooling layer. At the same time, the number of feature maps also decreases from 115 to 57 since we set the `COMPRESSION` parameter to 0.5.\ntext\n\n```\n# Codeblock 9 Output\noriginal : torch.Size([1, 115, 56, 56])\nafter second_transition : torch.Size([1, 57, 28, 28])\n```\n\n### The CSPDenseNet Model\nWith all the components ready, we can now build the entire CSPDenseNet architecture, which I break down in Codeblocks 10a, 10b, and 10c below. Let’s now focus on the Codeblock 10a first, where I initialize all the layers according to the structure given in Figure 8. Here you can see at line `#(1)` that we initialize a 7×7 convolution layer, which acts as the input layer of the network. This layer is then followed by a maxpooling layer (`#(2)`). These two layers use the stride of 2, meaning that the spatial dimensions of the input tensor will be reduced to one-fourth of its original size.\npython\n\n```\n# Codeblock 10a\nclass CSPDenseNet(nn.Module):\n def __init__(self):\n super().__init__()\n \n self.first_conv = nn.Conv2d(in_channels=3, #(1)\n out_channels=64, \n kernel_size=7, \n stride=2, \n padding=3, \n bias=False)\n self.first_pool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1) #(2)\n channel_count = 64\n \n \n \n ##### Stage 0\n self.dense_block_0 = DenseBlock(in_channels=channel_count//2, \n repeats=REPEATS[0])\n \n self.first_transition_0 = FirstTransition(in_channels=(channel_count//2)+(REPEATS[0]*GROWTH), \n out_channels=int(((channel_count//2)+(REPEATS[0]*GROWTH))*CHANNEL_POOLING))\n \n channel_count = (channel_count - (channel_count//2)) + int(((channel_count//2)+(REPEATS[0]*GROWTH))*CHANNEL_POOLING)\n \n self.second_transition_0 = SecondTransition(in_channels=channel_count, \n out_channels=int(channel_count*COMPRESSION))\n \n channel_count = int(channel_count*COMPRESSION)\n #####\n \n \n ##### Stage 1\n self.dense_block_1 = DenseBlock(in_channels=channel_count//2, \n repeats=REPEATS[1])\n \n self.first_transition_1 = FirstTransition(in_channels=(channel_count//2)+(REPEATS[1]*GROWTH), \n out_channels=int(((channel_count//2)+(REPEATS[1]*GROWTH))*CHANNEL_POOLING))\n \n channel_count = (channel_count - (channel_count//2)) + int(((channel_count//2)+(REPEATS[1]*GROWTH))*CHANNEL_POOLING)\n \n self.second_transition_1 = SecondTransition(in_channels=channel_count, \n out_channels=int(channel_count*COMPRESSION))\n \n channel_count = int(channel_count*COMPRESSION)\n #####\n \n \n ##### Stage 2\n self.dense_block_2 = DenseBlock(in_channels=channel_count//2, \n repeats=REPEATS[2])\n \n self.first_transition_2 = FirstTransition(in_channels=(channel_count//2)+(REPEATS[2]*GROWTH), \n out_channels=int(((channel_count//2)+(REPEATS[2]*GROWTH))*CHANNEL_POOLING))\n \n channel_count = (channel_count - (channel_count//2)) + int(((channel_count//2)+(REPEATS[2]*GROWTH))*CHANNEL_POOLING)\n \n self.second_transition_2 = SecondTransition(in_channels=channel_count, \n out_channels=int(channel_count*COMPRESSION))\n \n channel_count = int(channel_count*COMPRESSION)\n #####\n \n \n ##### Stage 3\n self.dense_block_3 = DenseBlock(in_channels=channel_count//2, \n repeats=REPEATS[3])\n \n self.first_transition_3 = FirstTransition(in_channels=(channel_count//2)+(REPEATS[3]*GROWTH), \n out_channels=int(((channel_count//2)+(REPEATS[3]*GROWTH))*CHANNEL_POOLING))\n \n channel_count = (channel_count - (channel_count//2)) + int(((channel_count//2)+(REPEATS[3]*GROWTH))*CHANNEL_POOLING)\n #####\n \n \n self.avgpool = nn.AdaptiveAvgPool2d(output_size=(1,1)) #(3)\n self.fc = nn.Linear(in_features=channel_count, out_features=1000) #(4)\n```\n\nStill with the above codeblock, here I group the layers I initialize based on the stage they belong to. Let’s now focus on the part I refer to as `Stage 0`. Here you can see that we got a dense block (`dense_block_0`) and the first transition layer (`first_transition_0`). These two components are responsible to process the _part 2_ tensor. Next, we initialize the second transition layer (`second_transition_0`), which is used to process the concatenation result of the _part 1_ and _part 2_ tensors. Since the channel count is dynamic depending on the `GROWTH`, `CHANNEL_POOLING`, `COMPRESSION`, and `REPEATS` variables, we need to keep track of the channel count after each step so that the model can adaptively adjust itself according to these variables. We do the same thing for all the remaining stages, except in `Stage 3` we don’t initialize the second transition layer since at that point we won’t reduce the channels and the spatial dimension any further. Instead, we will directly pass the concatenated `part 1` and `part 2` tensors to the average pooling (`#(3)`) and the classification (`#(4)`) layers. And that ends our discussion about the Codeblock 10a above.\nBefore we get into the `forward()` method, there is another function we need to create: `split_channels()`. As the name suggests, this function, which is written in Codeblock 10b below, is used to split a tensor into _part 1_ and _part 2_. The if-else statement here is used to check if the number of channels is odd or even. In fact, it would be very easy if the channel count is an even number as we can just divide them into two (`#(4)`). But if the channel count is odd, we need to manually determine the size of each part as seen at line `#(1)` and `#(2)` before eventually splitting them (`#(3)`).\npython\n\n```\n# Codeblock 10b\n def split_channels(self, x):\n\n channel_count = x.size(1)\n\n if channel_count%2 != 0:\n split_size_2 = channel_count // 2 #(1)\n split_size_1 = channel_count - split_size_2 #(2)\n return torch.split(x, [split_size_1, split_size_2], dim=1) #(3)\n\n else:\n return torch.split(x, channel_count // 2, dim=1) #(4)\n```\n\nAs we have finished defining the `__init__()` and the `split_channel()` methods, we can now implement the `forward()` method in Codeblock 10c below. Generally speaking, what we do here is simply forward the tensor sequentially. But now let’s pay attention to the part I refer to as `Stage 0`. Here you can see that after the tensor is passed through the `first_pool` layer (`#(1)`), we then split it into two using the `split_channels()` function we declared earlier (`#(2)`). From there, we now obtain the `part1` and `part2` tensors. We will leave the `part1` tensor as is all the way to the end of the stage. Meanwhile, for the `part2` tensor, we will process it with the dense block (`#(3)`) and the first transition layer (`#(4)`). Next, we concatenate the resulting tensor with the `part1` tensor to create the skip-connection (`#(5)`). And then, we finally pass it through the second transition layer (`#(6)`). The same steps are repeated for all stages until we eventually reach the output layer to make classification. Just remember that the `Stage 3` is quite different because here we don’t have the second transition layer.\npython\n\n```\n# Codeblock 10c\n def forward(self, x):\n print(f'original\\t\\t\\t: {x.size()}')\n \n x = self.first_conv(x)\n print(f'after first_conv\\t\\t: {x.size()}')\n \n x = self.first_pool(x) #(1)\n print(f'after first_pool\\t\\t: {x.size()}\\n')\n \n \n \n ##### Stage 0\n part1, part2 = self.split_channels(x) #(2)\n print(f'part1\\t\\t\\t\\t: {part1.size()}')\n print(f'part2\\t\\t\\t\\t: {part2.size()}')\n \n part2 = self.dense_block_0(part2) #(3)\n print(f'part2 after dense block 0\\t: {part2.size()}')\n \n part2 = self.first_transition_0(part2) #(4)\n print(f'part2 after first trans 0\\t: {part2.size()}')\n \n x = torch.cat((part1, part2), dim=1) #(5)\n print(f'after concatenate\\t\\t: {x.size()}')\n \n x = self.second_transition_0(x) #(6)\n print(f'after second transition 0\\t: {x.size()}\\n')\n \n \n \n ##### Stage 1\n part1, part2 = self.split_channels(x)\n print(f'part1\\t\\t\\t\\t: {part1.size()}')\n print(f'part2\\t\\t\\t\\t: {part2.size()}')\n \n part2 = self.dense_block_1(part2)\n print(f'part2 after dense block 1\\t: {part2.size()}')\n \n part2 = self.first_transition_1(part2)\n print(f'part2 after first trans 1\\t: {part2.size()}')\n \n x = torch.cat((part1, part2), dim=1)\n print(f'after concatenate\\t\\t: {x.size()}')\n \n x = self.second_transition_1(x)\n print(f'after second transition 1\\t: {x.size()}\\n')\n \n \n \n ##### Stage 2\n part1, part2 = self.split_channels(x)\n print(f'part1\\t\\t\\t\\t: {part1.size()}')\n print(f'part2\\t\\t\\t\\t: {part2.size()}')\n \n part2 = self.dense_block_2(part2)\n print(f'part2 after dense block 2\\t: {part2.size()}')\n \n part2 = self.first_transition_2(part2)\n print(f'part2 after first trans 2\\t: {part2.size()}')\n \n x = torch.cat((part1, part2), dim=1)\n print(f'after concatenate\\t\\t: {x.size()}')\n \n x = self.second_transition_2(x)\n print(f'after second transition 2\\t: {x.size()}\\n')\n \n \n \n ##### Stage 3\n part1, part2 = self.split_channels(x)\n print(f'part1\\t\\t\\t\\t: {part1.size()}')\n print(f'part2\\t\\t\\t\\t: {part2.size()}')\n \n part2 = self.dense_block_3(part2)\n print(f'part2 after dense block 2\\t: {part2.size()}')\n \n part2 = self.first_transition_3(part2)\n print(f'part2 after first trans 2\\t: {part2.size()}')\n \n x = torch.cat((part1, part2), dim=1)\n print(f'after concatenate\\t\\t: {x.size()}\\n')\n \n \n \n x = self.avgpool(x)\n print(f'after avgpool\\t\\t\\t: {x.size()}')\n \n x = torch.flatten(x, start_dim=1)\n print(f'after flatten\\t\\t\\t: {x.size()}')\n \n x = self.fc(x)\n print(f'after fc\\t\\t\\t: {x.size()}')\n \n return x\n```\n\nNow let’s test the CSPDenseNet class we just created by running the Codeblock 11 below. Here I use a dummy tensor of shape 3×224×224 to simulate a 224×224 RGB image passed through the network.\npython\n\n```\n# Codeblock 11\ncspdensenet = CSPDenseNet()\n\nx = torch.randn(1, 3, 224, 224)\nx = cspdensenet(x)\n```\n\nAnd below is what the output looks like. Here you can see that every time a tensor gets into a network, our `split_channels()` method correctly divides the tensor into two (`#(1–2)`). Then, the bottleneck block within each stage also correctly adds the number of channels of the _part 2_ tensor by 12 before eventually being passed through the first transition layer. The first transition layer itself successfully reduces the number of channels by 20% as seen at line `#(3)`, simulating the cross-channel pooling mechanism. Afterwards, the resulting tensor is then concatenated with the tensor from _part 1_ (`#(4)`) and passed through the second transition layer (`#(5)`) to further reduce the number of channels and halve the spatial dimension. We do the same thing for all stages until eventually we got the 1000-class prediction.\ntext\n\n```\n# Codeblock 11 Output\noriginal : torch.Size([1, 3, 224, 224])\nafter first_conv : torch.Size([1, 64, 112, 112])\nafter first_pool : torch.Size([1, 64, 56, 56])\n\npart1 : torch.Size([1, 32, 56, 56]) #(1)\npart2 : torch.Size([1, 32, 56, 56]) #(2)\nafter bottleneck #0 : torch.Size([1, 44, 56, 56])\nafter bottleneck #1 : torch.Size([1, 56, 56, 56])\nafter bottleneck #2 : torch.Size([1, 68, 56, 56])\nafter bottleneck #3 : torch.Size([1, 80, 56, 56])\nafter bottleneck #4 : torch.Size([1, 92, 56, 56])\nafter bottleneck #5 : torch.Size([1, 104, 56, 56])\npart2 after dense block 0 : torch.Size([1, 104, 56, 56])\npart2 after first trans 0 : torch.Size([1, 83, 56, 56]) #(3)\nafter concatenate : torch.Size([1, 115, 56, 56]) #(4)\nafter second transition 0 : torch.Size([1, 57, 28, 28]) #(5)\n\npart1 : torch.Size([1, 29, 28, 28])\npart2 : torch.Size([1, 28, 28, 28])\nafter bottleneck #0 : torch.Size([1, 40, 28, 28])\nafter bottleneck #1 : torch.Size([1, 52, 28, 28])\nafter bottleneck #2 : torch.Size([1, 64, 28, 28])\nafter bottleneck #3 : torch.Size([1, 76, 28, 28])\nafter bottleneck #4 : torch.Size([1, 88, 28, 28])\nafter bottleneck #5 : torch.Size([1, 100, 28, 28])\nafter bottleneck #6 : torch.Size([1, 112, 28, 28])\nafter bottleneck #7 : torch.Size([1, 124, 28, 28])\nafter bottleneck #8 : torch.Size([1, 136, 28, 28])\nafter bottleneck #9 : torch.Size([1, 148, 28, 28])\nafter bottleneck #10 : torch.Size([1, 160, 28, 28])\nafter bottleneck #11 : torch.Size([1, 172, 28, 28])\npart2 after dense block 1 : torch.Size([1, 172, 28, 28])\npart2 after first trans 1 : torch.Size([1, 137, 28, 28])\nafter concatenate : torch.Size([1, 166, 28, 28])\nafter second transition 1 : torch.Size([1, 83, 14, 14])\n\npart1 : torch.Size([1, 42, 14, 14])\npart2 : torch.Size([1, 41, 14, 14])\nafter bottleneck #0 : torch.Size([1, 53, 14, 14])\nafter bottleneck #1 : torch.Size([1, 65, 14, 14])\nafter bottleneck #2 : torch.Size([1, 77, 14, 14])\nafter bottleneck #3 : torch.Size([1, 89, 14, 14])\nafter bottleneck #4 : torch.Size([1, 101, 14, 14])\nafter bottleneck #5 : torch.Size([1, 113, 14, 14])\nafter bottleneck #6 : torch.Size([1, 125, 14, 14])\nafter bottleneck #7 : torch.Size([1, 137, 14, 14])\nafter bottleneck #8 : torch.Size([1, 149, 14, 14])\nafter bottleneck #9 : torch.Size([1, 161, 14, 14])\nafter bottleneck #10 : torch.Size([1, 173, 14, 14])\nafter bottleneck #11 : torch.Size([1, 185, 14, 14])\nafter bottleneck #12 : torch.Size([1, 197, 14, 14])\nafter bottleneck #13 : torch.Size([1, 209, 14, 14])\nafter bottleneck #14 : torch.Size([1, 221, 14, 14])\nafter bottleneck #15 : torch.Size([1, 233, 14, 14])\nafter bottleneck #16 : torch.Size([1, 245, 14, 14])\nafter bottleneck #17 : torch.Size([1, 257, 14, 14])\nafter bottleneck #18 : torch.Size([1, 269, 14, 14])\nafter bottleneck #19 : torch.Size([1, 281, 14, 14])\nafter bottleneck #20 : torch.Size([1, 293, 14, 14])\nafter bottleneck #21 : torch.Size([1, 305, 14, 14])\nafter bottleneck #22 : torch.Size([1, 317, 14, 14])\nafter bottleneck #23 : torch.Size([1, 329, 14, 14])\npart2 after dense block 2 : torch.Size([1, 329, 14, 14])\npart2 after first trans 2 : torch.Size([1, 263, 14, 14])\nafter concatenate : torch.Size([1, 305, 14, 14])\nafter second transition 2 : torch.Size([1, 152, 7, 7])\n\npart1 : torch.Size([1, 76, 7, 7])\npart2 : torch.Size([1, 76, 7, 7])\nafter bottleneck #0 : torch.Size([1, 88, 7, 7])\nafter bottleneck #1 : torch.Size([1, 100, 7, 7])\nafter bottleneck #2 : torch.Size([1, 112, 7, 7])\nafter bottleneck #3 : torch.Size([1, 124, 7, 7])\nafter bottleneck #4 : torch.Size([1, 136, 7, 7])\nafter bottleneck #5 : torch.Size([1, 148, 7, 7])\nafter bottleneck #6 : torch.Size([1, 160, 7, 7])\nafter bottleneck #7 : torch.Size([1, 172, 7, 7])\nafter bottleneck #8 : torch.Size([1, 184, 7, 7])\nafter bottleneck #9 : torch.Size([1, 196, 7, 7])\nafter bottleneck #10 : torch.Size([1, 208, 7, 7])\nafter bottleneck #11 : torch.Size([1, 220, 7, 7])\nafter bottleneck #12 : torch.Size([1, 232, 7, 7])\nafter bottleneck #13 : torch.Size([1, 244, 7, 7])\nafter bottleneck #14 : torch.Size([1, 256, 7, 7])\nafter bottleneck #15 : torch.Size([1, 268, 7, 7])\npart2 after dense block 2 : torch.Size([1, 268, 7, 7])\npart2 after first trans 2 : torch.Size([1, 214, 7, 7])\nafter concatenate : torch.Size([1, 290, 7, 7])\n\nafter avgpool : torch.Size([1, 290, 1, 1])\nafter flatten : torch.Size([1, 290])\nafter fc : torch.Size([1, 1000])\n```\n\n···\n## Ending\nAnd that’s it! We have successfully learned CSPNet and implemented it on DenseNet backbone. As I’ve mentioned earlier, we can actually use the idea of CSPNet to improve the performance of any other backbone models such as ResNet or ResNeXt. So here I challenge you to implement CSPNet on these models from scratch.\nTo be honest I cannot confirm that my implementation is 100% correct since the official GitHub repo [4] of the paper does not provide the PyTorch implementation — but that’s at least everything I understand from the manuscript. Please let me know if you find any mistake in the code or in my explanations. Thanks for reading, and see you again in my next article. Bye!\nBtw you can also find the code used in this article on my GitHub repo [5].\n···\n## References\n[1] Chien-Yao Wang _et al._ CSPnet: A New Backbone That Can Enhance Learning Capability of CNN. Arxiv. [Accessed October 1, 2025].\n[2] Gao Huang _et al._ Densely Connected Convolutional Networks. Arxiv. [Accessed September 18, 2025].\n[3] [Muhammad Ardi](https://medium.com/u/9801a58700ac). DenseNet Paper Walkthrough: All Connected. Towards Data Science. [Accessed April 26, 2026].\n[4] WongKinYiu. CrossStagePartialNetworks. GitHub. [Accessed October 1, 2025].\n[5] MuhammadArdiPutra. CSPNet. GitHub. [https://github.com/MuhammadArdiPutra/medium_articles/blob/main/DenseNet.ipynb](https://github.com/MuhammadArdiPutra/medium_articles/blob/main/Deep%20Learning%20From%20Scratch/CSPNet.ipynb) [Accessed October 1, 2025].\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](https://assets.insightmediagroup.io/media/1787657598287_taxhya.jpeg)\nWritten By\n## [Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\nDeep learning storyteller\n[See all from Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\n[Deep Dives](https://towardsdatascience.com/tag/deep-dives/)[Artificial Intelligence](https://towardsdatascience.com/tag/artificial-intelligence/)[Cnn](https://towardsdatascience.com/tag/cnn/)[Machine Learning](https://towardsdatascience.com/tag/machine-learning/)[Neural Network](https://towardsdatascience.com/tag/neural-network/)\nShare This Article\n * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fcspnet-paper-walkthrough-just-better-no-tradeoff%2F&title=CSPNet%20Paper%20Walkthrough%3A%20Just%20Better%2C%20No%20Tradeoffs)\n * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fcspnet-paper-walkthrough-just-better-no-tradeoff%2F&title=CSPNet%20Paper%20Walkthrough%3A%20Just%20Better%2C%20No%20Tradeoffs)\n * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fcspnet-paper-walkthrough-just-better-no-tradeoff%2F&text=CSPNet%20Paper%20Walkthrough%3A%20Just%20Better%2C%20No%20Tradeoffs)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/12/1ob_s3G7Kff-svStb3bsolA.jpeg)](https://towardsdatascience.com/how-to-create-a-celebrity-look-alike-demo-and-run-inference-on-an-npu-1e4e9de660cc/)\n## [How to create a celebrity-look-alike demo and run inference on an NPU](https://towardsdatascience.com/how-to-create-a-celebrity-look-alike-demo-and-run-inference-on-an-npu-1e4e9de660cc/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nFull detailed explanation on how to create, transform, and infer a celebrity-look-alike demo with TensorFlow on an i.MX 8M Plus NPU...\n[Jan Werth](https://towardsdatascience.com/author/janwerth/)\nDecember 9, 202015 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/12/13WhyN7c9c_3urpV0cndGiQ.jpeg)](https://towardsdatascience.com/nlp-gaining-insights-from-text-reviews-94ef955c58c0/)\n## [NLP: Gaining insights from text reviews](https://towardsdatascience.com/nlp-gaining-insights-from-text-reviews-94ef955c58c0/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nEvery time we express ourselves, either verbally or in written text, these expressions carry a lot of information. What subjects we talk...\n[Fredrik Olsson](https://towardsdatascience.com/author/fredrik-olsson/)\nDecember 17, 202011 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/03/1W8Iwg-wftbeMhwRNc0noUQ-scaled.jpeg)](https://towardsdatascience.com/how-to-use-large-language-models-llm-in-your-own-domains-b4dff2d08464/)\n## [How to Use Large Language Models (LLM) in Your Own Domains](https://towardsdatascience.com/how-to-use-large-language-models-llm-in-your-own-domains-b4dff2d08464/)\n[ChatGPT](https://towardsdatascience.com/category/artificial-intelligence/chatgpt/)\nCanonical and research-proven techniques to adapt large language models to domain specific tasks and the intuition of their effectiveness.\n[Eileen Pangu](https://towardsdatascience.com/author/eileen-code4fun/)\nMarch 20, 20239 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/07/el-youbi-akram-GjXyGXbLyeE-unsplash-scaled-1.jpg)](https://towardsdatascience.com/the-machine-learning-lessons-ive-learned-this-month-4/)\n## [The Machine Learning Lessons I’ve Learned This Month](https://towardsdatascience.com/the-machine-learning-lessons-ive-learned-this-month-4/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nFebruary 2026: exchange with others, documentation, and MLOps\n[Pascal Janetzky](https://towardsdatascience.com/author/pascaljanetzky/)\nMarch 2, 20265 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/06/0IMJzSuq7Le49kZbd.jpg)](https://towardsdatascience.com/stacked-autoencoders-f0a4391ae282/)\n## [Stacked Autoencoders.](https://towardsdatascience.com/stacked-autoencoders-f0a4391ae282/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nExtract important features from data using deep learning.\n[Rajas Bakshi](https://towardsdatascience.com/author/rajas-bakshi/)\nJune 28, 20216 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2026/03/blog2.png)](https://towardsdatascience.com/the-map-of-meaning-how-embedding-models-understand-human-language/)\n## [The Map of Meaning: How Embedding Models “Understand” Human Language](https://towardsdatascience.com/the-map-of-meaning-how-embedding-models-understand-human-language/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nA Non-Technical Deep Dive into How AI Uses Digital Fingerprints to Organize the World’s Information.\n[Gustavo Santos](https://towardsdatascience.com/author/gustavorsantos/)\nMarch 31, 202611 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2019/06/1r_ZHEI90PzPnyaHZXI_org.png)](https://towardsdatascience.com/automatic-text-summarization-simplified-3b7c10c4093a/)\n## [Automatic Text Summarization : Simplified](https://towardsdatascience.com/automatic-text-summarization-simplified-3b7c10c4093a/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nTake a peek into the world of Automatic Text Summarization\n[Prakhar Ganesh](https://towardsdatascience.com/author/prakhargannu/)\nJune 13, 20195 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2022/12/1m_Pg1RFBKAoCjDl6G409Vw.png)](https://towardsdatascience.com/fluorescent-neuronal-cells-dataset-part-iii-287c2a4f1a22/)\n## [Fluorescent Neuronal Cells dataset - part III](https://towardsdatascience.com/fluorescent-neuronal-cells-dataset-part-iii-287c2a4f1a22/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nBenchmark metrics for evaluation\n[Luca Clissa](https://towardsdatascience.com/author/luca.clissa/)\nDecember 13, 20225 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/09/1bx9qfZE5gb2GBmHJk9xnhA.png)](https://towardsdatascience.com/numpy-cheatsheet-for-essential-functions-python-2e7d8618d688/)\n## [Cheatsheet for NumPy: Essential and Lesser-Known Functions](https://towardsdatascience.com/numpy-cheatsheet-for-essential-functions-python-2e7d8618d688/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nLearn various essential functions of NumPy which you might be unaware of (with code)\n[Shantnu & Kartik](https://towardsdatascience.com/author/beginningofthefuture/)\nSeptember 3, 20208 min read\n\n\n * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n * [X](https://x.com/TDataScience)\n * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n * [Threads](https://www.threads.net/@towardsdatascience)\n * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n", "body_base_path": "attachments/0d18c97807dc/pages/", "body_markdown_path": "attachments/0d18c97807dc/pages/archived-body.md", "local_html": "attachments/0d18c97807dc/pages/archived-body.html", "body_source": "prior_capture" }