{ "source_file": "0930", "source_platform": "Medium", "author": "Muhammad Ardi Putra", "title_listed": "MobileNetV1 Paper Walkthrough: The Tiny Giant", "keywords": "", "domain": "图像处理", "content_type": "深度技术解析", "is_paid_flag": "否", "url": "https://towardsdatascience.com/the-tiny-giant-mobilenetv1/", "publish_date": "2025-09-04", "corpus_stage": "", "notes": "", "id": "c3cb7fd8c186", "collected_at": "2026-10-02T02:42:56", "title": "MobileNetV1 Paper Walkthrough: The Tiny Giant | Towards Data Science", "body_markdown": "![Revisit consent button](../images/body-df29beacd7496220cc94.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](../images/body-2159bb67f3de8ad870eb.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n * Cookie\nBCTempID\n * Duration\n10 minutes\n * Description\nNo description available.\n\n\n * Cookie\n__cf_bm\n * Duration\n1 hour\n * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n * Cookie\nAWSALBCORS\n * Duration\n7 days\n * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n * Cookie\n_cfuvid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n * Cookie\nli_gc\n * Duration\n6 months\n * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n * Cookie\n__hssrc\n * Duration\nsession\n * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n * Cookie\n__hssc\n * Duration\n1 hour\n * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n * Cookie\nwpEmojiSettingsSupports\n * Duration\nsession\n * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n * Cookie\nBCSessionID\n * Duration\n1 year 1 month 4 days\n * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n * Cookie\n_octo\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nlogged_in\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\n__Secure-YEC\n * Duration\npast\n * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n * Cookie\n__eoi\n * Duration\n6 months\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTGCORS\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\nlogin-status-p\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTG\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\ncsrf_token\n * Duration\nsession\n * Description\nNo description available.\n\n\n * Cookie\ntoken_v2\n * Duration\n1 day\n * Description\nDescription is currently not available.\n\n\n * Cookie\nD\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nPHPSESSID\n * Duration\nsession\n * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n * Cookie\nVISITOR_PRIVACY_METADATA\n * Duration\n6 months\n * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n * Cookie\ncookietest\n * Duration\nsession\n * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n * Cookie\n__Host-airtable-session\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\n__Host-airtable-session.sig\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\nm\n * Duration\n1 year 1 month 4 days\n * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n * Cookie\nBIGipServer*\n * Duration\nsession\n * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n * Cookie\n__cfruid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n * Cookie\n_GRECAPTCHA\n * Duration\n6 months\n * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n * Cookie\n__Secure-YNID\n * Duration\n6 months\n * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n * Cookie\ncookieyes-consent\n * Duration\n1 year\n * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n * Cookie\nlidc\n * Duration\n1 day\n * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n * Cookie\nbrw\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nbrwConsent\n * Duration\n5 minutes\n * Description\nDescription is currently not available.\n\n\n * Cookie\nWMF-Uniq\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nloom_anon_comment\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nloom_referral_video\n * Duration\nsession\n * Description\nDescription is currently not available.\n\n\n * Cookie\nVISITOR_INFO1_LIVE\n * Duration\n6 months\n * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n * Cookie\nyt-remote-connected-devices\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n * Duration\nNever Expires\n * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n * Cookie\nyt-remote-device-id\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-name\n * Duration\nsession\n * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\nyt-remote-fast-check-period\n * Duration\nsession\n * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-app\n * Duration\nsession\n * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n * Cookie\nyt-remote-cast-available\n * Duration\nsession\n * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n * Cookie\nyt-remote-cast-installed\n * Duration\nsession\n * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\ncp_session\n * Duration\n3 months\n * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n * Cookie\nloid\n * Duration\n1 year 1 month 4 days\n * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n * Cookie\n__hstc\n * Duration\n6 months\n * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n * Cookie\nhubspotutk\n * Duration\n6 months\n * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n * Cookie\n_ga\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n * Cookie\n_ga_*\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n * Cookie\n__Host-psifi.analyticsTrace\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n__Host-psifi.analyticsTraceV2\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n_gh_sess\n * Duration\nsession\n * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n * Cookie\nYSC\n * Duration\nsession\n * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n * Cookie\najs_anonymous_id\n * Duration\n1 year\n * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n * Cookie\nvuid\n * Duration\n1 year 1 month 4 days\n * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n * Cookie\nAWSALB\n * Duration\n7 days\n * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n * Cookie\nacq\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nacq.sig\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nptc\n * Duration\n2 years\n * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n * Cookie\nmuc_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n * Cookie\nguest_id_marketing\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\nguest_id_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\npersonalization_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n * Cookie\nguest_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n * Cookie\nbcookie\n * Duration\n1 year\n * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n * Cookie\n__Secure-ROLLOUT_TOKEN\n * Duration\n6 months\n * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n * Cookie\nyt.innertube::nextId\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nyt.innertube::requests\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nsession_tracker\n * Duration\nsession\n * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n * Cookie\nedgebucket\n * Duration\nsession\n * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n * Cookie\ndid\n * Duration\n1 year\n * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\n# MobileNetV1 Paper Walkthrough: The Tiny Giant\nUnderstanding and implementing MobileNetV1 from scratch with PyTorch\n[Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\nSeptember 4, 202526 min read\n![](../images/000.jpeg)\nPhoto by [Andreas Weilguny](https://unsplash.com/@aweilguny?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash) on [Unsplash](https://unsplash.com/photos/green-grass-field-under-blue-sky-during-daytime-JbSY1GKgFmA?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash)\n## Introduction\nDeep learning engineers used to focus on improving accuracy. They kept pushing the limit higher and higher until they eventually realized that the computational complexity of their models became more and more expensive. This was definitely a problem researchers needed to address because we want deep learning models to be able to work not only on high-end computers but also on small devices. To overcome this issue, Howard _et al._ back in 2017 proposed an extremely lightweight neural network model referred to as MobileNet which they introduced in a paper titled _MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications_ [1]. In fact, the model proposed in the paper is the first version of MobileNet, which is commonly known as MobileNetV1. Currently we already have four MobileNet versions: MobileNetV1 all the way to MobileNetV4. However, in this article we are only going to focus on MobileNetV1, covering the idea behind the architecture and how to implement it from scratch with PyTorch — I’ll save the later MobileNet versions for my upcoming articles.\n···\nLearn this step by step with the interactive [Machine Learning](https://roadmap.sh/machine-learning) roadmap.\n## Depthwise Separable Convolution\nIn order to achieve a lightweight model, MobileNet leverages the idea of _depthwise separable convolution_ , which is used nearly throughout the entire network. Figure 1 below displays the structural difference between this layer (right) and a standard convolution layer (left). You can see in the figure that depthwise separable convolution basically comprises two types of convolution layers: _depthwise convolution_ and _pointwise convolution_. In addition to that, we typically follow the _conv-BN-ReLU_ structure when it comes to constructing CNN-based models. This is essentially the reason that in the illustration we have batch normalization and ReLU right after each conv layer. We are going to discuss depthwise and pointwise convolutions more deeply in the subsequent sections.\n![](../images/001.png)\nFigure 1. The structure of a standard convolution layer (left) and a depthwise separable convolution layer (right) [1].\n### Depthwise Convolution\nA standard convolution layer is basically a convolution with the _group_ parameter set to 1. It is important to remember that in this case using a 3×3 kernel actually means applying a kernel of shape _C_ ×3×3 to the input tensor, where _C_ is the number of input channels. The use of this kernel shape allows us to aggregate information from all channels within each 3×3 patch at once. This is the reason why the standard convolution operation is computationally expensive, yet in return the output tensor contains a lot of information. If you take a closer look at Figure 2 below, a standard convolution layer corresponds to the one in the leftmost part of the tradeoff line.\n![](../images/002.png)\nFigure 2. The tradeoff between fewer and more convolution groups [2].\nIf you’re already familiar with _group convolution_ , depthwise convolution should be easy for you to understand. Group convolution is a method where we divide the channels of the input tensor according to the number of groups used and apply convolution independently within each group. For instance, suppose we have an input tensor of 64 channels and want to process it with 128 kernels grouped into 2. In such a case, the first 64 kernels are responsible for processing the first 32 channels of the input tensor, while the remaining 64 kernels process the last 32 channels of the input tensor. This mechanism results in 64 output channels for each group. The final output tensor is obtained by concatenating the resulting tensors from all groups along the channel dimension, resulting in a total of 128 channels in this example.\nAs we continue increasing the number of groups, we eventually reach the extreme case known as depthwise convolution, which is a special case of group convolution where the number of groups is set equal to the number of input channels. With this configuration, we basically have each channel processed independently of each other, causing every channel in the input to produce only a single output channel. By concatenating all the resulting 1-channel tensors, the final number of output channels remains exactly the same as that of the input. This mechanism requires us to use a kernel of size 1×3×3 instead of _C_ ×3×3, preventing us to perform information aggregation along channel axis. This allows us to have extremely lightweight computation, yet in return causing the output tensor to contain less information due to the absence of channel-wise information aggregation.\nSince the objective of MobileNet is to make the computation as fast as possible, we need to position ourselves at the rightmost part of the above tradeoff line despite capturing the least amount of information. This is definitely a problem that needs to be addressed, which is the reason why we employ pointwise convolution in the subsequent step.\n### Pointwise Convolution\nPointwise convolution is basically just a standard convolution, except that it uses kernels of size 1×1 — or to be more precise, it is actually _C_ ×1×1. This kernel shape allows us to aggregate information along the channel axis without being influenced by spatial information, effectively compensating for the limitation of depthwise convolution. Furthermore, remember that depthwise convolution alone can only output a tensor of the same number of channels as its input, which limits our flexibility in designing the model architecture. By applying pointwise convolution in the next step, we can set it to return as many channels as we want, allowing us to adapt the layer to the subsequent one as needed.\nWe can think of depthwise convolution and pointwise convolution as two complementary processes, where the former focuses on capturing spatial relationships while the latter captures channel relationships. These two processes might seem a bit inefficient at first glance since we can basically do the two processes at once using a standard convolution layer. However, if we take a closer look at the computational complexity, depthwise separable convolution is a lot more lightweight compared to the traditional convolution layer counterpart. In the next section I’ll discuss in more detail how we can calculate the number of parameters in these two models which definitely also affects the computational complexity.\n### Parameters Count Calculation\nSuppose we have an image of size 3× _H_ × _W_ , where _H_ and _W_ are the height and width of an image, respectively. For the sake of this example, let’s assume that we are about to process the image with 16 kernels of size 5×5, where the stride is set to 1 and the padding is set to 2 (which in this case is equivalent to _padding = same_). With this configuration, the size of the output tensor is going to be 16× _H_ × _W_. If we use a standard convolution layer, the number of parameters will be 5×5×3×16 = 1200 (without bias), in which this number is obtained based on the equation in Figure 3. The use of bias term is not strictly necessary in this case, but if we do the total number of parameters is going to be (5×5×3+1) × 16 = 1216.\n![](../images/body-d92420a14a6e7d3a5d87.png)\nFigure 3. Equation to calculate the number of parameters of a convolution layer [2].\nNow let’s calculate the parameter count of the depthwise separable convolution counterpart to produce the exact same tensor dimension. Following the same formula, we will have 5×5×1×3 = 75 for the depthwise convolution part (without bias). Or if we also account for the biases, then we will have (5×5×1+1) × 3 = 78 trainable params. In the case of depthwise convolution like this, the number of input channels is considered 1 since each kernel is responsible for processing a single channel only. To the pointwise convolution part, the number of parameters will be 1×1×3×16 = 48 (without bias) or (1×1×3+1) × 16 = 64 (with bias). Now to obtain the total number of parameters in the entire depthwise separable convolution process, we can simply calculate 75+48 = 123 (without bias) or 78+64 = 142 (with bias) — that’s nearly 90% reduction in parameter count if we compare it with the standard convolution! In theory, such an extreme drop in parameter count causes the model to have much lower capacity. But that’s just the theory. Later I will show you how MobileNet manages to keep up with other models in terms of accuracy.\n···\n## The Detailed MobileNetV1 Architecture\nFigure 4 below displays the entire MobileNetV1 architecture in detail. The depthwise convolution layers are the rows marked with _dw_ , while the pointwise convolutions are the ones having 1×1 filter shape. Notice that each _dw_ layer is always followed by a 1×1 convolution, indicating that the entire architecture mainly consists of depthwise separable convolutions. Furthermore, if you take a closer look at the architecture, you will see that spatial downsampling is done by the depthwise convolutions that have a stride of 2 (notice the rows with _s2_ in the table). Here you can see that every time we reduce the spatial dimension by half, the number of channels doubles to compensate for the loss of spatial information.\n![](../images/body-3a6fcdd5f45e7ede8f30.png)\nFigure 4. The entire MobileNetV1 architecture [1].\n### Width and Resolution Multiplier\nThe authors of MobileNet proposed a new parameter tuning mechanism by introducing the so-called _width_ and _resolution multiplier_ , which are formally denoted as _α_ and _ρ_ , respectively. The _α_ parameter can technically be adjusted freely, but authors suggest using either 1.0, 0.75, 0.5, or 0.25. This parameter works by reducing the number of channels produced by all convolution layers. For instance, if we set _α_ to 0.5, the first convolution layer in the network will turn 3-channel input into 16 instead of 32. On the other hand, _ρ_ is used to adjust the spatial dimension of the input tensor. It is important to note that even though ideally we should assign a floating-point number to this parameter, yet in practice it is more preferable to directly determine the actual resolution for the input image. In this case, authors recommend using either 224, 192, 160 and 128, in which the input size of 224×224 corresponds to _ρ_ = 1. The architecture displayed in Figure 4 above follows the default configuration where both _α_ and _ρ_ are set to 1.\n···\n## Experimental Results\nAuthors conducted plenty of experiments to prove the robustness of MobileNet. The first result to discuss is the one displayed in Figure 5 below, where in this experiment they tried to find out how the use of depthwise separable convolution layers affects performance. The second row of the table shows the result obtained by the architecture I showed you earlier in Figure 4, whereas the first row is the result when the layers are replaced with traditional convolutions. Here we can see that the accuracy of MobileNet with traditional CNN is indeed higher than that of the one using depthwise separable convolutions. However, if we take into account the number of multiplications and additions (_mult-adds_) as well as the parameter count, we can clearly see that the one with traditional convolution layers requires much more computational cost and memory usage just to make a slight improvement in accuracy. Thus, with depthwise separable convolutions, even though the model complexity of MobileNet significantly gets reduced, authors proved that the model capacity remains high.\n![](../images/body-e83b27c29ca3ccd29549.png)\nFigure 5. Performance comparison between MobileNet with depthwise separable convolution layers (second row) and its full-convolution counterpart (first row) [1].\nThe _α_ and _ρ_ parameters I explained earlier are mainly used to provide flexibility, considering that not all tasks require the highest MobileNet capability. Authors originally conducted experiments on 1000-class ImageNet dataset, but in practice, we might probably only need the model to perform classification on a dataset with fewer number of classes. In such a case, selecting lower values for the two parameters might be preferable as it can speed up the inference process while at the same time the model still has enough capacity to accommodate the classification task. Talking more specifically about _α_ , using smaller value for this parameter causes MobileNet to have lower accuracy. But that’s the result on 1000-class dataset. If our dataset is simpler and has fewer classes, using smaller _α_ might still be fine. In Figure 6 below the values 1.0, 0.75, 0.5, and 0.25 written next to each model correspond to the _α_ used.\n![](../images/body-b35c1847e38297cb8a22.png)\nFigure 6. How width multiplier affects model accuracy, number of operations, and parameter count [1].\nThe same thing also applies to the _ρ_ parameter, which is responsible for changing the resolution of the input image. Figure 7 below displays what the experimental results look like when we use different input resolutions. The results are somewhat similar to the ones in the previous figure, where the accuracy score decreases as we make the input image smaller. It is important to keep in mind that reducing input resolution like this also reduces the number of operations but does not affect the parameter count. This is essentially because the ones counted as parameters are the weights and biases, where in the case of CNN they correspond to the values inside the kernel. So, the parameter count will remain the same as long as we don’t change the configuration of the convolution layers. The number of operations, on the other hand, gets reduced in accordance with the decrease in input resolution since the number of pixels to be processed in smaller images is fewer than in larger images.\n![](../images/body-bb1b5f1fcc860dcd15b0.png)\nFigure 7. How input resolution affects model accuracy, number of operations, and parameter count [1].\nInstead of just comparing different values for _α_ and _ρ_ , authors also compared MobileNet with other popular models. We can see in Figure 8 that the largest MobileNet variant (the one using maximum _α_ and _ρ_) achieved comparable accuracy with GoogLeNet (InceptionV1) and VGG16 while maintaining the lowest computational complexity. This is basically the reason that I named this article _The Tiny Giant_ — lightweight yet powerful.\n![](../images/body-d74268ff6772a872f380.png)\nFigure 8. MobileNet achieves comparable accuracy to popular models while maintaining a much lower computational complexity and parameter count [1].\nAdditionally, authors also compared the smaller MobileNet variant with other small models. What’s to me interesting in Figure 9 is that even though the parameter count of SqueezeNet is lower than MobileNet, the number of operations in MobileNet is over 22 times smaller than SqueezeNet while still maintaining higher accuracy.\n![](../images/body-79a9b1858e694732dd9f.png)\nFigure 9. The performance of the smaller MobileNet variant compared to popular models [1].\n···\n## MobileNetV1 Implementation\nAs we have understood the idea behind MobileNetV1, we can now jump into the code. The architecture I am about to implement is based on the table in Figure 4. As always, the first thing we need to do is to import the required modules.\npython\n\n```\n# Codeblock 1\nimport torch\nimport torch.nn as nn\nfrom torchinfo import summary\n```\n\nNext, we initialize several configurable parameters so that we can adjust the model size according to our needs. In Codeblock 2 below, I denote _α_ as `ALPHA`, which the value can be changed to 0.75, 0.5 or 0.25 if we want the model to be smaller. We don’t specify any variable for _ρ_ since we can directly change `IMAGE_SIZE` to 192, 160 or 128 as we discussed earlier.\npython\n\n```\n# Codeblock 2\nBATCH_SIZE = 1\nIMAGE_SIZE = 224\nIN_CHANNELS = 3\nNUM_CLASSES = 1000\nALPHA = 1\n```\n\n···\n### First Convolution\nIf we go back to Figure 4, we can see that MobileNet essentially only consists of repeating patterns, i.e., a depthwise separable convolution followed by pointwise convolution. However, notice that the first row in the figure does not follow this pattern as it is actually just a standard convolution layer. Due to this reason, we need to create a separate class for this, which I refer to as `FirstConv` in Codeblock 3 below.\npython\n\n```\n# Codeblock 3\nclass FirstConv(nn.Module):\n def __init__(self):\n super().__init__()\n self.conv = nn.Conv2d(in_channels=3, \n out_channels=int(32*ALPHA), #(1)\n kernel_size=3, #(2)\n stride=2, #(3)\n padding=1, #(4)\n bias=False) #(5)\n self.bn = nn.BatchNorm2d(num_features=int(32*ALPHA))\n self.relu = nn.ReLU()\n \n def forward(self, x):\n x = self.relu(self.bn(self.conv(x)))\n return x\n```\n\nRemember that MobileNet follows the _conv-BN-ReLU_ structure. Thus, we need to initialize these three layers within the `__init__()` method of this class. The convolution layer itself is set to accept 3 input channels and output 32 channels. Since we want this number of output channels to be adjustable, we need to multiply it with `ALPHA` at the line marked with `#(1)`. Keep in mind that we need to change the datatype to integer after the multiplication since having a floating-point number for channel count is just nonsense. Next, at line `#(2)` and `#(3)` we set the kernel size to 3 and the stride to 2. With this configuration, the spatial dimension of the resulting tensor is going to be half that of the input. Additionally, using kernel of size 3×3 like this implicitly requires us to set the padding to 1 to implement _padding = same_ (`#(4)`). In this case we are not going to utilize the bias term, which is the reason that we set the `bias` parameter to `False` (`#(5)`). This is actually a standard practice when we use the _conv-BN-ReLU_ structure, since at the end of the day the value distribution of the convolution kernels will be centered around 0 again by the batch normalization layer, cancelling out the biases applied within the convolution kernel.\nIn order to find out whether the `FirstConv` class works properly, we are going to test it with the Codeblock 4 below. Here we initialize the layer and pass a tensor simulating a single RGB image of size 224×224. You can see in the resulting output that our convolution layer successfully downsampled the spatial dimension to 112×112 while at the same time expanding the number of channels to 32.\npython\n\n```\n# Codeblock 4\nfirst_conv = FirstConv()\nx = torch.randn((1, 3, 224, 224))\n\nout = first_conv(x)\nout.shape\n```\n\ntext\n\n```\n# Codeblock 4 Output\ntorch.Size([1, 32, 112, 112])\n```\n\n···\n### Depthwise Separable Convolutions\nAs the first convolution is done, we can now work on the repeating depthwise-pointwise layers. Since this pattern is the core idea of depthwise separable convolution, in the following code I wrap the two types of conv layers in a class called `DepthwiseSeparableConv`.\npython\n\n```\n# Codeblock 5\nclass DepthwiseSeparableConv(nn.Module):\n def __init__(self, in_channels, out_channels, downsample=False): #(1)\n super().__init__()\n \n in_channels = int(in_channels*ALPHA) #(2)\n out_channels = int(out_channels*ALPHA) #(3) \n \n if downsample: #(4)\n stride = 2\n else:\n stride = 1\n \n self.dwconv = nn.Conv2d(in_channels=in_channels,\n out_channels=in_channels, #(5)\n kernel_size=3, #(6)\n stride=stride, #(7)\n padding=1,\n groups=in_channels, #(8)\n bias=False)\n self.bn0 = nn.BatchNorm2d(num_features=in_channels) #(9)\n \n self.pwconv = nn.Conv2d(in_channels=in_channels, \n out_channels=out_channels, #(10)\n kernel_size=1, #(11)\n stride=1, #(12)\n padding=0, #(13)\n groups=1, #(14)\n bias=False)\n self.bn1 = nn.BatchNorm2d(num_features=out_channels) #(15)\n \n self.relu = nn.ReLU() #(16)\n\n def forward(self, x):\n print(f'original\\t: {x.size()}')\n \n x = self.relu(self.bn0(self.dwconv(x)))\n print(f'after dw conv\\t: {x.size()}')\n \n x = self.relu(self.bn1(self.pwconv(x)))\n print(f'after pw conv\\t: {x.size()}')\n \n return x\n```\n\nDifferent from `FirstConv` which does not take any input argument in the initialization phase, here we set the `DepthwiseSeparableConv` class to take several inputs as shown at line `#(1)` in Codeblock 5 above. I do this because we want the class to be reusable across all depthwise separable convolution layers throughout the entire network, in which each of them has slightly different behaviors from one another.\nWe can see in Figure 4 that after the 3-channel image is expanded to 32 by the first layer, this channel count increases to 64, 128, and so on all the way to 1024 in the subsequent processes. This is basically the reason that I set this class to accept the number of input and output channels (`in_channels` and `out_channels`) so that we can initialize the layer with flexible channel configurations. It is also important to keep in mind that we need to adjust this channel counts based on `ALPHA`. This can simply be done using the code at line `#(2)` and `#(3)`. Furthermore, here I also create a flag called `downsample` as the input parameter which by default is set to `False`. This flag is responsible to determine whether the layer will reduce the spatial dimension. Again, if you go back to Figure 4, you will notice that there are conditions where we reduce the spatial dimension by half and there are also some other conditions where the dimension is preserved. Every time we want to perform downsampling, we need to set the stride to 2, but if we don’t, we will set this parameter to 1 instead (`#(4)`).\nStill with the Codeblock 5 above, the next thing we need to do is to initialize the layers themselves. As we have discussed earlier, the depthwise convolution is responsible to capture spatial relationships between pixels, which is actually the reason that the kernel size is set to 3×3 (`#(6)`). In order for the input channels to be processed independently of each other, we can simply set the `groups` and `out_channels` parameters to be the same as the number of input channels itself (`#(8)` and `#(5)`). It is worth noting that if we set `out_channels` to be more than the number of input channels — say, twice as large — then we will have each channel processed by 2 kernels. Lastly for the depthwise convolution layer, the `stride` parameter at line `#(7)` can either be 1 or 2 which is determined according to the `downsampling` flag we discussed earlier.\nMeanwhile, the pointwise convolution uses 1×1 kernel (`#(11)`) since it is not intended to capture spatial information. This is actually the reason that we set the padding to 0 (`#(13)`) because there is no way this kernel size can reduce spatial dimension on its own. The `groups` parameter, on the other hand, is set to 1 (`#(14)`) because we want this layer to capture information from all channels at once. Unlike the depthwise convolution layer, here we can employ as many kernels as needed which corresponds to the number of channels in the resulting output tensor (`#(10)`). Meanwhile, the stride is set fixed to 1 (`#(12)`) since we will never perform downsampling with this layer.\nHere we need to initialize two separate batch normalization layers to be placed after the depthwise and pointwise convs (`#(9)` and `#(15)`). As for the ReLU activation function, we only need to initialize it once (`#(16)`) since it is just a mapping function without any trainable parameters. Thanks to this, we can reuse the same ReLU instance multiple times within the network.\nNow let’s see if our `DepthwiseSeparableConv` class works properly by passing a dummy tensor through it. Here I have prepared two test cases for this class. The first one is when we don’t perform downsampling and the second one is when we do. In Figure 10 below, the two tests I want to perform involve the use of the layers highlighted in green and blue, respectively.\n![](../images/body-8ebcb8827acff284768f.png)\nFigure 10. The layers highlighted in green and blue are the ones we are going to simulate to test the DepthwiseSeparableConv class [1][2].\nTo create the green part, we can simply use the `DepthwiseSeparableConv` class and set the number of input and output channels to 32 and 64 as seen in Codeblock 6 below (`#(1–2)`). Passing `downsample = False` is not quite necessary since we already set it as the default configuration (`#(3)`) — but I do this anyway just for the sake of clarity. The shape of the dummy tensor `x` is also configured to have the size of 32×112×112, in which it matches exactly with the input shape of the layer (`#(4)`).\npython\n\n```\n# Codeblock 6\ndepthwise_sep_conv = DepthwiseSeparableConv(in_channels=32, #(1)\n out_channels=64, #(2)\n downsample=False) #(3)\nx = torch.randn((1, int(32*ALPHA), 112, 112)) #(4)\n\nx = depthwise_sep_conv(x)\n```\n\nIf you run the above code, the following output should appear on your screen. Here you can see that the depthwise convolution layer returns a tensor of the exact same shape as the input (`#(1)`). The number of channels then doubles from 32 to 64 after the tensor being processed by the pointwise convolution (`#(2)`). This result proves that our `DepthwiseSeparableConv` class works properly for non-downsampling process. We will use this output tensor in the subsequent test as the input for the blue layer.\ntext\n\n```\n# Codeblock 6 Output\noriginal : torch.Size([1, 32, 112, 112])\nafter dw conv : torch.Size([1, 32, 112, 112]) #(1)\nafter pw conv : torch.Size([1, 64, 112, 112]) #(2)\n```\n\nThe second test is quite similar to the first one, except that here we need to configure the model based on the number of input and output channels of the blue layer. Not only that, the `downsample` parameter also needs to be set to `True` since we want the layer to reduce the spatial dimension by half. See Codeblock 7 below for the details.\npython\n\n```\n# Codeblock 7\ndepthwise_sep_conv = DepthwiseSeparableConv(in_channels=64, \n out_channels=128,\n downsample=True)\n\nx = depthwise_sep_conv(x)\n```\n\ntext\n\n```\n# Codeblock 7 Output\noriginal : torch.Size([1, 64, 112, 112])\nafter dw conv : torch.Size([1, 64, 56, 56]) #(1)\nafter pw conv : torch.Size([1, 128, 56, 56]) #(2)\n```\n\nWe can see in the above output that the spatial downsampling works properly as the depthwise convolution layer successfully converted the 112×112 image to 56×56 (`#(1)`). The channel axis is finally expanded to 128 with the help of the pointwise convolution layer (`#(2)`), making it ready to be fed into the subsequent layer. \nBased on the two tests I demonstrated above, it is proven that our `DepthwiseSeparableConv` class is correct and thus ready to be used to construct the entire MobileNetV1 architecture.\n···\n### The Entire MobileNetV1 Architecture\nI wrap everything within a class which I refer to as `MobileNetV1`. Since this class is quite long, I break it down into Codeblock 8a and 8b. If you want to run this code yourself, just ensure that these two codeblocks are written within the same notebook cell.\nNow let’s start from the `__init__()` method of this class. The first thing to do here is to initialize the `FirstConv` layer we created earlier (`#(1)`). The next layers we need to initialize are the core idea of MobileNet, i.e., the depthwise separable convolutions, in which every single of those layers consists of depthwise and pointwise convs. In this implementation I decided to name these pairs starting from `depthwise_sep_conv0` all the way to `depthwise_sep_conv8`. If you go back to Figure 4, you will notice that the downsampling process is done alternately with the non-downsampling layers. This can simply be implemented by setting the `downsample` flag to `True` for the layers number 1, 3, 5 and 7. The `depthwise_sep_conv6` is a bit special since it is actually not a standalone layer. Rather, it is a bunch of depthwise separable convolutions of the exact same specification repeated 5 times.\npython\n\n```\n# Codeblock 8a\nclass MobileNetV1(nn.Module):\n def __init__(self):\n super().__init__()\n \n self.first_conv = FirstConv() #(1)\n \n self.depthwise_sep_conv0 = DepthwiseSeparableConv(in_channels=32, \n out_channels=64)\n \n self.depthwise_sep_conv1 = DepthwiseSeparableConv(in_channels=64, \n out_channels=128, \n downsample=True)\n \n self.depthwise_sep_conv2 = DepthwiseSeparableConv(in_channels=128, \n out_channels=128)\n \n self.depthwise_sep_conv3 = DepthwiseSeparableConv(in_channels=128, \n out_channels=256, \n downsample=True)\n \n self.depthwise_sep_conv4 = DepthwiseSeparableConv(in_channels=256, \n out_channels=256)\n \n self.depthwise_sep_conv5 = DepthwiseSeparableConv(in_channels=256, \n out_channels=512, \n downsample=True)\n \n self.depthwise_sep_conv6 = nn.ModuleList(\n [DepthwiseSeparableConv(in_channels=512, out_channels=512) for _ in range(5)]\n )\n \n self.depthwise_sep_conv7 = DepthwiseSeparableConv(in_channels=512, \n out_channels=1024, \n downsample=True)\n \n self.depthwise_sep_conv8 = DepthwiseSeparableConv(in_channels=1024, #(2)\n out_channels=1024)\n \n num_out_channels = self.depthwise_sep_conv8.pwconv.out_channels #(3)\n \n self.avgpool = nn.AdaptiveAvgPool2d(output_size=(1,1)) #(4)\n self.fc = nn.Linear(in_features=num_out_channels, #(5)\n out_features=NUM_CLASSES)\n self.softmax = nn.Softmax(dim=1) #(6)\n```\n\nAs we have reached the very last `DepthwiseSeparableConv` layer (`#(2)`), what we need to do next is to initialize three more layers: an average pooling layer (`#(4)`), a fully-connected layer (`#(5)`), and a softmax activation function layer (`#(6)`). One thing that you need to keep in mind is that the number of output channels produced by the `depthwise_sep_conv8` is not always 1024 even though it appears to be fixed to that number. In fact, this output channel count will be different if we change the `ALPHA`. In order to make our implementation adaptive to such changes, we need to take the actual number of output channels generated using the code at line `#(3)`, which will then be used as the input size of the fully-connected layer (`#(5)`).\nRegarding the `forward()` method in Codeblock 8b, I think there is nothing I need to explain since what we basically do here is just to pass a tensor from one layer to the subsequent ones.\npython\n\n```\n# Codeblock 8b\n def forward(self, x):\n x = self.first_conv(x)\n print(f\"after first_conv\\t\\t: {x.shape}\")\n \n x = self.depthwise_sep_conv0(x)\n print(f\"after depthwise_sep_conv0\\t: {x.shape}\")\n \n x = self.depthwise_sep_conv1(x)\n print(f\"after depthwise_sep_conv1\\t: {x.shape}\")\n \n x = self.depthwise_sep_conv2(x)\n print(f\"after depthwise_sep_conv2\\t: {x.shape}\")\n \n x = self.depthwise_sep_conv3(x)\n print(f\"after depthwise_sep_conv3\\t: {x.shape}\")\n \n x = self.depthwise_sep_conv4(x)\n print(f\"after depthwise_sep_conv4\\t: {x.shape}\")\n \n x = self.depthwise_sep_conv5(x)\n print(f\"after depthwise_sep_conv5\\t: {x.shape}\")\n \n for i, layer in enumerate(self.depthwise_sep_conv6):\n x = layer(x)\n print(f\"after depthwise_sep_conv6 #{i}\\t: {x.shape}\")\n \n x = self.depthwise_sep_conv7(x)\n print(f\"after depthwise_sep_conv7\\t: {x.shape}\")\n \n x = self.depthwise_sep_conv8(x)\n print(f\"after depthwise_sep_conv8\\t: {x.shape}\")\n \n x = self.avgpool(x)\n print(f\"after avgpool\\t\\t\\t: {x.shape}\")\n \n x = torch.flatten(x, start_dim=1)\n print(f\"after flatten\\t\\t\\t: {x.shape}\")\n \n x = self.fc(x)\n print(f\"after fc\\t\\t\\t: {x.shape}\")\n \n x = self.softmax(x)\n print(f\"after softmax\\t\\t\\t: {x.shape}\")\n \n return x\n```\n\nNow let’s see if our MobileNetV1 works properly by running the following test code.\npython\n\n```\n# Codeblock 9\nmobilenetv1 = MobileNetV1()\nx = torch.randn((BATCH_SIZE, IN_CHANNELS, IMAGE_SIZE, IMAGE_SIZE))\n\nout = mobilenetv1(x)\n```\n\nAnd below is what the output looks like. Here we can see that our dummy image tensor successfully went through the `first_conv` layer all the way to the final output layer. During the convolution phase, we can see that the spatial dimension decreased as we get into the deeper layers while at the same time the number of channels increased. Afterwards, we apply an average pooling layer which works by taking the average value from each channel. We can say that at this point every single channel of size 7×7 is now represented as a single value, which is actually the reason that the spatial dimension dropped to 1×1 (`#(1)`). This tensor is then flattened (`#(2)`) so that we can process it further with the fully-connected layer (`#(3)`).\ntext\n\n```\n# Codeblock 9 Output\nafter first_conv : torch.Size([1, 32, 112, 112])\nafter depthwise_sep_conv0 : torch.Size([1, 64, 112, 112])\nafter depthwise_sep_conv1 : torch.Size([1, 128, 56, 56])\nafter depthwise_sep_conv2 : torch.Size([1, 128, 56, 56])\nafter depthwise_sep_conv3 : torch.Size([1, 256, 28, 28])\nafter depthwise_sep_conv4 : torch.Size([1, 256, 28, 28])\nafter depthwise_sep_conv5 : torch.Size([1, 512, 14, 14])\nafter depthwise_sep_conv6 #0 : torch.Size([1, 512, 14, 14])\nafter depthwise_sep_conv6 #1 : torch.Size([1, 512, 14, 14])\nafter depthwise_sep_conv6 #2 : torch.Size([1, 512, 14, 14])\nafter depthwise_sep_conv6 #3 : torch.Size([1, 512, 14, 14])\nafter depthwise_sep_conv6 #4 : torch.Size([1, 512, 14, 14])\nafter depthwise_sep_conv7 : torch.Size([1, 1024, 7, 7])\nafter depthwise_sep_conv8 : torch.Size([1, 1024, 7, 7])\nafter avgpool : torch.Size([1, 1024, 1, 1]) #(1)\nafter flatten : torch.Size([1, 1024]) #(2)\nafter fc : torch.Size([1, 1000]) #(3)\nafter softmax : torch.Size([1, 1000])\n```\n\nIf you want an even more detailed architecture, we can use the `summary()` function from `torchinfo` we imported earlier. If you scroll down the resulting output below, we can see that this model contains approximately 4.2 million trainable parameters, in which this number matches with the one written in Figure 5, 6, 7 and 8. I also tried to initialize the same model with different `ALPHA`, and I found that the numbers match with the table in Figure 6. Thanks to this reason, I think our MobileNetV1 implementation is correct.\npython\n\n```\n# Codeblock 10\nmobilenetv1 = MobileNetV1()\nsummary(mobilenetv1, input_size=(BATCH_SIZE, IN_CHANNELS, IMAGE_SIZE, IMAGE_SIZE))\n```\n\ntext\n\n```\n# Codeblock 10 Output\n==========================================================================================\nLayer (type:depth-idx) Output Shape Param #\n==========================================================================================\nMobileNetV1 [1, 1000] --\n├─FirstConv: 1-1 [1, 32, 112, 112] --\n│ └─Conv2d: 2-1 [1, 32, 112, 112] 864\n│ └─BatchNorm2d: 2-2 [1, 32, 112, 112] 64\n│ └─ReLU: 2-3 [1, 32, 112, 112] --\n├─DepthwiseSeparableConv: 1-2 [1, 64, 112, 112] --\n│ └─Conv2d: 2-4 [1, 32, 112, 112] 288\n│ └─BatchNorm2d: 2-5 [1, 32, 112, 112] 64\n│ └─ReLU: 2-6 [1, 32, 112, 112] --\n│ └─Conv2d: 2-7 [1, 64, 112, 112] 2,048\n│ └─BatchNorm2d: 2-8 [1, 64, 112, 112] 128\n│ └─ReLU: 2-9 [1, 64, 112, 112] --\n├─DepthwiseSeparableConv: 1-3 [1, 128, 56, 56] --\n│ └─Conv2d: 2-10 [1, 64, 56, 56] 576\n│ └─BatchNorm2d: 2-11 [1, 64, 56, 56] 128\n│ └─ReLU: 2-12 [1, 64, 56, 56] --\n│ └─Conv2d: 2-13 [1, 128, 56, 56] 8,192\n│ └─BatchNorm2d: 2-14 [1, 128, 56, 56] 256\n│ └─ReLU: 2-15 [1, 128, 56, 56] --\n├─DepthwiseSeparableConv: 1-4 [1, 128, 56, 56] --\n│ └─Conv2d: 2-16 [1, 128, 56, 56] 1,152\n│ └─BatchNorm2d: 2-17 [1, 128, 56, 56] 256\n│ └─ReLU: 2-18 [1, 128, 56, 56] --\n│ └─Conv2d: 2-19 [1, 128, 56, 56] 16,384\n│ └─BatchNorm2d: 2-20 [1, 128, 56, 56] 256\n│ └─ReLU: 2-21 [1, 128, 56, 56] --\n├─DepthwiseSeparableConv: 1-5 [1, 256, 28, 28] --\n│ └─Conv2d: 2-22 [1, 128, 28, 28] 1,152\n│ └─BatchNorm2d: 2-23 [1, 128, 28, 28] 256\n│ └─ReLU: 2-24 [1, 128, 28, 28] --\n│ └─Conv2d: 2-25 [1, 256, 28, 28] 32,768\n│ └─BatchNorm2d: 2-26 [1, 256, 28, 28] 512\n│ └─ReLU: 2-27 [1, 256, 28, 28] --\n├─DepthwiseSeparableConv: 1-6 [1, 256, 28, 28] --\n│ └─Conv2d: 2-28 [1, 256, 28, 28] 2,304\n│ └─BatchNorm2d: 2-29 [1, 256, 28, 28] 512\n│ └─ReLU: 2-30 [1, 256, 28, 28] --\n│ └─Conv2d: 2-31 [1, 256, 28, 28] 65,536\n│ └─BatchNorm2d: 2-32 [1, 256, 28, 28] 512\n│ └─ReLU: 2-33 [1, 256, 28, 28] --\n├─DepthwiseSeparableConv: 1-7 [1, 512, 14, 14] --\n│ └─Conv2d: 2-34 [1, 256, 14, 14] 2,304\n│ └─BatchNorm2d: 2-35 [1, 256, 14, 14] 512\n│ └─ReLU: 2-36 [1, 256, 14, 14] --\n│ └─Conv2d: 2-37 [1, 512, 14, 14] 131,072\n│ └─BatchNorm2d: 2-38 [1, 512, 14, 14] 1,024\n│ └─ReLU: 2-39 [1, 512, 14, 14] --\n├─ModuleList: 1-8 -- --\n│ └─DepthwiseSeparableConv: 2-40 [1, 512, 14, 14] --\n│ │ └─Conv2d: 3-1 [1, 512, 14, 14] 4,608\n│ │ └─BatchNorm2d: 3-2 [1, 512, 14, 14] 1,024\n│ │ └─ReLU: 3-3 [1, 512, 14, 14] --\n│ │ └─Conv2d: 3-4 [1, 512, 14, 14] 262,144\n│ │ └─BatchNorm2d: 3-5 [1, 512, 14, 14] 1,024\n│ │ └─ReLU: 3-6 [1, 512, 14, 14] --\n│ └─DepthwiseSeparableConv: 2-41 [1, 512, 14, 14] --\n│ │ └─Conv2d: 3-7 [1, 512, 14, 14] 4,608\n│ │ └─BatchNorm2d: 3-8 [1, 512, 14, 14] 1,024\n│ │ └─ReLU: 3-9 [1, 512, 14, 14] --\n│ │ └─Conv2d: 3-10 [1, 512, 14, 14] 262,144\n│ │ └─BatchNorm2d: 3-11 [1, 512, 14, 14] 1,024\n│ │ └─ReLU: 3-12 [1, 512, 14, 14] --\n│ └─DepthwiseSeparableConv: 2-42 [1, 512, 14, 14] --\n│ │ └─Conv2d: 3-13 [1, 512, 14, 14] 4,608\n│ │ └─BatchNorm2d: 3-14 [1, 512, 14, 14] 1,024\n│ │ └─ReLU: 3-15 [1, 512, 14, 14] --\n│ │ └─Conv2d: 3-16 [1, 512, 14, 14] 262,144\n│ │ └─BatchNorm2d: 3-17 [1, 512, 14, 14] 1,024\n│ │ └─ReLU: 3-18 [1, 512, 14, 14] --\n│ └─DepthwiseSeparableConv: 2-43 [1, 512, 14, 14] --\n│ │ └─Conv2d: 3-19 [1, 512, 14, 14] 4,608\n│ │ └─BatchNorm2d: 3-20 [1, 512, 14, 14] 1,024\n│ │ └─ReLU: 3-21 [1, 512, 14, 14] --\n│ │ └─Conv2d: 3-22 [1, 512, 14, 14] 262,144\n│ │ └─BatchNorm2d: 3-23 [1, 512, 14, 14] 1,024\n│ │ └─ReLU: 3-24 [1, 512, 14, 14] --\n│ └─DepthwiseSeparableConv: 2-44 [1, 512, 14, 14] --\n│ │ └─Conv2d: 3-25 [1, 512, 14, 14] 4,608\n│ │ └─BatchNorm2d: 3-26 [1, 512, 14, 14] 1,024\n│ │ └─ReLU: 3-27 [1, 512, 14, 14] --\n│ │ └─Conv2d: 3-28 [1, 512, 14, 14] 262,144\n│ │ └─BatchNorm2d: 3-29 [1, 512, 14, 14] 1,024\n│ │ └─ReLU: 3-30 [1, 512, 14, 14] --\n├─DepthwiseSeparableConv: 1-9 [1, 1024, 7, 7] --\n│ └─Conv2d: 2-45 [1, 512, 7, 7] 4,608\n│ └─BatchNorm2d: 2-46 [1, 512, 7, 7] 1,024\n│ └─ReLU: 2-47 [1, 512, 7, 7] --\n│ └─Conv2d: 2-48 [1, 1024, 7, 7] 524,288\n│ └─BatchNorm2d: 2-49 [1, 1024, 7, 7] 2,048\n│ └─ReLU: 2-50 [1, 1024, 7, 7] --\n├─DepthwiseSeparableConv: 1-10 [1, 1024, 7, 7] --\n│ └─Conv2d: 2-51 [1, 1024, 7, 7] 9,216\n│ └─BatchNorm2d: 2-52 [1, 1024, 7, 7] 2,048\n│ └─ReLU: 2-53 [1, 1024, 7, 7] --\n│ └─Conv2d: 2-54 [1, 1024, 7, 7] 1,048,576\n│ └─BatchNorm2d: 2-55 [1, 1024, 7, 7] 2,048\n│ └─ReLU: 2-56 [1, 1024, 7, 7] --\n├─AdaptiveAvgPool2d: 1-11 [1, 1024, 1, 1] --\n├─Linear: 1-12 [1, 1000] 1,025,000\n├─Softmax: 1-13 [1, 1000] --\n==========================================================================================\nTotal params: 4,231,976\nTrainable params: 4,231,976\nNon-trainable params: 0\nTotal mult-adds (Units.MEGABYTES): 568.76\n==========================================================================================\nInput size (MB): 0.60\nForward/backward pass size (MB): 80.69\nParams size (MB): 16.93\nEstimated Total Size (MB): 98.22\n==========================================================================================\n```\n\n···\n## Ending\nThat was pretty much everything about MobileNetV1. I do encourage you to play around with the above model. If you want to train it for image classification, you can adjust the number of neurons in the output layer according to the number of classes available in your dataset. You can also try to explore different _α_ and _ρ_ to find the values that suit best for your case in terms of accuracy and efficiency. Furthermore, since this implementation is literally done from scratch, it is also possible to change other things that are not explicitly mentioned in the paper, such as the number of repeats of the `depthwise_sep_conv6` layer, or even using _α_ and _ρ_ greater than 1. And well, there are basically lots of things to explore from our MobileNetV1 implementation! You can also access the code used in this article in my GitHub repository [3].\nFeel free to comment if you spot any mistake in my explanation or the code. Thanks for reading!\n···\n## References\n[1] Andrew G. Howard _et al._ MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. Arxiv. [Accessed April 7, 2025].\n[2] Image created originally by author.\n[3] MuhammadArdiPutra. The Tiny Giant — MobileNetV1. GitHub. [Accessed April 7, 2025].\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](../images/body-59e1c17daab1fcb582c5.jpeg)\nWritten By\n## [Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\nDeep learning storyteller\n[See all from Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\n[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Artificial Intelligence](https://towardsdatascience.com/tag/artificial-intelligence/)[Machine Learning](https://towardsdatascience.com/tag/machine-learning/)[Python](https://towardsdatascience.com/tag/python/)[Pytorch](https://towardsdatascience.com/tag/pytorch/)\nShare This Article\n * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fthe-tiny-giant-mobilenetv1%2F&title=MobileNetV1%20Paper%20Walkthrough%3A%20The%20Tiny%20Giant)\n * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fthe-tiny-giant-mobilenetv1%2F&title=MobileNetV1%20Paper%20Walkthrough%3A%20The%20Tiny%20Giant)\n * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fthe-tiny-giant-mobilenetv1%2F&text=MobileNetV1%20Paper%20Walkthrough%3A%20The%20Tiny%20Giant)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n * [![](../images/003.jpg)](https://towardsdatascience.com/new-perspectives-on-deep-learning-neural-networks-and-ai-a335547ac32c/)\n## [New Perspectives on Deep Learning, Neural Networks, and AI](https://towardsdatascience.com/new-perspectives-on-deep-learning-neural-networks-and-ai-a335547ac32c/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nOur weekly selection of must-read Editors' Picks and original features\n[TDS Editors](https://towardsdatascience.com/author/towardsdatascience/)\nJuly 14, 20223 min read\n * [![](../images/004.jpeg)](https://towardsdatascience.com/python-and-the-truth-90cf08380246/)\n## [Python and the Truth](https://towardsdatascience.com/python-and-the-truth-90cf08380246/)\n[Programming](https://towardsdatascience.com/category/programming/)\nWe will explore the definitions behind Truth values, how to operate on them and avoid nasty bugs, and how to tune the Truth of our classes.\n[Pere Miquel Brull](https://towardsdatascience.com/author/pmbrull/)\nApril 4, 20224 min read\n * [![](../images/005.png)](https://towardsdatascience.com/an-intuitive-explanation-of-self-attention-4f72709638e1/)\n## [An intuitive explanation of Self Attention](https://towardsdatascience.com/an-intuitive-explanation-of-self-attention-4f72709638e1/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nA step-by-step explanation of the multi-headed self-attention block\n[Saketh Kotamraju](https://towardsdatascience.com/author/saketh.kotamraju/)\nOctober 7, 202010 min read\n * [![](../images/006.jpeg)](https://towardsdatascience.com/some-key-challenges-in-building-an-ai-model-for-medical-diagnosis-63f7438f14a/)\n## [Some Key Challenges in Building an AI Model for Medical Diagnosis](https://towardsdatascience.com/some-key-challenges-in-building-an-ai-model-for-medical-diagnosis-63f7438f14a/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nBeginner's Guide to Working with AI in Medicine\n[Ishtiak Mahmud](https://towardsdatascience.com/author/ishtiakmahmud6/)\nSeptember 12, 20205 min read\n * [![](../images/007.jpg)](https://towardsdatascience.com/introduction-to-pca-in-python-with-sklearn-pandas-and-matplotlib-476880f30238/)\n## [Introduction to PCA in Python with Sklearn, Pandas, and Matplotlib](https://towardsdatascience.com/introduction-to-pca-in-python-with-sklearn-pandas-and-matplotlib-476880f30238/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nLearn the intuition behind PCA in Python and Sklearn by transforming a multidimensional dataset into an arbitrary number of dimensions and...\n[Andrea D'Agostino](https://towardsdatascience.com/author/thedrewdag/)\nSeptember 6, 202313 min read\n * [![](../images/008.jpg)](https://towardsdatascience.com/class-activation-mapping-using-transfer-learning-of-resnet50-e8ca7cfd657e/)\n## [Class Activation Mapping](https://towardsdatascience.com/class-activation-mapping-using-transfer-learning-of-resnet50-e8ca7cfd657e/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nA powerful method for object detection and debugging\n[Patrick Brus](https://towardsdatascience.com/author/brus-patrick63/)\nJuly 23, 20205 min read\n * [![](../images/009.jpeg)](https://towardsdatascience.com/try-these-5-tips-to-optimize-your-python-code-c7e0ccdf486a/)\n## [Try These 5 Tips To Optimize Your Python Code](https://towardsdatascience.com/try-these-5-tips-to-optimize-your-python-code-c7e0ccdf486a/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nQuick tips to write better python code\n[Pranjal Saxena](https://towardsdatascience.com/author/pranjalai/)\nSeptember 24, 20215 min read\n * [![](../images/010.jpeg)](https://towardsdatascience.com/concept-learning-making-your-network-interpretable-735a0698fc03/)\n## [Concept Learning: Making your network interpretable](https://towardsdatascience.com/concept-learning-making-your-network-interpretable-735a0698fc03/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nOver the last decade, neural networks have been showing superb performance across a large variety of datasets and problems. While metrics...\n[Lukas Huber](https://towardsdatascience.com/author/lukas.huber1/)\nAugust 2, 20225 min read\n * [![](../images/011.jpeg)](https://towardsdatascience.com/arma-forecasting-for-non-gaussian-time-series-data-using-copulas-45a3a28f69e5/)\n## [ARMA Forecasting for Non-Gaussian Time-Series Data Using Copulas](https://towardsdatascience.com/arma-forecasting-for-non-gaussian-time-series-data-using-copulas-45a3a28f69e5/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nIt's all fun and games until your time-series is not Normal anymore\n[Sarem Seitz](https://towardsdatascience.com/author/sarem-seitz/)\nJune 17, 20229 min read\n\n\n * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n * [X](https://x.com/TDataScience)\n * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n * [Threads](https://www.threads.net/@towardsdatascience)\n * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n#### Recommended Articles \nClose\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500) How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ \"How Relevance Models Foreshadowed Transformers for NLP\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n\n\n", "metadata": { "title": "MobileNetV1 Paper Walkthrough: The Tiny Giant | Towards Data Science", "description": "Understanding and implementing MobileNetV1 from scratch with PyTorch", "keywords": null, "author": "Muhammad Ardi Putra", "og:locale": "en_US", "og:type": "article", "og:title": "MobileNetV1 Paper Walkthrough: The Tiny Giant", "og:description": "Understanding and implementing MobileNetV1 from scratch with PyTorch", "og:url": "https://towardsdatascience.com/the-tiny-giant-mobilenetv1/", "og:site_name": "Towards Data Science", "og:image": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/09/andreas-weilguny-JbSY1GKgFmA-unsplash-scaled-1.jpeg", "og:image:width": "2560", "og:image:height": "1920", "og:image:type": "image/jpeg", "twitter:card": "summary_large_image", "twitter:site": "@TDataScience", "twitter:creator": "@TDataScience", "twitter:label1": "Written by", "twitter:data1": "Muhammad Ardi Putra", "twitter:label2": "Est. reading time", "twitter:data2": "26 min read", "article:published_time": "2025-09-04T11:36:09.000Z", "article:modified_time": "2025-09-04T11:36:09.000Z", "article:author": "https://towardsdatascience.com/author/muhammad_ardi/" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 202, "error": "" }, "code_blocks": [ { "language": "txt", "file": "attachments/c3cb7fd8c186/code/block_000.txt", "chars": 79 }, { "language": "txt", "file": "attachments/c3cb7fd8c186/code/block_001.txt", "chars": 99 }, { "language": "txt", "file": "attachments/c3cb7fd8c186/code/block_002.txt", "chars": 601 }, { "language": "txt", "file": "attachments/c3cb7fd8c186/code/block_003.txt", "chars": 104 }, { "language": "txt", "file": "attachments/c3cb7fd8c186/code/block_004.txt", "chars": 51 }, { "language": "txt", "file": "attachments/c3cb7fd8c186/code/block_005.txt", "chars": 1738 }, { "language": "txt", "file": "attachments/c3cb7fd8c186/code/block_006.txt", "chars": 317 }, { "language": "txt", "file": "attachments/c3cb7fd8c186/code/block_007.txt", "chars": 178 }, { "language": "txt", "file": "attachments/c3cb7fd8c186/code/block_008.txt", "chars": 225 }, { "language": "txt", "file": "attachments/c3cb7fd8c186/code/block_009.txt", "chars": 174 }, { "language": "txt", "file": "attachments/c3cb7fd8c186/code/block_010.txt", "chars": 2280 }, { "language": "txt", "file": "attachments/c3cb7fd8c186/code/block_011.txt", "chars": 1505 }, { "language": "txt", "file": "attachments/c3cb7fd8c186/code/block_012.txt", "chars": 131 }, { "language": "txt", "file": "attachments/c3cb7fd8c186/code/block_013.txt", "chars": 1115 }, { "language": "txt", "file": "attachments/c3cb7fd8c186/code/block_014.txt", "chars": 126 }, { "language": "txt", "file": "attachments/c3cb7fd8c186/code/block_015.txt", "chars": 7946 } ], "images": [ { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T18:03:22Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T18:03:22Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T18:03:22Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T18:03:22Z" } ], "audio": [], "videos": [ { "url": "https://www.youtube.com/c/TowardsDataScience", "kind": "channel_reference", "status": "reference_only" } ], "links": [ { "type": "internal", "href": "https://towardsdatascience.com/" }, { "type": "internal", "href": "https://towardsdatascience.com/latest/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/editors-pick/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/the-variable/" }, { "type": "internal", "href": "https://towardsdatascience.com/shipai/" }, { "type": "internal", "href": "https://towardsdatascience.com/submissions/" }, { "type": "internal", "href": "https://towardsdatascience.com/category/artificial-intelligence/deep-learning/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/muhammad_ardi/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/artificial-intelligence/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/machine-learning/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/python/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/pytorch/" }, { "type": "internal", "href": "https://towardsdatascience.com/new-perspectives-on-deep-learning-neural-networks-and-ai-a335547ac32c/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/towardsdatascience/" }, { "type": "internal", "href": "https://towardsdatascience.com/python-and-the-truth-90cf08380246/" }, { "type": "internal", "href": "https://towardsdatascience.com/category/programming/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/pmbrull/" }, { "type": "internal", "href": "https://towardsdatascience.com/an-intuitive-explanation-of-self-attention-4f72709638e1/" }, { "type": "internal", "href": "https://towardsdatascience.com/category/artificial-intelligence/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/saketh.kotamraju/" }, { "type": "external", "href": "https://write.insightmediagroup.io" }, { "type": "external", "href": "https://www.linkedin.com/company/towards-data-science/" }, { "type": "external", "href": "https://x.com/TDataScience" }, { "type": "external", "href": "https://unsplash.com/@aweilguny" }, { "type": "external", "href": "https://unsplash.com/photos/green-grass-field-under-blue-sky-during-daytime-JbSY1GKgFmA" }, { "type": "external", "href": "https://roadmap.sh/machine-learning" }, { "type": "external", "href": "https://arxiv.org/abs/1704.04861" }, { "type": "external", "href": "https://github.com/MuhammadArdiPutra/medium_articles/blob/main/The%20Tiny%20Giant%20-%20MobileNetV1.ipynb" }, { "type": "external", "href": "https://www.facebook.com/sharer/sharer.php?title=MobileNetV1+Paper+Walkthrough%3A+The+Tiny+Giant&u=https%3A%2F%2Ftowardsdatascience.com%2Fthe-tiny-giant-mobilenetv1%2F" }, { "type": "external", "href": "https://www.linkedin.com/shareArticle?mini=true&title=MobileNetV1+Paper+Walkthrough%3A+The+Tiny+Giant&url=https%3A%2F%2Ftowardsdatascience.com%2Fthe-tiny-giant-mobilenetv1%2F" }, { "type": "external", "href": "https://x.com/share?text=MobileNetV1+Paper+Walkthrough%3A+The+Tiny+Giant&url=https%3A%2F%2Ftowardsdatascience.com%2Fthe-tiny-giant-mobilenetv1%2F" }, { "type": "external", "href": "https://www.youtube.com/c/TowardsDataScience" }, { "type": "external", "href": "https://www.threads.net/@towardsdatascience" }, { "type": "external", "href": "https://bsky.app/profile/towardsdatascience.com" }, { "type": "external", "href": "https://thenewstack.io" }, { "type": "external", "href": "https://roadmap.sh" }, { "type": "external", "href": "https://insightmediagroup.io/creator-network" } ], "repair_version": 2, "repaired_at": "2026-10-20T17:31:04Z", "source_url": "https://towardsdatascience.com/the-tiny-giant-mobilenetv1/", "delivery_status": "unverified", "discovery_status": "http_403", "missing_reason": "源页面未重新核验:http_403；已按保留正文补采图片", "repair_error": "", "actual_video_files": [], "expected_saved_body_images": 4, "image_discovery": "saved_markdown", "source_body_markdown": "![Revisit consent button](https://cdn-cookieyes.com/assets/images/revisit.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](https://cdn-cookieyes.com/assets/images/close.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n * Cookie\nBCTempID\n * Duration\n10 minutes\n * Description\nNo description available.\n\n\n * Cookie\n__cf_bm\n * Duration\n1 hour\n * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n * Cookie\nAWSALBCORS\n * Duration\n7 days\n * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n * Cookie\n_cfuvid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n * Cookie\nli_gc\n * Duration\n6 months\n * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n * Cookie\n__hssrc\n * Duration\nsession\n * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n * Cookie\n__hssc\n * Duration\n1 hour\n * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n * Cookie\nwpEmojiSettingsSupports\n * Duration\nsession\n * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n * Cookie\nBCSessionID\n * Duration\n1 year 1 month 4 days\n * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n * Cookie\n_octo\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nlogged_in\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\n__Secure-YEC\n * Duration\npast\n * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n * Cookie\n__eoi\n * Duration\n6 months\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTGCORS\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\nlogin-status-p\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTG\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\ncsrf_token\n * Duration\nsession\n * Description\nNo description available.\n\n\n * Cookie\ntoken_v2\n * Duration\n1 day\n * Description\nDescription is currently not available.\n\n\n * Cookie\nD\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nPHPSESSID\n * Duration\nsession\n * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n * Cookie\nVISITOR_PRIVACY_METADATA\n * Duration\n6 months\n * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n * Cookie\ncookietest\n * Duration\nsession\n * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n * Cookie\n__Host-airtable-session\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\n__Host-airtable-session.sig\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\nm\n * Duration\n1 year 1 month 4 days\n * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n * Cookie\nBIGipServer*\n * Duration\nsession\n * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n * Cookie\n__cfruid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n * Cookie\n_GRECAPTCHA\n * Duration\n6 months\n * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n * Cookie\n__Secure-YNID\n * Duration\n6 months\n * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n * Cookie\ncookieyes-consent\n * Duration\n1 year\n * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n * Cookie\nlidc\n * Duration\n1 day\n * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n * Cookie\nbrw\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nbrwConsent\n * Duration\n5 minutes\n * Description\nDescription is currently not available.\n\n\n * Cookie\nWMF-Uniq\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nloom_anon_comment\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nloom_referral_video\n * Duration\nsession\n * Description\nDescription is currently not available.\n\n\n * Cookie\nVISITOR_INFO1_LIVE\n * Duration\n6 months\n * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n * Cookie\nyt-remote-connected-devices\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n * Duration\nNever Expires\n * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n * Cookie\nyt-remote-device-id\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-name\n * Duration\nsession\n * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\nyt-remote-fast-check-period\n * Duration\nsession\n * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-app\n * Duration\nsession\n * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n * Cookie\nyt-remote-cast-available\n * Duration\nsession\n * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n * Cookie\nyt-remote-cast-installed\n * Duration\nsession\n * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\ncp_session\n * Duration\n3 months\n * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n * Cookie\nloid\n * Duration\n1 year 1 month 4 days\n * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n * Cookie\n__hstc\n * Duration\n6 months\n * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n * Cookie\nhubspotutk\n * Duration\n6 months\n * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n * Cookie\n_ga\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n * Cookie\n_ga_*\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n * Cookie\n__Host-psifi.analyticsTrace\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n__Host-psifi.analyticsTraceV2\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n_gh_sess\n * Duration\nsession\n * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n * Cookie\nYSC\n * Duration\nsession\n * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n * Cookie\najs_anonymous_id\n * Duration\n1 year\n * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n * Cookie\nvuid\n * Duration\n1 year 1 month 4 days\n * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n * Cookie\nAWSALB\n * Duration\n7 days\n * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n * Cookie\nacq\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nacq.sig\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nptc\n * Duration\n2 years\n * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n * Cookie\nmuc_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n * Cookie\nguest_id_marketing\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\nguest_id_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\npersonalization_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n * Cookie\nguest_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n * Cookie\nbcookie\n * Duration\n1 year\n * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n * Cookie\n__Secure-ROLLOUT_TOKEN\n * Duration\n6 months\n * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n * Cookie\nyt.innertube::nextId\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nyt.innertube::requests\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nsession_tracker\n * Duration\nsession\n * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n * Cookie\nedgebucket\n * Duration\nsession\n * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n * Cookie\ndid\n * Duration\n1 year\n * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\n# MobileNetV1 Paper Walkthrough: The Tiny Giant\nUnderstanding and implementing MobileNetV1 from scratch with PyTorch\n[Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\nSeptember 4, 202526 min read\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/09/andreas-weilguny-JbSY1GKgFmA-unsplash-scaled-1.jpeg)\nPhoto by [Andreas Weilguny](https://unsplash.com/@aweilguny?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash) on [Unsplash](https://unsplash.com/photos/green-grass-field-under-blue-sky-during-daytime-JbSY1GKgFmA?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash)\n## Introduction\nDeep learning engineers used to focus on improving accuracy. They kept pushing the limit higher and higher until they eventually realized that the computational complexity of their models became more and more expensive. This was definitely a problem researchers needed to address because we want deep learning models to be able to work not only on high-end computers but also on small devices. To overcome this issue, Howard _et al._ back in 2017 proposed an extremely lightweight neural network model referred to as MobileNet which they introduced in a paper titled _MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications_ [1]. In fact, the model proposed in the paper is the first version of MobileNet, which is commonly known as MobileNetV1. Currently we already have four MobileNet versions: MobileNetV1 all the way to MobileNetV4. However, in this article we are only going to focus on MobileNetV1, covering the idea behind the architecture and how to implement it from scratch with PyTorch — I’ll save the later MobileNet versions for my upcoming articles.\n···\nLearn this step by step with the interactive [Machine Learning](https://roadmap.sh/machine-learning) roadmap.\n## Depthwise Separable Convolution\nIn order to achieve a lightweight model, MobileNet leverages the idea of _depthwise separable convolution_ , which is used nearly throughout the entire network. Figure 1 below displays the structural difference between this layer (right) and a standard convolution layer (left). You can see in the figure that depthwise separable convolution basically comprises two types of convolution layers: _depthwise convolution_ and _pointwise convolution_. In addition to that, we typically follow the _conv-BN-ReLU_ structure when it comes to constructing CNN-based models. This is essentially the reason that in the illustration we have batch normalization and ReLU right after each conv layer. We are going to discuss depthwise and pointwise convolutions more deeply in the subsequent sections.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/08/16WhmqbZcnnEl300tcWEKaA.png)\nFigure 1. The structure of a standard convolution layer (left) and a depthwise separable convolution layer (right) [1].\n### Depthwise Convolution\nA standard convolution layer is basically a convolution with the _group_ parameter set to 1. It is important to remember that in this case using a 3×3 kernel actually means applying a kernel of shape _C_ ×3×3 to the input tensor, where _C_ is the number of input channels. The use of this kernel shape allows us to aggregate information from all channels within each 3×3 patch at once. This is the reason why the standard convolution operation is computationally expensive, yet in return the output tensor contains a lot of information. If you take a closer look at Figure 2 below, a standard convolution layer corresponds to the one in the leftmost part of the tradeoff line.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/08/1T3MzD5izakd9OJGx6i2ctQ.png)\nFigure 2. The tradeoff between fewer and more convolution groups [2].\nIf you’re already familiar with _group convolution_ , depthwise convolution should be easy for you to understand. Group convolution is a method where we divide the channels of the input tensor according to the number of groups used and apply convolution independently within each group. For instance, suppose we have an input tensor of 64 channels and want to process it with 128 kernels grouped into 2. In such a case, the first 64 kernels are responsible for processing the first 32 channels of the input tensor, while the remaining 64 kernels process the last 32 channels of the input tensor. This mechanism results in 64 output channels for each group. The final output tensor is obtained by concatenating the resulting tensors from all groups along the channel dimension, resulting in a total of 128 channels in this example.\nAs we continue increasing the number of groups, we eventually reach the extreme case known as depthwise convolution, which is a special case of group convolution where the number of groups is set equal to the number of input channels. With this configuration, we basically have each channel processed independently of each other, causing every channel in the input to produce only a single output channel. By concatenating all the resulting 1-channel tensors, the final number of output channels remains exactly the same as that of the input. This mechanism requires us to use a kernel of size 1×3×3 instead of _C_ ×3×3, preventing us to perform information aggregation along channel axis. This allows us to have extremely lightweight computation, yet in return causing the output tensor to contain less information due to the absence of channel-wise information aggregation.\nSince the objective of MobileNet is to make the computation as fast as possible, we need to position ourselves at the rightmost part of the above tradeoff line despite capturing the least amount of information. This is definitely a problem that needs to be addressed, which is the reason why we employ pointwise convolution in the subsequent step.\n### Pointwise Convolution\nPointwise convolution is basically just a standard convolution, except that it uses kernels of size 1×1 — or to be more precise, it is actually _C_ ×1×1. This kernel shape allows us to aggregate information along the channel axis without being influenced by spatial information, effectively compensating for the limitation of depthwise convolution. Furthermore, remember that depthwise convolution alone can only output a tensor of the same number of channels as its input, which limits our flexibility in designing the model architecture. By applying pointwise convolution in the next step, we can set it to return as many channels as we want, allowing us to adapt the layer to the subsequent one as needed.\nWe can think of depthwise convolution and pointwise convolution as two complementary processes, where the former focuses on capturing spatial relationships while the latter captures channel relationships. These two processes might seem a bit inefficient at first glance since we can basically do the two processes at once using a standard convolution layer. However, if we take a closer look at the computational complexity, depthwise separable convolution is a lot more lightweight compared to the traditional convolution layer counterpart. In the next section I’ll discuss in more detail how we can calculate the number of parameters in these two models which definitely also affects the computational complexity.\n### Parameters Count Calculation\nSuppose we have an image of size 3× _H_ × _W_ , where _H_ and _W_ are the height and width of an image, respectively. For the sake of this example, let’s assume that we are about to process the image with 16 kernels of size 5×5, where the stride is set to 1 and the padding is set to 2 (which in this case is equivalent to _padding = same_). With this configuration, the size of the output tensor is going to be 16× _H_ × _W_. If we use a standard convolution layer, the number of parameters will be 5×5×3×16 = 1200 (without bias), in which this number is obtained based on the equation in Figure 3. The use of bias term is not strictly necessary in this case, but if we do the total number of parameters is going to be (5×5×3+1) × 16 = 1216.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/08/1SC_JZ3uHcNLfyfGrp9UZcQ.png)\nFigure 3. Equation to calculate the number of parameters of a convolution layer [2].\nNow let’s calculate the parameter count of the depthwise separable convolution counterpart to produce the exact same tensor dimension. Following the same formula, we will have 5×5×1×3 = 75 for the depthwise convolution part (without bias). Or if we also account for the biases, then we will have (5×5×1+1) × 3 = 78 trainable params. In the case of depthwise convolution like this, the number of input channels is considered 1 since each kernel is responsible for processing a single channel only. To the pointwise convolution part, the number of parameters will be 1×1×3×16 = 48 (without bias) or (1×1×3+1) × 16 = 64 (with bias). Now to obtain the total number of parameters in the entire depthwise separable convolution process, we can simply calculate 75+48 = 123 (without bias) or 78+64 = 142 (with bias) — that’s nearly 90% reduction in parameter count if we compare it with the standard convolution! In theory, such an extreme drop in parameter count causes the model to have much lower capacity. But that’s just the theory. Later I will show you how MobileNet manages to keep up with other models in terms of accuracy.\n···\n## The Detailed MobileNetV1 Architecture\nFigure 4 below displays the entire MobileNetV1 architecture in detail. The depthwise convolution layers are the rows marked with _dw_ , while the pointwise convolutions are the ones having 1×1 filter shape. Notice that each _dw_ layer is always followed by a 1×1 convolution, indicating that the entire architecture mainly consists of depthwise separable convolutions. Furthermore, if you take a closer look at the architecture, you will see that spatial downsampling is done by the depthwise convolutions that have a stride of 2 (notice the rows with _s2_ in the table). Here you can see that every time we reduce the spatial dimension by half, the number of channels doubles to compensate for the loss of spatial information.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/08/1cSzBXFoKQEm3bmwG_7ttAA.png)\nFigure 4. The entire MobileNetV1 architecture [1].\n### Width and Resolution Multiplier\nThe authors of MobileNet proposed a new parameter tuning mechanism by introducing the so-called _width_ and _resolution multiplier_ , which are formally denoted as _α_ and _ρ_ , respectively. The _α_ parameter can technically be adjusted freely, but authors suggest using either 1.0, 0.75, 0.5, or 0.25. This parameter works by reducing the number of channels produced by all convolution layers. For instance, if we set _α_ to 0.5, the first convolution layer in the network will turn 3-channel input into 16 instead of 32. On the other hand, _ρ_ is used to adjust the spatial dimension of the input tensor. It is important to note that even though ideally we should assign a floating-point number to this parameter, yet in practice it is more preferable to directly determine the actual resolution for the input image. In this case, authors recommend using either 224, 192, 160 and 128, in which the input size of 224×224 corresponds to _ρ_ = 1. The architecture displayed in Figure 4 above follows the default configuration where both _α_ and _ρ_ are set to 1.\n···\n## Experimental Results\nAuthors conducted plenty of experiments to prove the robustness of MobileNet. The first result to discuss is the one displayed in Figure 5 below, where in this experiment they tried to find out how the use of depthwise separable convolution layers affects performance. The second row of the table shows the result obtained by the architecture I showed you earlier in Figure 4, whereas the first row is the result when the layers are replaced with traditional convolutions. Here we can see that the accuracy of MobileNet with traditional CNN is indeed higher than that of the one using depthwise separable convolutions. However, if we take into account the number of multiplications and additions (_mult-adds_) as well as the parameter count, we can clearly see that the one with traditional convolution layers requires much more computational cost and memory usage just to make a slight improvement in accuracy. Thus, with depthwise separable convolutions, even though the model complexity of MobileNet significantly gets reduced, authors proved that the model capacity remains high.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/08/1lcuIPVr_F_pCv86-bJw52A.png)\nFigure 5. Performance comparison between MobileNet with depthwise separable convolution layers (second row) and its full-convolution counterpart (first row) [1].\nThe _α_ and _ρ_ parameters I explained earlier are mainly used to provide flexibility, considering that not all tasks require the highest MobileNet capability. Authors originally conducted experiments on 1000-class ImageNet dataset, but in practice, we might probably only need the model to perform classification on a dataset with fewer number of classes. In such a case, selecting lower values for the two parameters might be preferable as it can speed up the inference process while at the same time the model still has enough capacity to accommodate the classification task. Talking more specifically about _α_ , using smaller value for this parameter causes MobileNet to have lower accuracy. But that’s the result on 1000-class dataset. If our dataset is simpler and has fewer classes, using smaller _α_ might still be fine. In Figure 6 below the values 1.0, 0.75, 0.5, and 0.25 written next to each model correspond to the _α_ used.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/08/17wGywvUtfwFEiZXIXASxEQ.png)\nFigure 6. How width multiplier affects model accuracy, number of operations, and parameter count [1].\nThe same thing also applies to the _ρ_ parameter, which is responsible for changing the resolution of the input image. Figure 7 below displays what the experimental results look like when we use different input resolutions. The results are somewhat similar to the ones in the previous figure, where the accuracy score decreases as we make the input image smaller. It is important to keep in mind that reducing input resolution like this also reduces the number of operations but does not affect the parameter count. This is essentially because the ones counted as parameters are the weights and biases, where in the case of CNN they correspond to the values inside the kernel. So, the parameter count will remain the same as long as we don’t change the configuration of the convolution layers. The number of operations, on the other hand, gets reduced in accordance with the decrease in input resolution since the number of pixels to be processed in smaller images is fewer than in larger images.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/08/1swa0c5z2uWFt2dlTU38EOQ.png)\nFigure 7. How input resolution affects model accuracy, number of operations, and parameter count [1].\nInstead of just comparing different values for _α_ and _ρ_ , authors also compared MobileNet with other popular models. We can see in Figure 8 that the largest MobileNet variant (the one using maximum _α_ and _ρ_) achieved comparable accuracy with GoogLeNet (InceptionV1) and VGG16 while maintaining the lowest computational complexity. This is basically the reason that I named this article _The Tiny Giant_ — lightweight yet powerful.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/08/1Xcwg07330OEAIY9N8u002w.png)\nFigure 8. MobileNet achieves comparable accuracy to popular models while maintaining a much lower computational complexity and parameter count [1].\nAdditionally, authors also compared the smaller MobileNet variant with other small models. What’s to me interesting in Figure 9 is that even though the parameter count of SqueezeNet is lower than MobileNet, the number of operations in MobileNet is over 22 times smaller than SqueezeNet while still maintaining higher accuracy.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/08/1s7ZbSQEYZmWgNaD_Es7OMw.png)\nFigure 9. The performance of the smaller MobileNet variant compared to popular models [1].\n···\n## MobileNetV1 Implementation\nAs we have understood the idea behind MobileNetV1, we can now jump into the code. The architecture I am about to implement is based on the table in Figure 4. As always, the first thing we need to do is to import the required modules.\npython\n\n```\n# Codeblock 1\nimport torch\nimport torch.nn as nn\nfrom torchinfo import summary\n```\n\nNext, we initialize several configurable parameters so that we can adjust the model size according to our needs. In Codeblock 2 below, I denote _α_ as `ALPHA`, which the value can be changed to 0.75, 0.5 or 0.25 if we want the model to be smaller. We don’t specify any variable for _ρ_ since we can directly change `IMAGE_SIZE` to 192, 160 or 128 as we discussed earlier.\npython\n\n```\n# Codeblock 2\nBATCH_SIZE = 1\nIMAGE_SIZE = 224\nIN_CHANNELS = 3\nNUM_CLASSES = 1000\nALPHA = 1\n```\n\n···\n### First Convolution\nIf we go back to Figure 4, we can see that MobileNet essentially only consists of repeating patterns, i.e., a depthwise separable convolution followed by pointwise convolution. However, notice that the first row in the figure does not follow this pattern as it is actually just a standard convolution layer. Due to this reason, we need to create a separate class for this, which I refer to as `FirstConv` in Codeblock 3 below.\npython\n\n```\n# Codeblock 3\nclass FirstConv(nn.Module):\n def __init__(self):\n super().__init__()\n self.conv = nn.Conv2d(in_channels=3, \n out_channels=int(32*ALPHA), #(1)\n kernel_size=3, #(2)\n stride=2, #(3)\n padding=1, #(4)\n bias=False) #(5)\n self.bn = nn.BatchNorm2d(num_features=int(32*ALPHA))\n self.relu = nn.ReLU()\n \n def forward(self, x):\n x = self.relu(self.bn(self.conv(x)))\n return x\n```\n\nRemember that MobileNet follows the _conv-BN-ReLU_ structure. Thus, we need to initialize these three layers within the `__init__()` method of this class. The convolution layer itself is set to accept 3 input channels and output 32 channels. Since we want this number of output channels to be adjustable, we need to multiply it with `ALPHA` at the line marked with `#(1)`. Keep in mind that we need to change the datatype to integer after the multiplication since having a floating-point number for channel count is just nonsense. Next, at line `#(2)` and `#(3)` we set the kernel size to 3 and the stride to 2. With this configuration, the spatial dimension of the resulting tensor is going to be half that of the input. Additionally, using kernel of size 3×3 like this implicitly requires us to set the padding to 1 to implement _padding = same_ (`#(4)`). In this case we are not going to utilize the bias term, which is the reason that we set the `bias` parameter to `False` (`#(5)`). This is actually a standard practice when we use the _conv-BN-ReLU_ structure, since at the end of the day the value distribution of the convolution kernels will be centered around 0 again by the batch normalization layer, cancelling out the biases applied within the convolution kernel.\nIn order to find out whether the `FirstConv` class works properly, we are going to test it with the Codeblock 4 below. Here we initialize the layer and pass a tensor simulating a single RGB image of size 224×224. You can see in the resulting output that our convolution layer successfully downsampled the spatial dimension to 112×112 while at the same time expanding the number of channels to 32.\npython\n\n```\n# Codeblock 4\nfirst_conv = FirstConv()\nx = torch.randn((1, 3, 224, 224))\n\nout = first_conv(x)\nout.shape\n```\n\ntext\n\n```\n# Codeblock 4 Output\ntorch.Size([1, 32, 112, 112])\n```\n\n···\n### Depthwise Separable Convolutions\nAs the first convolution is done, we can now work on the repeating depthwise-pointwise layers. Since this pattern is the core idea of depthwise separable convolution, in the following code I wrap the two types of conv layers in a class called `DepthwiseSeparableConv`.\npython\n\n```\n# Codeblock 5\nclass DepthwiseSeparableConv(nn.Module):\n def __init__(self, in_channels, out_channels, downsample=False): #(1)\n super().__init__()\n \n in_channels = int(in_channels*ALPHA) #(2)\n out_channels = int(out_channels*ALPHA) #(3) \n \n if downsample: #(4)\n stride = 2\n else:\n stride = 1\n \n self.dwconv = nn.Conv2d(in_channels=in_channels,\n out_channels=in_channels, #(5)\n kernel_size=3, #(6)\n stride=stride, #(7)\n padding=1,\n groups=in_channels, #(8)\n bias=False)\n self.bn0 = nn.BatchNorm2d(num_features=in_channels) #(9)\n \n self.pwconv = nn.Conv2d(in_channels=in_channels, \n out_channels=out_channels, #(10)\n kernel_size=1, #(11)\n stride=1, #(12)\n padding=0, #(13)\n groups=1, #(14)\n bias=False)\n self.bn1 = nn.BatchNorm2d(num_features=out_channels) #(15)\n \n self.relu = nn.ReLU() #(16)\n\n def forward(self, x):\n print(f'original\\t: {x.size()}')\n \n x = self.relu(self.bn0(self.dwconv(x)))\n print(f'after dw conv\\t: {x.size()}')\n \n x = self.relu(self.bn1(self.pwconv(x)))\n print(f'after pw conv\\t: {x.size()}')\n \n return x\n```\n\nDifferent from `FirstConv` which does not take any input argument in the initialization phase, here we set the `DepthwiseSeparableConv` class to take several inputs as shown at line `#(1)` in Codeblock 5 above. I do this because we want the class to be reusable across all depthwise separable convolution layers throughout the entire network, in which each of them has slightly different behaviors from one another.\nWe can see in Figure 4 that after the 3-channel image is expanded to 32 by the first layer, this channel count increases to 64, 128, and so on all the way to 1024 in the subsequent processes. This is basically the reason that I set this class to accept the number of input and output channels (`in_channels` and `out_channels`) so that we can initialize the layer with flexible channel configurations. It is also important to keep in mind that we need to adjust this channel counts based on `ALPHA`. This can simply be done using the code at line `#(2)` and `#(3)`. Furthermore, here I also create a flag called `downsample` as the input parameter which by default is set to `False`. This flag is responsible to determine whether the layer will reduce the spatial dimension. Again, if you go back to Figure 4, you will notice that there are conditions where we reduce the spatial dimension by half and there are also some other conditions where the dimension is preserved. Every time we want to perform downsampling, we need to set the stride to 2, but if we don’t, we will set this parameter to 1 instead (`#(4)`).\nStill with the Codeblock 5 above, the next thing we need to do is to initialize the layers themselves. As we have discussed earlier, the depthwise convolution is responsible to capture spatial relationships between pixels, which is actually the reason that the kernel size is set to 3×3 (`#(6)`). In order for the input channels to be processed independently of each other, we can simply set the `groups` and `out_channels` parameters to be the same as the number of input channels itself (`#(8)` and `#(5)`). It is worth noting that if we set `out_channels` to be more than the number of input channels — say, twice as large — then we will have each channel processed by 2 kernels. Lastly for the depthwise convolution layer, the `stride` parameter at line `#(7)` can either be 1 or 2 which is determined according to the `downsampling` flag we discussed earlier.\nMeanwhile, the pointwise convolution uses 1×1 kernel (`#(11)`) since it is not intended to capture spatial information. This is actually the reason that we set the padding to 0 (`#(13)`) because there is no way this kernel size can reduce spatial dimension on its own. The `groups` parameter, on the other hand, is set to 1 (`#(14)`) because we want this layer to capture information from all channels at once. Unlike the depthwise convolution layer, here we can employ as many kernels as needed which corresponds to the number of channels in the resulting output tensor (`#(10)`). Meanwhile, the stride is set fixed to 1 (`#(12)`) since we will never perform downsampling with this layer.\nHere we need to initialize two separate batch normalization layers to be placed after the depthwise and pointwise convs (`#(9)` and `#(15)`). As for the ReLU activation function, we only need to initialize it once (`#(16)`) since it is just a mapping function without any trainable parameters. Thanks to this, we can reuse the same ReLU instance multiple times within the network.\nNow let’s see if our `DepthwiseSeparableConv` class works properly by passing a dummy tensor through it. Here I have prepared two test cases for this class. The first one is when we don’t perform downsampling and the second one is when we do. In Figure 10 below, the two tests I want to perform involve the use of the layers highlighted in green and blue, respectively.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/08/1E94NEh0QuFm_LiQRnp5VTw.png)\nFigure 10. The layers highlighted in green and blue are the ones we are going to simulate to test the DepthwiseSeparableConv class [1][2].\nTo create the green part, we can simply use the `DepthwiseSeparableConv` class and set the number of input and output channels to 32 and 64 as seen in Codeblock 6 below (`#(1–2)`). Passing `downsample = False` is not quite necessary since we already set it as the default configuration (`#(3)`) — but I do this anyway just for the sake of clarity. The shape of the dummy tensor `x` is also configured to have the size of 32×112×112, in which it matches exactly with the input shape of the layer (`#(4)`).\npython\n\n```\n# Codeblock 6\ndepthwise_sep_conv = DepthwiseSeparableConv(in_channels=32, #(1)\n out_channels=64, #(2)\n downsample=False) #(3)\nx = torch.randn((1, int(32*ALPHA), 112, 112)) #(4)\n\nx = depthwise_sep_conv(x)\n```\n\nIf you run the above code, the following output should appear on your screen. Here you can see that the depthwise convolution layer returns a tensor of the exact same shape as the input (`#(1)`). The number of channels then doubles from 32 to 64 after the tensor being processed by the pointwise convolution (`#(2)`). This result proves that our `DepthwiseSeparableConv` class works properly for non-downsampling process. We will use this output tensor in the subsequent test as the input for the blue layer.\ntext\n\n```\n# Codeblock 6 Output\noriginal : torch.Size([1, 32, 112, 112])\nafter dw conv : torch.Size([1, 32, 112, 112]) #(1)\nafter pw conv : torch.Size([1, 64, 112, 112]) #(2)\n```\n\nThe second test is quite similar to the first one, except that here we need to configure the model based on the number of input and output channels of the blue layer. Not only that, the `downsample` parameter also needs to be set to `True` since we want the layer to reduce the spatial dimension by half. See Codeblock 7 below for the details.\npython\n\n```\n# Codeblock 7\ndepthwise_sep_conv = DepthwiseSeparableConv(in_channels=64, \n out_channels=128,\n downsample=True)\n\nx = depthwise_sep_conv(x)\n```\n\ntext\n\n```\n# Codeblock 7 Output\noriginal : torch.Size([1, 64, 112, 112])\nafter dw conv : torch.Size([1, 64, 56, 56]) #(1)\nafter pw conv : torch.Size([1, 128, 56, 56]) #(2)\n```\n\nWe can see in the above output that the spatial downsampling works properly as the depthwise convolution layer successfully converted the 112×112 image to 56×56 (`#(1)`). The channel axis is finally expanded to 128 with the help of the pointwise convolution layer (`#(2)`), making it ready to be fed into the subsequent layer. \nBased on the two tests I demonstrated above, it is proven that our `DepthwiseSeparableConv` class is correct and thus ready to be used to construct the entire MobileNetV1 architecture.\n···\n### The Entire MobileNetV1 Architecture\nI wrap everything within a class which I refer to as `MobileNetV1`. Since this class is quite long, I break it down into Codeblock 8a and 8b. If you want to run this code yourself, just ensure that these two codeblocks are written within the same notebook cell.\nNow let’s start from the `__init__()` method of this class. The first thing to do here is to initialize the `FirstConv` layer we created earlier (`#(1)`). The next layers we need to initialize are the core idea of MobileNet, i.e., the depthwise separable convolutions, in which every single of those layers consists of depthwise and pointwise convs. In this implementation I decided to name these pairs starting from `depthwise_sep_conv0` all the way to `depthwise_sep_conv8`. If you go back to Figure 4, you will notice that the downsampling process is done alternately with the non-downsampling layers. This can simply be implemented by setting the `downsample` flag to `True` for the layers number 1, 3, 5 and 7. The `depthwise_sep_conv6` is a bit special since it is actually not a standalone layer. Rather, it is a bunch of depthwise separable convolutions of the exact same specification repeated 5 times.\npython\n\n```\n# Codeblock 8a\nclass MobileNetV1(nn.Module):\n def __init__(self):\n super().__init__()\n \n self.first_conv = FirstConv() #(1)\n \n self.depthwise_sep_conv0 = DepthwiseSeparableConv(in_channels=32, \n out_channels=64)\n \n self.depthwise_sep_conv1 = DepthwiseSeparableConv(in_channels=64, \n out_channels=128, \n downsample=True)\n \n self.depthwise_sep_conv2 = DepthwiseSeparableConv(in_channels=128, \n out_channels=128)\n \n self.depthwise_sep_conv3 = DepthwiseSeparableConv(in_channels=128, \n out_channels=256, \n downsample=True)\n \n self.depthwise_sep_conv4 = DepthwiseSeparableConv(in_channels=256, \n out_channels=256)\n \n self.depthwise_sep_conv5 = DepthwiseSeparableConv(in_channels=256, \n out_channels=512, \n downsample=True)\n \n self.depthwise_sep_conv6 = nn.ModuleList(\n [DepthwiseSeparableConv(in_channels=512, out_channels=512) for _ in range(5)]\n )\n \n self.depthwise_sep_conv7 = DepthwiseSeparableConv(in_channels=512, \n out_channels=1024, \n downsample=True)\n \n self.depthwise_sep_conv8 = DepthwiseSeparableConv(in_channels=1024, #(2)\n out_channels=1024)\n \n num_out_channels = self.depthwise_sep_conv8.pwconv.out_channels #(3)\n \n self.avgpool = nn.AdaptiveAvgPool2d(output_size=(1,1)) #(4)\n self.fc = nn.Linear(in_features=num_out_channels, #(5)\n out_features=NUM_CLASSES)\n self.softmax = nn.Softmax(dim=1) #(6)\n```\n\nAs we have reached the very last `DepthwiseSeparableConv` layer (`#(2)`), what we need to do next is to initialize three more layers: an average pooling layer (`#(4)`), a fully-connected layer (`#(5)`), and a softmax activation function layer (`#(6)`). One thing that you need to keep in mind is that the number of output channels produced by the `depthwise_sep_conv8` is not always 1024 even though it appears to be fixed to that number. In fact, this output channel count will be different if we change the `ALPHA`. In order to make our implementation adaptive to such changes, we need to take the actual number of output channels generated using the code at line `#(3)`, which will then be used as the input size of the fully-connected layer (`#(5)`).\nRegarding the `forward()` method in Codeblock 8b, I think there is nothing I need to explain since what we basically do here is just to pass a tensor from one layer to the subsequent ones.\npython\n\n```\n# Codeblock 8b\n def forward(self, x):\n x = self.first_conv(x)\n print(f\"after first_conv\\t\\t: {x.shape}\")\n \n x = self.depthwise_sep_conv0(x)\n print(f\"after depthwise_sep_conv0\\t: {x.shape}\")\n \n x = self.depthwise_sep_conv1(x)\n print(f\"after depthwise_sep_conv1\\t: {x.shape}\")\n \n x = self.depthwise_sep_conv2(x)\n print(f\"after depthwise_sep_conv2\\t: {x.shape}\")\n \n x = self.depthwise_sep_conv3(x)\n print(f\"after depthwise_sep_conv3\\t: {x.shape}\")\n \n x = self.depthwise_sep_conv4(x)\n print(f\"after depthwise_sep_conv4\\t: {x.shape}\")\n \n x = self.depthwise_sep_conv5(x)\n print(f\"after depthwise_sep_conv5\\t: {x.shape}\")\n \n for i, layer in enumerate(self.depthwise_sep_conv6):\n x = layer(x)\n print(f\"after depthwise_sep_conv6 #{i}\\t: {x.shape}\")\n \n x = self.depthwise_sep_conv7(x)\n print(f\"after depthwise_sep_conv7\\t: {x.shape}\")\n \n x = self.depthwise_sep_conv8(x)\n print(f\"after depthwise_sep_conv8\\t: {x.shape}\")\n \n x = self.avgpool(x)\n print(f\"after avgpool\\t\\t\\t: {x.shape}\")\n \n x = torch.flatten(x, start_dim=1)\n print(f\"after flatten\\t\\t\\t: {x.shape}\")\n \n x = self.fc(x)\n print(f\"after fc\\t\\t\\t: {x.shape}\")\n \n x = self.softmax(x)\n print(f\"after softmax\\t\\t\\t: {x.shape}\")\n \n return x\n```\n\nNow let’s see if our MobileNetV1 works properly by running the following test code.\npython\n\n```\n# Codeblock 9\nmobilenetv1 = MobileNetV1()\nx = torch.randn((BATCH_SIZE, IN_CHANNELS, IMAGE_SIZE, IMAGE_SIZE))\n\nout = mobilenetv1(x)\n```\n\nAnd below is what the output looks like. Here we can see that our dummy image tensor successfully went through the `first_conv` layer all the way to the final output layer. During the convolution phase, we can see that the spatial dimension decreased as we get into the deeper layers while at the same time the number of channels increased. Afterwards, we apply an average pooling layer which works by taking the average value from each channel. We can say that at this point every single channel of size 7×7 is now represented as a single value, which is actually the reason that the spatial dimension dropped to 1×1 (`#(1)`). This tensor is then flattened (`#(2)`) so that we can process it further with the fully-connected layer (`#(3)`).\ntext\n\n```\n# Codeblock 9 Output\nafter first_conv : torch.Size([1, 32, 112, 112])\nafter depthwise_sep_conv0 : torch.Size([1, 64, 112, 112])\nafter depthwise_sep_conv1 : torch.Size([1, 128, 56, 56])\nafter depthwise_sep_conv2 : torch.Size([1, 128, 56, 56])\nafter depthwise_sep_conv3 : torch.Size([1, 256, 28, 28])\nafter depthwise_sep_conv4 : torch.Size([1, 256, 28, 28])\nafter depthwise_sep_conv5 : torch.Size([1, 512, 14, 14])\nafter depthwise_sep_conv6 #0 : torch.Size([1, 512, 14, 14])\nafter depthwise_sep_conv6 #1 : torch.Size([1, 512, 14, 14])\nafter depthwise_sep_conv6 #2 : torch.Size([1, 512, 14, 14])\nafter depthwise_sep_conv6 #3 : torch.Size([1, 512, 14, 14])\nafter depthwise_sep_conv6 #4 : torch.Size([1, 512, 14, 14])\nafter depthwise_sep_conv7 : torch.Size([1, 1024, 7, 7])\nafter depthwise_sep_conv8 : torch.Size([1, 1024, 7, 7])\nafter avgpool : torch.Size([1, 1024, 1, 1]) #(1)\nafter flatten : torch.Size([1, 1024]) #(2)\nafter fc : torch.Size([1, 1000]) #(3)\nafter softmax : torch.Size([1, 1000])\n```\n\nIf you want an even more detailed architecture, we can use the `summary()` function from `torchinfo` we imported earlier. If you scroll down the resulting output below, we can see that this model contains approximately 4.2 million trainable parameters, in which this number matches with the one written in Figure 5, 6, 7 and 8. I also tried to initialize the same model with different `ALPHA`, and I found that the numbers match with the table in Figure 6. Thanks to this reason, I think our MobileNetV1 implementation is correct.\npython\n\n```\n# Codeblock 10\nmobilenetv1 = MobileNetV1()\nsummary(mobilenetv1, input_size=(BATCH_SIZE, IN_CHANNELS, IMAGE_SIZE, IMAGE_SIZE))\n```\n\ntext\n\n```\n# Codeblock 10 Output\n==========================================================================================\nLayer (type:depth-idx) Output Shape Param #\n==========================================================================================\nMobileNetV1 [1, 1000] --\n├─FirstConv: 1-1 [1, 32, 112, 112] --\n│ └─Conv2d: 2-1 [1, 32, 112, 112] 864\n│ └─BatchNorm2d: 2-2 [1, 32, 112, 112] 64\n│ └─ReLU: 2-3 [1, 32, 112, 112] --\n├─DepthwiseSeparableConv: 1-2 [1, 64, 112, 112] --\n│ └─Conv2d: 2-4 [1, 32, 112, 112] 288\n│ └─BatchNorm2d: 2-5 [1, 32, 112, 112] 64\n│ └─ReLU: 2-6 [1, 32, 112, 112] --\n│ └─Conv2d: 2-7 [1, 64, 112, 112] 2,048\n│ └─BatchNorm2d: 2-8 [1, 64, 112, 112] 128\n│ └─ReLU: 2-9 [1, 64, 112, 112] --\n├─DepthwiseSeparableConv: 1-3 [1, 128, 56, 56] --\n│ └─Conv2d: 2-10 [1, 64, 56, 56] 576\n│ └─BatchNorm2d: 2-11 [1, 64, 56, 56] 128\n│ └─ReLU: 2-12 [1, 64, 56, 56] --\n│ └─Conv2d: 2-13 [1, 128, 56, 56] 8,192\n│ └─BatchNorm2d: 2-14 [1, 128, 56, 56] 256\n│ └─ReLU: 2-15 [1, 128, 56, 56] --\n├─DepthwiseSeparableConv: 1-4 [1, 128, 56, 56] --\n│ └─Conv2d: 2-16 [1, 128, 56, 56] 1,152\n│ └─BatchNorm2d: 2-17 [1, 128, 56, 56] 256\n│ └─ReLU: 2-18 [1, 128, 56, 56] --\n│ └─Conv2d: 2-19 [1, 128, 56, 56] 16,384\n│ └─BatchNorm2d: 2-20 [1, 128, 56, 56] 256\n│ └─ReLU: 2-21 [1, 128, 56, 56] --\n├─DepthwiseSeparableConv: 1-5 [1, 256, 28, 28] --\n│ └─Conv2d: 2-22 [1, 128, 28, 28] 1,152\n│ └─BatchNorm2d: 2-23 [1, 128, 28, 28] 256\n│ └─ReLU: 2-24 [1, 128, 28, 28] --\n│ └─Conv2d: 2-25 [1, 256, 28, 28] 32,768\n│ └─BatchNorm2d: 2-26 [1, 256, 28, 28] 512\n│ └─ReLU: 2-27 [1, 256, 28, 28] --\n├─DepthwiseSeparableConv: 1-6 [1, 256, 28, 28] --\n│ └─Conv2d: 2-28 [1, 256, 28, 28] 2,304\n│ └─BatchNorm2d: 2-29 [1, 256, 28, 28] 512\n│ └─ReLU: 2-30 [1, 256, 28, 28] --\n│ └─Conv2d: 2-31 [1, 256, 28, 28] 65,536\n│ └─BatchNorm2d: 2-32 [1, 256, 28, 28] 512\n│ └─ReLU: 2-33 [1, 256, 28, 28] --\n├─DepthwiseSeparableConv: 1-7 [1, 512, 14, 14] --\n│ └─Conv2d: 2-34 [1, 256, 14, 14] 2,304\n│ └─BatchNorm2d: 2-35 [1, 256, 14, 14] 512\n│ └─ReLU: 2-36 [1, 256, 14, 14] --\n│ └─Conv2d: 2-37 [1, 512, 14, 14] 131,072\n│ └─BatchNorm2d: 2-38 [1, 512, 14, 14] 1,024\n│ └─ReLU: 2-39 [1, 512, 14, 14] --\n├─ModuleList: 1-8 -- --\n│ └─DepthwiseSeparableConv: 2-40 [1, 512, 14, 14] --\n│ │ └─Conv2d: 3-1 [1, 512, 14, 14] 4,608\n│ │ └─BatchNorm2d: 3-2 [1, 512, 14, 14] 1,024\n│ │ └─ReLU: 3-3 [1, 512, 14, 14] --\n│ │ └─Conv2d: 3-4 [1, 512, 14, 14] 262,144\n│ │ └─BatchNorm2d: 3-5 [1, 512, 14, 14] 1,024\n│ │ └─ReLU: 3-6 [1, 512, 14, 14] --\n│ └─DepthwiseSeparableConv: 2-41 [1, 512, 14, 14] --\n│ │ └─Conv2d: 3-7 [1, 512, 14, 14] 4,608\n│ │ └─BatchNorm2d: 3-8 [1, 512, 14, 14] 1,024\n│ │ └─ReLU: 3-9 [1, 512, 14, 14] --\n│ │ └─Conv2d: 3-10 [1, 512, 14, 14] 262,144\n│ │ └─BatchNorm2d: 3-11 [1, 512, 14, 14] 1,024\n│ │ └─ReLU: 3-12 [1, 512, 14, 14] --\n│ └─DepthwiseSeparableConv: 2-42 [1, 512, 14, 14] --\n│ │ └─Conv2d: 3-13 [1, 512, 14, 14] 4,608\n│ │ └─BatchNorm2d: 3-14 [1, 512, 14, 14] 1,024\n│ │ └─ReLU: 3-15 [1, 512, 14, 14] --\n│ │ └─Conv2d: 3-16 [1, 512, 14, 14] 262,144\n│ │ └─BatchNorm2d: 3-17 [1, 512, 14, 14] 1,024\n│ │ └─ReLU: 3-18 [1, 512, 14, 14] --\n│ └─DepthwiseSeparableConv: 2-43 [1, 512, 14, 14] --\n│ │ └─Conv2d: 3-19 [1, 512, 14, 14] 4,608\n│ │ └─BatchNorm2d: 3-20 [1, 512, 14, 14] 1,024\n│ │ └─ReLU: 3-21 [1, 512, 14, 14] --\n│ │ └─Conv2d: 3-22 [1, 512, 14, 14] 262,144\n│ │ └─BatchNorm2d: 3-23 [1, 512, 14, 14] 1,024\n│ │ └─ReLU: 3-24 [1, 512, 14, 14] --\n│ └─DepthwiseSeparableConv: 2-44 [1, 512, 14, 14] --\n│ │ └─Conv2d: 3-25 [1, 512, 14, 14] 4,608\n│ │ └─BatchNorm2d: 3-26 [1, 512, 14, 14] 1,024\n│ │ └─ReLU: 3-27 [1, 512, 14, 14] --\n│ │ └─Conv2d: 3-28 [1, 512, 14, 14] 262,144\n│ │ └─BatchNorm2d: 3-29 [1, 512, 14, 14] 1,024\n│ │ └─ReLU: 3-30 [1, 512, 14, 14] --\n├─DepthwiseSeparableConv: 1-9 [1, 1024, 7, 7] --\n│ └─Conv2d: 2-45 [1, 512, 7, 7] 4,608\n│ └─BatchNorm2d: 2-46 [1, 512, 7, 7] 1,024\n│ └─ReLU: 2-47 [1, 512, 7, 7] --\n│ └─Conv2d: 2-48 [1, 1024, 7, 7] 524,288\n│ └─BatchNorm2d: 2-49 [1, 1024, 7, 7] 2,048\n│ └─ReLU: 2-50 [1, 1024, 7, 7] --\n├─DepthwiseSeparableConv: 1-10 [1, 1024, 7, 7] --\n│ └─Conv2d: 2-51 [1, 1024, 7, 7] 9,216\n│ └─BatchNorm2d: 2-52 [1, 1024, 7, 7] 2,048\n│ └─ReLU: 2-53 [1, 1024, 7, 7] --\n│ └─Conv2d: 2-54 [1, 1024, 7, 7] 1,048,576\n│ └─BatchNorm2d: 2-55 [1, 1024, 7, 7] 2,048\n│ └─ReLU: 2-56 [1, 1024, 7, 7] --\n├─AdaptiveAvgPool2d: 1-11 [1, 1024, 1, 1] --\n├─Linear: 1-12 [1, 1000] 1,025,000\n├─Softmax: 1-13 [1, 1000] --\n==========================================================================================\nTotal params: 4,231,976\nTrainable params: 4,231,976\nNon-trainable params: 0\nTotal mult-adds (Units.MEGABYTES): 568.76\n==========================================================================================\nInput size (MB): 0.60\nForward/backward pass size (MB): 80.69\nParams size (MB): 16.93\nEstimated Total Size (MB): 98.22\n==========================================================================================\n```\n\n···\n## Ending\nThat was pretty much everything about MobileNetV1. I do encourage you to play around with the above model. If you want to train it for image classification, you can adjust the number of neurons in the output layer according to the number of classes available in your dataset. You can also try to explore different _α_ and _ρ_ to find the values that suit best for your case in terms of accuracy and efficiency. Furthermore, since this implementation is literally done from scratch, it is also possible to change other things that are not explicitly mentioned in the paper, such as the number of repeats of the `depthwise_sep_conv6` layer, or even using _α_ and _ρ_ greater than 1. And well, there are basically lots of things to explore from our MobileNetV1 implementation! You can also access the code used in this article in my GitHub repository [3].\nFeel free to comment if you spot any mistake in my explanation or the code. Thanks for reading!\n···\n## References\n[1] Andrew G. Howard _et al._ MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. Arxiv. [Accessed April 7, 2025].\n[2] Image created originally by author.\n[3] MuhammadArdiPutra. The Tiny Giant — MobileNetV1. GitHub. [Accessed April 7, 2025].\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](https://assets.insightmediagroup.io/media/1787657598287_taxhya.jpeg)\nWritten By\n## [Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\nDeep learning storyteller\n[See all from Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\n[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Artificial Intelligence](https://towardsdatascience.com/tag/artificial-intelligence/)[Machine Learning](https://towardsdatascience.com/tag/machine-learning/)[Python](https://towardsdatascience.com/tag/python/)[Pytorch](https://towardsdatascience.com/tag/pytorch/)\nShare This Article\n * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fthe-tiny-giant-mobilenetv1%2F&title=MobileNetV1%20Paper%20Walkthrough%3A%20The%20Tiny%20Giant)\n * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fthe-tiny-giant-mobilenetv1%2F&title=MobileNetV1%20Paper%20Walkthrough%3A%20The%20Tiny%20Giant)\n * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fthe-tiny-giant-mobilenetv1%2F&text=MobileNetV1%20Paper%20Walkthrough%3A%20The%20Tiny%20Giant)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2022/07/0eBPEJGriFHYPpBnA-scaled.jpg)](https://towardsdatascience.com/new-perspectives-on-deep-learning-neural-networks-and-ai-a335547ac32c/)\n## [New Perspectives on Deep Learning, Neural Networks, and AI](https://towardsdatascience.com/new-perspectives-on-deep-learning-neural-networks-and-ai-a335547ac32c/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nOur weekly selection of must-read Editors' Picks and original features\n[TDS Editors](https://towardsdatascience.com/author/towardsdatascience/)\nJuly 14, 20223 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2022/04/1lhDKcrT7AY1-E_EfFiT77A-scaled.jpeg)](https://towardsdatascience.com/python-and-the-truth-90cf08380246/)\n## [Python and the Truth](https://towardsdatascience.com/python-and-the-truth-90cf08380246/)\n[Programming](https://towardsdatascience.com/category/programming/)\nWe will explore the definitions behind Truth values, how to operate on them and avoid nasty bugs, and how to tune the Truth of our classes.\n[Pere Miquel Brull](https://towardsdatascience.com/author/pmbrull/)\nApril 4, 20224 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/10/1mHWENrcV7kAF-81OCWqhNw.png)](https://towardsdatascience.com/an-intuitive-explanation-of-self-attention-4f72709638e1/)\n## [An intuitive explanation of Self Attention](https://towardsdatascience.com/an-intuitive-explanation-of-self-attention-4f72709638e1/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nA step-by-step explanation of the multi-headed self-attention block\n[Saketh Kotamraju](https://towardsdatascience.com/author/saketh.kotamraju/)\nOctober 7, 202010 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2022/05/1KKYZK_GTHja7MIKqVO26Sw-scaled.jpeg)](https://towardsdatascience.com/some-key-challenges-in-building-an-ai-model-for-medical-diagnosis-63f7438f14a/)\n## [Some Key Challenges in Building an AI Model for Medical Diagnosis](https://towardsdatascience.com/some-key-challenges-in-building-an-ai-model-for-medical-diagnosis-63f7438f14a/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nBeginner's Guide to Working with AI in Medicine\n[Ishtiak Mahmud](https://towardsdatascience.com/author/ishtiakmahmud6/)\nSeptember 12, 20205 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/09/0KYPDob2wP1N6BIxD-scaled.jpg)](https://towardsdatascience.com/introduction-to-pca-in-python-with-sklearn-pandas-and-matplotlib-476880f30238/)\n## [Introduction to PCA in Python with Sklearn, Pandas, and Matplotlib](https://towardsdatascience.com/introduction-to-pca-in-python-with-sklearn-pandas-and-matplotlib-476880f30238/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nLearn the intuition behind PCA in Python and Sklearn by transforming a multidimensional dataset into an arbitrary number of dimensions and...\n[Andrea D'Agostino](https://towardsdatascience.com/author/thedrewdag/)\nSeptember 6, 202313 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/07/0fNLyrYP0O5f127WW-scaled.jpg)](https://towardsdatascience.com/class-activation-mapping-using-transfer-learning-of-resnet50-e8ca7cfd657e/)\n## [Class Activation Mapping](https://towardsdatascience.com/class-activation-mapping-using-transfer-learning-of-resnet50-e8ca7cfd657e/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nA powerful method for object detection and debugging\n[Patrick Brus](https://towardsdatascience.com/author/brus-patrick63/)\nJuly 23, 20205 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/09/1VwsZyLljie10avMyw_fZUA-scaled.jpeg)](https://towardsdatascience.com/try-these-5-tips-to-optimize-your-python-code-c7e0ccdf486a/)\n## [Try These 5 Tips To Optimize Your Python Code](https://towardsdatascience.com/try-these-5-tips-to-optimize-your-python-code-c7e0ccdf486a/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nQuick tips to write better python code\n[Pranjal Saxena](https://towardsdatascience.com/author/pranjalai/)\nSeptember 24, 20215 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2022/08/1AIVs5FP7voKmNtChT61beQ-scaled.jpeg)](https://towardsdatascience.com/concept-learning-making-your-network-interpretable-735a0698fc03/)\n## [Concept Learning: Making your network interpretable](https://towardsdatascience.com/concept-learning-making-your-network-interpretable-735a0698fc03/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nOver the last decade, neural networks have been showing superb performance across a large variety of datasets and problems. While metrics...\n[Lukas Huber](https://towardsdatascience.com/author/lukas.huber1/)\nAugust 2, 20225 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2022/06/1mcRohIUCivp8yu-jXFDG_A-scaled.jpeg)](https://towardsdatascience.com/arma-forecasting-for-non-gaussian-time-series-data-using-copulas-45a3a28f69e5/)\n## [ARMA Forecasting for Non-Gaussian Time-Series Data Using Copulas](https://towardsdatascience.com/arma-forecasting-for-non-gaussian-time-series-data-using-copulas-45a3a28f69e5/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nIt's all fun and games until your time-series is not Normal anymore\n[Sarem Seitz](https://towardsdatascience.com/author/sarem-seitz/)\nJune 17, 20229 min read\n\n\n * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n * [X](https://x.com/TDataScience)\n * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n * [Threads](https://www.threads.net/@towardsdatascience)\n * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n#### Recommended Articles \nClose\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500) How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ \"How Relevance Models Foreshadowed Transformers for NLP\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n\n\n", "body_base_path": "attachments/c3cb7fd8c186/pages/", "body_markdown_path": "attachments/c3cb7fd8c186/pages/archived-body.md", "local_html": "attachments/c3cb7fd8c186/pages/archived-body.html", "body_source": "prior_capture" }