![Revisit consent button](../images/body-df29beacd7496220cc94.svg) We value your privacy We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies. Customise Reject All Accept All Customise Consent Preferences ![](../images/body-2159bb67f3de8ad870eb.svg) We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below. The cookies that are categorised as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more NecessaryAlways Active Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data. * Cookie BCTempID * Duration 10 minutes * Description No description available. * Cookie __cf_bm * Duration 1 hour * Description This cookie, set by Cloudflare, is used to support Cloudflare Bot Management. * Cookie AWSALBCORS * Duration 7 days * Description Amazon Web Services set this cookie for load balancing. * Cookie _cfuvid * Duration session * Description Cloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services * Cookie li_gc * Duration 6 months * Description Linkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes. * Cookie __hssrc * Duration session * Description This cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session. * Cookie __hssc * Duration 1 hour * Description HubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie. * Cookie wpEmojiSettingsSupports * Duration session * Description WordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly. * Cookie BCSessionID * Duration 1 year 1 month 4 days * Description Blueconic sets this cookie as a unique identifier for the BlueConic profile. * Cookie _octo * Duration 1 year * Description No description available. * Cookie logged_in * Duration 1 year * Description No description available. * Cookie __Secure-YEC * Duration past * Description YouTube sets this cookie to stores the user's video player preferences using embedded YouTube video * Cookie __eoi * Duration 6 months * Description Description is currently not available. * Cookie AWSALBTGCORS * Duration 7 days * Description No description available. * Cookie login-status-p * Duration past * Description Description is currently not available. * Cookie AWSALBTG * Duration 7 days * Description No description available. * Cookie csrf_token * Duration session * Description No description available. * Cookie token_v2 * Duration 1 day * Description Description is currently not available. * Cookie D * Duration 1 year * Description Description is currently not available. * Cookie PHPSESSID * Duration session * Description This cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed. * Cookie VISITOR_PRIVACY_METADATA * Duration 6 months * Description YouTube sets this cookie to store the user's cookie consent state for the current domain. * Cookie cookietest * Duration session * Description The cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience. * Cookie __Host-airtable-session * Duration 1 year * Description This cookie is used to enable us to integrate the services of Airtable. * Cookie __Host-airtable-session.sig * Duration 1 year * Description This cookie is used to enable us to integrate the services of Airtable. * Cookie m * Duration 1 year 1 month 4 days * Description Stripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly. * Cookie BIGipServer* * Duration session * Description Marketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. * Cookie __cfruid * Duration session * Description Cloudflare sets this cookie to identify trusted web traffic. * Cookie _GRECAPTCHA * Duration 6 months * Description Google Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks. * Cookie __Secure-YNID * Duration 6 months * Description Google cookie used to protect user security and prevent fraud, especially during the login process. * Cookie cookieyes-consent * Duration 1 year * Description CookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors. Functional Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features. * Cookie lidc * Duration 1 day * Description LinkedIn sets the lidc cookie to facilitate data center selection. * Cookie brw * Duration 1 year * Description No description available. * Cookie brwConsent * Duration 5 minutes * Description Description is currently not available. * Cookie WMF-Uniq * Duration 1 year * Description Description is currently not available. * Cookie loom_anon_comment * Duration 1 year * Description No description available. * Cookie loom_referral_video * Duration session * Description Description is currently not available. * Cookie VISITOR_INFO1_LIVE * Duration 6 months * Description A cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface. * Cookie yt-remote-connected-devices * Duration Never Expires * Description YouTube sets this cookie to store the user's video preferences using embedded YouTube videos. * Cookie ytidb::LAST_RESULT_ENTRY_KEY * Duration Never Expires * Description The cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future. * Cookie yt-remote-device-id * Duration Never Expires * Description YouTube sets this cookie to store the user's video preferences using embedded YouTube videos. * Cookie yt-remote-session-name * Duration session * Description The yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video. * Cookie yt-remote-fast-check-period * Duration session * Description The yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos. * Cookie yt-remote-session-app * Duration session * Description The yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player. * Cookie yt-remote-cast-available * Duration session * Description The yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player. * Cookie yt-remote-cast-installed * Duration session * Description The yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video. * Cookie cp_session * Duration 3 months * Description Codepen sets this cookie for Help systems found in the website. * Cookie loid * Duration 1 year 1 month 4 days * Description This cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform. Analytics Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc. * Cookie __hstc * Duration 6 months * Description Hubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session). * Cookie hubspotutk * Duration 6 months * Description HubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts. * Cookie _ga * Duration 1 year 1 month 4 days * Description Google Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors. * Cookie _ga_* * Duration 1 year 1 month 4 days * Description Google Analytics sets this cookie to store and count page views. * Cookie __Host-psifi.analyticsTrace * Duration 6 hours * Description Description is currently not available. * Cookie __Host-psifi.analyticsTraceV2 * Duration 6 hours * Description Description is currently not available. * Cookie _gh_sess * Duration session * Description GitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form. * Cookie YSC * Duration session * Description YSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages. * Cookie ajs_anonymous_id * Duration 1 year * Description This cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before. * Cookie vuid * Duration 1 year 1 month 4 days * Description Vimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. Performance Performance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors. * Cookie AWSALB * Duration 7 days * Description AWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target. * Cookie acq * Duration past * Description Description is currently not available. * Cookie acq.sig * Duration past * Description Description is currently not available. * Cookie ptc * Duration 2 years * Description No description available. Advertisement Advertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns. * Cookie muc_ads * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to collect user behaviour and interaction data to optimize the website. * Cookie guest_id_marketing * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. * Cookie guest_id_ads * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. * Cookie personalization_id * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting. * Cookie guest_id * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences. * Cookie bcookie * Duration 1 year * Description LinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs. * Cookie __Secure-ROLLOUT_TOKEN * Duration 6 months * Description YouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment. * Cookie yt.innertube::nextId * Duration Never Expires * Description YouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen. * Cookie yt.innertube::requests * Duration Never Expires * Description YouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen. * Cookie session_tracker * Duration session * Description This cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website. * Cookie edgebucket * Duration session * Description Reddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content. * Cookie did * Duration 1 year * Description Arbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year. Uncategorised Other uncategorised cookies are those that are being analysed and have not been classified into a category as yet. No cookies to display. Reject All Save My Preferences Accept All Toggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation [](https://towardsdatascience.com/) Publish AI, ML & data-science insights to a global community of data professionals. Sign in[Submit an Article](https://write.insightmediagroup.io) [Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/) * * * [Write For TDS](https://towardsdatascience.com/submissions/) [LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search [Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/) # Replace Manual Normalization with Batch Normalization in Vision AI Models A neat trick to avoid expensive manual pixel normalization for Vision (Image/Video) AI models is to stick a Batch normalization layer as... [Dhruv Matani](https://towardsdatascience.com/author/dhruvbird/) May 18, 20239 min read ### A neat trick to avoid expensive manual pixel normalization for Vision (Image/Video) AI models is to stick a Batch normalization layer as the first layer of your model Co-authored with [Naresh](https://towardsdatascience.com/replace-manual-normalization-with-batch-normalization-in-vision-ai-models-e7782e82193c/None). ![Photo by Kevin Ku on Unsplash](../images/000.jpg) Photo by [Kevin Ku](https://unsplash.com/ko/@ikukevk?utm_source=medium&utm_medium=referral) on [Unsplash](https://unsplash.com?utm_source=medium&utm_medium=referral) * * * Learn this step by step with the interactive [Machine Learning](https://roadmap.sh/machine-learning) roadmap. ## Channel Normalization for image pre-processing Colour images usually have 3 channels (RGB). It's common for vision AI models to pre-process image pixels and normalize them so that the pixels in a given channel are normalized to have a mean of 0.0 and a variance of 1.0. Normalization is performed per-channel since each channel can have its own statistics. The use of batch normalization is a general best practice used in vision models to avoid a phenomenon known as covariate shift. ### What is covariate shift? [Covariate shift](https://en.wikipedia.org/wiki/Batch_normalization) is a phenomenon that occurs when the distribution of the input features (i.e., covariates) changes between the training and testing phases of a machine learning model. This can lead to degraded performance of the model, as the assumptions made during training may not hold true during testing. Covariate shift can occur due to changes in the data collection process, changes in the population being sampled, or changes in the environment in which the model is being used. In order to address covariate shift, techniques such as [domain adaptation](https://en.wikipedia.org/wiki/Domain_adaptation) and importance weighting can be used to adjust the model's predictions based on the changes in the input distribution. However, these techniques are fairly complex, and require a deeper understanding of the input data distribution. ### How does batch normalization avoid covariate shift? [Batch normalization](https://en.wikipedia.org/wiki/Batch_normalization) helps with covariate shift by normalizing the activations of each layer in a neural network. This means that the mean and variance of the activations are maintained at a fixed value, regardless of the distribution of the inputs to that layer. By doing so, batch normalization reduces the effect of covariate shift between the training and testing datasets. Batch normalization can be applied to any domain and scales well to various use cases without any modifications. More specifically, during training, batch normalization centers and scales the activations of each layer based on the mean and variance of the activations computed over the current batch. This normalizes the activations to have zero mean and unit variance, which helps to stabilize and accelerate the training process. A running mean and variance is tracked during the training phase. During testing, the mean and variance computed during training are used to normalize the activations. This ensures that the normalization is consistent with the training data and reduces the effect of covariate shift. By reducing the effect of covariate shift, batch normalization can improve the generalization performance of neural networks and make them more robust to changes in the input distribution. * * * ## Why to pre-process inputs? This is done to allow a model to converge faster and generalize well to the input data. Models perform relatively better when the input data distribution is in a consistent and well-specified range. Specifically, it has the following benefits: 1. **To prevent overflow or underflow** : Machine learning models often involve mathematical operations such as addition, multiplication, and exponentiation. If the input values are too large or too small, the operations can result in overflow or underflow, leading to inaccurate or undefined results. For example, adding a small floating point (fp32) number to a large one can end up [ignoring the small number](https://stackoverflow.com/questions/22186589/why-does-adding-a-small-float-to-a-large-float-just-drop-the-small-one). 2. **To ensure efficient learning** : Neural networks often use a backpropagation algorithm to update the weights in the network. This algorithm relies on calculating gradients, which can become very small or very large if the input values are not normalized, making the learning process slower and less efficient. 3. **To improve generalization** : When a model is trained on a specific range of input values, it may not perform well on inputs outside of that range. Normalizing the inputs can help the model generalize to new and unseen data. **Batch normalization** can help address some of these issues by normalizing the inputs to each layer of the network during training. This can improve the stability of the model and make it less sensitive to the scale of the inputs. However, Batch normalization is not always a drop-in replacement for input normalization done manually with hard-coded constants. * * * ## What is Batch normalization? The topic of what Batch normalization is and how it helps models has been covered extensively in numerous articles, so we shall link to the ones that provide the most detailed insights and allow the reader to develop an intuition for the operation. We also provide some links that compare batch normalization with other normalization techniques. * [Batch normalization in 3 levels of understanding](https://towardsdatascience.com/batch-normalization-in-3-levels-of-understanding-14c2da90a338) * [Batch Normalisation Explained](https://towardsdatascience.com/batch-normalisation-explained-5f4bd9de5feb) * [Dive Into Deep Learning: Batch Normalization](https://d2l.ai/chapter_convolutional-modern/batch-norm.html) * [What are the consequences of layer norm vs batch norm?](https://ai.stackexchange.com/questions/27309/what-are-the-consequences-of-layer-norm-vs-batch-norm) * * * ## How is input normalization typically done? Typically, the person training the model is responsible for computing the per-channel statistics (mean and variance) for the entire training dataset, and normalizing the input before training the vision AI model. This normalization is also expected to be performed during inference. The code for this pre-processing may look like this when using[ torchvision transforms](https://pytorch.org/vision/main/transforms.html). python ``` transforms = torch.nn.Sequential( transforms.CenterCrop(10), transforms.Normalize( # Channel means # R, G, B (0.485, 0.456, 0.406), # Channel standard deviation (0.229, 0.224, 0.225), ), ) ``` You can read more about the [Normalize transform here](https://pytorch.org/vision/main/generated/torchvision.transforms.Normalize.html#torchvision.transforms.Normalize). You'll notice that the **means and standard deviations are pre-computed** and **then hard-coded** into the **pre-processing pipeline**. There's a lot of **discussion online** about how to do this both **correctly** and **efficiently**. For example, 1. [Computing the mean and std of dataset](https://discuss.pytorch.org/t/computing-the-mean-and-std-of-dataset/34949) 2. [Kaggle Notebook: Computing Dataset Mean and STD (using PyTorch)](https://www.kaggle.com/code/kozodoi/computing-dataset-mean-and-std/notebook) 3. [How to calculate mean and standard deviation of images in PyTorch](https://www.binarystudy.com/2021/04/how-to-calculate-mean-standard-deviation-images-pytorch.html) * * * ## How to make this less painful? Now that we know how painful this process is, we'll see a neat trick to make this less painful for you! Simply stick a [BatchNorm2d](https://pytorch.org/docs/stable/generated/torch.nn.BatchNorm2d.html) layer as the first layer of your vision AI model, and drop the [Normalize](https://pytorch.org/vision/main/generated/torchvision.transforms.Normalize.html#torchvision.transforms.Normalize) transform from your pre-processing steps! By using batch normalization as the first layer of the model, the input data will be **normalized automatically during the training process** , and you won't need to manually normalize the image pixels. This can save you some coding time and reduce the chances of introducing errors in the normalization process. * * * ## The rough equivalence of manual normalization and Batch normalization Here we'll see some code that can convince us about the rough equivalence of the 2 approaches. We'll take 1000 batches of a randomly generated 1x1 image with 3 channels, and see if the manually computed mean and variance are similar to the ones computed using PyTorch's BatchNorm2d layer. python ``` torch.manual_seed(21) num_channels = 3 # Example tensor so that we can use randn_like() below. y = torch.randn(20, num_channels, 1, 1) model = nn.BatchNorm2d(num_channels) # nb is a dict containing the buffers (non-trainable parameters) # of the BatchNorm2d layer. Since these are non-trainable # parameters, we don't need to run a backward pass to update # these values. They will be updated during the forward pass itself. nb = dict(model.named_buffers()) print(f"Buffers in BatchNorm2d: {nb.keys()}n") stacked = torch.tensor([]).reshape(0, num_channels, 1, 1) for i in range(2000): x = torch.randn_like(y) y_hat = model(x) # Save all the input tensor into 'stacked' so that # we can compute the mean and variance later. stacked = torch.cat([stacked, x], dim=0) # end for print(f"Shape of stackend tensor: {stacked.shape}n") smean = stacked.mean(dim=(0, 2, 3)) svar = stacked.var(dim=(0, 2, 3)) print(f"Manually Computed:") print(f"------------------") print(f"Mean: {smean}nVariance: {svar}n") print(f"Computed by BatchNorm2d:") print(f"------------------------") rm, rv = nb['running_mean'], nb['running_var'] print(f"Mean: {rm}nVariance: {rv}n") print(f"Mean Absolute Differences:") print(f"--------------------------") print(f"Mean: {(smean-rm).abs().mean():.4f}, Variance: {(svar-rv).abs().mean():.4f}") ``` You can see that output of the code cell below. text ``` Buffers in BatchNorm2d: dict_keys(['running_mean', 'running_var', 'num_batches_tracked']) Shape of stackend tensor: torch.Size([40000, 3, 1, 1]) Manually Computed: ------------------ Mean: tensor([0.0039, 0.0015, 0.0095]) Variance: tensor([1.0029, 1.0026, 0.9947]) Computed by BatchNorm2d: ------------------------ Mean: tensor([-0.0628, 0.0649, 0.0600]) Variance: tensor([1.0812, 1.0318, 1.0721]) Mean Absolute Differences: -------------------------- Mean: 0.0602, Variance: 0.0616 ``` We started with a random tensor initialized using [torch.randn_like()](https://pytorch.org/docs/stable/generated/torch.randn_like.html), so we expect that over a sufficiently large (40k) number of samples, the mean and variance will tend to 0.0 and 1.0 respectively, since that's what we expect torch.randn_like() to generate. We see that the difference between the manually computed mean and variance over the entire input and the mean and variance computed using BatchNorm2d's rolling average based method is close enough for all practical purposes. We can see that the means computed using BatchNorm2d are consistently higher or lower (by up to 40x) than those computed manually. However, in practical terms, this should not matter. * * * ## Caveats There are definitely pros and cons of using this Batch normalization substitution in place of manual normalization, and this article wouldn't be complete without a detailed comparison between the 2 in terms of when each one may or may not be applicable. ### Transfer learning When using transfer learning, it is often recommended to keep the normalization method used in the[ pre-trained model](https://pytorch.org/vision/main/models.html) to avoid introducing unnecessary changes. In this case, it may not be appropriate to replace manual normalization with batch normalization. For example, here's what the torchvision page has to say on this subject. > "Before using the pre-trained models, one must preprocess the image (resize with right resolution/interpolation, apply inference transforms, rescale the values etc). There is no standard way to do this as it depends on how a given model was trained. It can vary across model families, variants or even weight versions. Using the correct preprocessing method is critical and failing to do so may lead to decreased accuracy or incorrect outputs." ### Training Efficiency When considering training efficiency, it may be beneficial to pre-compute the mean and variance of your dataset and hard-code it as a pre-training normalization step. This prevents the repeated computation of these statistics during training. Note that using either method uses an equal amount of compute during inference since you'll normalize the inputs using the computed mean and variance either before feeding the inputs into the model (manually computed) or as the first step of the model (using Batch normalization). ### Human Efficiency Based on what we've seen above, it's much easier for a human to stick a Batch normalization layer instead of manually computing statistics. ### Data Augmentation When augmenting inputs before feeding them into the model, one must be careful to apply the normalization **after** all the augmentation and other pre-processing steps are done to avoid computing incorrect statistics. For example, if you're using a[ ColorJitter transform](https://pytorch.org/vision/main/generated/torchvision.transforms.ColorJitter.html#torchvision.transforms.ColorJitter), then it will change the computed statistics meaningfully. This leads us to another interesting question _**"When should the mean and variance of the dataset be computed when using data augmentation?"**_ This is tricky since manually computing the mean and variance accurately across augmented inputs would require you to know a priori which images will be augmented with which transforms, and then requires you to compute the statistics and apply augmentations during model training in a consistent manner. This is somewhat hard to do in general since augmentations are applied to input images at random. In addition, the same image is augmented differently at every training epoch to prevent the model from overfitting on the training dataset. Hence, in this case, using Batch normalization would result in better model accuracy as well, since it's computing the mean and variance on the augmented image and not on the original un-augmented images. ### Mini-batch distribution Since Batch normalization computes the mean and variance of the mini-batch for normalizing values during training, it is important to randomize the input data to ensure the mean and variance of the mini-batch are somewhat representative of the entire training dataset. If your mini-batches are small or biased, then you should consider un-biasing them or using manual normalization. This problem doesn't exist at test or inference time. * * * ## Conclusion Normalization is an important pre-processing step in any vision AI pipeline. Computing it correctly and efficiently via manual steps can be tedious and error prone. Using Batch normalization as the first layer of your vision AI model is a viable substitute for manual normalization in many scenarios. ## Register Now to Continue Reading ### Unlock Towards Data Science with a Free Account All articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more. Register for freeSign in to your account * * * ![](../images/body-a3403563a948ce1eab7b.jpeg) Written By ## [Dhruv Matani](https://towardsdatascience.com/author/dhruvbird/) [See all from Dhruv Matani](https://towardsdatascience.com/author/dhruvbird/) [Batch Normalization](https://towardsdatascience.com/tag/batch-normalization/)[Deep Learning](https://towardsdatascience.com/tag/deep-learning/)[Neural Networks](https://towardsdatascience.com/tag/neural-networks/)[Pytorch](https://towardsdatascience.com/tag/pytorch/)[Vision](https://towardsdatascience.com/tag/vision/) Share This Article * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Freplace-manual-normalization-with-batch-normalization-in-vision-ai-models-e7782e82193c%2F&title=Replace%20Manual%20Normalization%20with%20Batch%20Normalization%20in%20Vision%20AI%20Models) * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Freplace-manual-normalization-with-batch-normalization-in-vision-ai-models-e7782e82193c%2F&title=Replace%20Manual%20Normalization%20with%20Batch%20Normalization%20in%20Vision%20AI%20Models) * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Freplace-manual-normalization-with-batch-normalization-in-vision-ai-models-e7782e82193c%2F&text=Replace%20Manual%20Normalization%20with%20Batch%20Normalization%20in%20Vision%20AI%20Models) Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program. [Write for TDS](https://towardsdatascience.com/submissions/) ## Related Articles * [![](../images/001.png)](https://towardsdatascience.com/think-your-data-different-ddc435f70850/) ## [Think your Data Different](https://towardsdatascience.com/think-your-data-different-ddc435f70850/) [Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/) Learn how node2vec works, and what kind of information it captures that word2vec doesn't - includes case study [Yoel Zeldes](https://towardsdatascience.com/author/yoelzeldes/) January 21, 20195 min read * [![](../images/002.jpeg)](https://towardsdatascience.com/10-days-with-deep-learning-for-coders-759b34b9336b/) ## [10 Days With "Deep Learning for Coders"](https://towardsdatascience.com/10-days-with-deep-learning-for-coders-759b34b9336b/) [Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/) My experience with the course and the book from "fastai" [Arnuld On Data](https://towardsdatascience.com/author/arnuldondata/) September 16, 20204 min read * [![](../images/003.jpg)](https://towardsdatascience.com/uncertainty-in-deep-learning-aleatoric-uncertainty-and-maximum-likelihood-estimation-c7449ee13712/) ## [Uncertainty in Deep Learning - Aleatoric Uncertainty and Maximum Likelihood Estimation](https://towardsdatascience.com/uncertainty-in-deep-learning-aleatoric-uncertainty-and-maximum-likelihood-estimation-c7449ee13712/) [Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/) In the previous article we discussed about softmax outputs and about uncertainty in Deep Learning. Now, we extend those using TensorFlow... [Kaan Bıçakcı](https://towardsdatascience.com/author/frightera/) February 8, 20222 min read * [![](../images/004.jpg)](https://towardsdatascience.com/ai-papers-to-read-in-2025/) ## [AI Papers to Read in 2025](https://towardsdatascience.com/ai-papers-to-read-in-2025/) [Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/) And Why They Matter for Anyone Working With AI [Ygor Serpa](https://towardsdatascience.com/author/ygorserpa/) November 5, 202515 min read * [![](../images/005.png)](https://towardsdatascience.com/classical-classifier-combination-techniques-voting-approaches-borda-counts-and-5b047faaffbc/) ## [Classical classifier combination techniques: Voting approaches, Borda Counts and...](https://towardsdatascience.com/classical-classifier-combination-techniques-voting-approaches-borda-counts-and-5b047faaffbc/) [Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/) In this article, we review classical classifier combination techniques that are usually used as benchmarks against a newly proposed method. [Mahmoud Albardan](https://towardsdatascience.com/author/mahmoudalbardan/) November 1, 20206 min read * [![](../images/006.png)](https://towardsdatascience.com/keep-track-of-your-experiments-with-hydra-b29937a99fc9/) ## [Keep Track of your Experiments with Hydra](https://towardsdatascience.com/keep-track-of-your-experiments-with-hydra-b29937a99fc9/) [Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/) Configure hyperparameters using YAML files and speed up your research! [Marcello Politi](https://towardsdatascience.com/author/marcellopoliti/) August 1, 20235 min read * [![](../images/007.jpg)](https://towardsdatascience.com/neural-networks-made-fun-with-tensorflow-playground-4e681a0c4529/) ## [Neural Networks Made Fun With TensorFlow Playground!](https://towardsdatascience.com/neural-networks-made-fun-with-tensorflow-playground-4e681a0c4529/) [Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/) Using TensorFlow Playground to explore and have fun with Neural Networks [Bharath K](https://towardsdatascience.com/author/bharath-k1297/) September 29, 20205 min read * [![](../images/008.jpeg)](https://towardsdatascience.com/getting-started-with-tensorflow-e33999defdbf/) ## [Getting started with Tensorflow](https://towardsdatascience.com/getting-started-with-tensorflow-e33999defdbf/) [Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/) Learn the basics through examples [André Ribeiro](https://towardsdatascience.com/author/andrefsr/) June 27, 20213 min read * [![](../images/009.jpeg)](https://towardsdatascience.com/hyperparameter-tuning-of-neural-networks-with-optuna-and-pytorch-22e179efc837/) ## [Hyperparameter Tuning of Neural Networks with Optuna and PyTorch](https://towardsdatascience.com/hyperparameter-tuning-of-neural-networks-with-optuna-and-pytorch-22e179efc837/) [Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/) How to find that perfect neural networks model for our use case with the help of Optuna [Ruben Winastwan](https://towardsdatascience.com/author/marcellusruben/) December 14, 202110 min read * [YouTube](https://www.youtube.com/c/TowardsDataScience) * [X](https://x.com/TDataScience) * [LinkedIn](https://www.linkedin.com/company/towards-data-science/) * [Threads](https://www.threads.net/@towardsdatascience) * [Bluesky](https://bsky.app/profile/towardsdatascience.com) [](https://towardsdatascience.com/) Your home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals. Subscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/) © Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network) #### Recommended Articles Close * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ "Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1") * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ "The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs") * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ "Does Your Company Have a Data Strategy?") * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/4970/image?etag=1780565225463&width=500&height=500) Productivity Tips, Data Career Insights, and Other Recent Must-Reads ](https://towardsdatascience.com/productivity-tips-data-career-insights-and-other-recent-must-reads-5c47eb7a2fe0/ "Productivity Tips, Data Career Insights, and Other Recent Must-Reads")