{ "source_file": "0930", "source_platform": "Medium", "author": "James Andrew Godwin", "title_listed": "Bayesian Statistics", "keywords": "", "domain": "机器学习", "content_type": "深度技术解析", "is_paid_flag": "否", "url": "https://towardsdatascience.com/bayesian-statistics-11f225174d5a/", "publish_date": "2021-02-25", "corpus_stage": "", "notes": "", "id": "249f20a1f7a8", "collected_at": "2026-10-02T08:47:27", "title": "Bayesian Statistics | Towards Data Science", "body_markdown": "![Revisit consent button](../images/body-df29beacd7496220cc94.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](../images/body-2159bb67f3de8ad870eb.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n * Cookie\nBCTempID\n * Duration\n10 minutes\n * Description\nNo description available.\n\n\n * Cookie\n__cf_bm\n * Duration\n1 hour\n * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n * Cookie\nAWSALBCORS\n * Duration\n7 days\n * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n * Cookie\n_cfuvid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n * Cookie\nli_gc\n * Duration\n6 months\n * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n * Cookie\n__hssrc\n * Duration\nsession\n * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n * Cookie\n__hssc\n * Duration\n1 hour\n * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n * Cookie\nwpEmojiSettingsSupports\n * Duration\nsession\n * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n * Cookie\nBCSessionID\n * Duration\n1 year 1 month 4 days\n * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n * Cookie\n_octo\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nlogged_in\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\n__Secure-YEC\n * Duration\npast\n * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n * Cookie\n__eoi\n * Duration\n6 months\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTGCORS\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\nlogin-status-p\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTG\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\ncsrf_token\n * Duration\nsession\n * Description\nNo description available.\n\n\n * Cookie\ntoken_v2\n * Duration\n1 day\n * Description\nDescription is currently not available.\n\n\n * Cookie\nD\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nPHPSESSID\n * Duration\nsession\n * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n * Cookie\nVISITOR_PRIVACY_METADATA\n * Duration\n6 months\n * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n * Cookie\ncookietest\n * Duration\nsession\n * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n * Cookie\n__Host-airtable-session\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\n__Host-airtable-session.sig\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\nm\n * Duration\n1 year 1 month 4 days\n * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n * Cookie\nBIGipServer*\n * Duration\nsession\n * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n * Cookie\n__cfruid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n * Cookie\n_GRECAPTCHA\n * Duration\n6 months\n * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n * Cookie\n__Secure-YNID\n * Duration\n6 months\n * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n * Cookie\ncookieyes-consent\n * Duration\n1 year\n * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n * Cookie\nlidc\n * Duration\n1 day\n * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n * Cookie\nbrw\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nbrwConsent\n * Duration\n5 minutes\n * Description\nDescription is currently not available.\n\n\n * Cookie\nWMF-Uniq\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nloom_anon_comment\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nloom_referral_video\n * Duration\nsession\n * Description\nDescription is currently not available.\n\n\n * Cookie\nVISITOR_INFO1_LIVE\n * Duration\n6 months\n * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n * Cookie\nyt-remote-connected-devices\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n * Duration\nNever Expires\n * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n * Cookie\nyt-remote-device-id\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-name\n * Duration\nsession\n * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\nyt-remote-fast-check-period\n * Duration\nsession\n * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-app\n * Duration\nsession\n * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n * Cookie\nyt-remote-cast-available\n * Duration\nsession\n * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n * Cookie\nyt-remote-cast-installed\n * Duration\nsession\n * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\ncp_session\n * Duration\n3 months\n * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n * Cookie\nloid\n * Duration\n1 year 1 month 4 days\n * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n * Cookie\n__hstc\n * Duration\n6 months\n * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n * Cookie\nhubspotutk\n * Duration\n6 months\n * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n * Cookie\n_ga\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n * Cookie\n_ga_*\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n * Cookie\n__Host-psifi.analyticsTrace\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n__Host-psifi.analyticsTraceV2\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n_gh_sess\n * Duration\nsession\n * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n * Cookie\nYSC\n * Duration\nsession\n * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n * Cookie\najs_anonymous_id\n * Duration\n1 year\n * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n * Cookie\nvuid\n * Duration\n1 year 1 month 4 days\n * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n * Cookie\nAWSALB\n * Duration\n7 days\n * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n * Cookie\nacq\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nacq.sig\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nptc\n * Duration\n2 years\n * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n * Cookie\nmuc_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n * Cookie\nguest_id_marketing\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\nguest_id_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\npersonalization_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n * Cookie\nguest_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n * Cookie\nbcookie\n * Duration\n1 year\n * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n * Cookie\n__Secure-ROLLOUT_TOKEN\n * Duration\n6 months\n * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n * Cookie\nyt.innertube::nextId\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nyt.innertube::requests\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nsession_tracker\n * Duration\nsession\n * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n * Cookie\nedgebucket\n * Duration\nsession\n * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n * Cookie\ndid\n * Duration\n1 year\n * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Data Science](https://towardsdatascience.com/category/data-science/)\n# Bayesian Statistics\nHard to believe there was once a controversy over probabilistic statistics\n[James Andrew Godwin](https://towardsdatascience.com/author/james-andrew/)\nFebruary 25, 202117 min read\n### [Getting Started](https://towardsdatascience.com/tagged/getting-started)\n![Frequentists vs Bayesians | Photo by Lucas Benjamin](../images/000.jpeg)\nFrequentists vs Bayesians | Photo by [Lucas Benjamin](https://unsplash.com/@aznbokchoy)\nThis article builds on my previous article about [Bootstrap Resampling](https://towardsdatascience.com/bootstrap-resampling-2b453bb036ec).\nLearn this step by step with the interactive [AI and Data Scientist](https://roadmap.sh/ai-data-scientist) roadmap.\n## Introduction to Bayes Models\nBayesian models are a rich class of models, which can provide attractive alternatives to Frequentist models. Arguably the most well-known feature of Bayesian statistics is Bayes theorem, more on this later. With the recent advent of greater computational power and general acceptance, Bayes methods are now widely used in areas ranging from medical research to natural language processing (NLP) to understanding to web searches.\nIn the early 20th century there was a big debate about the legitimacy of what is now called Bayesian, which is essentially a probabilistic way of doing some statistics - in contrast to the \"Frequentist\" camp that we are all intimately familiar with. To use the parlance of our times, it was the #Team Jacob vs #Team Edward debate of its time, among statisticians. The argument is now moot as almost everyone is both Bayesian and Frequentist. A good heuristic is that some problems are better handled by Frequentist methods and some with Bayesian methods.\nBy the end of this article I hope you can appreciate this joke: A Bayesian is one who, vaguely expecting a horse, and catching a glimpse of a donkey, strongly believes she has seen a mule!\n### History\nA limited version of Bayes theorem was proposed by the eponymous Reverend Thomas Bayes (1702–1761). Bayes' interest was in probabilities of gambling games. He also was an avid supporter of Sir Isaac Newton's newfangled theory of calculus with his amazingly entitled publication, \"_An Introduction to the Doctrine of Fluxions, and a Defence of the Mathematicians Against the Objections of the Author of The Analyst_.\"\nIn 1814, Pierre-Simon Laplace published a version of Bayes theorem similar to its modern form in the \"_Essai philosophique sur les probabilités_.\" Laplace applied Bayesian methods to problems in celestial mechanics. Solving these problems had great practical implications for ships in the late 18th/early 19th centuries. The geophysicist and mathematician Harold Jeffreys extensively used Bayes' methods.\nWWII saw many successful applications of Bayesian methods:\n * Andrey Kolmagorov, a Russian (née Soviet) statistician used Bayes methods to greatly improve artillery accuracy\n * Alan Turing used Bayesian models to break German U-boat codes\n * Bernard Koopman, French born American mathematician, helped the Allies ability to locate German U-boats by intercepting directional radio transmissions\n\n\nThe latter half of the 20th century lead to the following notable advances in computational Bayesian methods:\n * Statistical sampling using Monte Carlo methods; Stanislaw Ulam, John von Neuman; 1946, 1947\n * MCMC, or [Markov Chain Monte Carlo](https://en.wikipedia.org/wiki/Markov_chain_Monte_Carlo); Metropolis et al. (1953) Journal of Chemical Physics\n * Hastings (1970), Monte Carlo sampling methods using Markov chains and their application\n * Geman and Geman (1984) Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images\n * Duane, Kennedy, Pendleton, and Roweth (1987), Hamiltonian MCMC\n * Gelfand and Smith (1990), Sampling-based approaches to calculating marginal densities.\n\n\n### Bayesian vs Frequentist views\nWith greater computational power and general acceptance, Bayes methods are now widely used in areas ranging from medical research to natural language understanding to web search. Among pragmatists, the common belief today is that some problems are better handled by frequentist methods and some with Bayesian methods.\nFrom my first article on the [Central Limit Theorem](https://medium.com/swlh/on-the-importance-of-the-central-limit-theorem-e8cce1f4d253) (CLT), I established that you need \"enough data\" for the CLT to apply. Bootstrap sampling is a convenient way of obtaining more distributions that approach your underlying distribution. These are Frequentist tools; but what happens if you have too little data to reliably apply the CLT? As a probabilistic Bayesian statistician, I already have some estimates, admittedly, with a little bit of guesswork, but there is a way of combining that guesswork in an intelligent fashion with my data. And that is what Bayesian statistics is basically all about - you combine it and basically, that combination is a simple multiplication of the two probable probability distributions, the one that you guessed at, and the other one, that for which you have evidence.\nThe more data you have, the prior distribution becomes less and less important and Frequentist methods can be used. But, with less data, Bayesian's will give you a better answer.\nSo how then do Bayesian and Frequentist methodologies differ? Simply stated:\n * Bayesian methodologies use **prior distributions** to quantify what is known about the parameters\n * Frequentists do not quantify anything about the parameters; instead using **p-values** and **confidence intervals** (CI's) to express the unknowns about parameters\n\n\nBoth methods are useful, converging with more and more data - but contrasted:\n**Strict Frequentist approach**\n * **Goal is a point estimate and CI**\n * Start from observations\n * Re-compute model given new observations\n * Examples: Mean estimate, t-test, ANOVA\n\n\n**Bayesian approach**\n * **Goal is posterior distribution**\n * Start from prior distribution (the guesswork)\n * Update posterior (the belief/hypothesis) given new observations\n * Examples: posterior distribution of mean, highest density interval (HDI) overlap\n\n\nToday no one is a strict Frequentist. Bayesian methodologies simply work, that is fact.\n### Bayes theorem\nBayes theorem describes the probability of an event, based on **prior knowledge (our guesswork)** of conditions that might be related to the event, **our data**. Bayes theorem tells us how to combine these two probabilities. For example, if developing a disease like Alzheimer's is related to age, then, using Bayes' theorem, a person's age can be used to more reliably assess the probability that they have Alzheimer's, or cancer, or any other age-related disease.\nBayes theorem is used to update the probability for a hypothesis; the hypothesis is this guesswork, this prior distribution thing. Prior means your prior belief, the belief that you have before you have evidence, and it is a way to update it. Because you get more information, you update your hypothesis, and so on and so forth.\nFor those who do not know, or who have forgotten, let us derive Bayes theorem from the rule for conditional probability:\n![Conditional probability](../images/001.png)\nConditional probability\nTranslates to \"the probability of A happening given B is the case.\"\n![](../images/body-f5d02f1059349724a72b.png)\n\"The probability of B happening given A happens.\"\nEliminating 𝑃(𝐴∩𝐵) and doing very minor algebra:\n![](../images/body-fc8e91466a4bd20fcb86.png)\nor, rearranging terms\n![](../images/body-878c76c437dfa68d6a18.png)\nWhich is Bayes theorem! This describes how one finds the conditional probability of event A given event B.\n### Applications\nYou have a disease test, and the probability that you will get a positive test result given that you have the disease is really, really high; in other words the test has a very high accuracy rate. The problem is that there is a probability that you will get a positive test result even if you do not have the disease. And that you can simply calculate from Bayes law. The big point is, is that these probabilities are not the same as the probability that you will get a positive result given the disease **is not the same** as the probability that you will have the disease given a positive result.\n![The test sensitivity does not match the rate of incidence of the disease](../images/002.png)\nThe test sensitivity does not match the rate of incidence of the disease\nThese are two different probability distributions. And what makes them so different is the probability of disease and the probability of a positive test result. So if the disease is rare, the probability of disease will be very, very small.\nDisease testing: A = Have disease, B = Tested positive.\n![](../images/body-e6369e05d7d88cd7aef5.png)\n### Example\n![The sensitivity of this test is quite bad despite the high accuracy of the test](../images/003.png)\nThe sensitivity of this test is quite bad despite the high accuracy of the test\nUsing Bayes theorem we have deduced that a positive test result for this disease indicates that only 10% of the time the person will actually have the disease, because the incidence rate of the disease is an order of magnitude lower than the false positive rate.\n### Python example: probabilities of hair and eye color\nA sample population has the following probabilities of hair and eye color combinations. I will run the code below in a Jupyter Notebook to find the conditional probabilities:\ntext\n\n```\n# begin with the imports\nimport pandas as pd\nimport numpy as np\nimport matplotlib\nfrom matplotlib import pyplot as plt\nimport scipy\nimport seaborn as sns\nimport itertools\n```\n\ntext\n\n```\n%matplotlib inline\n```\n\ntext\n\n```\n# create the dataframe\nhair_eye = pd.DataFrame({\n 'black': [0.11, 0.03, 0.03, 0.01],\n 'blond': [0.01, 0.16, 0.02, 0.03],\n 'brunette': [0.2, 0.14, 0.09, 0.05],\n 'red': [0.04, 0.03, 0.02, 0.02],\n\n}, index=['brown', 'blue', 'hazel', 'green'])\n```\n\ntext\n\n```\nhair_eye\n```\n\n![](../images/body-f92d594c0b768ce92081.jpeg)\nN.B: this is string index for eye color rather than a numeric zero-based index.\ntext\n\n```\nhair_eye.loc['hazel', 'red']\n```\n\n![2% of people have hazel eyes given red hair](../images/004.jpeg)\n2% of people have hazel eyes given red hair\nThe figures in the table above are the **conditional probabilities**. Note that in this case:\n𝑃(hair|eye)=𝑃(eye|hair)P(hair|eye)=P(eye|hair)\nGiven these conditional probabilities, it is easy to compute the marginal probabilities by summing the probabilities in the rows and columns. The **marginal probability** is the probability along one variable (one margin) of the distribution. The marginal distributions must sum to unity (1).\ntext\n\n```\n## Compute the marginal distribution of each eye color\nhair_eye['marginal_eye'] = hair_eye.sum(axis=1)\n```\n\ntext\n\n```\nhair_eye\n```\n\n![Added marginal_eye column](../images/005.jpeg)\nAdded marginal_eye column\ntext\n\n```\nhair_eye.sum(axis=0)\n```\n\n![The hair color probabilities sum to unity](../images/006.png)\nThe hair color probabilities sum to unity\ntext\n\n```\nhair_eye.loc['marginal_hair'] = hair_eye.sum(axis=0)\nhair_eye.describe\n```\n\n![The eye color probabilities also sum to 1.0](../images/007.jpeg)\nThe eye color probabilities also sum to 1.0\nI have blue eyes. I wonder what the marginal probabilities of hair color are given that they have blue eyes?\ntext\n\n```\n# probability of blue eyes given any hair color divided by total probability of blue eyes\n```\n\ntext\n\n```\n# marginal probabilities of hair color given blue eyes\n#blue_black(.03/.36) + blue_brunette(.14/.36) + blue_red(.03/.36) + blue_blond(.16/.36)\nblue = hair_eye.loc['blue',:]/hair_eye.loc['blue','marginal_eye']\nblue\n```\n\n![For a given person with blue eyes, it is 44% likely they have blond hair, like me](../images/008.jpeg)\nFor a given person with blue eyes, it is 44% likely they have blond hair, like me\n## Applying Bayes theorem\nThere is a formulation of Bayes theorem that is convenient to use for computational problems because we do not want to sum all of the possibilities to compute **P(B)** , like in the example above.\nHere are more facts and relationships about conditional probabilities, largely arising from the fact that P(B) is so hard to calculate:\n![](../images/body-9798bb024bb4fcf1aa07.png)\nRewriting Bayes theorem:\n![](../images/body-2196d51c56da3e05d960.png)\nThis is a mess, but we do not always need the complicated denominator, which is the probability of B. **P(B)** is our actual evidence, which is hard to come by. Remember that this is probabilistic and that we are working with smaller data samples than we are normally used to.\nRewriting we get:\n![](../images/body-d63ea1dfaf38a7f88a0a.png)\nIgnoring the constant **𝑘, b** ecause it is given that the evidence - P(B) - is presumed to be constant, we get:\n![We do not need to know the probability distribution of the actual evidence, of the data](../images/009.png)\nWe do not need to know the probability distribution of the actual evidence, of the data\n### Applying the reduced relationship Bayes theorem\nWe interpret the reduced relationship above as follows:\n![Proportional Bayes](../images/010.png)\nProportional Bayes\nThe proportional relationships apply to the observed data distributions, or to parameters in a model (partial slopes, intercept, error distributions, lasso constant, e.g.,). The above equations tell you how to combine your prior belief if you want to call that, which I call the prior distribution with your evidence.\nAn important nuance is that while we cannot speak about exact values we can talk about distributions; we can talk about the shapes of the curves of the probability distribution curves. What we want is the so-called posterior distribution. That is what we want. What we do is guess at the prior distribution: what is the probability of A? What is the probability that something is true given the data that I have? What is the probability of A given B? what is the probability that something is true given the data I have? Some people will rephrase this as what is the probability that my hypothesis A is true given the data B?\nI might have very little data. Too little to do any Frequentist approaches; but I can estimate that with the little small amount of data I have, called the **likelihood** , I can estimate the probability that this data would be correct, given my hypothesis that I can calculate directly. Because I have the data, not a lot of data, it is too little for me to actually believe, but I have some data. And I have my hypothesis. And so I can do my calculations on my distributions.\nThe prior distribution is the probability of my hypothesis, which I do not know - but I can guess it. If I am really bad at guessing, I am going to say that this prior distribution is a **uniform distribution** , it could be a Gaussian or any other shape. Now if I guess a uniform distribution I am still doing probabilistic statistics, Bayesian statistics, but I am getting really, really close to what Frequentists would do.\nIn Bayesian, I will update this prior distribution as I get more data because I will say, \"Hey!\" at first I thought it was uniform. Now I see that this thing, this probability distribution actually has a mean somewhere in the middle. So I am now going to choose a different distribution. Or I'm going to modify my distribution to accommodate what I am seeing in my posterior. I am trying to make this prior distribution look similar in shape to my posterior distribution. And when I say similar in shape, that is actually very misleading. What I am trying to do is find a mathematical formula that will mix the probability of A and the probability of A given B similarly. Similar in type that the math, that the equation looks similar, as in the formulaic of the question and not the actual probabilities; we call that getting the conjugate prior. This is when you search for a distribution that is mathematically similar to what you believe to be the final distribution. This is formally referred to as searching for a conjugate prior. A beta distribution, because of its flexibility is often, but not always, used as the prior.\nTo reiterate, the **hypothesis is formalized as parameters in the model** , P(A), and the likelihood, **P(B|A)** , is the probability of the data given those parameters - this is easy to do, t-tests can do this!\n### Creating Bayes models\nGiven prior assumptions about the behavior of the parameters (the prior), produce a model which tells us the probability of observing our data, to compute new probability of our parameters. Thus the steps for working with a Bayes model are:\n * Identify data relevant to the research question: e.g., measurement scales of the data\n * Define a descriptive model for the data. For instance, pick a linear model formula or a beta distribution of (2,1)\n * Specify a prior distribution of the parameters - this the formalization of the hypothesis. For example, thinking the error in the linear model is normally distributed as 𝑁(𝜃,𝜎²)\n * Choose the Bayesian inference formula to compute posterior parameter probabilities\n * Update Bayes model if more data is observed. **This is key!** The posterior distribution naturally updates as more data is added\n * Simulate data values from realizations of the posterior distribution of the parameters\n\n\n### Choosing a prior\nThe choice of the prior distribution is very important to Bayesian analysis. A prior should be convincing to a **skeptical audience**. Some heuristics to consider are:\n * Domain knowledge (SME)\n * Prior observations\n * If poor knowledge use less informative prior\n * **Caution:** the uniform prior is informative, you need to set the limits on the range of values\n\n\nOne analytically and computationally simple choice is the before mentioned conjugate prior. When a likelihood is multiplied by a conjugate prior the distribution of the posterior is the same as the likelihood. Most named distributions have conjugates:\n![A conjugate prior is not always used](../images/011.png)\nA conjugate prior is not always used\n### Python Example: COVID-19 random sampling\nAs a health official for your city, you are interested in analyzing COVID-19 infection rates. You decide to sample 10 people at an intersection while the light is red and determine if they test positive for Covid - the test is fast and to avoid sampling bias we take the test to them. The data are binomially distributed; a person tests positive or tests negative - for the sake of simplicity, assume the test is 100% accurate.\nIn this notebook example:\n * Select a prior for the parameter 𝑝, the probability of having COVID-19.\n * Using data, compute the likelihood.\n * Compute the posterior and posterior distributions.\n * Try another prior distribution.\n * Add more data to our data set to updated the posterior distribution.\n\n\nThe likelihood of the data and the posterior distribution are binomially distributed. The binomial distribution has one parameter we need to estimate, 𝑝, the probability. We can write this formally for 𝑘 successes in 𝑁 trials:\n![](../images/body-bd0c8031bd97698ba86e.png)\nThe following code computes some basic summary statistics:\ntext\n\n```\nsampled = ['yes','no','yes','no','no','yes','no','no','no','yes']\npositive = [1 if x is 'yes' else 0 for x in sampled]\npositive\n```\n\n![](../images/body-f9e0892b6e63f625a95a.png)\ntext\n\n```\nN = len(positive) # sample size\nn_positive = sum(positive) # number of positive drivers\nn_not = N - n_positive # number negative\nprint('Tested positive= %d Tested negative= %d'\n 'nProbability of having COVID-19= %.1f' \n % (n_positive, n_not, n_positive / (n_positive+ n_not)))\n```\n\n![](../images/body-25a6a4ba5108887d4cc2.png)\nFor the prior I will start with a uniform distribution as I do not know what to expect re Covid rates:\ntext\n\n```\nN = 100\np = np.linspace(.01, .99, num=N)\npp = [1./N] * N\nplt.plot(p, pp, linewidth=2, color='blue')\nplt.show()\n```\n\nNext step: compute the likelihood. The likelihood is the probability of the data given the parameter, **𝑃(𝑋|𝑝). E** ach observation of each test as \"positive\" or \"not\" is a Bernoulli trial, so I choose the binomial distribution.\ntext\n\n```\ndef likelihood(p, data):\n k = sum(data)\n N = len(data)\n # Compute Binomial likelihood\n l = scipy.special.comb(N, k) * p**k * (1-p)**(N-k)\n # Normalize the likelihood to sum to unity\n return l/sum(l)\n```\n\ntext\n\n```\nl = likelihood(p, positive)\nplt.plot(p, l)\nplt.title('Likelihood function')\nplt.xlabel('Parameter')\nplt.ylabel('Likelihood')\nplt.show()\n```\n\n![The probability our data is true given our hypothesis](../images/body-c7669c2a0aaf7b5fd296.png)\nThe probability our data is true given our hypothesis\nNow that we have a prior and a likelihood we can compute the posterior distribution of the parameter 𝑝: 𝑃**(𝑝|𝑋).**\nN.B. The computational methods used here are simplified for the purpose of illustration. Computationally efficient code must be used!\ntext\n\n```\ndef posterior(prior, like):\n post = prior * like # compute the product of the probabilities\n return post / sum(post) # normalize the distribution\n```\n\ntext\n\n```\ndef plot_post(prior, like, post, x):\n maxy = max(max(prior), max(like), max(post))\n plt.figure(figsize=(12, 4))\n plt.plot(x, like, label='likelihood', linewidth=12, color='black', alpha=.2)\n plt.plot(x, prior, label='prior')\n plt.plot(x, post, label='posterior', color='green')\n plt.ylim(0, maxy)\n plt.xlim(0, 1)\n plt.title('Density of prior, likelihood and posterior')\n plt.xlabel('Parameter value')\n plt.ylabel('Density')\n plt.legend()\n\npost = posterior(pp, l)\nplot_post(pp, l, post, p)\n```\n\n![](../images/body-ff15cbde8059999aed7a.png)\ntext\n\n```\nprint('Maximum of the prior density = %.3f' % max(pp))\nprint('Maximum likelihood = %.3f' % max(l))\nprint('MAP = %.3f' % max(post))\n```\n\n![](../images/body-2695f25583f04754aa52.png)\nWith a uniform prior distribution, the **posterior is just the likelihood**. The key point is that the Frequentist probabilities are identical to the Bayesian posterior distribution given a uniform prior.\n### Trying a different prior\nFrom the chart above, I chose the **conjugate prior** of the Binomial distribution which is the beta distribution. The beta distribution is defined on the interval 0 ≤ Beta(P|A, B)≤10 ≤ Beta(P|A, B)≤ 1. The beta distribution has two parameters, 𝑎 and 𝑏, which determine the shape.\ntext\n\n```\nplt.figure(figsize=(12, 8))\n```\n\ntext\n\n```\nalpha = [.5, 1, 2, 3, 4]\nbeta = alpha[:]\nx = np.linspace(.001, .999, num=100) #100 samples\n```\n\ntext\n\n```\nfor i, (a, b) in enumerate(itertools.product(alpha, beta)):\n plt.subplot(len(alpha), len(beta), i+1)\n plt.plot(x, scipy.stats.beta.pdf(x, a, b))\n plt.title('(a, b) = ({}, {})'.format(a,b))\nplt.tight_layout()\n```\n\n![Note how varied these beta distributions can become with slightly different parameters](../images/body-b09258c46bf84ea5aae1.png)\nNote how varied these beta distributions can become with slightly different parameters\nI still do not know a lot about the behavior of drivers at this intersection who may or may not have COVID-19, so I pick a rather uninformative or broad beta distribution as the prior for the hypothesis. I choose a symmetric prior with a=2 and b=2:\ntext\n\n```\ndef beta_prior(x, a, b):\n l = scipy.stats.beta.pdf(p, a, b) # compute likelihood\n return l / l.sum() # normalize and return\n```\n\ntext\n\n```\npp = beta_prior(p, 2, 2)\npost = posterior(pp, l)\nplot_post(pp, l, post, p)\n```\n\n![](../images/body-ff4837c037e98724e453.png)\nTake note that the mode of the posterior is close to the mode of the likelihood, but has shifted toward the mode of the prior. This behavior of Bayesian posteriors to be shifted toward the prior is known as the **shrinkage** property: the tendency of the maximum likelihood point of the posterior is said to shrink toward the maximum likelihood point of the prior.\n### Updating the Bayesian model\nLet us update the model with 10 new observations to our data set. Note that the more observations added to the model moves the posterior distribution closer to the likelihood.\nN.B. With large datasets, it might require **HUGE** amounts of data to see convergence in behavior of the posterior and the likelihood.\ntext\n\n```\nnew_samples = ['yes','no','no','no','no',\n 'yes','no','yes','no','no'] # new data to update model, n = 20\nnew_positive = [1 if x is 'yes' else 0 for x in new_samples]\n```\n\ntext\n\n```\nl = likelihood(p, positive+ new_positive)\npost = posterior(pp, l)\nplot_post(pp, l, post, p)\n```\n\n![Compared to the first one, note the convergence of the two distributions as we updated with more data](../images/body-680ef54b211dd5cd40df.png)\nCompared to the first one, note the convergence of the two distributions as we updated with more data\n### Credible Intervals\nA **credible interval** is an interval on the Bayesian posterior distribution. The credible interval is sometimes called the highest density interval (HDI). For instance, the 90% credible interval encompasses 90% of the posterior distribution with the highest probability density. It can be confusing since both credible intervals of Bayesian and confidence intervals of Frequentist are abbreviated the same (CI).\nLuckily, the **credible interval is the Bayesian analog of the Frequentist confidence interval**. However, they are conceptually different. The confidence interval is chosen on the distribution of a test statistic, whereas the credible interval is computed on the posterior distribution of the parameter. For symmetric distributions, the credible interval can be numerically the same as the confidence interval. However, in the general case, these two quantities can be quite different.\nNow we plot the posterior distribution of the parameter of the binomial distribution parameter p. The 95% credible interval, or HDI, is also computed and displayed:\ntext\n\n```\nnum_samples = 100000\nlower_q, upper_q = [.025, .975]\n```\n\ntext\n\n```\n# Caution, this code assumes a symmetric prior distribution and will not work in the general case\n```\n\ntext\n\n```\ndef plot_ci(p, post, num_samples, lower_q, upper_q):\n # Compute a large sample by resampling with replacement\n samples = np.random.choice(p, size=num_samples, replace=True, p=post)\n ci = scipy.percentile(samples, [lower_q*100, upper_q*100]) # compute the quantiles\n\n interval = upper_q - lower_q\n plt.title('Posterior density with %.3f credible interval' % interval)\n plt.plot(p, post, color='blue')\n plt.xlabel('Parameter value')\n plt.ylabel('Density')\n plt.axvline(x=ci[0], color='red')\n plt.axvline(x=ci[1], color='red')\n print('The %.3f credible interval is %.3f to %.3f' \n % (interval, lower_q, upper_q))\n\nplot_ci(p, post, num_samples, lower_q, upper_q)\n```\n\n![](../images/body-d2f797e72c615f0f0792.png)\nBecause the **posterior distribution above is symmetric** , like the beta prior we chose, **the analysis is identical to CI's in the frequentist approach**. We can be 95% confident that around 40% of driver's tested at this intersection during a red light will have COVID-19! This info is very useful for health agencies to allocate resources appropriately.\nIf you remember the joke in the third paragraph, about the Bayesian who vaguely suspected a horse and caught a fleeting glimpse of a donkey and strongly suspects seeing a mule - I hope you can now assign each clause of the joke the appropriate part Bayes theorem.\nCheck out my next piece on [linear regression](https://towardsdatascience.com/an-introduction-to-linear-regression-9cbb64b52d23) and using [bootstrap with regression models](https://towardsdatascience.com/linear-regression-with-bootstrapping-4924c05d2a9).\nFind me on [Linkedin](https://www.linkedin.com/in/james-a-w-godwin/)\n_Physicist cum Data Scientist- Available for new opportunity | SaaS | Sports | Start-ups | Scale-ups_\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](../images/body-92fdfcb3140ff02db487.jpeg)\nWritten By\n## [James Andrew Godwin](https://towardsdatascience.com/author/james-andrew/)\n[See all from James Andrew Godwin](https://towardsdatascience.com/author/james-andrew/)\n[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Bayes Theorem](https://towardsdatascience.com/tag/bayes-theorem/)[Bayesian Statistics](https://towardsdatascience.com/tag/bayesian-statistics/)[Data Science](https://towardsdatascience.com/tag/data-science/)[Getting Started](https://towardsdatascience.com/tag/getting-started/)\nShare This Article\n * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fbayesian-statistics-11f225174d5a%2F&title=Bayesian%20Statistics)\n * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fbayesian-statistics-11f225174d5a%2F&title=Bayesian%20Statistics)\n * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fbayesian-statistics-11f225174d5a%2F&text=Bayesian%20Statistics)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n * [![](../images/012.jpg)](https://towardsdatascience.com/semantic-chunking-for-rag-35b7675ffafd/)\n## [Why Does Position-Based Chunking Lead to Poor Performance in RAGs?](https://towardsdatascience.com/semantic-chunking-for-rag-35b7675ffafd/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nHow to implement semantic chunking and gain better results.\n[Thuwarakesh Murallie](https://towardsdatascience.com/author/thuwarakesh/)\nAugust 22, 202411 min read\n * [![](../images/013.jpg)](https://towardsdatascience.com/why-you-should-learn-machine-learning-engineering-and-not-data-science-e616141ea866/)\n## [Why You Should Learn Machine Learning Engineering and Not Data Science](https://towardsdatascience.com/why-you-should-learn-machine-learning-engineering-and-not-data-science-e616141ea866/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nAnd how you can start\n[Patrick Brus](https://towardsdatascience.com/author/brus-patrick63/)\nSeptember 28, 20227 min read\n * [![](../images/014.png)](https://towardsdatascience.com/the-lottery-data-behind-the-luck-ambition-and-a-billion-dollar-dream-c31fdc17d170/)\n## [Data behind the Luck, Ambition, and a Billion-Dollar Dream: Lottery](https://towardsdatascience.com/the-lottery-data-behind-the-luck-ambition-and-a-billion-dollar-dream-c31fdc17d170/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nUsing Seattle's local retail store data for consumer patterns of the lottery (SQL, Python)\n[Sung Ahn](https://towardsdatascience.com/author/sahn1998/)\nJanuary 4, 202512 min read\n * [![](../images/015.jpg)](https://towardsdatascience.com/the-best-machine-learning-algorithm-for-handwritten-digits-recognition-2c6089ad8f09/)\n## [The Best Machine Learning Algorithm for Handwritten Digits Recognition](https://towardsdatascience.com/the-best-machine-learning-algorithm-for-handwritten-digits-recognition-2c6089ad8f09/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nImplementing Machine Learning Classification Algorithms to Recognize Handwritten Digits\n[Mahnoor Javed](https://towardsdatascience.com/author/mahnoorjaved/)\nNovember 21, 20208 min read\n * [![](../images/016.jpg)](https://towardsdatascience.com/creating-dynamic-dashboards-with-streamlit-747b98a68ab5/)\n## [Creating dynamic dashboards with Streamlit](https://towardsdatascience.com/creating-dynamic-dashboards-with-streamlit-747b98a68ab5/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nUsing Streamlit to develop dynamically updated data visualizations\n[Mohammad Khorasani](https://towardsdatascience.com/author/khorasani/)\nDecember 4, 20203 min read\n * [![](../images/017.jpg)](https://towardsdatascience.com/how-to-benchmark-deepseek-r1-distilled-models-on-gpqa-using-ollama-and-openais-simple-evals/)\n## [How to Benchmark DeepSeek-R1 Distilled Models on GPQA Using Ollama and OpenAI’s simple-evals](https://towardsdatascience.com/how-to-benchmark-deepseek-r1-distilled-models-on-gpqa-using-ollama-and-openais-simple-evals/)\n[Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/)\nSet up and run the GPQA-Diamond benchmark on DeepSeek-R1's distilled models locally to evaluate its reasoning capabilities.\n[Kenneth Leung](https://towardsdatascience.com/author/kennethleungty/)\nApril 23, 20253 min read\n * [![](../images/018.jpeg)](https://towardsdatascience.com/two-simple-things-you-need-to-steal-from-agile-for-data-and-analytics-work-d3130050f0e5/)\n## [Two Simple Things You Need to Steal from Agile for Data and Analytics Work](https://towardsdatascience.com/two-simple-things-you-need-to-steal-from-agile-for-data-and-analytics-work-d3130050f0e5/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nPeer Review and Definition of Done: Small Changes, BIG Impact\n[Jon Loyens](https://towardsdatascience.com/author/jonloyens/)\nOctober 4, 20217 min read\n * [![](../images/019.jpeg)](https://towardsdatascience.com/failed-in-data-science-interview-here-are-some-tips-to-help-you-succeed-2e9a6bb2b307/)\n## [Failed in Data Science Interview? Here Are Some Tips to Help You Succeed](https://towardsdatascience.com/failed-in-data-science-interview-here-are-some-tips-to-help-you-succeed-2e9a6bb2b307/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nBased on 10,000+ hours of experience and many failures!\n[Sharan Kumar Ravindran](https://towardsdatascience.com/author/rsharankumar/)\nJuly 29, 202210 min read\n * [![](../images/020.jpg)](https://towardsdatascience.com/analyze-scientific-publications-with-e-utilities-and-python-56f76de22959/)\n## [Analyze Scientific Publications with E-utilities and Python](https://towardsdatascience.com/analyze-scientific-publications-with-e-utilities-and-python-56f76de22959/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nHow to gather data about scientific literature and discover trends\n[Jozsef Meszaros](https://towardsdatascience.com/author/neuraljojo/)\nMay 19, 202316 min read\n\n\n * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n * [X](https://x.com/TDataScience)\n * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n * [Threads](https://www.threads.net/@towardsdatascience)\n * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n#### Recommended Articles \nClose\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500) How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ \"How Relevance Models Foreshadowed Transformers for NLP\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n\n\n", "metadata": { "title": "Bayesian Statistics | Towards Data Science", "description": "Hard to believe there was once a controversy over probabilistic statistics", "keywords": null, "author": "James Andrew Godwin", "og:locale": "en_US", "og:type": "article", "og:title": "Bayesian Statistics", "og:description": "Hard to believe there was once a controversy over probabilistic statistics", "og:url": "https://towardsdatascience.com/bayesian-statistics-11f225174d5a/", "og:site_name": "Towards Data Science", "og:image": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1bbNOuUHe8wFQF1v2sdul3Q.jpeg", "og:image:width": "640", "og:image:height": "427", "og:image:type": "image/jpeg", "twitter:card": "summary_large_image", "twitter:site": "@TDataScience", "twitter:creator": "@TDataScience", "twitter:label1": "Written by", "twitter:data1": "James Andrew Godwin", "twitter:label2": "Est. reading time", "twitter:data2": "17 min read", "article:published_time": "2021-02-25T01:38:11.000Z", "article:modified_time": "2021-02-25T01:38:11.000Z", "article:author": "https://towardsdatascience.com/author/james-andrew/" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 202, "error": "" }, "code_blocks": [ { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_000.txt", "chars": 171 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_001.txt", "chars": 19 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_002.txt", "chars": 252 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_003.txt", "chars": 9 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_004.txt", "chars": 29 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_005.txt", "chars": 103 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_006.txt", "chars": 9 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_007.txt", "chars": 21 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_008.txt", "chars": 71 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_009.txt", "chars": 90 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_010.txt", "chars": 214 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_011.txt", "chars": 128 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_012.txt", "chars": 277 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_013.txt", "chars": 111 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_014.txt", "chars": 199 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_015.txt", "chars": 136 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_016.txt", "chars": 146 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_017.txt", "chars": 489 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_018.txt", "chars": 131 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_019.txt", "chars": 28 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_020.txt", "chars": 91 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_021.txt", "chars": 208 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_022.txt", "chars": 124 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_023.txt", "chars": 75 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_024.txt", "chars": 167 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_025.txt", "chars": 92 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_026.txt", "chars": 53 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_027.txt", "chars": 98 }, { "language": "txt", "file": "attachments/249f20a1f7a8/code/block_028.txt", "chars": 659 } ], "images": [ { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T17:42:13Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T17:42:13Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T17:42:14Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T17:42:14Z" } ], "audio": [], "videos": [ { "url": "https://www.youtube.com/c/TowardsDataScience", "kind": "channel_reference", "status": "reference_only" } ], "links": [ { "type": "internal", "href": "https://towardsdatascience.com/" }, { "type": "internal", "href": "https://towardsdatascience.com/latest/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/editors-pick/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/the-variable/" }, { "type": "internal", "href": "https://towardsdatascience.com/shipai/" }, { "type": "internal", "href": "https://towardsdatascience.com/submissions/" }, { "type": "internal", "href": "https://towardsdatascience.com/category/data-science/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/james-andrew/" }, { "type": "internal", "href": "https://towardsdatascience.com/tagged/getting-started" }, { "type": "internal", "href": "https://towardsdatascience.com/bootstrap-resampling-2b453bb036ec" }, { "type": "internal", "href": "https://towardsdatascience.com/an-introduction-to-linear-regression-9cbb64b52d23" }, { "type": "internal", "href": "https://towardsdatascience.com/linear-regression-with-bootstrapping-4924c05d2a9" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/bayes-theorem/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/bayesian-statistics/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/data-science/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/getting-started/" }, { "type": "internal", "href": "https://towardsdatascience.com/semantic-chunking-for-rag-35b7675ffafd/" }, { "type": "internal", "href": "https://towardsdatascience.com/category/artificial-intelligence/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/thuwarakesh/" }, { "type": "internal", "href": "https://towardsdatascience.com/why-you-should-learn-machine-learning-engineering-and-not-data-science-e616141ea866/" }, { "type": "external", "href": "https://write.insightmediagroup.io" }, { "type": "external", "href": "https://www.linkedin.com/company/towards-data-science/" }, { "type": "external", "href": "https://x.com/TDataScience" }, { "type": "external", "href": "https://unsplash.com/@aznbokchoy" }, { "type": "external", "href": "https://roadmap.sh/ai-data-scientist" }, { "type": "external", "href": "https://en.wikipedia.org/wiki/Markov_chain_Monte_Carlo" }, { "type": "external", "href": "https://medium.com/swlh/on-the-importance-of-the-central-limit-theorem-e8cce1f4d253" }, { "type": "external", "href": "https://www.linkedin.com/in/james-a-w-godwin/" }, { "type": "external", "href": "https://www.facebook.com/sharer/sharer.php?title=Bayesian+Statistics&u=https%3A%2F%2Ftowardsdatascience.com%2Fbayesian-statistics-11f225174d5a%2F" }, { "type": "external", "href": "https://www.linkedin.com/shareArticle?mini=true&title=Bayesian+Statistics&url=https%3A%2F%2Ftowardsdatascience.com%2Fbayesian-statistics-11f225174d5a%2F" }, { "type": "external", "href": "https://x.com/share?text=Bayesian+Statistics&url=https%3A%2F%2Ftowardsdatascience.com%2Fbayesian-statistics-11f225174d5a%2F" }, { "type": "external", "href": "https://www.youtube.com/c/TowardsDataScience" }, { "type": "external", "href": "https://www.threads.net/@towardsdatascience" }, { "type": "external", "href": "https://bsky.app/profile/towardsdatascience.com" }, { "type": "external", "href": "https://thenewstack.io" }, { "type": "external", "href": "https://roadmap.sh" }, { "type": "external", "href": "https://insightmediagroup.io/creator-network" } ], "repair_version": 2, "repaired_at": "2026-10-20T17:25:29Z", "source_url": "https://towardsdatascience.com/bayesian-statistics-11f225174d5a/", "delivery_status": "unverified", "discovery_status": "http_403", "missing_reason": "源页面未重新核验:http_403；已按保留正文补采图片", "repair_error": "", "actual_video_files": [], "expected_saved_body_images": 4, "image_discovery": "saved_markdown", "source_body_markdown": "![Revisit consent button](https://cdn-cookieyes.com/assets/images/revisit.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](https://cdn-cookieyes.com/assets/images/close.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n * Cookie\nBCTempID\n * Duration\n10 minutes\n * Description\nNo description available.\n\n\n * Cookie\n__cf_bm\n * Duration\n1 hour\n * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n * Cookie\nAWSALBCORS\n * Duration\n7 days\n * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n * Cookie\n_cfuvid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n * Cookie\nli_gc\n * Duration\n6 months\n * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n * Cookie\n__hssrc\n * Duration\nsession\n * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n * Cookie\n__hssc\n * Duration\n1 hour\n * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n * Cookie\nwpEmojiSettingsSupports\n * Duration\nsession\n * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n * Cookie\nBCSessionID\n * Duration\n1 year 1 month 4 days\n * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n * Cookie\n_octo\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nlogged_in\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\n__Secure-YEC\n * Duration\npast\n * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n * Cookie\n__eoi\n * Duration\n6 months\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTGCORS\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\nlogin-status-p\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTG\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\ncsrf_token\n * Duration\nsession\n * Description\nNo description available.\n\n\n * Cookie\ntoken_v2\n * Duration\n1 day\n * Description\nDescription is currently not available.\n\n\n * Cookie\nD\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nPHPSESSID\n * Duration\nsession\n * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n * Cookie\nVISITOR_PRIVACY_METADATA\n * Duration\n6 months\n * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n * Cookie\ncookietest\n * Duration\nsession\n * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n * Cookie\n__Host-airtable-session\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\n__Host-airtable-session.sig\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\nm\n * Duration\n1 year 1 month 4 days\n * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n * Cookie\nBIGipServer*\n * Duration\nsession\n * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n * Cookie\n__cfruid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n * Cookie\n_GRECAPTCHA\n * Duration\n6 months\n * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n * Cookie\n__Secure-YNID\n * Duration\n6 months\n * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n * Cookie\ncookieyes-consent\n * Duration\n1 year\n * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n * Cookie\nlidc\n * Duration\n1 day\n * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n * Cookie\nbrw\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nbrwConsent\n * Duration\n5 minutes\n * Description\nDescription is currently not available.\n\n\n * Cookie\nWMF-Uniq\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nloom_anon_comment\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nloom_referral_video\n * Duration\nsession\n * Description\nDescription is currently not available.\n\n\n * Cookie\nVISITOR_INFO1_LIVE\n * Duration\n6 months\n * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n * Cookie\nyt-remote-connected-devices\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n * Duration\nNever Expires\n * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n * Cookie\nyt-remote-device-id\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-name\n * Duration\nsession\n * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\nyt-remote-fast-check-period\n * Duration\nsession\n * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-app\n * Duration\nsession\n * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n * Cookie\nyt-remote-cast-available\n * Duration\nsession\n * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n * Cookie\nyt-remote-cast-installed\n * Duration\nsession\n * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\ncp_session\n * Duration\n3 months\n * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n * Cookie\nloid\n * Duration\n1 year 1 month 4 days\n * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n * Cookie\n__hstc\n * Duration\n6 months\n * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n * Cookie\nhubspotutk\n * Duration\n6 months\n * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n * Cookie\n_ga\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n * Cookie\n_ga_*\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n * Cookie\n__Host-psifi.analyticsTrace\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n__Host-psifi.analyticsTraceV2\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n_gh_sess\n * Duration\nsession\n * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n * Cookie\nYSC\n * Duration\nsession\n * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n * Cookie\najs_anonymous_id\n * Duration\n1 year\n * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n * Cookie\nvuid\n * Duration\n1 year 1 month 4 days\n * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n * Cookie\nAWSALB\n * Duration\n7 days\n * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n * Cookie\nacq\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nacq.sig\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nptc\n * Duration\n2 years\n * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n * Cookie\nmuc_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n * Cookie\nguest_id_marketing\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\nguest_id_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\npersonalization_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n * Cookie\nguest_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n * Cookie\nbcookie\n * Duration\n1 year\n * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n * Cookie\n__Secure-ROLLOUT_TOKEN\n * Duration\n6 months\n * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n * Cookie\nyt.innertube::nextId\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nyt.innertube::requests\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nsession_tracker\n * Duration\nsession\n * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n * Cookie\nedgebucket\n * Duration\nsession\n * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n * Cookie\ndid\n * Duration\n1 year\n * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Data Science](https://towardsdatascience.com/category/data-science/)\n# Bayesian Statistics\nHard to believe there was once a controversy over probabilistic statistics\n[James Andrew Godwin](https://towardsdatascience.com/author/james-andrew/)\nFebruary 25, 202117 min read\n### [Getting Started](https://towardsdatascience.com/tagged/getting-started)\n![Frequentists vs Bayesians | Photo by Lucas Benjamin](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1bbNOuUHe8wFQF1v2sdul3Q.jpeg)\nFrequentists vs Bayesians | Photo by [Lucas Benjamin](https://unsplash.com/@aznbokchoy)\nThis article builds on my previous article about [Bootstrap Resampling](https://towardsdatascience.com/bootstrap-resampling-2b453bb036ec).\nLearn this step by step with the interactive [AI and Data Scientist](https://roadmap.sh/ai-data-scientist) roadmap.\n## Introduction to Bayes Models\nBayesian models are a rich class of models, which can provide attractive alternatives to Frequentist models. Arguably the most well-known feature of Bayesian statistics is Bayes theorem, more on this later. With the recent advent of greater computational power and general acceptance, Bayes methods are now widely used in areas ranging from medical research to natural language processing (NLP) to understanding to web searches.\nIn the early 20th century there was a big debate about the legitimacy of what is now called Bayesian, which is essentially a probabilistic way of doing some statistics - in contrast to the \"Frequentist\" camp that we are all intimately familiar with. To use the parlance of our times, it was the #Team Jacob vs #Team Edward debate of its time, among statisticians. The argument is now moot as almost everyone is both Bayesian and Frequentist. A good heuristic is that some problems are better handled by Frequentist methods and some with Bayesian methods.\nBy the end of this article I hope you can appreciate this joke: A Bayesian is one who, vaguely expecting a horse, and catching a glimpse of a donkey, strongly believes she has seen a mule!\n### History\nA limited version of Bayes theorem was proposed by the eponymous Reverend Thomas Bayes (1702–1761). Bayes' interest was in probabilities of gambling games. He also was an avid supporter of Sir Isaac Newton's newfangled theory of calculus with his amazingly entitled publication, \"_An Introduction to the Doctrine of Fluxions, and a Defence of the Mathematicians Against the Objections of the Author of The Analyst_.\"\nIn 1814, Pierre-Simon Laplace published a version of Bayes theorem similar to its modern form in the \"_Essai philosophique sur les probabilités_.\" Laplace applied Bayesian methods to problems in celestial mechanics. Solving these problems had great practical implications for ships in the late 18th/early 19th centuries. The geophysicist and mathematician Harold Jeffreys extensively used Bayes' methods.\nWWII saw many successful applications of Bayesian methods:\n * Andrey Kolmagorov, a Russian (née Soviet) statistician used Bayes methods to greatly improve artillery accuracy\n * Alan Turing used Bayesian models to break German U-boat codes\n * Bernard Koopman, French born American mathematician, helped the Allies ability to locate German U-boats by intercepting directional radio transmissions\n\n\nThe latter half of the 20th century lead to the following notable advances in computational Bayesian methods:\n * Statistical sampling using Monte Carlo methods; Stanislaw Ulam, John von Neuman; 1946, 1947\n * MCMC, or [Markov Chain Monte Carlo](https://en.wikipedia.org/wiki/Markov_chain_Monte_Carlo); Metropolis et al. (1953) Journal of Chemical Physics\n * Hastings (1970), Monte Carlo sampling methods using Markov chains and their application\n * Geman and Geman (1984) Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images\n * Duane, Kennedy, Pendleton, and Roweth (1987), Hamiltonian MCMC\n * Gelfand and Smith (1990), Sampling-based approaches to calculating marginal densities.\n\n\n### Bayesian vs Frequentist views\nWith greater computational power and general acceptance, Bayes methods are now widely used in areas ranging from medical research to natural language understanding to web search. Among pragmatists, the common belief today is that some problems are better handled by frequentist methods and some with Bayesian methods.\nFrom my first article on the [Central Limit Theorem](https://medium.com/swlh/on-the-importance-of-the-central-limit-theorem-e8cce1f4d253) (CLT), I established that you need \"enough data\" for the CLT to apply. Bootstrap sampling is a convenient way of obtaining more distributions that approach your underlying distribution. These are Frequentist tools; but what happens if you have too little data to reliably apply the CLT? As a probabilistic Bayesian statistician, I already have some estimates, admittedly, with a little bit of guesswork, but there is a way of combining that guesswork in an intelligent fashion with my data. And that is what Bayesian statistics is basically all about - you combine it and basically, that combination is a simple multiplication of the two probable probability distributions, the one that you guessed at, and the other one, that for which you have evidence.\nThe more data you have, the prior distribution becomes less and less important and Frequentist methods can be used. But, with less data, Bayesian's will give you a better answer.\nSo how then do Bayesian and Frequentist methodologies differ? Simply stated:\n * Bayesian methodologies use **prior distributions** to quantify what is known about the parameters\n * Frequentists do not quantify anything about the parameters; instead using **p-values** and **confidence intervals** (CI's) to express the unknowns about parameters\n\n\nBoth methods are useful, converging with more and more data - but contrasted:\n**Strict Frequentist approach**\n * **Goal is a point estimate and CI**\n * Start from observations\n * Re-compute model given new observations\n * Examples: Mean estimate, t-test, ANOVA\n\n\n**Bayesian approach**\n * **Goal is posterior distribution**\n * Start from prior distribution (the guesswork)\n * Update posterior (the belief/hypothesis) given new observations\n * Examples: posterior distribution of mean, highest density interval (HDI) overlap\n\n\nToday no one is a strict Frequentist. Bayesian methodologies simply work, that is fact.\n### Bayes theorem\nBayes theorem describes the probability of an event, based on **prior knowledge (our guesswork)** of conditions that might be related to the event, **our data**. Bayes theorem tells us how to combine these two probabilities. For example, if developing a disease like Alzheimer's is related to age, then, using Bayes' theorem, a person's age can be used to more reliably assess the probability that they have Alzheimer's, or cancer, or any other age-related disease.\nBayes theorem is used to update the probability for a hypothesis; the hypothesis is this guesswork, this prior distribution thing. Prior means your prior belief, the belief that you have before you have evidence, and it is a way to update it. Because you get more information, you update your hypothesis, and so on and so forth.\nFor those who do not know, or who have forgotten, let us derive Bayes theorem from the rule for conditional probability:\n![Conditional probability](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/0v5UDmwkYM37-XheO.png)\nConditional probability\nTranslates to \"the probability of A happening given B is the case.\"\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/0LADNbajB7xaa0DAz.png)\n\"The probability of B happening given A happens.\"\nEliminating 𝑃(𝐴∩𝐵) and doing very minor algebra:\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/0gfTIDgijb56AsCwk.png)\nor, rearranging terms\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/07JzIc55IFq6idLy3.png)\nWhich is Bayes theorem! This describes how one finds the conditional probability of event A given event B.\n### Applications\nYou have a disease test, and the probability that you will get a positive test result given that you have the disease is really, really high; in other words the test has a very high accuracy rate. The problem is that there is a probability that you will get a positive test result even if you do not have the disease. And that you can simply calculate from Bayes law. The big point is, is that these probabilities are not the same as the probability that you will get a positive result given the disease **is not the same** as the probability that you will have the disease given a positive result.\n![The test sensitivity does not match the rate of incidence of the disease](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/03CCMNmF1tJM7GmiR.png)\nThe test sensitivity does not match the rate of incidence of the disease\nThese are two different probability distributions. And what makes them so different is the probability of disease and the probability of a positive test result. So if the disease is rare, the probability of disease will be very, very small.\nDisease testing: A = Have disease, B = Tested positive.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/0bs-SsfYiDtoyq0VX.png)\n### Example\n![The sensitivity of this test is quite bad despite the high accuracy of the test](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/0ZlfT_9A8LJ6RrrB0.png)\nThe sensitivity of this test is quite bad despite the high accuracy of the test\nUsing Bayes theorem we have deduced that a positive test result for this disease indicates that only 10% of the time the person will actually have the disease, because the incidence rate of the disease is an order of magnitude lower than the false positive rate.\n### Python example: probabilities of hair and eye color\nA sample population has the following probabilities of hair and eye color combinations. I will run the code below in a Jupyter Notebook to find the conditional probabilities:\ntext\n\n```\n# begin with the imports\nimport pandas as pd\nimport numpy as np\nimport matplotlib\nfrom matplotlib import pyplot as plt\nimport scipy\nimport seaborn as sns\nimport itertools\n```\n\ntext\n\n```\n%matplotlib inline\n```\n\ntext\n\n```\n# create the dataframe\nhair_eye = pd.DataFrame({\n 'black': [0.11, 0.03, 0.03, 0.01],\n 'blond': [0.01, 0.16, 0.02, 0.03],\n 'brunette': [0.2, 0.14, 0.09, 0.05],\n 'red': [0.04, 0.03, 0.02, 0.02],\n\n}, index=['brown', 'blue', 'hazel', 'green'])\n```\n\ntext\n\n```\nhair_eye\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1sAVP1CrcwbnAcoiikc7S3w.jpeg)\nN.B: this is string index for eye color rather than a numeric zero-based index.\ntext\n\n```\nhair_eye.loc['hazel', 'red']\n```\n\n![2% of people have hazel eyes given red hair](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1WSVcCrqgIvKX0nyuzCd3-Q.jpeg)\n2% of people have hazel eyes given red hair\nThe figures in the table above are the **conditional probabilities**. Note that in this case:\n𝑃(hair|eye)=𝑃(eye|hair)P(hair|eye)=P(eye|hair)\nGiven these conditional probabilities, it is easy to compute the marginal probabilities by summing the probabilities in the rows and columns. The **marginal probability** is the probability along one variable (one margin) of the distribution. The marginal distributions must sum to unity (1).\ntext\n\n```\n## Compute the marginal distribution of each eye color\nhair_eye['marginal_eye'] = hair_eye.sum(axis=1)\n```\n\ntext\n\n```\nhair_eye\n```\n\n![Added marginal_eye column](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1PhSToVTXSnWKktNSl4YeEQ.jpeg)\nAdded marginal_eye column\ntext\n\n```\nhair_eye.sum(axis=0)\n```\n\n![The hair color probabilities sum to unity](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1NDgZfkSz_O8dHB6j3E1gug.png)\nThe hair color probabilities sum to unity\ntext\n\n```\nhair_eye.loc['marginal_hair'] = hair_eye.sum(axis=0)\nhair_eye.describe\n```\n\n![The eye color probabilities also sum to 1.0](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1XQZgtlj870v-QlyXkKmOfg.jpeg)\nThe eye color probabilities also sum to 1.0\nI have blue eyes. I wonder what the marginal probabilities of hair color are given that they have blue eyes?\ntext\n\n```\n# probability of blue eyes given any hair color divided by total probability of blue eyes\n```\n\ntext\n\n```\n# marginal probabilities of hair color given blue eyes\n#blue_black(.03/.36) + blue_brunette(.14/.36) + blue_red(.03/.36) + blue_blond(.16/.36)\nblue = hair_eye.loc['blue',:]/hair_eye.loc['blue','marginal_eye']\nblue\n```\n\n![For a given person with blue eyes, it is 44% likely they have blond hair, like me](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1Ex7g2ZoRPejHAyjtzyjUOA.jpeg)\nFor a given person with blue eyes, it is 44% likely they have blond hair, like me\n## Applying Bayes theorem\nThere is a formulation of Bayes theorem that is convenient to use for computational problems because we do not want to sum all of the possibilities to compute **P(B)** , like in the example above.\nHere are more facts and relationships about conditional probabilities, largely arising from the fact that P(B) is so hard to calculate:\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1BYaPhxDZ0UG__V3EhAI16A.png)\nRewriting Bayes theorem:\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/00p3PNqnGNv6j7_QW.png)\nThis is a mess, but we do not always need the complicated denominator, which is the probability of B. **P(B)** is our actual evidence, which is hard to come by. Remember that this is probabilistic and that we are working with smaller data samples than we are normally used to.\nRewriting we get:\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/0qEmsDvp9GBOtH29l.png)\nIgnoring the constant **𝑘, b** ecause it is given that the evidence - P(B) - is presumed to be constant, we get:\n![We do not need to know the probability distribution of the actual evidence, of the data](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/0HtJbgUnD_Rzr6i1o.png)\nWe do not need to know the probability distribution of the actual evidence, of the data\n### Applying the reduced relationship Bayes theorem\nWe interpret the reduced relationship above as follows:\n![Proportional Bayes](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1vDt2ltstA8VRxEme8vIWXA.png)\nProportional Bayes\nThe proportional relationships apply to the observed data distributions, or to parameters in a model (partial slopes, intercept, error distributions, lasso constant, e.g.,). The above equations tell you how to combine your prior belief if you want to call that, which I call the prior distribution with your evidence.\nAn important nuance is that while we cannot speak about exact values we can talk about distributions; we can talk about the shapes of the curves of the probability distribution curves. What we want is the so-called posterior distribution. That is what we want. What we do is guess at the prior distribution: what is the probability of A? What is the probability that something is true given the data that I have? What is the probability of A given B? what is the probability that something is true given the data I have? Some people will rephrase this as what is the probability that my hypothesis A is true given the data B?\nI might have very little data. Too little to do any Frequentist approaches; but I can estimate that with the little small amount of data I have, called the **likelihood** , I can estimate the probability that this data would be correct, given my hypothesis that I can calculate directly. Because I have the data, not a lot of data, it is too little for me to actually believe, but I have some data. And I have my hypothesis. And so I can do my calculations on my distributions.\nThe prior distribution is the probability of my hypothesis, which I do not know - but I can guess it. If I am really bad at guessing, I am going to say that this prior distribution is a **uniform distribution** , it could be a Gaussian or any other shape. Now if I guess a uniform distribution I am still doing probabilistic statistics, Bayesian statistics, but I am getting really, really close to what Frequentists would do.\nIn Bayesian, I will update this prior distribution as I get more data because I will say, \"Hey!\" at first I thought it was uniform. Now I see that this thing, this probability distribution actually has a mean somewhere in the middle. So I am now going to choose a different distribution. Or I'm going to modify my distribution to accommodate what I am seeing in my posterior. I am trying to make this prior distribution look similar in shape to my posterior distribution. And when I say similar in shape, that is actually very misleading. What I am trying to do is find a mathematical formula that will mix the probability of A and the probability of A given B similarly. Similar in type that the math, that the equation looks similar, as in the formulaic of the question and not the actual probabilities; we call that getting the conjugate prior. This is when you search for a distribution that is mathematically similar to what you believe to be the final distribution. This is formally referred to as searching for a conjugate prior. A beta distribution, because of its flexibility is often, but not always, used as the prior.\nTo reiterate, the **hypothesis is formalized as parameters in the model** , P(A), and the likelihood, **P(B|A)** , is the probability of the data given those parameters - this is easy to do, t-tests can do this!\n### Creating Bayes models\nGiven prior assumptions about the behavior of the parameters (the prior), produce a model which tells us the probability of observing our data, to compute new probability of our parameters. Thus the steps for working with a Bayes model are:\n * Identify data relevant to the research question: e.g., measurement scales of the data\n * Define a descriptive model for the data. For instance, pick a linear model formula or a beta distribution of (2,1)\n * Specify a prior distribution of the parameters - this the formalization of the hypothesis. For example, thinking the error in the linear model is normally distributed as 𝑁(𝜃,𝜎²)\n * Choose the Bayesian inference formula to compute posterior parameter probabilities\n * Update Bayes model if more data is observed. **This is key!** The posterior distribution naturally updates as more data is added\n * Simulate data values from realizations of the posterior distribution of the parameters\n\n\n### Choosing a prior\nThe choice of the prior distribution is very important to Bayesian analysis. A prior should be convincing to a **skeptical audience**. Some heuristics to consider are:\n * Domain knowledge (SME)\n * Prior observations\n * If poor knowledge use less informative prior\n * **Caution:** the uniform prior is informative, you need to set the limits on the range of values\n\n\nOne analytically and computationally simple choice is the before mentioned conjugate prior. When a likelihood is multiplied by a conjugate prior the distribution of the posterior is the same as the likelihood. Most named distributions have conjugates:\n![A conjugate prior is not always used](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1IFAqEDajqVyCukbxAGwFdA.png)\nA conjugate prior is not always used\n### Python Example: COVID-19 random sampling\nAs a health official for your city, you are interested in analyzing COVID-19 infection rates. You decide to sample 10 people at an intersection while the light is red and determine if they test positive for Covid - the test is fast and to avoid sampling bias we take the test to them. The data are binomially distributed; a person tests positive or tests negative - for the sake of simplicity, assume the test is 100% accurate.\nIn this notebook example:\n * Select a prior for the parameter 𝑝, the probability of having COVID-19.\n * Using data, compute the likelihood.\n * Compute the posterior and posterior distributions.\n * Try another prior distribution.\n * Add more data to our data set to updated the posterior distribution.\n\n\nThe likelihood of the data and the posterior distribution are binomially distributed. The binomial distribution has one parameter we need to estimate, 𝑝, the probability. We can write this formally for 𝑘 successes in 𝑁 trials:\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1aClYuCdGoLDj_umkkoTvog.png)\nThe following code computes some basic summary statistics:\ntext\n\n```\nsampled = ['yes','no','yes','no','no','yes','no','no','no','yes']\npositive = [1 if x is 'yes' else 0 for x in sampled]\npositive\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/18oNPhxb-nuos7y51YfvAVQ.png)\ntext\n\n```\nN = len(positive) # sample size\nn_positive = sum(positive) # number of positive drivers\nn_not = N - n_positive # number negative\nprint('Tested positive= %d Tested negative= %d'\n 'nProbability of having COVID-19= %.1f' \n % (n_positive, n_not, n_positive / (n_positive+ n_not)))\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1T7bGWDWHyrEw4EXftJdWHw.png)\nFor the prior I will start with a uniform distribution as I do not know what to expect re Covid rates:\ntext\n\n```\nN = 100\np = np.linspace(.01, .99, num=N)\npp = [1./N] * N\nplt.plot(p, pp, linewidth=2, color='blue')\nplt.show()\n```\n\nNext step: compute the likelihood. The likelihood is the probability of the data given the parameter, **𝑃(𝑋|𝑝). E** ach observation of each test as \"positive\" or \"not\" is a Bernoulli trial, so I choose the binomial distribution.\ntext\n\n```\ndef likelihood(p, data):\n k = sum(data)\n N = len(data)\n # Compute Binomial likelihood\n l = scipy.special.comb(N, k) * p**k * (1-p)**(N-k)\n # Normalize the likelihood to sum to unity\n return l/sum(l)\n```\n\ntext\n\n```\nl = likelihood(p, positive)\nplt.plot(p, l)\nplt.title('Likelihood function')\nplt.xlabel('Parameter')\nplt.ylabel('Likelihood')\nplt.show()\n```\n\n![The probability our data is true given our hypothesis](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/196QVhTBNEsVk4VMJF-gLMg.png)\nThe probability our data is true given our hypothesis\nNow that we have a prior and a likelihood we can compute the posterior distribution of the parameter 𝑝: 𝑃**(𝑝|𝑋).**\nN.B. The computational methods used here are simplified for the purpose of illustration. Computationally efficient code must be used!\ntext\n\n```\ndef posterior(prior, like):\n post = prior * like # compute the product of the probabilities\n return post / sum(post) # normalize the distribution\n```\n\ntext\n\n```\ndef plot_post(prior, like, post, x):\n maxy = max(max(prior), max(like), max(post))\n plt.figure(figsize=(12, 4))\n plt.plot(x, like, label='likelihood', linewidth=12, color='black', alpha=.2)\n plt.plot(x, prior, label='prior')\n plt.plot(x, post, label='posterior', color='green')\n plt.ylim(0, maxy)\n plt.xlim(0, 1)\n plt.title('Density of prior, likelihood and posterior')\n plt.xlabel('Parameter value')\n plt.ylabel('Density')\n plt.legend()\n\npost = posterior(pp, l)\nplot_post(pp, l, post, p)\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1rQ9UPVUhCikpG5OToQ4gaw.png)\ntext\n\n```\nprint('Maximum of the prior density = %.3f' % max(pp))\nprint('Maximum likelihood = %.3f' % max(l))\nprint('MAP = %.3f' % max(post))\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1JU-a6EpDYNjuJouSZlWagw.png)\nWith a uniform prior distribution, the **posterior is just the likelihood**. The key point is that the Frequentist probabilities are identical to the Bayesian posterior distribution given a uniform prior.\n### Trying a different prior\nFrom the chart above, I chose the **conjugate prior** of the Binomial distribution which is the beta distribution. The beta distribution is defined on the interval 0 ≤ Beta(P|A, B)≤10 ≤ Beta(P|A, B)≤ 1. The beta distribution has two parameters, 𝑎 and 𝑏, which determine the shape.\ntext\n\n```\nplt.figure(figsize=(12, 8))\n```\n\ntext\n\n```\nalpha = [.5, 1, 2, 3, 4]\nbeta = alpha[:]\nx = np.linspace(.001, .999, num=100) #100 samples\n```\n\ntext\n\n```\nfor i, (a, b) in enumerate(itertools.product(alpha, beta)):\n plt.subplot(len(alpha), len(beta), i+1)\n plt.plot(x, scipy.stats.beta.pdf(x, a, b))\n plt.title('(a, b) = ({}, {})'.format(a,b))\nplt.tight_layout()\n```\n\n![Note how varied these beta distributions can become with slightly different parameters](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1tKGns3ITagawxeX5bvqWow.png)\nNote how varied these beta distributions can become with slightly different parameters\nI still do not know a lot about the behavior of drivers at this intersection who may or may not have COVID-19, so I pick a rather uninformative or broad beta distribution as the prior for the hypothesis. I choose a symmetric prior with a=2 and b=2:\ntext\n\n```\ndef beta_prior(x, a, b):\n l = scipy.stats.beta.pdf(p, a, b) # compute likelihood\n return l / l.sum() # normalize and return\n```\n\ntext\n\n```\npp = beta_prior(p, 2, 2)\npost = posterior(pp, l)\nplot_post(pp, l, post, p)\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1x2fT3QybkxvZW7tyhN8lwQ.png)\nTake note that the mode of the posterior is close to the mode of the likelihood, but has shifted toward the mode of the prior. This behavior of Bayesian posteriors to be shifted toward the prior is known as the **shrinkage** property: the tendency of the maximum likelihood point of the posterior is said to shrink toward the maximum likelihood point of the prior.\n### Updating the Bayesian model\nLet us update the model with 10 new observations to our data set. Note that the more observations added to the model moves the posterior distribution closer to the likelihood.\nN.B. With large datasets, it might require **HUGE** amounts of data to see convergence in behavior of the posterior and the likelihood.\ntext\n\n```\nnew_samples = ['yes','no','no','no','no',\n 'yes','no','yes','no','no'] # new data to update model, n = 20\nnew_positive = [1 if x is 'yes' else 0 for x in new_samples]\n```\n\ntext\n\n```\nl = likelihood(p, positive+ new_positive)\npost = posterior(pp, l)\nplot_post(pp, l, post, p)\n```\n\n![Compared to the first one, note the convergence of the two distributions as we updated with more data](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1NF9Xto4UwBJH_O-mg0F_Yw.png)\nCompared to the first one, note the convergence of the two distributions as we updated with more data\n### Credible Intervals\nA **credible interval** is an interval on the Bayesian posterior distribution. The credible interval is sometimes called the highest density interval (HDI). For instance, the 90% credible interval encompasses 90% of the posterior distribution with the highest probability density. It can be confusing since both credible intervals of Bayesian and confidence intervals of Frequentist are abbreviated the same (CI).\nLuckily, the **credible interval is the Bayesian analog of the Frequentist confidence interval**. However, they are conceptually different. The confidence interval is chosen on the distribution of a test statistic, whereas the credible interval is computed on the posterior distribution of the parameter. For symmetric distributions, the credible interval can be numerically the same as the confidence interval. However, in the general case, these two quantities can be quite different.\nNow we plot the posterior distribution of the parameter of the binomial distribution parameter p. The 95% credible interval, or HDI, is also computed and displayed:\ntext\n\n```\nnum_samples = 100000\nlower_q, upper_q = [.025, .975]\n```\n\ntext\n\n```\n# Caution, this code assumes a symmetric prior distribution and will not work in the general case\n```\n\ntext\n\n```\ndef plot_ci(p, post, num_samples, lower_q, upper_q):\n # Compute a large sample by resampling with replacement\n samples = np.random.choice(p, size=num_samples, replace=True, p=post)\n ci = scipy.percentile(samples, [lower_q*100, upper_q*100]) # compute the quantiles\n\n interval = upper_q - lower_q\n plt.title('Posterior density with %.3f credible interval' % interval)\n plt.plot(p, post, color='blue')\n plt.xlabel('Parameter value')\n plt.ylabel('Density')\n plt.axvline(x=ci[0], color='red')\n plt.axvline(x=ci[1], color='red')\n print('The %.3f credible interval is %.3f to %.3f' \n % (interval, lower_q, upper_q))\n\nplot_ci(p, post, num_samples, lower_q, upper_q)\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1zus9GoXEr2vVt8ukTd-_mA.png)\nBecause the **posterior distribution above is symmetric** , like the beta prior we chose, **the analysis is identical to CI's in the frequentist approach**. We can be 95% confident that around 40% of driver's tested at this intersection during a red light will have COVID-19! This info is very useful for health agencies to allocate resources appropriately.\nIf you remember the joke in the third paragraph, about the Bayesian who vaguely suspected a horse and caught a fleeting glimpse of a donkey and strongly suspects seeing a mule - I hope you can now assign each clause of the joke the appropriate part Bayes theorem.\nCheck out my next piece on [linear regression](https://towardsdatascience.com/an-introduction-to-linear-regression-9cbb64b52d23) and using [bootstrap with regression models](https://towardsdatascience.com/linear-regression-with-bootstrapping-4924c05d2a9).\nFind me on [Linkedin](https://www.linkedin.com/in/james-a-w-godwin/)\n_Physicist cum Data Scientist- Available for new opportunity | SaaS | Sports | Start-ups | Scale-ups_\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](https://assets.insightmediagroup.io/media/1787657234707_1wi608.jpeg)\nWritten By\n## [James Andrew Godwin](https://towardsdatascience.com/author/james-andrew/)\n[See all from James Andrew Godwin](https://towardsdatascience.com/author/james-andrew/)\n[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Bayes Theorem](https://towardsdatascience.com/tag/bayes-theorem/)[Bayesian Statistics](https://towardsdatascience.com/tag/bayesian-statistics/)[Data Science](https://towardsdatascience.com/tag/data-science/)[Getting Started](https://towardsdatascience.com/tag/getting-started/)\nShare This Article\n * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fbayesian-statistics-11f225174d5a%2F&title=Bayesian%20Statistics)\n * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fbayesian-statistics-11f225174d5a%2F&title=Bayesian%20Statistics)\n * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fbayesian-statistics-11f225174d5a%2F&text=Bayesian%20Statistics)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/08/0V4Th8PGgGokuP8Rg-scaled.jpg)](https://towardsdatascience.com/semantic-chunking-for-rag-35b7675ffafd/)\n## [Why Does Position-Based Chunking Lead to Poor Performance in RAGs?](https://towardsdatascience.com/semantic-chunking-for-rag-35b7675ffafd/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nHow to implement semantic chunking and gain better results.\n[Thuwarakesh Murallie](https://towardsdatascience.com/author/thuwarakesh/)\nAugust 22, 202411 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2022/09/04-7EssdS2374Og7m-scaled.jpg)](https://towardsdatascience.com/why-you-should-learn-machine-learning-engineering-and-not-data-science-e616141ea866/)\n## [Why You Should Learn Machine Learning Engineering and Not Data Science](https://towardsdatascience.com/why-you-should-learn-machine-learning-engineering-and-not-data-science-e616141ea866/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nAnd how you can start\n[Patrick Brus](https://towardsdatascience.com/author/brus-patrick63/)\nSeptember 28, 20227 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/01/1Ibo72DSqEuv55qfBuVT0Ag.png)](https://towardsdatascience.com/the-lottery-data-behind-the-luck-ambition-and-a-billion-dollar-dream-c31fdc17d170/)\n## [Data behind the Luck, Ambition, and a Billion-Dollar Dream: Lottery](https://towardsdatascience.com/the-lottery-data-behind-the-luck-ambition-and-a-billion-dollar-dream-c31fdc17d170/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nUsing Seattle's local retail store data for consumer patterns of the lottery (SQL, Python)\n[Sung Ahn](https://towardsdatascience.com/author/sahn1998/)\nJanuary 4, 202512 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/11/0uQhoT2C87TDr8EEz.jpg)](https://towardsdatascience.com/the-best-machine-learning-algorithm-for-handwritten-digits-recognition-2c6089ad8f09/)\n## [The Best Machine Learning Algorithm for Handwritten Digits Recognition](https://towardsdatascience.com/the-best-machine-learning-algorithm-for-handwritten-digits-recognition-2c6089ad8f09/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nImplementing Machine Learning Classification Algorithms to Recognize Handwritten Digits\n[Mahnoor Javed](https://towardsdatascience.com/author/mahnoorjaved/)\nNovember 21, 20208 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/12/0id9v0rtmFRiQeRVV-scaled.jpg)](https://towardsdatascience.com/creating-dynamic-dashboards-with-streamlit-747b98a68ab5/)\n## [Creating dynamic dashboards with Streamlit](https://towardsdatascience.com/creating-dynamic-dashboards-with-streamlit-747b98a68ab5/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nUsing Streamlit to develop dynamically updated data visualizations\n[Mohammad Khorasani](https://towardsdatascience.com/author/khorasani/)\nDecember 4, 20203 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/04/todd-cravens-lwACYK8ScmA-unsplash.jpg)](https://towardsdatascience.com/how-to-benchmark-deepseek-r1-distilled-models-on-gpqa-using-ollama-and-openais-simple-evals/)\n## [How to Benchmark DeepSeek-R1 Distilled Models on GPQA Using Ollama and OpenAI’s simple-evals](https://towardsdatascience.com/how-to-benchmark-deepseek-r1-distilled-models-on-gpqa-using-ollama-and-openais-simple-evals/)\n[Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/)\nSet up and run the GPQA-Diamond benchmark on DeepSeek-R1's distilled models locally to evaluate its reasoning capabilities.\n[Kenneth Leung](https://towardsdatascience.com/author/kennethleungty/)\nApril 23, 20253 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/10/1ftiDLeJNLEQsbTdoa4yMeA.jpeg)](https://towardsdatascience.com/two-simple-things-you-need-to-steal-from-agile-for-data-and-analytics-work-d3130050f0e5/)\n## [Two Simple Things You Need to Steal from Agile for Data and Analytics Work](https://towardsdatascience.com/two-simple-things-you-need-to-steal-from-agile-for-data-and-analytics-work-d3130050f0e5/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nPeer Review and Definition of Done: Small Changes, BIG Impact\n[Jon Loyens](https://towardsdatascience.com/author/jonloyens/)\nOctober 4, 20217 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2022/07/1kRxMVB-G-AjFUjpwhNOXaA-scaled.jpeg)](https://towardsdatascience.com/failed-in-data-science-interview-here-are-some-tips-to-help-you-succeed-2e9a6bb2b307/)\n## [Failed in Data Science Interview? Here Are Some Tips to Help You Succeed](https://towardsdatascience.com/failed-in-data-science-interview-here-are-some-tips-to-help-you-succeed-2e9a6bb2b307/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nBased on 10,000+ hours of experience and many failures!\n[Sharan Kumar Ravindran](https://towardsdatascience.com/author/rsharankumar/)\nJuly 29, 202210 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/05/0dyLU0htgzeBYD6L3-scaled.jpg)](https://towardsdatascience.com/analyze-scientific-publications-with-e-utilities-and-python-56f76de22959/)\n## [Analyze Scientific Publications with E-utilities and Python](https://towardsdatascience.com/analyze-scientific-publications-with-e-utilities-and-python-56f76de22959/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nHow to gather data about scientific literature and discover trends\n[Jozsef Meszaros](https://towardsdatascience.com/author/neuraljojo/)\nMay 19, 202316 min read\n\n\n * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n * [X](https://x.com/TDataScience)\n * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n * [Threads](https://www.threads.net/@towardsdatascience)\n * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n#### Recommended Articles \nClose\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500) How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ \"How Relevance Models Foreshadowed Transformers for NLP\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n\n\n", "body_base_path": "attachments/249f20a1f7a8/pages/", "body_markdown_path": "attachments/249f20a1f7a8/pages/archived-body.md", "local_html": "attachments/249f20a1f7a8/pages/archived-body.html", "body_source": "prior_capture" }