{
  "source_file": "0930",
  "source_platform": "Medium",
  "author": "James Andrew Godwin",
  "title_listed": "An Introduction to Linear Regression",
  "keywords": "",
  "domain": "机器学习",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://towardsdatascience.com/an-introduction-to-linear-regression-9cbb64b52d23/",
  "publish_date": "2021-03-09",
  "corpus_stage": "",
  "notes": "",
  "id": "d3f227ccb337",
  "collected_at": "2026-10-02T08:45:15",
  "title": "An Introduction to Linear Regression | Towards Data Science",
  "body_markdown": "![Revisit consent button](../images/body-df29beacd7496220cc94.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](../images/body-2159bb67f3de8ad870eb.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n  * Cookie\nBCTempID\n  * Duration\n10 minutes\n  * Description\nNo description available.\n\n\n  * Cookie\n__cf_bm\n  * Duration\n1 hour\n  * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n  * Cookie\nAWSALBCORS\n  * Duration\n7 days\n  * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n  * Cookie\n_cfuvid\n  * Duration\nsession\n  * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n  * Cookie\nli_gc\n  * Duration\n6 months\n  * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n  * Cookie\n__hssrc\n  * Duration\nsession\n  * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n  * Cookie\n__hssc\n  * Duration\n1 hour\n  * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n  * Cookie\nwpEmojiSettingsSupports\n  * Duration\nsession\n  * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n  * Cookie\nBCSessionID\n  * Duration\n1 year 1 month 4 days\n  * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n  * Cookie\n_octo\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nlogged_in\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\n__Secure-YEC\n  * Duration\npast\n  * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n  * Cookie\n__eoi\n  * Duration\n6 months\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nAWSALBTGCORS\n  * Duration\n7 days\n  * Description\nNo description available.\n\n\n  * Cookie\nlogin-status-p\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nAWSALBTG\n  * Duration\n7 days\n  * Description\nNo description available.\n\n\n  * Cookie\ncsrf_token\n  * Duration\nsession\n  * Description\nNo description available.\n\n\n  * Cookie\ntoken_v2\n  * Duration\n1 day\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nD\n  * Duration\n1 year\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nPHPSESSID\n  * Duration\nsession\n  * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n  * Cookie\nVISITOR_PRIVACY_METADATA\n  * Duration\n6 months\n  * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n  * Cookie\ncookietest\n  * Duration\nsession\n  * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n  * Cookie\n__Host-airtable-session\n  * Duration\n1 year\n  * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n  * Cookie\n__Host-airtable-session.sig\n  * Duration\n1 year\n  * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n  * Cookie\nm\n  * Duration\n1 year 1 month 4 days\n  * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n  * Cookie\nBIGipServer*\n  * Duration\nsession\n  * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n  * Cookie\n__cfruid\n  * Duration\nsession\n  * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n  * Cookie\n_GRECAPTCHA\n  * Duration\n6 months\n  * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n  * Cookie\n__Secure-YNID\n  * Duration\n6 months\n  * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n  * Cookie\ncookieyes-consent\n  * Duration\n1 year\n  * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n  * Cookie\nlidc\n  * Duration\n1 day\n  * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n  * Cookie\nbrw\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nbrwConsent\n  * Duration\n5 minutes\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nWMF-Uniq\n  * Duration\n1 year\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nloom_anon_comment\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nloom_referral_video\n  * Duration\nsession\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nVISITOR_INFO1_LIVE\n  * Duration\n6 months\n  * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n  * Cookie\nyt-remote-connected-devices\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n  * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n  * Duration\nNever Expires\n  * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n  * Cookie\nyt-remote-device-id\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n  * Cookie\nyt-remote-session-name\n  * Duration\nsession\n  * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n  * Cookie\nyt-remote-fast-check-period\n  * Duration\nsession\n  * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n  * Cookie\nyt-remote-session-app\n  * Duration\nsession\n  * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n  * Cookie\nyt-remote-cast-available\n  * Duration\nsession\n  * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n  * Cookie\nyt-remote-cast-installed\n  * Duration\nsession\n  * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n  * Cookie\ncp_session\n  * Duration\n3 months\n  * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n  * Cookie\nloid\n  * Duration\n1 year 1 month 4 days\n  * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n  * Cookie\n__hstc\n  * Duration\n6 months\n  * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n  * Cookie\nhubspotutk\n  * Duration\n6 months\n  * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n  * Cookie\n_ga\n  * Duration\n1 year 1 month 4 days\n  * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n  * Cookie\n_ga_*\n  * Duration\n1 year 1 month 4 days\n  * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n  * Cookie\n__Host-psifi.analyticsTrace\n  * Duration\n6 hours\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\n__Host-psifi.analyticsTraceV2\n  * Duration\n6 hours\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\n_gh_sess\n  * Duration\nsession\n  * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n  * Cookie\nYSC\n  * Duration\nsession\n  * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n  * Cookie\najs_anonymous_id\n  * Duration\n1 year\n  * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n  * Cookie\nvuid\n  * Duration\n1 year 1 month 4 days\n  * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n  * Cookie\nAWSALB\n  * Duration\n7 days\n  * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n  * Cookie\nacq\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nacq.sig\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nptc\n  * Duration\n2 years\n  * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n  * Cookie\nmuc_ads\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n  * Cookie\nguest_id_marketing\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n  * Cookie\nguest_id_ads\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n  * Cookie\npersonalization_id\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n  * Cookie\nguest_id\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n  * Cookie\nbcookie\n  * Duration\n1 year\n  * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n  * Cookie\n__Secure-ROLLOUT_TOKEN\n  * Duration\n6 months\n  * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n  * Cookie\nyt.innertube::nextId\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n  * Cookie\nyt.innertube::requests\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n  * Cookie\nsession_tracker\n  * Duration\nsession\n  * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n  * Cookie\nedgebucket\n  * Duration\nsession\n  * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n  * Cookie\ndid\n  * Duration\n1 year\n  * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Data Science](https://towardsdatascience.com/category/data-science/)\n# An Introduction to Linear Regression\nRegression is used primarily if you wish to predict or explain numeric data\n[James Andrew Godwin](https://towardsdatascience.com/author/james-andrew/)\nMarch 8, 202119 min read\n![Regression | Photo by Hao Wang](../images/000.jpeg)\nRegression | Photo by [Hao Wang](https://unsplash.com/@danranwanghao)\nThis article is a continuation of series that last covered [Bayes statistics](https://towardsdatascience.com/bayesian-statistics-11f225174d5a).\nVolatility is inherent within the stock market, but perhaps the rise of \"meme stocks\" is something that is beyond traditional comprehension. This article is not about such \"stonks\" - which is the ironic misspelling of stocks that encapsulates the new investing zeitgeist amongst Millennials (broadly speaking). Rather, I was pondering the fabulous rollercoaster of a ride of $GME as a timely example of something to apply linear regression to; one can apply a linear regression model to a given stock to predict its price in the future. This article and subsequent articles will expand upon the concepts of linear regression and several methods associated with it, its associated errors for model analysis as well as touching on residuals.\nBut what is regression? The goal of regression is to produce a model that represents the best fit for some observed data. A better, more specific way to state this is that regression is used to predict or explain numeric data **** - smarter people than me have been applying such models to financial data (like $GME) for decades. But, if you want to predict whether or not somebody is going to come down with a disease or whether or not somebody will be wearing black, or whether an asset will appreciate or depreciate in value those would not be regressions, those would be classifications. A** regression is either an explanatory model or a predictive model for numeric dat**a. And for the most part, we will be dealing with ones where we can describe a function - that means an output - we can describe the relationship between one or more input variables and a single output variable (for linear regression).\n## History\nRegression is based on the method of least squares or the method of minimum mean square error. The ideas around least squares or averaging errors have evolved over nearly three centuries. The first known publication of a 'Method of Averages' was by the German astronomer Tobias Mayer in 1750. Pierre-Simon Laplace published in 1788 a similar method. The first publication of the method of least squares was by the French mathematician Adrien-Marie Legendre in 1805.\nAs a historical aside, it is very likely that the German physicist and mathematician Carl Friedrich Gauss developed the method of least-squares as early as 1795, but he did not publish this method until 1809. Legendre firmly opposed any notion that Gauss had used the method earlier than the 1805 publishing date.\nFrancis Galton, a cousin of Charles Darwin, coined the term regression in 1886. Galton was interested in determining which traits of plants and animals, including humans, could be said to be inherited. While Galton invented a modern form of regression, it was **Karl Pearson** who put regression and multiple regression on a firm mathematical footing. He created the world's first university statistics department at UCL. Pearson's 1898 publication proposed a method of regression as we understand it today.\n![Karl Pearson | Source Wikipedia](../images/001.jpeg)\nKarl Pearson | Source [Wikipedia](https://en.wikipedia.org/wiki/Karl_Pearson)\nNumerous others have expanded the theory of regression over the last century since Pearson's pivotal paper. Notably, Joseph Berkson published the logistic regression method in 1944, one of the first classification algorithms. In recent times the explosion of interest in machine learning (ML) has lead to a rapid increase in the numbers and types of regression models.\n### Types of regression\nThere are a variety of types of regressions including many that are nonlinear and have absolutely nothing to do with what I will cover; they are used to predict numeric outcomes but use different underlying technologies. Some common regression types are:\n  * Linear - most common\n  * Logistic - binary outcomes (success/failure)\n  * Polynomial - when a straight line is not the best fit\n  * Stepwise Ridge, and Lasso (common in ML)\n\n\n### Least-squares linear regression\nLeast-squares regression is the type I will be expounding on today. Caution, most people will conflate this with linear regression. Now the two need not be the same, but just warning you that most people will conflate these two. Linear regression just means that you are going to do something using a **linear collection of parameters**. There are a variety of other ways to do regressions and those would not use those linear collections of parameters; where each parameter is associated with one variable.\n![Linear Regression | Source Wikipedia](../images/002.png)\nLinear Regression | Source [Wikipedia](https://en.wikipedia.org/wiki/File:Linear_least_squares_example2.svg)\nWe have our linear regression, which is most commonly used to describe a straight line like the picture above - but that would also be true for logistic and polynomial. And probably also for most cases of a stepwise ridge and lasso. So there are **two definitions of linear** - and it will be confusing.\nThe one definition of linear is a **line that goes through your data**. Said another way, a line is fitted to your data. The other definition of linear is when you have a **linear collection of parameters** , with each parameter being associated with one variable. Said differently: the linearly predictable behavior of the dependent (target) value based on the explanatory variables.\nThis is a fancy way of saying:\n![Linear collection of parameters](../images/body-f782333c32cdbec94e65.img)\nLinear collection of parameters\nIn the example above the parameters a, b, c have a linear association with the input variables x1, x2, and x3 - x0 is the offset. When I say variables, that means that **they are inputs to a regression**.\nThe least-squares linear regression was derived with the help of linear algebra and some calculus. I believe that understanding linear models is the basis for understanding the behavior of a whole bunch of statistical and ML models. In fact, I believe that about half of all machine learning models will use some of this form of linear regression, at least in their starting points.\n### Model selection: 11 dots\n![Model 1 | First-order linear regression, a simple line of best fit](../images/003.png)\nModel 1 | First-order linear regression, a simple line of best fit\nA very reasonable and acceptable way to do a regression for some data (like the above example) is just to eyeball it and draw your line through it. And you want to call that a regression, that is fine. I doubt it can be called a linear regression anymore, but it is certainly a regression. And it is a very valid one. Now we are going to be talking about regressions where we can show mathematical rules for going from data points to a line of best fit.\n![Mathematically defining the line above](../images/004.png)\nMathematically defining the line above\nEarlier I defined linear regression, now I will show an example of a quadratic equation that is still linear the way we have defined it:\n![Model 2 | Quadratic regression of the second order](../images/005.jpeg)\nModel 2 | Quadratic regression of the second order\n![](../images/body-5e4433b9a635a1cc3903.png)\nequivalently:\n![](../images/body-cb7e4be904cf7061464f.png)\nWhat we are going to do that or what we could do is we can say, \"okay\", y is not the output from a function that takes in one variable, namely x1, but rather y is the output from a function that takes in two variables; the first one being x1, the second one being x1². In fact, we can just call x1 times x1 the variable x2.\n![By relabeling x1² as x2, we have recovered a linear regression](../images/006.png)\nBy relabeling x1² as x2, we have recovered a linear regression\nWe can have non-straight lines fit a linear equation! Extending from this we can basically go to any polynomial. The following graph has the same data as the quadratic example, but a different line of best fit.\n![Model 3 | Higher-order regression using the same data as quadratic example](../images/007.jpeg)\nModel 3 | Higher-order regression using the same data as quadratic example\nThe only thing really changing is the line that we are drawing through them. In both cases, we are using what we call a linear relationship - well people who are in this regression business (mathematicians/physicists) call it such. They know full well that this is not a straight line. But they claim that in higher dimensions, this is actually straight; calling it not a line but a hyperplane.\nThe hyperplane is not curved in those dimensions. The dimensions would likely be:\n![Maybe a fourth-order is needed.](../images/008.png)\nMaybe a fourth-order is needed.\nThis is adjacent to the mathematical field of topology, very relevant to particle physicists and Einstein's General Relativity, in general (haha)! Physicists will say that the path of a hit baseball is a straight line. In high school, we learned that this path is a parabola but in curved space-time, the path is a straight line geodesic (a geodesic being a \"straight\" line in a curved topological landscape). Geodesics you are likely familiar with include the lines of latitude on a globe.\nSo here the relevant question being asked is which of these three proceeding models is correct? The first order linear one? The second-order linear one? The fourth-order linear one? Well, the better question is which of these three is more useful, and that would naturally require that we have knowledge of what the model is being used for.\nThe general rule is, and this is a pretty important rule, that **the simpler the model, the more useful it is**. Remember this heuristic! Later I will write about regularization, which is just a way of making models simpler. When you simplify your linear model (well any model, but the context here is linear models) it is not just easier to understand, more importantly, they work better!\nNo data are perfect, errors will always exist. Another useful heuristic to remember is \"all models are wrong, but some are useful!\" The fact is that some of the data above could just be errors. The 11 dots above are the training data. When we look at new data, it might turn out, or probably will turn out, that they will not follow the funny curve, our higher-order model. Chances are that the \"hyperplane\" model is probably an example of what we call overfitting. And the reverse process of overfitting or the process that undoes overfitting is called regularization. As such, the balance of probability says that our first model is the most useful model to conduct further analysis on.\n### Terminology\n  * **Response** (Dependent) variable: the variable of primary interest, the variable we are trying to predict or explain. Like in our Jupyter notebook examples above, the response is the lefthand side of the equal sign -'y' is DEPENDENT on the values on the righthand side of the equals sign\n  * **Explanatory** (Independent) variable: the variable that attempts to explain the observed outcomes of the response variable. In an equation, if you change the 'y' (dependent) variable, you are not changing the righthand side 'x' variables - hence they are INDEPENDENT of changes in 'y'\n\n\nThere are two types of parameters in linear models:\n  * **y-intercept,** referred to above as the offset\n  * **slope** , rise over run, change in y divided by change in x. If there are multiple inputs, then we have multiple slopes (we call these partial slopes)\n\n\nAs an aside, but I want to mention that data is often not linear, and as such the data need to be transformed to apply regression analysis. An important such transformation is doing a log transformation when dealing with nonlinear data.\n![Country area vs population | Source Wikipedia](../images/009.jpeg)\nCountry area vs population | Source [Wikipedia](https://en.wikipedia.org/wiki/Data_transformation_\\(statistics\\))\nThis raw data is not linear, however, we can make it appear linear by applying a logarithmic change, here it will be log base 10:\n![The same data, transformed | Source Wikipedia](../images/010.jpeg)\nThe same data, transformed | Source [Wikipedia](https://en.wikipedia.org/wiki/Data_transformation_\\(statistics\\))\nThe reason for doing the log transformation is because now your errors are going to probably be consistent throughout and that is very probably going to be better. It is possible to fit an exponential line to the data, doing non-parametric line fitting, but then the errors would not be consistent, they would not be of the same scale - which makes doing downstream analysis much harder.\n### Error terminology\nBefore explaining least-squares regression, we need to understand what the errors are:\n![epsilon for error, i for which data point it is \\(first, second, etc.\\)](../images/011.png)\nepsilon for error, i for which data point it is (first, second, etc.)\nThe E(error) is the distance between each point and the regression line. We select the line with the smallest sum of squared differences between each point and the line.\n![Regression line and errors for each point](../images/012.jpeg)\nRegression line and errors for each point\nThe line above is a pretty good fit for the five dots above, but it is not perfect. What we want to know is how much error is there in the dependent variable. The epsilon term is added to bridge the gap between our predicted value(y_hat), which is where the red dot follows the green line and meets the regression line, and the actual observed values. The error is the difference between the predicted and observed variables. This is further explained in the code analysis below.\nWith a little bit of calculus and linear algebra, we try to find the minimum value of the sum of the errors for all possible regression lines. This summation is the best fit line. This is incredibly laborious to calculate. If instead, you say you want to minimize the square of this value, it turns out that there are beautiful, analytical solutions. And this is why we do least-squares fitting. So we do not try to minimize those dotted lines, or the sum of those dotted lines, we try instead to minimize the sum of the squares. So what is the sum of squares?\n![Visualization of the sum of squared errors for the regression line](../images/013.png)\nVisualization of the sum of squared errors for the regression line\nThe squares are now summed. The smallest total area of these squares is what we believe to be our best fit line. The advantage of this method, besides the beautiful analytical solutions, is that it penalizes larger errors - it punishes outliers. In the example above say that the largest error is 6 times larger than the smallest error. Well in least-squares regression that error is 36 (6 x 6)times more weighted than the smallest square.\n![Explicit solutions exist using calculus](../images/014.jpeg)\nExplicit solutions exist using calculus\nWe let the computers solve the equation above to find the sum of least-squares by playing around with the slope and intercept. The predicted (theoretical) values are subtracted from the actual (observed) values and summed.\n## Python example: regression model\nThe following code can be run in a Jupyter Notebook to help get a feel and a visualization for least-squares linear regression.\nN.B. we use synthetic data because we have a lot of control over what happens. Synthetic data is not always the best.\ntext\n\n```\n# relevant imports\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\n%matplotlib inline\n```\n\ntext\n\n```\n# synthetic data points between 0 and 10 with standard deviation of #the errors of 1 (std deviation of y should be around 3-the square #root of 10)\nn_points = 80\nx_start, x_end = 0, 10\ny_start, y_end = 0, 10\ny_sd = 1 # std deviation of the errors\n```\n\ntext\n\n```\n# data columns, the y values are the errors added to the x values\nx_data = np.linspace(x_start, x_end, n_points)\ny_error = np.random.normal(loc=0, scale=y_sd, size=n_points)\ny_data = np.linspace(y_start, y_end, n_points) + y_error\n```\n\ntext\n\n```\n# create dataframe\nsyn_data = pd.DataFrame({'x':x_data, 'y':y_data})\n```\n\ntext\n\n```\nsyn_data.head()\n```\n\n![The first five lines of our 80 synthetic data points](../images/015.jpeg)\nThe first five lines of our 80 synthetic data points\nLet us now visualize the data - note the strong linearity of the data. Remember that the point of this exercise is to answer the question \"if we know the value of X can we predict the value of Y?\" We believe that by drawing the line of best fit that if someone gives us an X, say an X that we do not already have, then we can figure out what the Y value would be.\ntext\n\n```\nplt.plot(syn_data['x'], syn_data['y'], 'ko') # ko = black points\nplt.grid(True)\nplt.xlabel('x')\nplt.ylabel('y')\nplt.title('x vs y')\nplt.show()\n```\n\n![Our synthetic data - for the most part, the x values are approximately the y values](../images/016.jpeg)\nOur synthetic data - for the most part, the x values are approximately the y values\n### Building linear regression model\nFollowing a simple template, we can build our model. You are likely familiar with one common package that can do this: Sci-Kit Learn. A quirk of `sklearn` is that works best with NumPy arrays, it is just what it is used to. The following template reshapes and converts the data for model training and evaluation:\ntext\n\n```\n# standard lin reg template\n```\n\ntext\n\n```\n# imports\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom sklearn.linear_model import LinearRegression\nfrom sklearn.metrics import mean_squared_error, r2_score\n```\n\ntext\n\n```\n# create numpy arrays for the x (independent) and y (dependent) #variables\nx = syn_data.x.values.reshape(-1,1)\ny = syn_data.y.values.reshape(-1,1)\n```\n\ntext\n\n```\n# model initialization\nregression_model = LinearRegression()\n```\n\ntext\n\n```\n# train the model (fit data)\nregression_model.fit(x, y)\n\ny_predicted = regression_model.predict(x)\n```\n\ntext\n\n```\n# model evaluation\nrmse = mean_squared_error(y, y_predicted)\nr2 = r2_score(y, y_predicted)\n```\n\ntext\n\n```\n# printing values\nprint('Slope:' ,regression_model.coef_)\nprint('Intercept:', regression_model.intercept_)\nprint('Root mean squared error: ', rmse)\nprint('R2 score: ', r2)\n```\n\ntext\n\n```\n# plotting values\nplt.scatter(x, y, s=10)\nplt.xlabel('x')\nplt.ylabel('y')\n```\n\ntext\n\n```\nplt.plot(x, y_predicted, color='r')\nplt.show()\n```\n\n![The line is a good fit for the data](../images/017.jpeg)\nThe line is a good fit for the data\nThe statistical values are explained in greater detail below when we evaluate the model's performance.\n### Plot residuals\nThe following is an important tool in understanding how good your regression model is - a residual plot. What is happening is that for every point, the difference between the observed and predicted values is plotted.\ntext\n\n```\nimport seaborn as sns\nsns.residplot(x, y)\nplt.show()\n```\n\n![The predicted y value differences from actual y values](../images/018.jpeg)\nThe predicted y value differences from actual y values\nThe residuals are the errors, which we plotted earlier. The x-axis is the x values. The y axis is the discrepancy between the actual y values and the predicted y values, the errors.\ntext\n\n```\nfrom sklearn import linear_model\n# from sklearn.linear_model import LinearRegression\n```\n\ntext\n\n```\n# initialize the model.\nlinear_model = linear_model.LinearRegression()\n```\n\ntext\n\n```\n# define x and y\nx_input = syn_data['x'].values.reshape(n_points, 1)\ny_output = syn_data['y'].values.reshape(n_points, 1)\n```\n\ntext\n\n```\n# fit the model to the existing data with lm.fit(x,y)\nlinear_model.fit(x_input, y_output)\n```\n\ntext\n\n```\n# fit model to make predictions\ny_pred = linear_model.predict(x_input)\n```\n\ntext\n\n```\n# plot\nplt.scatter(x_input, y_output)\nplt.plot(x_input, y_pred, linewidth=2)\nplt.grid(True)\nplt.xlabel('x')\nplt.ylabel('y')\nplt.title('x vs y')\nplt.show()\n```\n\ntext\n\n```\n# model parameters\nprint('Intercept: {0:.5f}'.format(linear_model.intercept_[0]))\nprint('Slope : {0:.5f}'.format(linear_model.coef_[0][0]))\n```\n\n![](../images/body-f1ede7bb5b17b84721b2.jpeg)\n### Stats Model package\nAnother, older and lesser-used Python package is Stats Model `sm`. The following code duplicates the Sci-Kit Learn work. Note the subtle differences between the two w/r to how the outputs are displayed.\nN.B. The statistics are the same, but the presentations and the robustness thereof differ between the packages. `sklearn` is the preferred package to learn these days.\ntext\n\n```\nimport statsmodels.formula.api as sm\n```\n\ntext\n\n```\n# ols = ordinary least squares\nols_model = sm.ols(formula = 'y ~ x', data=syn_data)\n```\n\ntext\n\n```\n# fit the model\nresults = ols_model.fit()\n```\n\ntext\n\n```\nprint('Intercept: {0:.5f}'.format(results.params.Intercept))\nprint('Slope : {0:.5f}'.format(results.params.x))\n```\n\n![](../images/body-fb04cccaab77ba207c9e.jpeg)\ntext\n\n```\n# add predicted to dataframe\nsyn_data['predicted'] = y_pred\n```\n\ntext\n\n```\n# add residuals to dataframe\nsyn_data['resids'] = y_output - y_pred\n```\n\ntext\n\n```\n# verify data is as expected\nsyn_data.head()\n```\n\n![](../images/body-c69ca99db268eaf80f7e.jpeg)\ntext\n\n```\n# coefficients of line of best fit\nm = linear_model.coef_[0]\nb = linear_model.intercept_\nprint('m = {}'.format(m[0]))\nprint('b = {}'.format(b[0]))\n```\n\n![](../images/body-4d694aab54cd83d83af1.jpeg)\n### Interpreting the parameters\nIt is very important to know how to interpret the parameters (your results will be different because of the random seed in creating the data).\n  * **y-intercept (b)** : when x is zero, y is -0.13...\n  * **slope (m)** : When we increase x by 1 then we expect that y will go up by 1.03....\n\n\nSimple enough, I bet most readers will have a strong grasp on these basics, but it is important to refresh everyone and to build up from a solid and universal foundation.\n### Model summary statistics\nThis is where SM shines - the beautiful results summary. A lot more data is shown natively than we need to understand least - squares regression at the moment. The residuals are also plotted after the OLS report.\ntext\n\n```\nimport statsmodels.formula.api as sm\nimport seaborn as sns\n```\n\ntext\n\n```\nols_model = sm.ols(formula = 'y ~ x', data=syn_data)\n```\n\ntext\n\n```\nresults = ols_model.fit()\n```\n\ntext\n\n```\n# print slope (m) and y-intercept (b)\nprint('Intercept, Slope : {}'.format(results.params))\n```\n\ntext\n\n```\nprint('nSSE, SST, SSR, and RMSE:')\nmean_y = np.mean(y_output)\nsst = np.sum((y_output - mean_y)**2)\nssr = np.sum((y_pred - y_output)**2)\nsse = np.sum((y_pred - mean_y)**2)\n```\n\ntext\n\n```\nprint('Sum of Squares Explained (SSE): {}'.format(sse))\nprint('Total Sum of Squares (SST): {}'.format(sst))\nprint('Sum of Squared Residuals (SSR): {}'.format(ssr))\nprint('RMSE: {}'.format(np.sqrt(results.mse_resid)))\n```\n\ntext\n\n```\n# print linear regression statistics\nprint('n', results.summary())\n```\n\n![](../images/body-c750c298e3002d739ff8.jpeg)\ntext\n\n```\n# plot histogram of the residuals\nsns.distplot(results.resid, hist=True)\nplt.xlabel('Residual')\nplt.ylabel('Frequency')\nplt.title('Residual Histogram')\nplt.show()\n```\n\n![](../images/body-1606bd421e24fa9da3f7.jpeg)\nThe first thing to note about our residuals plotted above is that they are roughly normal. This is to be expected because the synthetic data was created to be such, so this a bit of a self-fulfilling prophecy. The y data is created from errors added to the randomly generated x data, with the y data having a standard deviation of +/- one.\n### Evaluation of regression models\nNow that we have built a regression model, we can quantitatively evaluate the performance of our regression model. The evaluation of regression models is based on measurements of the errors.\nIf we want to know how well we did, we must consider what a good benchmark would be. A common easy benchmark is to see if we can predict better than just the **mean of the target variable**.\nWe define the following terms:\n![](../images/body-a94a9129a915d7330bf1.png)\n![](../images/body-1cb63a799bdc086ea865.img)\n![](../images/body-e3ca39ee76ab683ceb57.img)\nNow we need to define the following error metrics:\n  * **Sum of Squared Residuals (SSR)** : This is the value that the method of least squares was able to minimize to with the best fit line. The goal of regression is to minimize this quantity.\n\n\n![The goal of regression - minimize this](../images/body-da2557b140ec22fe60fe.png)\nThe goal of regression - minimize this\n  * **Sum of Squared Total (SST)** : This is a measure of how bad a prediction the target-mean would be\n\n\n![](../images/body-42d37a6438b9b4034dda.png)\n  * **Sum of Squared Explained (SSE)** : This is a sum of a squared pointwise difference of the (SST - SSR). You can also think of this as a measurement of how much better we are explaining the variation than the mean\n\n\n![](../images/body-9b6068683496c91beebb.png)\nNotice that **SST = SSR + SSE**\n  * This formula is only true if you do your regression on your training data! If you do it on test data, this formula is no longer true. This formula is true for the cases where your coefficient of determination (COD)is the same as your Pearson's R², the correlation coefficient. **They are not the same!** They can sometimes be the same, but only in the situation for at least-squares fitting on your training data, not for when you use either your test data or not when you do a non least-squares fitting. SSR can become larger than SST, leading to a negative R²!\n\n\n![Visualization of the errors of a linear regression model](../images/body-3e9964ce8693644c9046.jpeg)\nVisualization of the errors of a linear regression model\nWe can now talk about the **Root-Mean-Squared-Error (RMSE)**. The reason we do this is that the \"sum-squared-error\" (also referred to as the mean-squared error) is in units of **x²**. To convert the error to units of **x** , we need the square root of the mean-squared-error or the square root of the sum-squared residuals. This is the RMSE:\n![](../images/body-e28444b8125b0c151315.png)\nRemember that the goal of regression is to minimize the residual error, **SSR**. Specifically, we wish to explain the maximum amount of the variance in the original data as possible with our model. We can quantify this idea with the coefficient of determination (COD)also known as **R²** :\n![ONLY CORRECT WITH LEAST-SQUARES FITTING AND MEASURING TRAINING DATA](../images/body-92200700ea281e4f18bd.png)\nONLY CORRECT WITH LEAST-SQUARES FITTING AND MEASURING TRAINING DATA\nso as\n![](../images/body-26c0fb8f1e818795ab22.png)\n![](../images/body-ea410eb6d5d3076b332c.png)\n**R² is the fraction of the variance of the original data explained by the model**. A model that perfectly explains the data has R² = 1. A model which does not explain the data at all has R² = 0. This means that the training data is spread out so evenly that you cannot draw any best fit line through it. You can draw a line through the data, it just would not be the best fit line.\nHowever, R² is not perfect:\n  * R² is not bias adjusted for degrees of freedom\n  * More importantly, there is no adjustment for the number of model parameters. As the number of model parameters increases, SSR will generally decrease. Without an adjustment, you will get a false sense of model performance\n\n\nWe address this by defining **Adjusted R² (R² adj)** :\n![](../images/body-b62f97922d9326f1a840.png)\n![](../images/body-cf7b0a8c1748210a0981.img)\nwhere\n![](../images/body-b6e8ae9f60e0b07ef88e.img)\n![](../images/body-3f3a206b3596bcd90e12.img)\nThis gives **R²adj** as:\n![](../images/body-1a90813e0122caa70ffa.png)\nwhere\n![](../images/body-d553c80dfe70e4a2706c.png)\n![](../images/body-8f14ff618d9a34367a78.img)\nWhich can be rewritten:\n![](../images/body-d63862085290588b8c14.png)\nNow we have a metric that decreases with more added parameters, rewarding simplicity - remembering from earlier that simpler models are preferred to complex models.\n## Conclusion\nWe have covered many regression concepts so far:\n  * The single regression equation of a line is\n\n\n![](../images/011.png)\n  * Use the Python model object `linear_model.LinearRegression()` to initialize the model\n  * Use the `fit` method to fit the model to the data\n  * Use `predict` method to compute scores(predict values) for the dependent value (y)\n  * Pull the model parameters - m(slope) with `coef` and b(y-intercept) - with `intercept`\n  * The residuals are the difference between the y_output and the y_predicted\n  * Use the `statsmodels` and `seaborn` libraries to get summary statistics and make diagnostic plots\n  * `sklearn` is preferable to the `sm` package, it is simpler to use and more robust\n  * What residuals are and why we plot them\n\n\nWe evaluated the results from the OLS Regression Results with `summary` method\n  * **SSR** (Sum of the Squared Residuals) is the difference in error from the regression line (try to minimize)\n  * **SST (** Sum of the Squared Total) is the difference to the target mean\n  * **SSE** (Sum of the Squared Explained) is the variation of the regression line to the mean\n  * **RMSE** (Root Mean Squared Error) is the square root of the SSR\n  * **R²** is the fraction of the variance of the original data explained by the model, want a number close to 1\n  * **SST = SSR + SSE** is ONLY true for the cases where your coefficient of determination (COD) is the same as your Pearson's R², the correlation coefficient. For least-squares regression on training data, yes it is true. Outside of this case, on test data or when SSR > SST the formula does not hold\n\n\nIn my next article, I cover [multivariant regression](https://towardsdatascience.com/multivariant-linear-regression-e636a4f99b40) and then [bootstrapping wiht regression models](https://towardsdatascience.com/linear-regression-with-bootstrapping-4924c05d2a9)!\nFind me on [Linkedin](https://www.linkedin.com/in/james-a-w-godwin/)\n_Physicist cum Data Scientist- Available for new opportunity | SaaS | Sports | Start-ups | Scale-ups_\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](../images/body-92fdfcb3140ff02db487.jpeg)\nWritten By\n## [James Andrew Godwin](https://towardsdatascience.com/author/james-andrew/)\n[See all from James Andrew Godwin](https://towardsdatascience.com/author/james-andrew/)\n[Data Science](https://towardsdatascience.com/tag/data-science/)[Linear Regression Python](https://towardsdatascience.com/tag/linear-regression-python/)[Machine Learning](https://towardsdatascience.com/tag/machine-learning/)[Towards Data Science](https://towardsdatascience.com/tag/towards-data-science/)\nShare This Article\n  * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fan-introduction-to-linear-regression-9cbb64b52d23%2F&title=An%20Introduction%20to%20Linear%20Regression)\n  * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fan-introduction-to-linear-regression-9cbb64b52d23%2F&title=An%20Introduction%20to%20Linear%20Regression)\n  * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fan-introduction-to-linear-regression-9cbb64b52d23%2F&text=An%20Introduction%20to%20Linear%20Regression)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n  * [![](../images/019.png)](https://towardsdatascience.com/reinforcement-learning-applied-to-the-mountain-car-problem-1c4fb16729ba/)\n## [Reinforcement Learning Applied to the Mountain Car Problem](https://towardsdatascience.com/reinforcement-learning-applied-to-the-mountain-car-problem-1c4fb16729ba/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nHow I Taught a Machine to Play this Simple Video-Game\n[Diego Unzueta](https://towardsdatascience.com/author/diegounzuetaruedas/)\nFebruary 3, 202210 min read\n  * [![](../images/020.png)](https://towardsdatascience.com/multimodal-search-engine-agents-powered-by-blip-2-and-gemini/)\n## [Multimodal Search Engine Agents Powered by BLIP-2 and Gemini](https://towardsdatascience.com/multimodal-search-engine-agents-powered-by-blip-2-and-gemini/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nBuilding Multimodal Fashion Assistant Agents with Text and Image-Based Search\n[Luís Roque](https://towardsdatascience.com/author/luisroque/)\nFebruary 19, 202514 min read\n  * [![](../images/021.jpeg)](https://towardsdatascience.com/pricing-objects-in-mercari-machine-learning-deep-learning-perspectives-6aa80514b2c8/)\n## [Pricing Objects in Mercari - Machine learning & Deep learning perspectives](https://towardsdatascience.com/pricing-objects-in-mercari-machine-learning-deep-learning-perspectives-6aa80514b2c8/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nSupplementary endeavor to solve NLP-based regression of Kaggle's Mercari dataset as part of a pioneering self-case study - Applied AI...\n[Yogeeshwari S](https://towardsdatascience.com/author/syogeeshwari/)\nAugust 19, 202019 min read\n  * [![](../images/022.png)](https://towardsdatascience.com/reinforcement-learning-explained-visually-part-5-deep-q-networks-step-by-step-5a5317197f4b/)\n## [Reinforcement Learning Explained Visually (Part 5): Deep Q Networks, step-by-step](https://towardsdatascience.com/reinforcement-learning-explained-visually-part-5-deep-q-networks-step-by-step-5a5317197f4b/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nA Gentle Guide to DQNs with Experience Replay, in Plain English\n[Ketan Doshi](https://towardsdatascience.com/author/ketanhdoshi/)\nDecember 19, 202011 min read\n  * [![](../images/023.png)](https://towardsdatascience.com/time-series-signals-the-fourier-transform-f68e8a97c1c2/)\n## [Time Series, Signals, & the Fourier Transform](https://towardsdatascience.com/time-series-signals-the-fourier-transform-f68e8a97c1c2/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nAn introduction and primer for future posts\n[Shaw Talebi](https://towardsdatascience.com/author/shawhin/)\nNovember 16, 20204 min read\n  * [![](../images/024.jpg)](https://towardsdatascience.com/reinforcement-learning-from-scratch-deep-q-networks-0a8d33ce165b/)\n## [Reinforcement Learning: Deep Q-Networks](https://towardsdatascience.com/reinforcement-learning-from-scratch-deep-q-networks-0a8d33ce165b/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nTeaching a shuttle to land on the moon using Deep Q-Networks in Python. A mathematical deep dive into Reinforcement Learning.\n[Cristian Leo](https://towardsdatascience.com/author/cristianleo120/)\nMay 23, 202429 min read\n  * [![](../images/025.jpeg)](https://towardsdatascience.com/free-gpus-for-training-your-deep-learning-models-c1ce47863350/)\n## [Free GPUs for Training Your Deep Learning Models](https://towardsdatascience.com/free-gpus-for-training-your-deep-learning-models-c1ce47863350/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nYes you heard it right. It's free.\n[Edwin Tan](https://towardsdatascience.com/author/edwin.tan/)\nAugust 20, 20215 min read\n  * [![](../images/026.png)](https://towardsdatascience.com/bessels-correction-why-do-we-divide-by-n-1-instead-of-n-in-sample-variance-30b074503bd9/)\n## [Bessel's Correction: Why Do We Divide by n−1 Instead of n in Sample Variance?](https://towardsdatascience.com/bessels-correction-why-do-we-divide-by-n-1-instead-of-n-in-sample-variance-30b074503bd9/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nUnderstanding the Unbiased Estimation of Population Variance\n[Aman Agrawal](https://towardsdatascience.com/author/amannagrawall002/)\nNovember 11, 20248 min read\n  * [![](../images/027.jpeg)](https://towardsdatascience.com/best-library-to-simplify-math-for-machine-learning-ed64cbe536ac/)\n## [Best Library To Simplify Math For Machine Learning!](https://towardsdatascience.com/best-library-to-simplify-math-for-machine-learning-ed64cbe536ac/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nUsing the sympy Python library to simplify differential and integral calculus for Machine Learning\n[Bharath K](https://towardsdatascience.com/author/bharath-k1297/)\nApril 21, 20217 min read\n\n\n  * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n  * [X](https://x.com/TDataScience)\n  * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n  * [Threads](https://www.threads.net/@towardsdatascience)\n  * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n####  Recommended Articles \nClose\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500) How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ \"How Relevance Models Foreshadowed Transformers for NLP\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n\n\n",
  "metadata": {
    "title": "An Introduction to Linear Regression | Towards Data Science",
    "description": "Regression is used primarily if you wish to predict or explain numeric data",
    "keywords": null,
    "author": "James Andrew Godwin",
    "og:locale": "en_US",
    "og:type": "article",
    "og:title": "An Introduction to Linear Regression",
    "og:description": "Regression is used primarily if you wish to predict or explain numeric data",
    "og:url": "https://towardsdatascience.com/an-introduction-to-linear-regression-9cbb64b52d23/",
    "og:site_name": "Towards Data Science",
    "og:image": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1id8GiZhuwEw7swLwUq1pIg.jpeg",
    "og:image:width": "640",
    "og:image:height": "427",
    "og:image:type": "image/jpeg",
    "twitter:card": "summary_large_image",
    "twitter:site": "@TDataScience",
    "twitter:creator": "@TDataScience",
    "twitter:label1": "Written by",
    "twitter:data1": "James Andrew Godwin",
    "twitter:label2": "Est. reading time",
    "twitter:data2": "19 min read",
    "article:published_time": "2021-03-08T21:52:05.000Z",
    "article:modified_time": "2021-03-08T21:52:05.000Z",
    "article:author": "https://towardsdatascience.com/author/james-andrew/"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 202,
    "error": ""
  },
  "code_blocks": [
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_000.txt",
      "chars": 109
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_001.txt",
      "chars": 247
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_002.txt",
      "chars": 231
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_003.txt",
      "chars": 69
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_004.txt",
      "chars": 16
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_005.txt",
      "chars": 143
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_006.txt",
      "chars": 28
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_007.txt",
      "chars": 168
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_008.txt",
      "chars": 147
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_009.txt",
      "chars": 61
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_010.txt",
      "chars": 99
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_011.txt",
      "chars": 91
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_012.txt",
      "chars": 172
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_013.txt",
      "chars": 74
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_014.txt",
      "chars": 47
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_015.txt",
      "chars": 53
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_016.txt",
      "chars": 85
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_017.txt",
      "chars": 71
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_018.txt",
      "chars": 122
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_019.txt",
      "chars": 90
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_020.txt",
      "chars": 71
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_021.txt",
      "chars": 155
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_022.txt",
      "chars": 140
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_023.txt",
      "chars": 37
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_024.txt",
      "chars": 84
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_025.txt",
      "chars": 42
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_026.txt",
      "chars": 111
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_027.txt",
      "chars": 60
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_028.txt",
      "chars": 68
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_029.txt",
      "chars": 45
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_030.txt",
      "chars": 147
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_031.txt",
      "chars": 59
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_032.txt",
      "chars": 53
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_033.txt",
      "chars": 26
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_034.txt",
      "chars": 92
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_035.txt",
      "chars": 171
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_036.txt",
      "chars": 217
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_037.txt",
      "chars": 67
    },
    {
      "language": "txt",
      "file": "attachments/d3f227ccb337/code/block_038.txt",
      "chars": 163
    }
  ],
  "images": [
    {
      "status": "http_404",
      "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500",
      "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500",
      "alt": "",
      "downloaded": false,
      "discovery": "saved_markdown",
      "retried_at": "2026-10-20T17:52:05Z"
    },
    {
      "status": "http_404",
      "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500",
      "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500",
      "alt": "",
      "downloaded": false,
      "discovery": "saved_markdown",
      "retried_at": "2026-10-20T17:52:05Z"
    },
    {
      "status": "http_404",
      "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500",
      "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500",
      "alt": "",
      "downloaded": false,
      "discovery": "saved_markdown",
      "retried_at": "2026-10-20T17:52:05Z"
    },
    {
      "status": "http_404",
      "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500",
      "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500",
      "alt": "",
      "downloaded": false,
      "discovery": "saved_markdown",
      "retried_at": "2026-10-20T17:52:05Z"
    }
  ],
  "audio": [],
  "videos": [
    {
      "url": "https://www.youtube.com/c/TowardsDataScience",
      "kind": "channel_reference",
      "status": "reference_only"
    }
  ],
  "links": [
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/latest/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/editors-pick/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/the-variable/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/shipai/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/submissions/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/category/data-science/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/author/james-andrew/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/bayesian-statistics-11f225174d5a"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/multivariant-linear-regression-e636a4f99b40"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/linear-regression-with-bootstrapping-4924c05d2a9"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/data-science/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/linear-regression-python/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/machine-learning/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/towards-data-science/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/reinforcement-learning-applied-to-the-mountain-car-problem-1c4fb16729ba/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/category/artificial-intelligence/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/author/diegounzuetaruedas/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/multimodal-search-engine-agents-powered-by-blip-2-and-gemini/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/category/artificial-intelligence/machine-learning/"
    },
    {
      "type": "external",
      "href": "https://write.insightmediagroup.io"
    },
    {
      "type": "external",
      "href": "https://www.linkedin.com/company/towards-data-science/"
    },
    {
      "type": "external",
      "href": "https://x.com/TDataScience"
    },
    {
      "type": "external",
      "href": "https://unsplash.com/@danranwanghao"
    },
    {
      "type": "external",
      "href": "https://en.wikipedia.org/wiki/Karl_Pearson"
    },
    {
      "type": "external",
      "href": "https://en.wikipedia.org/wiki/File:Linear_least_squares_example2.svg"
    },
    {
      "type": "external",
      "href": "https://en.wikipedia.org/wiki/Data_transformation_(statistics)"
    },
    {
      "type": "external",
      "href": "https://www.linkedin.com/in/james-a-w-godwin/"
    },
    {
      "type": "external",
      "href": "https://www.facebook.com/sharer/sharer.php?title=An+Introduction+to+Linear+Regression&u=https%3A%2F%2Ftowardsdatascience.com%2Fan-introduction-to-linear-regression-9cbb64b52d23%2F"
    },
    {
      "type": "external",
      "href": "https://www.linkedin.com/shareArticle?mini=true&title=An+Introduction+to+Linear+Regression&url=https%3A%2F%2Ftowardsdatascience.com%2Fan-introduction-to-linear-regression-9cbb64b52d23%2F"
    },
    {
      "type": "external",
      "href": "https://x.com/share?text=An+Introduction+to+Linear+Regression&url=https%3A%2F%2Ftowardsdatascience.com%2Fan-introduction-to-linear-regression-9cbb64b52d23%2F"
    },
    {
      "type": "external",
      "href": "https://www.youtube.com/c/TowardsDataScience"
    },
    {
      "type": "external",
      "href": "https://www.threads.net/@towardsdatascience"
    },
    {
      "type": "external",
      "href": "https://bsky.app/profile/towardsdatascience.com"
    },
    {
      "type": "external",
      "href": "https://thenewstack.io"
    },
    {
      "type": "external",
      "href": "https://roadmap.sh"
    },
    {
      "type": "external",
      "href": "https://insightmediagroup.io/creator-network"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-20T17:27:48Z",
  "source_url": "https://towardsdatascience.com/an-introduction-to-linear-regression-9cbb64b52d23/",
  "delivery_status": "unverified",
  "discovery_status": "http_403",
  "missing_reason": "源页面未重新核验:http_403；已按保留正文补采图片",
  "repair_error": "",
  "actual_video_files": [],
  "expected_saved_body_images": 4,
  "image_discovery": "saved_markdown",
  "source_body_markdown": "![Revisit consent button](https://cdn-cookieyes.com/assets/images/revisit.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](https://cdn-cookieyes.com/assets/images/close.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n  * Cookie\nBCTempID\n  * Duration\n10 minutes\n  * Description\nNo description available.\n\n\n  * Cookie\n__cf_bm\n  * Duration\n1 hour\n  * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n  * Cookie\nAWSALBCORS\n  * Duration\n7 days\n  * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n  * Cookie\n_cfuvid\n  * Duration\nsession\n  * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n  * Cookie\nli_gc\n  * Duration\n6 months\n  * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n  * Cookie\n__hssrc\n  * Duration\nsession\n  * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n  * Cookie\n__hssc\n  * Duration\n1 hour\n  * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n  * Cookie\nwpEmojiSettingsSupports\n  * Duration\nsession\n  * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n  * Cookie\nBCSessionID\n  * Duration\n1 year 1 month 4 days\n  * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n  * Cookie\n_octo\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nlogged_in\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\n__Secure-YEC\n  * Duration\npast\n  * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n  * Cookie\n__eoi\n  * Duration\n6 months\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nAWSALBTGCORS\n  * Duration\n7 days\n  * Description\nNo description available.\n\n\n  * Cookie\nlogin-status-p\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nAWSALBTG\n  * Duration\n7 days\n  * Description\nNo description available.\n\n\n  * Cookie\ncsrf_token\n  * Duration\nsession\n  * Description\nNo description available.\n\n\n  * Cookie\ntoken_v2\n  * Duration\n1 day\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nD\n  * Duration\n1 year\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nPHPSESSID\n  * Duration\nsession\n  * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n  * Cookie\nVISITOR_PRIVACY_METADATA\n  * Duration\n6 months\n  * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n  * Cookie\ncookietest\n  * Duration\nsession\n  * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n  * Cookie\n__Host-airtable-session\n  * Duration\n1 year\n  * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n  * Cookie\n__Host-airtable-session.sig\n  * Duration\n1 year\n  * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n  * Cookie\nm\n  * Duration\n1 year 1 month 4 days\n  * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n  * Cookie\nBIGipServer*\n  * Duration\nsession\n  * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n  * Cookie\n__cfruid\n  * Duration\nsession\n  * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n  * Cookie\n_GRECAPTCHA\n  * Duration\n6 months\n  * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n  * Cookie\n__Secure-YNID\n  * Duration\n6 months\n  * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n  * Cookie\ncookieyes-consent\n  * Duration\n1 year\n  * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n  * Cookie\nlidc\n  * Duration\n1 day\n  * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n  * Cookie\nbrw\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nbrwConsent\n  * Duration\n5 minutes\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nWMF-Uniq\n  * Duration\n1 year\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nloom_anon_comment\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nloom_referral_video\n  * Duration\nsession\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nVISITOR_INFO1_LIVE\n  * Duration\n6 months\n  * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n  * Cookie\nyt-remote-connected-devices\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n  * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n  * Duration\nNever Expires\n  * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n  * Cookie\nyt-remote-device-id\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n  * Cookie\nyt-remote-session-name\n  * Duration\nsession\n  * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n  * Cookie\nyt-remote-fast-check-period\n  * Duration\nsession\n  * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n  * Cookie\nyt-remote-session-app\n  * Duration\nsession\n  * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n  * Cookie\nyt-remote-cast-available\n  * Duration\nsession\n  * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n  * Cookie\nyt-remote-cast-installed\n  * Duration\nsession\n  * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n  * Cookie\ncp_session\n  * Duration\n3 months\n  * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n  * Cookie\nloid\n  * Duration\n1 year 1 month 4 days\n  * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n  * Cookie\n__hstc\n  * Duration\n6 months\n  * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n  * Cookie\nhubspotutk\n  * Duration\n6 months\n  * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n  * Cookie\n_ga\n  * Duration\n1 year 1 month 4 days\n  * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n  * Cookie\n_ga_*\n  * Duration\n1 year 1 month 4 days\n  * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n  * Cookie\n__Host-psifi.analyticsTrace\n  * Duration\n6 hours\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\n__Host-psifi.analyticsTraceV2\n  * Duration\n6 hours\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\n_gh_sess\n  * Duration\nsession\n  * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n  * Cookie\nYSC\n  * Duration\nsession\n  * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n  * Cookie\najs_anonymous_id\n  * Duration\n1 year\n  * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n  * Cookie\nvuid\n  * Duration\n1 year 1 month 4 days\n  * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n  * Cookie\nAWSALB\n  * Duration\n7 days\n  * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n  * Cookie\nacq\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nacq.sig\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nptc\n  * Duration\n2 years\n  * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n  * Cookie\nmuc_ads\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n  * Cookie\nguest_id_marketing\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n  * Cookie\nguest_id_ads\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n  * Cookie\npersonalization_id\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n  * Cookie\nguest_id\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n  * Cookie\nbcookie\n  * Duration\n1 year\n  * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n  * Cookie\n__Secure-ROLLOUT_TOKEN\n  * Duration\n6 months\n  * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n  * Cookie\nyt.innertube::nextId\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n  * Cookie\nyt.innertube::requests\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n  * Cookie\nsession_tracker\n  * Duration\nsession\n  * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n  * Cookie\nedgebucket\n  * Duration\nsession\n  * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n  * Cookie\ndid\n  * Duration\n1 year\n  * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Data Science](https://towardsdatascience.com/category/data-science/)\n# An Introduction to Linear Regression\nRegression is used primarily if you wish to predict or explain numeric data\n[James Andrew Godwin](https://towardsdatascience.com/author/james-andrew/)\nMarch 8, 202119 min read\n![Regression | Photo by Hao Wang](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1id8GiZhuwEw7swLwUq1pIg.jpeg)\nRegression | Photo by [Hao Wang](https://unsplash.com/@danranwanghao)\nThis article is a continuation of series that last covered [Bayes statistics](https://towardsdatascience.com/bayesian-statistics-11f225174d5a).\nVolatility is inherent within the stock market, but perhaps the rise of \"meme stocks\" is something that is beyond traditional comprehension. This article is not about such \"stonks\" - which is the ironic misspelling of stocks that encapsulates the new investing zeitgeist amongst Millennials (broadly speaking). Rather, I was pondering the fabulous rollercoaster of a ride of $GME as a timely example of something to apply linear regression to; one can apply a linear regression model to a given stock to predict its price in the future. This article and subsequent articles will expand upon the concepts of linear regression and several methods associated with it, its associated errors for model analysis as well as touching on residuals.\nBut what is regression? The goal of regression is to produce a model that represents the best fit for some observed data. A better, more specific way to state this is that regression is used to predict or explain numeric data **** - smarter people than me have been applying such models to financial data (like $GME) for decades. But, if you want to predict whether or not somebody is going to come down with a disease or whether or not somebody will be wearing black, or whether an asset will appreciate or depreciate in value those would not be regressions, those would be classifications. A** regression is either an explanatory model or a predictive model for numeric dat**a. And for the most part, we will be dealing with ones where we can describe a function - that means an output - we can describe the relationship between one or more input variables and a single output variable (for linear regression).\n## History\nRegression is based on the method of least squares or the method of minimum mean square error. The ideas around least squares or averaging errors have evolved over nearly three centuries. The first known publication of a 'Method of Averages' was by the German astronomer Tobias Mayer in 1750. Pierre-Simon Laplace published in 1788 a similar method. The first publication of the method of least squares was by the French mathematician Adrien-Marie Legendre in 1805.\nAs a historical aside, it is very likely that the German physicist and mathematician Carl Friedrich Gauss developed the method of least-squares as early as 1795, but he did not publish this method until 1809. Legendre firmly opposed any notion that Gauss had used the method earlier than the 1805 publishing date.\nFrancis Galton, a cousin of Charles Darwin, coined the term regression in 1886. Galton was interested in determining which traits of plants and animals, including humans, could be said to be inherited. While Galton invented a modern form of regression, it was **Karl Pearson** who put regression and multiple regression on a firm mathematical footing. He created the world's first university statistics department at UCL. Pearson's 1898 publication proposed a method of regression as we understand it today.\n![Karl Pearson | Source Wikipedia](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1HGGcWwZmm4Y0jCyHIKy_dA.jpeg)\nKarl Pearson | Source [Wikipedia](https://en.wikipedia.org/wiki/Karl_Pearson)\nNumerous others have expanded the theory of regression over the last century since Pearson's pivotal paper. Notably, Joseph Berkson published the logistic regression method in 1944, one of the first classification algorithms. In recent times the explosion of interest in machine learning (ML) has lead to a rapid increase in the numbers and types of regression models.\n### Types of regression\nThere are a variety of types of regressions including many that are nonlinear and have absolutely nothing to do with what I will cover; they are used to predict numeric outcomes but use different underlying technologies. Some common regression types are:\n  * Linear - most common\n  * Logistic - binary outcomes (success/failure)\n  * Polynomial - when a straight line is not the best fit\n  * Stepwise Ridge, and Lasso (common in ML)\n\n\n### Least-squares linear regression\nLeast-squares regression is the type I will be expounding on today. Caution, most people will conflate this with linear regression. Now the two need not be the same, but just warning you that most people will conflate these two. Linear regression just means that you are going to do something using a **linear collection of parameters**. There are a variety of other ways to do regressions and those would not use those linear collections of parameters; where each parameter is associated with one variable.\n![Linear Regression | Source Wikipedia](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1oEV2AHBfeHD8wRlWb50O4Q.png)\nLinear Regression | Source [Wikipedia](https://en.wikipedia.org/wiki/File:Linear_least_squares_example2.svg)\nWe have our linear regression, which is most commonly used to describe a straight line like the picture above - but that would also be true for logistic and polynomial. And probably also for most cases of a stepwise ridge and lasso. So there are **two definitions of linear** - and it will be confusing.\nThe one definition of linear is a **line that goes through your data**. Said another way, a line is fitted to your data. The other definition of linear is when you have a **linear collection of parameters** , with each parameter being associated with one variable. Said differently: the linearly predictable behavior of the dependent (target) value based on the explanatory variables.\nThis is a fancy way of saying:\n![Linear collection of parameters](https://assets.insightmediagroup.io/media/0*1eneKB_3oeD_Vx0g)\nLinear collection of parameters\nIn the example above the parameters a, b, c have a linear association with the input variables x1, x2, and x3 - x0 is the offset. When I say variables, that means that **they are inputs to a regression**.\nThe least-squares linear regression was derived with the help of linear algebra and some calculus. I believe that understanding linear models is the basis for understanding the behavior of a whole bunch of statistical and ML models. In fact, I believe that about half of all machine learning models will use some of this form of linear regression, at least in their starting points.\n### Model selection: 11 dots\n![Model 1 | First-order linear regression, a simple line of best fit](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1wQ4KtBsmSb__mZlSQXr6ug.png)\nModel 1 | First-order linear regression, a simple line of best fit\nA very reasonable and acceptable way to do a regression for some data (like the above example) is just to eyeball it and draw your line through it. And you want to call that a regression, that is fine. I doubt it can be called a linear regression anymore, but it is certainly a regression. And it is a very valid one. Now we are going to be talking about regressions where we can show mathematical rules for going from data points to a line of best fit.\n![Mathematically defining the line above](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/0cys9O79-XirsvOB9.png)\nMathematically defining the line above\nEarlier I defined linear regression, now I will show an example of a quadratic equation that is still linear the way we have defined it:\n![Model 2 | Quadratic regression of the second order](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1AitknPmbVzY34wYr0-B9kQ.jpeg)\nModel 2 | Quadratic regression of the second order\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/0HiK8LzpUj7wM4TnB.png)\nequivalently:\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/0JxCa2d_ruO2yoitD.png)\nWhat we are going to do that or what we could do is we can say, \"okay\", y is not the output from a function that takes in one variable, namely x1, but rather y is the output from a function that takes in two variables; the first one being x1, the second one being x1². In fact, we can just call x1 times x1 the variable x2.\n![By relabeling x1² as x2, we have recovered a linear regression](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/09GnO9lMR9dGznBVT.png)\nBy relabeling x1² as x2, we have recovered a linear regression\nWe can have non-straight lines fit a linear equation! Extending from this we can basically go to any polynomial. The following graph has the same data as the quadratic example, but a different line of best fit.\n![Model 3 | Higher-order regression using the same data as quadratic example](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1OPrwwJny-EMIR0pNjl0F8g.jpeg)\nModel 3 | Higher-order regression using the same data as quadratic example\nThe only thing really changing is the line that we are drawing through them. In both cases, we are using what we call a linear relationship - well people who are in this regression business (mathematicians/physicists) call it such. They know full well that this is not a straight line. But they claim that in higher dimensions, this is actually straight; calling it not a line but a hyperplane.\nThe hyperplane is not curved in those dimensions. The dimensions would likely be:\n![Maybe a fourth-order is needed.](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/0255k_pa8ftIygUq3.png)\nMaybe a fourth-order is needed.\nThis is adjacent to the mathematical field of topology, very relevant to particle physicists and Einstein's General Relativity, in general (haha)! Physicists will say that the path of a hit baseball is a straight line. In high school, we learned that this path is a parabola but in curved space-time, the path is a straight line geodesic (a geodesic being a \"straight\" line in a curved topological landscape). Geodesics you are likely familiar with include the lines of latitude on a globe.\nSo here the relevant question being asked is which of these three proceeding models is correct? The first order linear one? The second-order linear one? The fourth-order linear one? Well, the better question is which of these three is more useful, and that would naturally require that we have knowledge of what the model is being used for.\nThe general rule is, and this is a pretty important rule, that **the simpler the model, the more useful it is**. Remember this heuristic! Later I will write about regularization, which is just a way of making models simpler. When you simplify your linear model (well any model, but the context here is linear models) it is not just easier to understand, more importantly, they work better!\nNo data are perfect, errors will always exist. Another useful heuristic to remember is \"all models are wrong, but some are useful!\" The fact is that some of the data above could just be errors. The 11 dots above are the training data. When we look at new data, it might turn out, or probably will turn out, that they will not follow the funny curve, our higher-order model. Chances are that the \"hyperplane\" model is probably an example of what we call overfitting. And the reverse process of overfitting or the process that undoes overfitting is called regularization. As such, the balance of probability says that our first model is the most useful model to conduct further analysis on.\n### Terminology\n  * **Response** (Dependent) variable: the variable of primary interest, the variable we are trying to predict or explain. Like in our Jupyter notebook examples above, the response is the lefthand side of the equal sign -'y' is DEPENDENT on the values on the righthand side of the equals sign\n  * **Explanatory** (Independent) variable: the variable that attempts to explain the observed outcomes of the response variable. In an equation, if you change the 'y' (dependent) variable, you are not changing the righthand side 'x' variables - hence they are INDEPENDENT of changes in 'y'\n\n\nThere are two types of parameters in linear models:\n  * **y-intercept,** referred to above as the offset\n  * **slope** , rise over run, change in y divided by change in x. If there are multiple inputs, then we have multiple slopes (we call these partial slopes)\n\n\nAs an aside, but I want to mention that data is often not linear, and as such the data need to be transformed to apply regression analysis. An important such transformation is doing a log transformation when dealing with nonlinear data.\n![Country area vs population | Source Wikipedia](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1QoAwCnVY3JMDAw1iXGGm-Q.jpeg)\nCountry area vs population | Source [Wikipedia](https://en.wikipedia.org/wiki/Data_transformation_\\(statistics\\))\nThis raw data is not linear, however, we can make it appear linear by applying a logarithmic change, here it will be log base 10:\n![The same data, transformed | Source Wikipedia](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1thRXTm88gOV4zymRrQF0-w.jpeg)\nThe same data, transformed | Source [Wikipedia](https://en.wikipedia.org/wiki/Data_transformation_\\(statistics\\))\nThe reason for doing the log transformation is because now your errors are going to probably be consistent throughout and that is very probably going to be better. It is possible to fit an exponential line to the data, doing non-parametric line fitting, but then the errors would not be consistent, they would not be of the same scale - which makes doing downstream analysis much harder.\n### Error terminology\nBefore explaining least-squares regression, we need to understand what the errors are:\n![epsilon for error, i for which data point it is \\(first, second, etc.\\)](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/0wrv1rlRy7l88NetW.png)\nepsilon for error, i for which data point it is (first, second, etc.)\nThe E(error) is the distance between each point and the regression line. We select the line with the smallest sum of squared differences between each point and the line.\n![Regression line and errors for each point](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1icgMTS3If8VilWykm6kkHA.jpeg)\nRegression line and errors for each point\nThe line above is a pretty good fit for the five dots above, but it is not perfect. What we want to know is how much error is there in the dependent variable. The epsilon term is added to bridge the gap between our predicted value(y_hat), which is where the red dot follows the green line and meets the regression line, and the actual observed values. The error is the difference between the predicted and observed variables. This is further explained in the code analysis below.\nWith a little bit of calculus and linear algebra, we try to find the minimum value of the sum of the errors for all possible regression lines. This summation is the best fit line. This is incredibly laborious to calculate. If instead, you say you want to minimize the square of this value, it turns out that there are beautiful, analytical solutions. And this is why we do least-squares fitting. So we do not try to minimize those dotted lines, or the sum of those dotted lines, we try instead to minimize the sum of the squares. So what is the sum of squares?\n![Visualization of the sum of squared errors for the regression line](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1ThlNNj4XJhCq6sGKQzSiyw.png)\nVisualization of the sum of squared errors for the regression line\nThe squares are now summed. The smallest total area of these squares is what we believe to be our best fit line. The advantage of this method, besides the beautiful analytical solutions, is that it penalizes larger errors - it punishes outliers. In the example above say that the largest error is 6 times larger than the smallest error. Well in least-squares regression that error is 36 (6 x 6)times more weighted than the smallest square.\n![Explicit solutions exist using calculus](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/16WVaj6MpLM4IvsqCsO0c1w.jpeg)\nExplicit solutions exist using calculus\nWe let the computers solve the equation above to find the sum of least-squares by playing around with the slope and intercept. The predicted (theoretical) values are subtracted from the actual (observed) values and summed.\n## Python example: regression model\nThe following code can be run in a Jupyter Notebook to help get a feel and a visualization for least-squares linear regression.\nN.B. we use synthetic data because we have a lot of control over what happens. Synthetic data is not always the best.\ntext\n\n```\n# relevant imports\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\n%matplotlib inline\n```\n\ntext\n\n```\n# synthetic data points between 0 and 10 with standard deviation of #the errors of 1 (std deviation of y should be around 3-the square #root of 10)\nn_points = 80\nx_start, x_end = 0, 10\ny_start, y_end = 0, 10\ny_sd = 1 # std deviation of the errors\n```\n\ntext\n\n```\n# data columns, the y values are the errors added to the x values\nx_data = np.linspace(x_start, x_end, n_points)\ny_error = np.random.normal(loc=0, scale=y_sd, size=n_points)\ny_data = np.linspace(y_start, y_end, n_points) + y_error\n```\n\ntext\n\n```\n# create dataframe\nsyn_data = pd.DataFrame({'x':x_data, 'y':y_data})\n```\n\ntext\n\n```\nsyn_data.head()\n```\n\n![The first five lines of our 80 synthetic data points](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/15SacGAWKYatODiYrUUwJew.jpeg)\nThe first five lines of our 80 synthetic data points\nLet us now visualize the data - note the strong linearity of the data. Remember that the point of this exercise is to answer the question \"if we know the value of X can we predict the value of Y?\" We believe that by drawing the line of best fit that if someone gives us an X, say an X that we do not already have, then we can figure out what the Y value would be.\ntext\n\n```\nplt.plot(syn_data['x'], syn_data['y'], 'ko') # ko = black points\nplt.grid(True)\nplt.xlabel('x')\nplt.ylabel('y')\nplt.title('x vs y')\nplt.show()\n```\n\n![Our synthetic data - for the most part, the x values are approximately the y values](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1zQIEDkC2jC_6zKuVqbiS0w.jpeg)\nOur synthetic data - for the most part, the x values are approximately the y values\n### Building linear regression model\nFollowing a simple template, we can build our model. You are likely familiar with one common package that can do this: Sci-Kit Learn. A quirk of `sklearn` is that works best with NumPy arrays, it is just what it is used to. The following template reshapes and converts the data for model training and evaluation:\ntext\n\n```\n# standard lin reg template\n```\n\ntext\n\n```\n# imports\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom sklearn.linear_model import LinearRegression\nfrom sklearn.metrics import mean_squared_error, r2_score\n```\n\ntext\n\n```\n# create numpy arrays for the x (independent) and y (dependent) #variables\nx = syn_data.x.values.reshape(-1,1)\ny = syn_data.y.values.reshape(-1,1)\n```\n\ntext\n\n```\n# model initialization\nregression_model = LinearRegression()\n```\n\ntext\n\n```\n# train the model (fit data)\nregression_model.fit(x, y)\n\ny_predicted = regression_model.predict(x)\n```\n\ntext\n\n```\n# model evaluation\nrmse = mean_squared_error(y, y_predicted)\nr2 = r2_score(y, y_predicted)\n```\n\ntext\n\n```\n# printing values\nprint('Slope:' ,regression_model.coef_)\nprint('Intercept:', regression_model.intercept_)\nprint('Root mean squared error: ', rmse)\nprint('R2 score: ', r2)\n```\n\ntext\n\n```\n# plotting values\nplt.scatter(x, y, s=10)\nplt.xlabel('x')\nplt.ylabel('y')\n```\n\ntext\n\n```\nplt.plot(x, y_predicted, color='r')\nplt.show()\n```\n\n![The line is a good fit for the data](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1ma7Yht7jvBZ_HZjJzsuauw.jpeg)\nThe line is a good fit for the data\nThe statistical values are explained in greater detail below when we evaluate the model's performance.\n### Plot residuals\nThe following is an important tool in understanding how good your regression model is - a residual plot. What is happening is that for every point, the difference between the observed and predicted values is plotted.\ntext\n\n```\nimport seaborn as sns\nsns.residplot(x, y)\nplt.show()\n```\n\n![The predicted y value differences from actual y values](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1uxCijMpxiBsi4GaKkRFZdA.jpeg)\nThe predicted y value differences from actual y values\nThe residuals are the errors, which we plotted earlier. The x-axis is the x values. The y axis is the discrepancy between the actual y values and the predicted y values, the errors.\ntext\n\n```\nfrom sklearn import linear_model\n# from sklearn.linear_model import LinearRegression\n```\n\ntext\n\n```\n# initialize the model.\nlinear_model = linear_model.LinearRegression()\n```\n\ntext\n\n```\n# define x and y\nx_input = syn_data['x'].values.reshape(n_points, 1)\ny_output = syn_data['y'].values.reshape(n_points, 1)\n```\n\ntext\n\n```\n# fit the model to the existing data with lm.fit(x,y)\nlinear_model.fit(x_input, y_output)\n```\n\ntext\n\n```\n# fit model to make predictions\ny_pred = linear_model.predict(x_input)\n```\n\ntext\n\n```\n# plot\nplt.scatter(x_input, y_output)\nplt.plot(x_input, y_pred, linewidth=2)\nplt.grid(True)\nplt.xlabel('x')\nplt.ylabel('y')\nplt.title('x vs y')\nplt.show()\n```\n\ntext\n\n```\n# model parameters\nprint('Intercept: {0:.5f}'.format(linear_model.intercept_[0]))\nprint('Slope : {0:.5f}'.format(linear_model.coef_[0][0]))\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1dGkJWcBnFJpHls4Zj-553Q.jpeg)\n### Stats Model package\nAnother, older and lesser-used Python package is Stats Model `sm`. The following code duplicates the Sci-Kit Learn work. Note the subtle differences between the two w/r to how the outputs are displayed.\nN.B. The statistics are the same, but the presentations and the robustness thereof differ between the packages. `sklearn` is the preferred package to learn these days.\ntext\n\n```\nimport statsmodels.formula.api as sm\n```\n\ntext\n\n```\n# ols = ordinary least squares\nols_model = sm.ols(formula = 'y ~ x', data=syn_data)\n```\n\ntext\n\n```\n# fit the model\nresults = ols_model.fit()\n```\n\ntext\n\n```\nprint('Intercept: {0:.5f}'.format(results.params.Intercept))\nprint('Slope : {0:.5f}'.format(results.params.x))\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1DqKw1qN_pBwFk-CIEIhZXA.jpeg)\ntext\n\n```\n# add predicted to dataframe\nsyn_data['predicted'] = y_pred\n```\n\ntext\n\n```\n# add residuals to dataframe\nsyn_data['resids'] = y_output - y_pred\n```\n\ntext\n\n```\n# verify data is as expected\nsyn_data.head()\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1GR_tOqB-Xn2XYFWgyOProw.jpeg)\ntext\n\n```\n# coefficients of line of best fit\nm = linear_model.coef_[0]\nb = linear_model.intercept_\nprint('m = {}'.format(m[0]))\nprint('b = {}'.format(b[0]))\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1oe1ImkbT9QwOzamdtGb_rQ.jpeg)\n### Interpreting the parameters\nIt is very important to know how to interpret the parameters (your results will be different because of the random seed in creating the data).\n  * **y-intercept (b)** : when x is zero, y is -0.13...\n  * **slope (m)** : When we increase x by 1 then we expect that y will go up by 1.03....\n\n\nSimple enough, I bet most readers will have a strong grasp on these basics, but it is important to refresh everyone and to build up from a solid and universal foundation.\n### Model summary statistics\nThis is where SM shines - the beautiful results summary. A lot more data is shown natively than we need to understand least - squares regression at the moment. The residuals are also plotted after the OLS report.\ntext\n\n```\nimport statsmodels.formula.api as sm\nimport seaborn as sns\n```\n\ntext\n\n```\nols_model = sm.ols(formula = 'y ~ x', data=syn_data)\n```\n\ntext\n\n```\nresults = ols_model.fit()\n```\n\ntext\n\n```\n# print slope (m) and y-intercept (b)\nprint('Intercept, Slope : {}'.format(results.params))\n```\n\ntext\n\n```\nprint('nSSE, SST, SSR, and RMSE:')\nmean_y = np.mean(y_output)\nsst = np.sum((y_output - mean_y)**2)\nssr = np.sum((y_pred - y_output)**2)\nsse = np.sum((y_pred - mean_y)**2)\n```\n\ntext\n\n```\nprint('Sum of Squares Explained (SSE): {}'.format(sse))\nprint('Total Sum of Squares (SST): {}'.format(sst))\nprint('Sum of Squared Residuals (SSR): {}'.format(ssr))\nprint('RMSE: {}'.format(np.sqrt(results.mse_resid)))\n```\n\ntext\n\n```\n# print linear regression statistics\nprint('n', results.summary())\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/16nQcAVIC3sxD2oIw7qnRkA.jpeg)\ntext\n\n```\n# plot histogram of the residuals\nsns.distplot(results.resid, hist=True)\nplt.xlabel('Residual')\nplt.ylabel('Frequency')\nplt.title('Residual Histogram')\nplt.show()\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1P5Ln9GqZuxD5BMIXMn6ySQ.jpeg)\nThe first thing to note about our residuals plotted above is that they are roughly normal. This is to be expected because the synthetic data was created to be such, so this a bit of a self-fulfilling prophecy. The y data is created from errors added to the randomly generated x data, with the y data having a standard deviation of +/- one.\n### Evaluation of regression models\nNow that we have built a regression model, we can quantitatively evaluate the performance of our regression model. The evaluation of regression models is based on measurements of the errors.\nIf we want to know how well we did, we must consider what a good benchmark would be. A common easy benchmark is to see if we can predict better than just the **mean of the target variable**.\nWe define the following terms:\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/0nh0P7yOwoyCwJRXg.png)\n![](https://assets.insightmediagroup.io/media/0*oiHucUFFurtYFsms)\n![](https://assets.insightmediagroup.io/media/0*RdlLehjgVCDB1w_f)\nNow we need to define the following error metrics:\n  * **Sum of Squared Residuals (SSR)** : This is the value that the method of least squares was able to minimize to with the best fit line. The goal of regression is to minimize this quantity.\n\n\n![The goal of regression - minimize this](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/0-1_2DDBuagoVNIXh.png)\nThe goal of regression - minimize this\n  * **Sum of Squared Total (SST)** : This is a measure of how bad a prediction the target-mean would be\n\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/0R_Qew1M7JkYZGH_E.png)\n  * **Sum of Squared Explained (SSE)** : This is a sum of a squared pointwise difference of the (SST - SSR). You can also think of this as a measurement of how much better we are explaining the variation than the mean\n\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/0S9XpWRQN8T2PR7aq.png)\nNotice that **SST = SSR + SSE**\n  * This formula is only true if you do your regression on your training data! If you do it on test data, this formula is no longer true. This formula is true for the cases where your coefficient of determination (COD)is the same as your Pearson's R², the correlation coefficient. **They are not the same!** They can sometimes be the same, but only in the situation for at least-squares fitting on your training data, not for when you use either your test data or not when you do a non least-squares fitting. SSR can become larger than SST, leading to a negative R²!\n\n\n![Visualization of the errors of a linear regression model](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1EPCzO70fMR7Z2l-6TJB8wA.jpeg)\nVisualization of the errors of a linear regression model\nWe can now talk about the **Root-Mean-Squared-Error (RMSE)**. The reason we do this is that the \"sum-squared-error\" (also referred to as the mean-squared error) is in units of **x²**. To convert the error to units of **x** , we need the square root of the mean-squared-error or the square root of the sum-squared residuals. This is the RMSE:\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/0Npqht8Wn55wd4A_4.png)\nRemember that the goal of regression is to minimize the residual error, **SSR**. Specifically, we wish to explain the maximum amount of the variance in the original data as possible with our model. We can quantify this idea with the coefficient of determination (COD)also known as **R²** :\n![ONLY CORRECT WITH LEAST-SQUARES FITTING AND MEASURING TRAINING DATA](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/0iXsHZdVgsw3H1N9f.png)\nONLY CORRECT WITH LEAST-SQUARES FITTING AND MEASURING TRAINING DATA\nso as\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/0GhTXfT2VneYdxLfH.png)\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/0d7wZmSverEnNDww7.png)\n**R² is the fraction of the variance of the original data explained by the model**. A model that perfectly explains the data has R² = 1. A model which does not explain the data at all has R² = 0. This means that the training data is spread out so evenly that you cannot draw any best fit line through it. You can draw a line through the data, it just would not be the best fit line.\nHowever, R² is not perfect:\n  * R² is not bias adjusted for degrees of freedom\n  * More importantly, there is no adjustment for the number of model parameters. As the number of model parameters increases, SSR will generally decrease. Without an adjustment, you will get a false sense of model performance\n\n\nWe address this by defining **Adjusted R² (R² adj)** :\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/0i7dyfnwpOPtnXM1U.png)\n![](https://assets.insightmediagroup.io/media/0*FPc84gNj0bmqV3iD)\nwhere\n![](https://assets.insightmediagroup.io/media/0*dD_wftTW9LPGCT2d)\n![](https://assets.insightmediagroup.io/media/0*L2moAVkUnrpbbwvH)\nThis gives **R²adj** as:\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/0SPkkcji5NqODvBc9.png)\nwhere\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/0xjWzCR1Jtw7b9GiM.png)\n![](https://assets.insightmediagroup.io/media/0*71_MXxHFhUk0nOu4)\nWhich can be rewritten:\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/028ZA0HI3v9q9bIUs.png)\nNow we have a metric that decreases with more added parameters, rewarding simplicity - remembering from earlier that simpler models are preferred to complex models.\n## Conclusion\nWe have covered many regression concepts so far:\n  * The single regression equation of a line is\n\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/0wrv1rlRy7l88NetW.png)\n  * Use the Python model object `linear_model.LinearRegression()` to initialize the model\n  * Use the `fit` method to fit the model to the data\n  * Use `predict` method to compute scores(predict values) for the dependent value (y)\n  * Pull the model parameters - m(slope) with `coef` and b(y-intercept) - with `intercept`\n  * The residuals are the difference between the y_output and the y_predicted\n  * Use the `statsmodels` and `seaborn` libraries to get summary statistics and make diagnostic plots\n  * `sklearn` is preferable to the `sm` package, it is simpler to use and more robust\n  * What residuals are and why we plot them\n\n\nWe evaluated the results from the OLS Regression Results with `summary` method\n  * **SSR** (Sum of the Squared Residuals) is the difference in error from the regression line (try to minimize)\n  * **SST (** Sum of the Squared Total) is the difference to the target mean\n  * **SSE** (Sum of the Squared Explained) is the variation of the regression line to the mean\n  * **RMSE** (Root Mean Squared Error) is the square root of the SSR\n  * **R²** is the fraction of the variance of the original data explained by the model, want a number close to 1\n  * **SST = SSR + SSE** is ONLY true for the cases where your coefficient of determination (COD) is the same as your Pearson's R², the correlation coefficient. For least-squares regression on training data, yes it is true. Outside of this case, on test data or when SSR > SST the formula does not hold\n\n\nIn my next article, I cover [multivariant regression](https://towardsdatascience.com/multivariant-linear-regression-e636a4f99b40) and then [bootstrapping wiht regression models](https://towardsdatascience.com/linear-regression-with-bootstrapping-4924c05d2a9)!\nFind me on [Linkedin](https://www.linkedin.com/in/james-a-w-godwin/)\n_Physicist cum Data Scientist- Available for new opportunity | SaaS | Sports | Start-ups | Scale-ups_\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](https://assets.insightmediagroup.io/media/1787657234707_1wi608.jpeg)\nWritten By\n## [James Andrew Godwin](https://towardsdatascience.com/author/james-andrew/)\n[See all from James Andrew Godwin](https://towardsdatascience.com/author/james-andrew/)\n[Data Science](https://towardsdatascience.com/tag/data-science/)[Linear Regression Python](https://towardsdatascience.com/tag/linear-regression-python/)[Machine Learning](https://towardsdatascience.com/tag/machine-learning/)[Towards Data Science](https://towardsdatascience.com/tag/towards-data-science/)\nShare This Article\n  * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fan-introduction-to-linear-regression-9cbb64b52d23%2F&title=An%20Introduction%20to%20Linear%20Regression)\n  * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fan-introduction-to-linear-regression-9cbb64b52d23%2F&title=An%20Introduction%20to%20Linear%20Regression)\n  * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fan-introduction-to-linear-regression-9cbb64b52d23%2F&text=An%20Introduction%20to%20Linear%20Regression)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2022/02/1VNhHRv5sY1fwVkJ6jF7vmQ.png)](https://towardsdatascience.com/reinforcement-learning-applied-to-the-mountain-car-problem-1c4fb16729ba/)\n## [Reinforcement Learning Applied to the Mountain Car Problem](https://towardsdatascience.com/reinforcement-learning-applied-to-the-mountain-car-problem-1c4fb16729ba/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nHow I Taught a Machine to Play this Simple Video-Game\n[Diego Unzueta](https://towardsdatascience.com/author/diegounzuetaruedas/)\nFebruary 3, 202210 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/Screenshot-2025-02-19-at-1.10.54%E2%80%AFPM.png)](https://towardsdatascience.com/multimodal-search-engine-agents-powered-by-blip-2-and-gemini/)\n## [Multimodal Search Engine Agents Powered by BLIP-2 and Gemini](https://towardsdatascience.com/multimodal-search-engine-agents-powered-by-blip-2-and-gemini/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nBuilding Multimodal Fashion Assistant Agents with Text and Image-Based Search\n[Luís Roque](https://towardsdatascience.com/author/luisroque/)\nFebruary 19, 202514 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/08/1voAKyYBQ2Vo3obrev18Y4g-scaled.jpeg)](https://towardsdatascience.com/pricing-objects-in-mercari-machine-learning-deep-learning-perspectives-6aa80514b2c8/)\n## [Pricing Objects in Mercari - Machine learning & Deep learning perspectives](https://towardsdatascience.com/pricing-objects-in-mercari-machine-learning-deep-learning-perspectives-6aa80514b2c8/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nSupplementary endeavor to solve NLP-based regression of Kaggle's Mercari dataset as part of a pioneering self-case study - Applied AI...\n[Yogeeshwari S](https://towardsdatascience.com/author/syogeeshwari/)\nAugust 19, 202019 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/12/1c9XPmcLESZMOTkBJG901Lw.png)](https://towardsdatascience.com/reinforcement-learning-explained-visually-part-5-deep-q-networks-step-by-step-5a5317197f4b/)\n## [Reinforcement Learning Explained Visually (Part 5): Deep Q Networks, step-by-step](https://towardsdatascience.com/reinforcement-learning-explained-visually-part-5-deep-q-networks-step-by-step-5a5317197f4b/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nA Gentle Guide to DQNs with Experience Replay, in Plain English\n[Ketan Doshi](https://towardsdatascience.com/author/ketanhdoshi/)\nDecember 19, 202011 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/11/1KSGfeQYxRbI9tGjpMbu87g.png)](https://towardsdatascience.com/time-series-signals-the-fourier-transform-f68e8a97c1c2/)\n## [Time Series, Signals, & the Fourier Transform](https://towardsdatascience.com/time-series-signals-the-fourier-transform-f68e8a97c1c2/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nAn introduction and primer for future posts\n[Shaw Talebi](https://towardsdatascience.com/author/shawhin/)\nNovember 16, 20204 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/05/0JMEYoKAW7QN8T-Ey.jpg)](https://towardsdatascience.com/reinforcement-learning-from-scratch-deep-q-networks-0a8d33ce165b/)\n## [Reinforcement Learning: Deep Q-Networks](https://towardsdatascience.com/reinforcement-learning-from-scratch-deep-q-networks-0a8d33ce165b/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nTeaching a shuttle to land on the moon using Deep Q-Networks in Python. A mathematical deep dive into Reinforcement Learning.\n[Cristian Leo](https://towardsdatascience.com/author/cristianleo120/)\nMay 23, 202429 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/08/15MHIzqGOqvTrJBb3m9drlw.jpeg)](https://towardsdatascience.com/free-gpus-for-training-your-deep-learning-models-c1ce47863350/)\n## [Free GPUs for Training Your Deep Learning Models](https://towardsdatascience.com/free-gpus-for-training-your-deep-learning-models-c1ce47863350/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nYes you heard it right. It's free.\n[Edwin Tan](https://towardsdatascience.com/author/edwin.tan/)\nAugust 20, 20215 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/11/1YwZBX1nxU7aTHJQFWOTDCg.png)](https://towardsdatascience.com/bessels-correction-why-do-we-divide-by-n-1-instead-of-n-in-sample-variance-30b074503bd9/)\n## [Bessel's Correction: Why Do We Divide by n−1 Instead of n in Sample Variance?](https://towardsdatascience.com/bessels-correction-why-do-we-divide-by-n-1-instead-of-n-in-sample-variance-30b074503bd9/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nUnderstanding the Unbiased Estimation of Population Variance\n[Aman Agrawal](https://towardsdatascience.com/author/amannagrawall002/)\nNovember 11, 20248 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/04/1p5SAw7kckaClkvwDu8BxoA.jpeg)](https://towardsdatascience.com/best-library-to-simplify-math-for-machine-learning-ed64cbe536ac/)\n## [Best Library To Simplify Math For Machine Learning!](https://towardsdatascience.com/best-library-to-simplify-math-for-machine-learning-ed64cbe536ac/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nUsing the sympy Python library to simplify differential and integral calculus for Machine Learning\n[Bharath K](https://towardsdatascience.com/author/bharath-k1297/)\nApril 21, 20217 min read\n\n\n  * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n  * [X](https://x.com/TDataScience)\n  * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n  * [Threads](https://www.threads.net/@towardsdatascience)\n  * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n####  Recommended Articles \nClose\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500) How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ \"How Relevance Models Foreshadowed Transformers for NLP\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n\n\n",
  "body_base_path": "attachments/d3f227ccb337/pages/",
  "body_markdown_path": "attachments/d3f227ccb337/pages/archived-body.md",
  "local_html": "attachments/d3f227ccb337/pages/archived-body.html",
  "body_source": "prior_capture"
}