{ "source_file": "0930", "source_platform": "Medium", "author": "Cristian Leo", "title_listed": "The Math Behind KAN - Kolmogorov-Arnold Networks", "keywords": "", "domain": "大模型算法", "content_type": "深度技术解析", "is_paid_flag": "否", "url": "https://towardsdatascience.com/the-math-behind-kan-kolmogorov-arnold-networks-7c12a164ba95/", "publish_date": "2024-06-12", "corpus_stage": "", "notes": "", "id": "8f333daf7214", "collected_at": "2026-10-02T06:10:03", "title": "The Math Behind KAN - Kolmogorov-Arnold Networks | Towards Data Science", "body_markdown": "![Revisit consent button](../images/body-df29beacd7496220cc94.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](../images/body-2159bb67f3de8ad870eb.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n * Cookie\nBCTempID\n * Duration\n10 minutes\n * Description\nNo description available.\n\n\n * Cookie\n__cf_bm\n * Duration\n1 hour\n * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n * Cookie\nAWSALBCORS\n * Duration\n7 days\n * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n * Cookie\n_cfuvid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n * Cookie\nli_gc\n * Duration\n6 months\n * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n * Cookie\n__hssrc\n * Duration\nsession\n * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n * Cookie\n__hssc\n * Duration\n1 hour\n * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n * Cookie\nwpEmojiSettingsSupports\n * Duration\nsession\n * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n * Cookie\nBCSessionID\n * Duration\n1 year 1 month 4 days\n * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n * Cookie\n_octo\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nlogged_in\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\n__Secure-YEC\n * Duration\npast\n * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n * Cookie\n__eoi\n * Duration\n6 months\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTGCORS\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\nlogin-status-p\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTG\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\ncsrf_token\n * Duration\nsession\n * Description\nNo description available.\n\n\n * Cookie\ntoken_v2\n * Duration\n1 day\n * Description\nDescription is currently not available.\n\n\n * Cookie\nD\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nPHPSESSID\n * Duration\nsession\n * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n * Cookie\nVISITOR_PRIVACY_METADATA\n * Duration\n6 months\n * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n * Cookie\ncookietest\n * Duration\nsession\n * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n * Cookie\n__Host-airtable-session\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\n__Host-airtable-session.sig\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\nm\n * Duration\n1 year 1 month 4 days\n * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n * Cookie\nBIGipServer*\n * Duration\nsession\n * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n * Cookie\n__cfruid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n * Cookie\n_GRECAPTCHA\n * Duration\n6 months\n * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n * Cookie\n__Secure-YNID\n * Duration\n6 months\n * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n * Cookie\ncookieyes-consent\n * Duration\n1 year\n * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n * Cookie\nlidc\n * Duration\n1 day\n * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n * Cookie\nbrw\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nbrwConsent\n * Duration\n5 minutes\n * Description\nDescription is currently not available.\n\n\n * Cookie\nWMF-Uniq\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nloom_anon_comment\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nloom_referral_video\n * Duration\nsession\n * Description\nDescription is currently not available.\n\n\n * Cookie\nVISITOR_INFO1_LIVE\n * Duration\n6 months\n * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n * Cookie\nyt-remote-connected-devices\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n * Duration\nNever Expires\n * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n * Cookie\nyt-remote-device-id\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-name\n * Duration\nsession\n * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\nyt-remote-fast-check-period\n * Duration\nsession\n * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-app\n * Duration\nsession\n * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n * Cookie\nyt-remote-cast-available\n * Duration\nsession\n * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n * Cookie\nyt-remote-cast-installed\n * Duration\nsession\n * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\ncp_session\n * Duration\n3 months\n * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n * Cookie\nloid\n * Duration\n1 year 1 month 4 days\n * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n * Cookie\n__hstc\n * Duration\n6 months\n * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n * Cookie\nhubspotutk\n * Duration\n6 months\n * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n * Cookie\n_ga\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n * Cookie\n_ga_*\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n * Cookie\n__Host-psifi.analyticsTrace\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n__Host-psifi.analyticsTraceV2\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n_gh_sess\n * Duration\nsession\n * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n * Cookie\nYSC\n * Duration\nsession\n * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n * Cookie\najs_anonymous_id\n * Duration\n1 year\n * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n * Cookie\nvuid\n * Duration\n1 year 1 month 4 days\n * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n * Cookie\nAWSALB\n * Duration\n7 days\n * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n * Cookie\nacq\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nacq.sig\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nptc\n * Duration\n2 years\n * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n * Cookie\nmuc_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n * Cookie\nguest_id_marketing\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\nguest_id_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\npersonalization_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n * Cookie\nguest_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n * Cookie\nbcookie\n * Duration\n1 year\n * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n * Cookie\n__Secure-ROLLOUT_TOKEN\n * Duration\n6 months\n * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n * Cookie\nyt.innertube::nextId\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nyt.innertube::requests\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nsession_tracker\n * Duration\nsession\n * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n * Cookie\nedgebucket\n * Duration\nsession\n * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n * Cookie\ndid\n * Duration\n1 year\n * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Data Science](https://towardsdatascience.com/category/data-science/)\n# The Math Behind KAN - Kolmogorov-Arnold Networks\nA new alternative to the classic Multi-Layer Perceptron is out. Why is it more accurate and interpretable? Math and Code Deep Dive.\n[Cristian Leo](https://towardsdatascience.com/author/cristianleo120/)\nJune 12, 202413 min read\n![Image generated by DALL-E](../images/000.jpg)\nImage generated by DALL-E\nIn today's world of AI, neural networks drive countless innovations and advancements. At the heart of many breakthroughs is the Multi-Layer Perceptron (MLP), a type of neural network known for its ability to approximate complex functions. But as we push the boundaries of what AI can achieve, we must ask: Can we do better than the classic MLP?\nHere's Kolmogorov-Arnold Networks (KANs), a new approach to neural networks inspired by the Kolmogorov-Arnold representation theorem. Unlike traditional MLPs, which use fixed activation functions at each neuron, KANs use learnable activation functions on the edges (weights) of the network. This simple shift opens up new possibilities in accuracy, interpretability, and efficiency.\nThis article explores why KANs are a revolutionary advancement in neural network design. We'll dive into their mathematical foundations, highlight the key differences from MLPs, and show how KANs can outperform traditional methods.\nLearn this step by step with the interactive [AI and Data Scientist](https://roadmap.sh/ai-data-scientist), [Computer Science](https://roadmap.sh/computer-science) and [Machine Learning](https://roadmap.sh/machine-learning) roadmaps.\n## 1: Limitations of MLPs\n![MLP with input layer with 3 nodes, 2 hidden layers with 10 nodes each, and an output layer with 1 node - Image by Author](../images/001.png)\nMLP with input layer with 3 nodes, 2 hidden layers with 10 nodes each, and an output layer with 1 node - Image by Author\nMulti-Layer Perceptrons (MLPs) are a core component of modern neural networks. They consist of layers of interconnected nodes, or \"neurons,\" designed to approximate complex, non-linear functions by learning from data. Each neuron uses a fixed activation function on the weighted sum of its inputs, transforming input data into the desired output through multiple layers of abstraction. MLPs have driven breakthroughs in various fields, from computer vision to speech recognition.\nHowever, MLPs have some significant limitations:\n 1. **Fixed Activation Functions on Nodes** : Each node in an MLP has a predetermined activation function, like ReLU or Sigmoid. While effective in many cases, these fixed functions limit the network's flexibility and adaptability. This can make it challenging for MLPs to optimize certain types of functions or adapt to specific data characteristics.\n 2. **Interpretability Issues** : MLPs are often criticized for being \"black boxes.\" As they become more complex, understanding their decision-making process becomes harder. The fixed activation functions and intricate weight matrices obscure the network's inner workings, making it difficult to interpret and trust the model's predictions without extensive analysis.\n\n\nThese drawbacks highlight the need for alternatives that offer greater flexibility and interpretability, paving the way for innovations like Kolmogorov-Arnold Networks (KANs).\n## 2: Kolmogorov-Arnold Networks (KANs)\n![MLP vs KAN comparison -Imake by Kan Github Repo \\(License\\)](../images/002.png)\nMLP vs KAN comparison -Imake by [Kan Github Repo](https://github.com/KindXiaoming/pykan/tree/master?tab=readme-ov-file) ([License](https://creativecommons.org/licenses/by/4.0/))\nThe Kolmogorov-Arnold representation theorem, formulated by mathematicians Andrey Kolmogorov and Vladimir Arnold, states that any multivariate continuous function can be represented as a finite composition of continuous functions of a single variable and the operation of addition. Think of this theorem as breaking down a complex recipe into individual, simple steps that anyone can follow. Instead of dealing with the entire recipe at once, you handle each step separately, making the overall process more manageable. This theorem implies that complex, high-dimensional functions can be broken down into simpler, univariate functions.\n> For neural networks, this insight is revolutionary, it suggests that a network could be designed to learn these univariate functions and their compositions, potentially improving both accuracy and interpretability.\nKANs leverage the power of the Kolmogorov-Arnold theorem by fundamentally altering the structure of neural networks. Unlike traditional MLPs, where fixed activation functions are applied at each node, KANs place learnable activation functions on the edges (weights) of the network. This key difference means that instead of having a static set of activation functions, KANs adaptively learn the best functions to apply during training. Each edge in a KAN represents a univariate function parameterized as a spline, allowing for dynamic and fine-grained adjustments based on the data.\nThis change enhances the network's flexibility and ability to capture complex patterns in data, providing a more interpretable and powerful alternative to traditional MLPs. By focusing on learnable activation functions on edges, KANs effectively utilize the Kolmogorov-Arnold theorem to transform neural network design, leading to improved performance in various AI tasks.\n## 3: Mathematical Foundations\nAt the core of Kolmogorov-Arnold Networks (KANs) is a set of equations that define how these networks process and transform input data. The foundation of KANs lies in the Kolmogorov-Arnold representation theorem, which inspires the structure and learning process of the network.\nImagine you have an input vector x=[x1,x2,...,xn], which represents data points that you want to process. Think of this input vector as a list of ingredients for a recipe.\nThe theorem states that any complex recipe (high-dimensional function) can be broken down into simpler steps (univariate functions). For KANs, each ingredient (input value) is transformed through a series of simple steps (univariate functions) placed on the edges of the network. Mathematically, this can be represented as:\n![KAN Formula - Image by Author](../images/003.png)\nKAN Formula - Image by Author\nHere, ϕ_q,p​ are univariate functions that are learned during training. Think of ϕ_q,p​ as individual cooking techniques for each ingredient, and Φ_q as the final assembly step that combines these prepared ingredients.\nEach layer of a KAN applies these cooking techniques to transform the ingredients further. For layer l, the transformation is given by:\n![Kan Layer Transformation Formula - Image by Author](../images/004.png)\nKan Layer Transformation Formula - Image by Author\nHere, x(l) denotes the transformed ingredients at layer l, and ϕ_l,i,j are the learnable univariate functions on the edges between layer l and l+1. Think of this as applying different cooking techniques to the ingredients at each step to get intermediate dishes.\nThe output of a KAN is a composition of these layer transformations. Just as you would combine intermediate dishes to create a final meal, KANs combine the transformations to produce the final output:\n![KAN Output Formula - Image by Author](../images/005.png)\nKAN Output Formula - Image by Author\nHere, Φl represents the matrix of univariate functions at layer l. The overall function of the KAN is a composition of these layers, each refining the transformation further.\n**MLPs Structure** In traditional MLPs, each node applies a fixed activation function (like ReLU or sigmoid) to its inputs. Think of this as using the same cooking technique for all ingredients, regardless of their nature.\nMLPs use linear transformations followed by these fixed non-linear activations:\n![MLP Formula - Image by Author](../images/006.png)\nMLP Formula - Image by Author\nwhere W represents the weight matrices, and σ represents the fixed activation functions.\n### Grid Extension Technique\n![Left: Notations of activations that flow through the network. Right: an activation function is parameterized as a B-spline, which allows switching between coarse-grained and fine-grained grids - Imagine extracted by "KAN: Kolmogorov-Arnold Networks" \\(License\\)](../images/007.png)\nLeft: Notations of activations that flow through the network. Right: an activation function is parameterized as a B-spline, which allows switching between coarse-grained and fine-grained grids - Imagine extracted by \"[KAN: Kolmogorov-Arnold Networks](https://arxiv.org/abs/2404.19756)\" ([License](https://creativecommons.org/licenses/by/4.0/))\nGrid extension is a powerful technique used to improve the accuracy of Kolmogorov-Arnold Networks (KANs) by refining the spline grids on which the univariate functions are defined. This process allows the network to learn increasingly detailed patterns in the data without requiring complete retraining.\nThese B-splines are a series of polynomial functions that are pieced together to form a smooth curve. They are used in KANs to represent the univariate functions on the edges. The spline is defined over a series of intervals called grid points. The more grid points there are, the finer the detail that the spline can capture\n![KAN with 2D input, 1 hidden layer with 5 nodes, and an output layer with 1 node - Imagine extracted by "KAN: Kolmogorov-Arnold Networks" \\(License\\)](../images/008.png)\nKAN with 2D input, 1 hidden layer with 5 nodes, and an output layer with 1 node - Imagine extracted by \"[KAN: Kolmogorov-Arnold Networks](https://arxiv.org/abs/2404.19756)\" ([License](https://creativecommons.org/licenses/by/4.0/))\nInitially, the network starts with a coarse grid, which means there are fewer intervals between grid points. This allows the network to learn the basic structure of the data without getting bogged down in details. Think of this like sketching a rough outline before filling in the fine details.\nAs training progresses, the number of grid points is gradually increased. This process is known as grid refinement. By adding more grid points, the spline becomes more detailed and can capture finer patterns in the data. This is similar to progressively adding more detail to your initial sketch, turning it into a detailed drawing.\nEach increase introduces new B-spline basis functions B′_m​(x). The coefficients c'_m for these new basis functions are adjusted to ensure that the new, finer spline closely matches the original, coarser spline.\nTo achieve this match, least squares optimization is used. This method adjusts the coefficients c'_m​ to minimize the difference between the original spline and the refined spline.\n![Least Square Optimization Formula - Image by Author](../images/009.png)\nLeast Square Optimization Formula - Image by Author\nEssentially, this process ensures that the refined spline continues to accurately represent the data patterns learned by the coarse spline.\n### Simplification Techniques\nTo enhance the interpretability of KANs, several simplification techniques can be employed, making the network easier to understand and visualize.\n**Sparsification and Pruning** This technique involves adding a penalty to the loss function based on the L1 norm of the activation functions. The L1 norm for a function ϕ is defined as the average magnitude of the function across all input samples:\n![L1 Norm Formula - Image by Author](../images/010.png)\nL1 Norm Formula - Image by Author\nHere, N_p​ is the number of input samples, and ϕ(x_s​) represents the value of the function ϕ for the input sample x_s​.\nThink of sparsification like decluttering a room. By removing unnecessary items (or reducing the influence of less important functions), you make the space (or network) more organized and easier to navigate.\nAfter applying L1 regularization, the L1 norms of the activation functions are evaluated. Neurons and edges with norms below a certain threshold are considered insignificant and are pruned away. The threshold for pruning is a hyperparameter that determines how aggressive the pruning should be.\nPruning is like trimming a tree. By cutting away the weak and unnecessary branches, you allow the tree to focus its resources on the stronger, more vital parts, leading to a healthier and more manageable structure.\n**Symbolification** Another approach is to replace learned univariate functions with known symbolic forms to make the network more interpretable.\nThe task is to identify potential symbolic forms (e.g., sin⁡, exp) that can approximate the learned functions. This step involves analyzing the learned functions and suggesting symbolic candidates based on their shape and behavior.\nOnce symbolic candidates are identified, use grid search and linear regression to fit parameters such that the symbolic function closely approximates the learned function.\n## 4: KAN vs MLP in Python\nTo demonstrate the capabilities of Kolmogorov-Arnold Networks (KANs) compared to traditional Multi-Layer Perceptrons (MLPs), we will fit a function-generated dataset to both a KAN model and MLP model (leveraging PyTorch), to see what their performances look like.\nThe function we will be using is the same one used by the authors of the paper to show KAN's capabilities vs MLP ([Original paper example](https://github.com/KindXiaoming/pykan/blob/master/tutorials/Example_1_function_fitting.ipynb)). However, the code will be different. You can find all the code we will cover today in this Notebook:\n> [**models-from-scratch-python/KAN - Kolmogorov-Arnold Networks/demo.ipynb at main ·...**](https://github.com/cristianleoo/models-from-scratch-python/blob/main/KAN%20-%20Kolmogorov-Arnold%20Networks/demo.ipynb)\nLet's import the required libraries, and generate the dataset\njavascript\n\n```\nimport numpy as np\nimport torch\nimport torch.nn as nn\nfrom torchsummary import summary\nfrom kan import KAN, create_dataset\nimport matplotlib.pyplot as plt\n```\n\nHere, we use:\n * `numpy`: For numerical operations.\n * `torch`: For PyTorch, which is used for building and training neural networks.\n * `torch.nn`: For neural network modules in PyTorch.\n * `torchsummary`: For summarizing the model structure.\n * `kan`: Custom library containing the KAN model and dataset creation functions.\n * `matplotlib.pyplot`: For plotting and visualizations.\n\n\npython\n\n```\n# Define the dataset generation function\nf = lambda x: torch.exp(torch.sin(torch.pi * x[:, [0]]) + x[:, [1]] ** 2)\n```\n\nThis function includes both sinusoidal (`sin`) and exponential (`exp`) components. It takes a 2D input `x` and computes the output using the formula:\n![Dataset generation Function - Image by Author](../images/011.png)\nDataset generation Function - Image by Author\nLet's now fit a tensor of 100 points uniformly distributed between [-2, 2] to this function, to see what it looks like:\n![Plot generated by fitting 100 points uniformly distributed between \\[-2, 2\\] to Dataset generation function - Image by Author](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1ap4SLcVIX_cqpdHsEMfjTA-2.png)\nPlot generated by fitting 100 points uniformly distributed between [-2, 2] to Dataset generation function - Image by Author\npython\n\n```\n# Create the dataset\ndataset = create_dataset(f, n_var=2)\n```\n\n`create_dataset` generates a dataset based on the function `f`. The dataset includes input-output pairs that will be used for training and testing the neural networks.\nNow let's build a KAN model and train it on the dataset. We will start with a coarse grid (5 points) and gradually refine it (up to 100 points). This improves the model's accuracy by capturing finer details in the data.\npython\n\n```\ngrids = np.array([5, 10, 20, 50, 100])\ntrain_losses_kan = []\ntest_losses_kan = []\nsteps = 50\nk = 3\n\nfor i in range(grids.shape[0]):\n if i == 0:\n model = KAN(width=[2, 1, 1], grid=grids[i], k=k)\n else:\n model = KAN(width=[2, 1, 1], grid=grids[i], k=k).initialize_from_another_model(model, dataset['train_input'])\n results = model.train(dataset, opt=\"LBFGS\", steps=steps, stop_grid_update_step=30)\n train_losses_kan += results['train_loss']\n test_losses_kan += results['test_loss']\n\n print(f\"Train RMSE: {results['train_loss'][-1]:.8f} | Test RMSE: {results['test_loss'][-1]:.8f}\")\n```\n\nIn this example, we define an array called `grids` with values [5, 10, 20, 50, 100]. We iterate over these grids to fit models sequentially, meaning each new model is initialized using the previous one.\nFor each iteration, we define a model with k=3, where k is the order of the B-spline. We set the number of training steps (or epochs) to 50. The model's architecture consists of an input layer with 2 nodes, one hidden layer with 1 node, and an output layer with 1 node. We use the LFGBS optimizer for training.\nHere are the training and test losses during the training process:\n![KAN Train and Test Losses Plot - Image by Author](../images/013.png)\nKAN Train and Test Losses Plot - Image by Author\nLet's now define and train a traditional MLP for comparison.\npython\n\n```\n# Define the MLP\nclass MLP(nn.Module):\n def __init__(self):\n super(MLP, self).__init__()\n self.layers = nn.Sequential(\n nn.Linear(dataset['train_input'].shape[1], 64),\n nn.ReLU(),\n nn.Linear(64, 64),\n nn.ReLU(),\n nn.Linear(64, 1)\n )\n def forward(self, x):\n return self.layers(x)\n\n# Instantiate the model\nmodel = MLP()\nsummary(model, input_size=(dataset['train_input'].shape[1],))\n```\n\nThe MLP has an input layer, two hidden layers with 64 neurons each, and an output layer. ReLU activation is used between the layers.\npython\n\n```\ncriterion = nn.MSELoss()\noptimizer = torch.optim.Adam(model.parameters(), lr=1e-2)\ntrain_loss_mlp = []\ntest_loss_mlp = []\n\nepochs = 250\nfor epoch in range(epochs):\n optimizer.zero_grad()\n output = model(dataset['train_input']).squeeze()\n loss = criterion(output, dataset['train_label'])\n loss.backward()\n optimizer.step()\n train_loss_mlp.append(loss.item()**0.5)\n\n # Test the model\n model.eval()\n with torch.no_grad():\n output = model(dataset['test_input']).squeeze()\n loss = criterion(output, dataset['test_label'])\n test_loss_mlp.append(loss.item()**0.5)\n\n print(f'Epoch {epoch+1}/{epochs}, Train Loss: {train_loss_mlp[-1]:.2f}, Test Loss: {test_loss_mlp[-1]:.2f}', end='r')\n```\n\nWe use mean squared error (MSE) loss and Adam optimizer, and train the model for 250 epochs, recording the training and testing losses.\nThis is what the train and test RMSE look like in MLP:\n![MLP Train and Test Losses Plot - Image by Author](../images/body-0ff31caafc2616a8432b.png)\nMLP Train and Test Losses Plot - Image by Author\nLet's put side to side the loss plots for a comparison:\n![KAN vs MLP training losses plot \\(left\\) and test losses plot \\(right\\) - Image by Author](../images/body-c0d336304496f75133dd.png)\nKAN vs MLP training losses plot (left) and test losses plot (right) - Image by Author\nThe plot shows that the KAN model achieves lower training RMSE than the MLP model, indicating better function-fitting capability. Similarly, the KAN model outperforms the MLP on the test set, demonstrating its superior generalization ability.\nThis example illustrates how KANs can more accurately fit complex functions than traditional MLPs, thanks to their flexible and adaptive structure. By refining the grid and employing learnable univariate functions on the edges, KANs capture intricate patterns in the data that MLPs may miss, leading to improved performance in function-fitting tasks.\nDoes this mean we should switch to KAN models permanently? Not necessarily.\nKANs showed great results in this example, but when I tested them on other scenarios with real data, MLPs often performed better. One thing you'll notice when working with KAN models is their sensitivity to hyperparameter optimization. Also, KANs have primarily been tested using spline functions, which work well for smoothly varying data like our example but might not perform as well in other situations.\nIn summary, KANs are definitely intriguing and have a lot of potential, but they need more study, especially regarding different datasets and the algorithm's inner workings, to really make them work effectively.\n## 5: Advantages of KANs\n### Accuracy\nOne of the standout advantages of Kolmogorov-Arnold Networks (KANs) is their ability to achieve higher accuracy with fewer parameters compared to traditional Multi-Layer Perceptrons (MLPs). This is primarily due to the learnable activation functions on the edges, which allow KANs to better capture complex patterns and relationships in the data.\nUnlike MLPs that use fixed activation functions at each node, KANs use univariate functions on the edges, making the network more flexible and capable of fine-tuning its learning process to the data.\nBecause KANs can adjust the functions between layers dynamically, they can achieve comparable or even superior accuracy with a smaller number of parameters. This efficiency is particularly beneficial for tasks with limited data or computational resources.\n### Interpretability\nKANs offer significant improvements in interpretability over traditional MLPs. This enhanced interpretability is crucial for applications where understanding the decision-making process is as important as the outcome.\nKANs can be simplified through techniques like sparsification and pruning, which remove unnecessary functions and parameters. These techniques not only improve interpretability but also enhance the network's performance by focusing on the most relevant components.\nFor some functions, it is possible to identify symbolic forms of the activation functions, making it easier to understand the mathematical transformations within the network.\n### Scalability\nKANs exhibit faster neural scaling laws compared to MLPs, meaning they improve more rapidly as the number of parameters increases.\nKANs benefit from more favorable scaling laws due to their ability to decompose complex functions into simpler, univariate functions. This allows them to achieve lower error rates with increasing model complexity more efficiently than MLPs.\nKANs can start with a coarser grid and extend it to finer grids during training, which helps in balancing computational efficiency and accuracy. This approach allows KANs to scale up more gracefully than MLPs, which often require complete retraining when increasing model size.\n## Conclusion\nKolmogorov-Arnold Networks (KANs) present a groundbreaking alternative to traditional Multi-Layer Perceptrons (MLPs), offering several key innovations that address the limitations of their predecessors. By leveraging learnable activation functions on the edges rather than fixed functions at the nodes, KANs introduce a new level of flexibility and adaptability. This structural change leads to:\n * **Enhanced Accuracy** : KANs achieve higher accuracy with fewer parameters, making them more efficient and effective for a wide range of tasks.\n * **Improved Interpretability** : The ability to visualize and simplify KANs aids in understanding the decision-making process, which is crucial for critical applications in healthcare, finance, and autonomous systems.\n * **Better Scalability** : KANs exhibit faster neural scaling laws, allowing them to handle increasing complexity more gracefully than MLPs.\n\n\nThe introduction of Kolmogorov-Arnold Networks marks an exciting development in the field of neural networks, opening up new possibilities for AI and machine learning.\n## References\n * Ziming Liu, et al**.** \"KAN: Kolmogorov-Arnold Networks\". \n\n\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](../images/body-8f73ac5a40c91a7b503f.jpeg)\nWritten By\n## [Cristian Leo](https://towardsdatascience.com/author/cristianleo120/)\n[See all from Cristian Leo](https://towardsdatascience.com/author/cristianleo120/)\n[Data Science](https://towardsdatascience.com/tag/data-science/)[Machine Learning](https://towardsdatascience.com/tag/machine-learning/)[Mathematics](https://towardsdatascience.com/tag/mathematics/)[Programming](https://towardsdatascience.com/tag/programming/)[Thoughts And Theory](https://towardsdatascience.com/tag/thoughts-and-theory/)\nShare This Article\n * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fthe-math-behind-kan-kolmogorov-arnold-networks-7c12a164ba95%2F&title=The%20Math%20Behind%20KAN%20-%20Kolmogorov-Arnold%20Networks)\n * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fthe-math-behind-kan-kolmogorov-arnold-networks-7c12a164ba95%2F&title=The%20Math%20Behind%20KAN%20-%20Kolmogorov-Arnold%20Networks)\n * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fthe-math-behind-kan-kolmogorov-arnold-networks-7c12a164ba95%2F&text=The%20Math%20Behind%20KAN%20-%20Kolmogorov-Arnold%20Networks)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n * [![](../images/body-63beb50f664a055ed739.jpg)](https://towardsdatascience.com/reinforcement-learning-with-sarsa-a-good-alternative-to-q-learning-algorithm-bf35b209e1c/)\n## [Reinforcement Learning with SARSA - A Good Alternative to Q-Learning Algorithm](https://towardsdatascience.com/reinforcement-learning-with-sarsa-a-good-alternative-to-q-learning-algorithm-bf35b209e1c/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nHow to teach an intelligent agent to play simple games using State-Action-Reward-State-Action (SARSA) algorithm\n[Saul Dobilas](https://towardsdatascience.com/author/solclover/)\nOctober 20, 202213 min read\n * [![](../images/015.png)](https://towardsdatascience.com/from-outdated-to-updated-2df4b43fcc38/)\n## [From Outdated to Updated](https://towardsdatascience.com/from-outdated-to-updated-2df4b43fcc38/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nHow to Create a Searchable Archive of PDFs with OCR, Python, & SQLite\n[Brandon Cosley](https://towardsdatascience.com/author/brandoncosley/)\nAugust 10, 20219 min read\n * [![](../images/016.jpeg)](https://towardsdatascience.com/4-different-ways-of-creating-a-new-column-with-pyspark-1a6782fe764e/)\n## [4 Different Ways of Creating a New Column with PySpark](https://towardsdatascience.com/4-different-ways-of-creating-a-new-column-with-pyspark-1a6782fe764e/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nLearn how to create new columns in Spark data frames\n[Soner Yıldırım](https://towardsdatascience.com/author/sonery/)\nDecember 20, 20214 min read\n * [![](../images/017.jpeg)](https://towardsdatascience.com/data-democratisation-5-data-for-all-strategies-embraced-by-large-companies-e62fad52c250/)\n## [Data Democratisation: 5 'Data For All' Strategies Embraced by Large Companies](https://towardsdatascience.com/data-democratisation-5-data-for-all-strategies-embraced-by-large-companies-e62fad52c250/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nIt's about Data, Skills, Tools...and especially Culture!\n[Col Jung](https://towardsdatascience.com/author/col-jung/)\nJuly 19, 202316 min read\n * [![](../images/018.jpg)](https://towardsdatascience.com/boosting-algorithms-in-machine-learning-part-ii-gradient-boosting-c155ae505fe9/)\n## [Boosting Algorithms in Machine Learning, Part II: Gradient Boosting](https://towardsdatascience.com/boosting-algorithms-in-machine-learning-part-ii-gradient-boosting-c155ae505fe9/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nUncovering a simple yet powerful, award-winning machine learning algorithm\n[Gurjinder Kaur](https://towardsdatascience.com/author/gurjinderkaur95/)\nNovember 12, 20249 min read\n * [![](../images/019.jpg)](https://towardsdatascience.com/linear-regression-what-is-the-sum-of-squares-3746db90a05d/)\n## [Linear Regression: What is the Sum of Squares?](https://towardsdatascience.com/linear-regression-what-is-the-sum-of-squares-3746db90a05d/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nA closer look at understanding why the Sum of Squares is important to Data Science\n[Mike Wolfe](https://towardsdatascience.com/author/miketechgame/)\nJune 14, 20225 min read\n * [![](../images/020.jpeg)](https://towardsdatascience.com/a-hands-on-guide-to-sorting-dataframes-in-pandas-384996ca6bb8/)\n## [A hands-on guide to 'sorting' dataframes in Pandas](https://towardsdatascience.com/a-hands-on-guide-to-sorting-dataframes-in-pandas-384996ca6bb8/)\n[Programming](https://towardsdatascience.com/category/programming/)\nTen easy ways to sort data in pandas\n[Parul Pandey](https://towardsdatascience.com/author/pandeyparul/)\nMarch 22, 20217 min read\n * [![](../images/021.png)](https://towardsdatascience.com/te-papa-tongarewa-museum-of-new-zealand-sentimental-footing-d062e41e430f/)\n## [Te Papa Tongarewa (Museum of New Zealand) Sentimental Footing](https://towardsdatascience.com/te-papa-tongarewa-museum-of-new-zealand-sentimental-footing-d062e41e430f/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nI paid a visit to this museum about four months ago on a tour of Wellington, the windiest capital of the world. The Maori name, 'Te Papa...\n[Herman Wandabwa](https://towardsdatascience.com/author/hermanwandabwa/)\nJuly 22, 20192 min read\n * [![](../images/022.jpeg)](https://towardsdatascience.com/bad-assumptions-the-downfall-of-even-experienced-data-scientists-8fe00726f849/)\n## [Bad Assumptions - The Downfall of Even Experienced Data Scientists](https://towardsdatascience.com/bad-assumptions-the-downfall-of-even-experienced-data-scientists-8fe00726f849/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nData can be deceptive, so be on your toes!\n[Michael Clayton](https://towardsdatascience.com/author/maclayton/)\nAugust 16, 202410 min read\n\n\n * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n * [X](https://x.com/TDataScience)\n * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n * [Threads](https://www.threads.net/@towardsdatascience)\n * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n#### Recommended Articles \nClose\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500) How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ \"How Relevance Models Foreshadowed Transformers for NLP\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n\n\n", "metadata": { "title": "The Math Behind KAN - Kolmogorov-Arnold Networks | Towards Data Science", "description": "A new alternative to the classic Multi-Layer Perceptron is out. Why is it more accurate and interpretable? Math and Code Deep Dive.", "keywords": null, "author": "Cristian Leo", "og:locale": "en_US", "og:type": "article", "og:title": "The Math Behind KAN - Kolmogorov-Arnold Networks", "og:description": "A new alternative to the classic Multi-Layer Perceptron is out. Why is it more accurate and interpretable? Math and Code Deep Dive.", "og:url": "https://towardsdatascience.com/the-math-behind-kan-kolmogorov-arnold-networks-7c12a164ba95/", "og:site_name": "Towards Data Science", "og:image": "https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/0HO-vPqDcT_AnH8Gw.jpg", "og:image:width": "1792", "og:image:height": "1024", "og:image:type": "image/jpeg", "twitter:card": "summary_large_image", "twitter:site": "@TDataScience", "twitter:creator": "@TDataScience", "twitter:label1": "Written by", "twitter:data1": "Cristian Leo", "twitter:label2": "Est. reading time", "twitter:data2": "13 min read", "article:published_time": "2024-06-12T03:45:09.000Z", "article:modified_time": "2024-06-12T03:45:09.000Z", "article:author": "https://towardsdatascience.com/author/cristianleo120/" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 202, "error": "" }, "code_blocks": [ { "language": "txt", "file": "attachments/8f333daf7214/code/block_000.txt", "chars": 155 }, { "language": "txt", "file": "attachments/8f333daf7214/code/block_001.txt", "chars": 115 }, { "language": "txt", "file": "attachments/8f333daf7214/code/block_002.txt", "chars": 58 }, { "language": "txt", "file": "attachments/8f333daf7214/code/block_003.txt", "chars": 612 }, { "language": "txt", "file": "attachments/8f333daf7214/code/block_004.txt", "chars": 469 }, { "language": "txt", "file": "attachments/8f333daf7214/code/block_005.txt", "chars": 728 } ], "images": [ { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T17:46:44Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T17:46:44Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T17:46:44Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T17:46:44Z" } ], "audio": [], "videos": [ { "url": "https://www.youtube.com/c/TowardsDataScience", "kind": "channel_reference", "status": "reference_only" } ], "links": [ { "type": "internal", "href": "https://towardsdatascience.com/" }, { "type": "internal", "href": "https://towardsdatascience.com/latest/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/editors-pick/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/the-variable/" }, { "type": "internal", "href": "https://towardsdatascience.com/shipai/" }, { "type": "internal", "href": "https://towardsdatascience.com/submissions/" }, { "type": "internal", "href": "https://towardsdatascience.com/category/data-science/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/cristianleo120/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/data-science/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/machine-learning/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/mathematics/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/programming/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/thoughts-and-theory/" }, { "type": "internal", "href": "https://towardsdatascience.com/reinforcement-learning-with-sarsa-a-good-alternative-to-q-learning-algorithm-bf35b209e1c/" }, { "type": "internal", "href": "https://towardsdatascience.com/category/artificial-intelligence/machine-learning/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/solclover/" }, { "type": "internal", "href": "https://towardsdatascience.com/from-outdated-to-updated-2df4b43fcc38/" }, { "type": "internal", "href": "https://towardsdatascience.com/category/artificial-intelligence/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/brandoncosley/" }, { "type": "internal", "href": "https://towardsdatascience.com/4-different-ways-of-creating-a-new-column-with-pyspark-1a6782fe764e/" }, { "type": "external", "href": "https://write.insightmediagroup.io" }, { "type": "external", "href": "https://www.linkedin.com/company/towards-data-science/" }, { "type": "external", "href": "https://x.com/TDataScience" }, { "type": "external", "href": "https://roadmap.sh/ai-data-scientist" }, { "type": "external", "href": "https://roadmap.sh/computer-science" }, { "type": "external", "href": "https://roadmap.sh/machine-learning" }, { "type": "external", "href": "https://github.com/KindXiaoming/pykan/tree/master?tab=readme-ov-file" }, { "type": "external", "href": "https://creativecommons.org/licenses/by/4.0/" }, { "type": "external", "href": "https://arxiv.org/abs/2404.19756" }, { "type": "external", "href": "https://github.com/KindXiaoming/pykan/blob/master/tutorials/Example_1_function_fitting.ipynb" }, { "type": "external", "href": "https://github.com/cristianleoo/models-from-scratch-python/blob/main/KAN%20-%20Kolmogorov-Arnold%20Networks/demo.ipynb" }, { "type": "external", "href": "https://www.facebook.com/sharer/sharer.php?title=The+Math+Behind+KAN+-+Kolmogorov-Arnold+Networks&u=https%3A%2F%2Ftowardsdatascience.com%2Fthe-math-behind-kan-kolmogorov-arnold-networks-7c12a164ba95%2F" }, { "type": "external", "href": "https://www.linkedin.com/shareArticle?mini=true&title=The+Math+Behind+KAN+-+Kolmogorov-Arnold+Networks&url=https%3A%2F%2Ftowardsdatascience.com%2Fthe-math-behind-kan-kolmogorov-arnold-networks-7c12a164ba95%2F" }, { "type": "external", "href": "https://x.com/share?text=The+Math+Behind+KAN+-+Kolmogorov-Arnold+Networks&url=https%3A%2F%2Ftowardsdatascience.com%2Fthe-math-behind-kan-kolmogorov-arnold-networks-7c12a164ba95%2F" }, { "type": "external", "href": "https://www.youtube.com/c/TowardsDataScience" }, { "type": "external", "href": "https://www.threads.net/@towardsdatascience" }, { "type": "external", "href": "https://bsky.app/profile/towardsdatascience.com" }, { "type": "external", "href": "https://thenewstack.io" }, { "type": "external", "href": "https://roadmap.sh" }, { "type": "external", "href": "https://insightmediagroup.io/creator-network" } ], "repair_version": 2, "repaired_at": "2026-10-20T17:26:13Z", "source_url": "https://towardsdatascience.com/the-math-behind-kan-kolmogorov-arnold-networks-7c12a164ba95/", "delivery_status": "unverified", "discovery_status": "http_403", "missing_reason": "源页面未重新核验:http_403；已按保留正文补采图片", "repair_error": "", "actual_video_files": [], "expected_saved_body_images": 4, "image_discovery": "saved_markdown", "source_body_markdown": "![Revisit consent button](https://cdn-cookieyes.com/assets/images/revisit.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](https://cdn-cookieyes.com/assets/images/close.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n * Cookie\nBCTempID\n * Duration\n10 minutes\n * Description\nNo description available.\n\n\n * Cookie\n__cf_bm\n * Duration\n1 hour\n * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n * Cookie\nAWSALBCORS\n * Duration\n7 days\n * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n * Cookie\n_cfuvid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n * Cookie\nli_gc\n * Duration\n6 months\n * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n * Cookie\n__hssrc\n * Duration\nsession\n * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n * Cookie\n__hssc\n * Duration\n1 hour\n * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n * Cookie\nwpEmojiSettingsSupports\n * Duration\nsession\n * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n * Cookie\nBCSessionID\n * Duration\n1 year 1 month 4 days\n * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n * Cookie\n_octo\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nlogged_in\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\n__Secure-YEC\n * Duration\npast\n * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n * Cookie\n__eoi\n * Duration\n6 months\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTGCORS\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\nlogin-status-p\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTG\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\ncsrf_token\n * Duration\nsession\n * Description\nNo description available.\n\n\n * Cookie\ntoken_v2\n * Duration\n1 day\n * Description\nDescription is currently not available.\n\n\n * Cookie\nD\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nPHPSESSID\n * Duration\nsession\n * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n * Cookie\nVISITOR_PRIVACY_METADATA\n * Duration\n6 months\n * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n * Cookie\ncookietest\n * Duration\nsession\n * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n * Cookie\n__Host-airtable-session\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\n__Host-airtable-session.sig\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\nm\n * Duration\n1 year 1 month 4 days\n * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n * Cookie\nBIGipServer*\n * Duration\nsession\n * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n * Cookie\n__cfruid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n * Cookie\n_GRECAPTCHA\n * Duration\n6 months\n * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n * Cookie\n__Secure-YNID\n * Duration\n6 months\n * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n * Cookie\ncookieyes-consent\n * Duration\n1 year\n * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n * Cookie\nlidc\n * Duration\n1 day\n * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n * Cookie\nbrw\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nbrwConsent\n * Duration\n5 minutes\n * Description\nDescription is currently not available.\n\n\n * Cookie\nWMF-Uniq\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nloom_anon_comment\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nloom_referral_video\n * Duration\nsession\n * Description\nDescription is currently not available.\n\n\n * Cookie\nVISITOR_INFO1_LIVE\n * Duration\n6 months\n * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n * Cookie\nyt-remote-connected-devices\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n * Duration\nNever Expires\n * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n * Cookie\nyt-remote-device-id\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-name\n * Duration\nsession\n * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\nyt-remote-fast-check-period\n * Duration\nsession\n * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-app\n * Duration\nsession\n * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n * Cookie\nyt-remote-cast-available\n * Duration\nsession\n * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n * Cookie\nyt-remote-cast-installed\n * Duration\nsession\n * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\ncp_session\n * Duration\n3 months\n * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n * Cookie\nloid\n * Duration\n1 year 1 month 4 days\n * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n * Cookie\n__hstc\n * Duration\n6 months\n * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n * Cookie\nhubspotutk\n * Duration\n6 months\n * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n * Cookie\n_ga\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n * Cookie\n_ga_*\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n * Cookie\n__Host-psifi.analyticsTrace\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n__Host-psifi.analyticsTraceV2\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n_gh_sess\n * Duration\nsession\n * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n * Cookie\nYSC\n * Duration\nsession\n * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n * Cookie\najs_anonymous_id\n * Duration\n1 year\n * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n * Cookie\nvuid\n * Duration\n1 year 1 month 4 days\n * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n * Cookie\nAWSALB\n * Duration\n7 days\n * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n * Cookie\nacq\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nacq.sig\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nptc\n * Duration\n2 years\n * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n * Cookie\nmuc_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n * Cookie\nguest_id_marketing\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\nguest_id_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\npersonalization_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n * Cookie\nguest_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n * Cookie\nbcookie\n * Duration\n1 year\n * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n * Cookie\n__Secure-ROLLOUT_TOKEN\n * Duration\n6 months\n * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n * Cookie\nyt.innertube::nextId\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nyt.innertube::requests\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nsession_tracker\n * Duration\nsession\n * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n * Cookie\nedgebucket\n * Duration\nsession\n * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n * Cookie\ndid\n * Duration\n1 year\n * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Data Science](https://towardsdatascience.com/category/data-science/)\n# The Math Behind KAN - Kolmogorov-Arnold Networks\nA new alternative to the classic Multi-Layer Perceptron is out. Why is it more accurate and interpretable? Math and Code Deep Dive.\n[Cristian Leo](https://towardsdatascience.com/author/cristianleo120/)\nJune 12, 202413 min read\n![Image generated by DALL-E](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/0HO-vPqDcT_AnH8Gw.jpg)\nImage generated by DALL-E\nIn today's world of AI, neural networks drive countless innovations and advancements. At the heart of many breakthroughs is the Multi-Layer Perceptron (MLP), a type of neural network known for its ability to approximate complex functions. But as we push the boundaries of what AI can achieve, we must ask: Can we do better than the classic MLP?\nHere's Kolmogorov-Arnold Networks (KANs), a new approach to neural networks inspired by the Kolmogorov-Arnold representation theorem. Unlike traditional MLPs, which use fixed activation functions at each neuron, KANs use learnable activation functions on the edges (weights) of the network. This simple shift opens up new possibilities in accuracy, interpretability, and efficiency.\nThis article explores why KANs are a revolutionary advancement in neural network design. We'll dive into their mathematical foundations, highlight the key differences from MLPs, and show how KANs can outperform traditional methods.\nLearn this step by step with the interactive [AI and Data Scientist](https://roadmap.sh/ai-data-scientist), [Computer Science](https://roadmap.sh/computer-science) and [Machine Learning](https://roadmap.sh/machine-learning) roadmaps.\n## 1: Limitations of MLPs\n![MLP with input layer with 3 nodes, 2 hidden layers with 10 nodes each, and an output layer with 1 node - Image by Author](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1UKIOBnw4N5EVG9AA9oIYMw-2.png)\nMLP with input layer with 3 nodes, 2 hidden layers with 10 nodes each, and an output layer with 1 node - Image by Author\nMulti-Layer Perceptrons (MLPs) are a core component of modern neural networks. They consist of layers of interconnected nodes, or \"neurons,\" designed to approximate complex, non-linear functions by learning from data. Each neuron uses a fixed activation function on the weighted sum of its inputs, transforming input data into the desired output through multiple layers of abstraction. MLPs have driven breakthroughs in various fields, from computer vision to speech recognition.\nHowever, MLPs have some significant limitations:\n 1. **Fixed Activation Functions on Nodes** : Each node in an MLP has a predetermined activation function, like ReLU or Sigmoid. While effective in many cases, these fixed functions limit the network's flexibility and adaptability. This can make it challenging for MLPs to optimize certain types of functions or adapt to specific data characteristics.\n 2. **Interpretability Issues** : MLPs are often criticized for being \"black boxes.\" As they become more complex, understanding their decision-making process becomes harder. The fixed activation functions and intricate weight matrices obscure the network's inner workings, making it difficult to interpret and trust the model's predictions without extensive analysis.\n\n\nThese drawbacks highlight the need for alternatives that offer greater flexibility and interpretability, paving the way for innovations like Kolmogorov-Arnold Networks (KANs).\n## 2: Kolmogorov-Arnold Networks (KANs)\n![MLP vs KAN comparison -Imake by Kan Github Repo \\(License\\)](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1imne0qCLt5xlJUnsn3m9pA-2.png)\nMLP vs KAN comparison -Imake by [Kan Github Repo](https://github.com/KindXiaoming/pykan/tree/master?tab=readme-ov-file) ([License](https://creativecommons.org/licenses/by/4.0/))\nThe Kolmogorov-Arnold representation theorem, formulated by mathematicians Andrey Kolmogorov and Vladimir Arnold, states that any multivariate continuous function can be represented as a finite composition of continuous functions of a single variable and the operation of addition. Think of this theorem as breaking down a complex recipe into individual, simple steps that anyone can follow. Instead of dealing with the entire recipe at once, you handle each step separately, making the overall process more manageable. This theorem implies that complex, high-dimensional functions can be broken down into simpler, univariate functions.\n> For neural networks, this insight is revolutionary, it suggests that a network could be designed to learn these univariate functions and their compositions, potentially improving both accuracy and interpretability.\nKANs leverage the power of the Kolmogorov-Arnold theorem by fundamentally altering the structure of neural networks. Unlike traditional MLPs, where fixed activation functions are applied at each node, KANs place learnable activation functions on the edges (weights) of the network. This key difference means that instead of having a static set of activation functions, KANs adaptively learn the best functions to apply during training. Each edge in a KAN represents a univariate function parameterized as a spline, allowing for dynamic and fine-grained adjustments based on the data.\nThis change enhances the network's flexibility and ability to capture complex patterns in data, providing a more interpretable and powerful alternative to traditional MLPs. By focusing on learnable activation functions on edges, KANs effectively utilize the Kolmogorov-Arnold theorem to transform neural network design, leading to improved performance in various AI tasks.\n## 3: Mathematical Foundations\nAt the core of Kolmogorov-Arnold Networks (KANs) is a set of equations that define how these networks process and transform input data. The foundation of KANs lies in the Kolmogorov-Arnold representation theorem, which inspires the structure and learning process of the network.\nImagine you have an input vector x=[x1,x2,...,xn], which represents data points that you want to process. Think of this input vector as a list of ingredients for a recipe.\nThe theorem states that any complex recipe (high-dimensional function) can be broken down into simpler steps (univariate functions). For KANs, each ingredient (input value) is transformed through a series of simple steps (univariate functions) placed on the edges of the network. Mathematically, this can be represented as:\n![KAN Formula - Image by Author](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1cYgPygULLe1NyEItbkVvjg-2.png)\nKAN Formula - Image by Author\nHere, ϕ_q,p​ are univariate functions that are learned during training. Think of ϕ_q,p​ as individual cooking techniques for each ingredient, and Φ_q as the final assembly step that combines these prepared ingredients.\nEach layer of a KAN applies these cooking techniques to transform the ingredients further. For layer l, the transformation is given by:\n![Kan Layer Transformation Formula - Image by Author](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1rfucBSYhBXFSDeaL3jly8Q-2.png)\nKan Layer Transformation Formula - Image by Author\nHere, x(l) denotes the transformed ingredients at layer l, and ϕ_l,i,j are the learnable univariate functions on the edges between layer l and l+1. Think of this as applying different cooking techniques to the ingredients at each step to get intermediate dishes.\nThe output of a KAN is a composition of these layer transformations. Just as you would combine intermediate dishes to create a final meal, KANs combine the transformations to produce the final output:\n![KAN Output Formula - Image by Author](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1tsGnFbXCZeyRdwYgz-UYrw-2.png)\nKAN Output Formula - Image by Author\nHere, Φl represents the matrix of univariate functions at layer l. The overall function of the KAN is a composition of these layers, each refining the transformation further.\n**MLPs Structure** In traditional MLPs, each node applies a fixed activation function (like ReLU or sigmoid) to its inputs. Think of this as using the same cooking technique for all ingredients, regardless of their nature.\nMLPs use linear transformations followed by these fixed non-linear activations:\n![MLP Formula - Image by Author](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1NJlIC_SSaSUDTr0cRHyzEQ-2.png)\nMLP Formula - Image by Author\nwhere W represents the weight matrices, and σ represents the fixed activation functions.\n### Grid Extension Technique\n![Left: Notations of activations that flow through the network. Right: an activation function is parameterized as a B-spline, which allows switching between coarse-grained and fine-grained grids - Imagine extracted by "KAN: Kolmogorov-Arnold Networks" \\(License\\)](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1rI6hR46zEh_ykjWPrGsAlQ-2.png)\nLeft: Notations of activations that flow through the network. Right: an activation function is parameterized as a B-spline, which allows switching between coarse-grained and fine-grained grids - Imagine extracted by \"[KAN: Kolmogorov-Arnold Networks](https://arxiv.org/abs/2404.19756)\" ([License](https://creativecommons.org/licenses/by/4.0/))\nGrid extension is a powerful technique used to improve the accuracy of Kolmogorov-Arnold Networks (KANs) by refining the spline grids on which the univariate functions are defined. This process allows the network to learn increasingly detailed patterns in the data without requiring complete retraining.\nThese B-splines are a series of polynomial functions that are pieced together to form a smooth curve. They are used in KANs to represent the univariate functions on the edges. The spline is defined over a series of intervals called grid points. The more grid points there are, the finer the detail that the spline can capture\n![KAN with 2D input, 1 hidden layer with 5 nodes, and an output layer with 1 node - Imagine extracted by "KAN: Kolmogorov-Arnold Networks" \\(License\\)](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1GGUGmQiJ3a5JfKkXIe4tMQ-2.png)\nKAN with 2D input, 1 hidden layer with 5 nodes, and an output layer with 1 node - Imagine extracted by \"[KAN: Kolmogorov-Arnold Networks](https://arxiv.org/abs/2404.19756)\" ([License](https://creativecommons.org/licenses/by/4.0/))\nInitially, the network starts with a coarse grid, which means there are fewer intervals between grid points. This allows the network to learn the basic structure of the data without getting bogged down in details. Think of this like sketching a rough outline before filling in the fine details.\nAs training progresses, the number of grid points is gradually increased. This process is known as grid refinement. By adding more grid points, the spline becomes more detailed and can capture finer patterns in the data. This is similar to progressively adding more detail to your initial sketch, turning it into a detailed drawing.\nEach increase introduces new B-spline basis functions B′_m​(x). The coefficients c'_m for these new basis functions are adjusted to ensure that the new, finer spline closely matches the original, coarser spline.\nTo achieve this match, least squares optimization is used. This method adjusts the coefficients c'_m​ to minimize the difference between the original spline and the refined spline.\n![Least Square Optimization Formula - Image by Author](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1TVPgLdsD8EpwX-h2p95ETw-2.png)\nLeast Square Optimization Formula - Image by Author\nEssentially, this process ensures that the refined spline continues to accurately represent the data patterns learned by the coarse spline.\n### Simplification Techniques\nTo enhance the interpretability of KANs, several simplification techniques can be employed, making the network easier to understand and visualize.\n**Sparsification and Pruning** This technique involves adding a penalty to the loss function based on the L1 norm of the activation functions. The L1 norm for a function ϕ is defined as the average magnitude of the function across all input samples:\n![L1 Norm Formula - Image by Author](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/13k9nBq-k4b9AuBZT59mw5g-2.png)\nL1 Norm Formula - Image by Author\nHere, N_p​ is the number of input samples, and ϕ(x_s​) represents the value of the function ϕ for the input sample x_s​.\nThink of sparsification like decluttering a room. By removing unnecessary items (or reducing the influence of less important functions), you make the space (or network) more organized and easier to navigate.\nAfter applying L1 regularization, the L1 norms of the activation functions are evaluated. Neurons and edges with norms below a certain threshold are considered insignificant and are pruned away. The threshold for pruning is a hyperparameter that determines how aggressive the pruning should be.\nPruning is like trimming a tree. By cutting away the weak and unnecessary branches, you allow the tree to focus its resources on the stronger, more vital parts, leading to a healthier and more manageable structure.\n**Symbolification** Another approach is to replace learned univariate functions with known symbolic forms to make the network more interpretable.\nThe task is to identify potential symbolic forms (e.g., sin⁡, exp) that can approximate the learned functions. This step involves analyzing the learned functions and suggesting symbolic candidates based on their shape and behavior.\nOnce symbolic candidates are identified, use grid search and linear regression to fit parameters such that the symbolic function closely approximates the learned function.\n## 4: KAN vs MLP in Python\nTo demonstrate the capabilities of Kolmogorov-Arnold Networks (KANs) compared to traditional Multi-Layer Perceptrons (MLPs), we will fit a function-generated dataset to both a KAN model and MLP model (leveraging PyTorch), to see what their performances look like.\nThe function we will be using is the same one used by the authors of the paper to show KAN's capabilities vs MLP ([Original paper example](https://github.com/KindXiaoming/pykan/blob/master/tutorials/Example_1_function_fitting.ipynb)). However, the code will be different. You can find all the code we will cover today in this Notebook:\n> [**models-from-scratch-python/KAN - Kolmogorov-Arnold Networks/demo.ipynb at main ·...**](https://github.com/cristianleoo/models-from-scratch-python/blob/main/KAN%20-%20Kolmogorov-Arnold%20Networks/demo.ipynb)\nLet's import the required libraries, and generate the dataset\njavascript\n\n```\nimport numpy as np\nimport torch\nimport torch.nn as nn\nfrom torchsummary import summary\nfrom kan import KAN, create_dataset\nimport matplotlib.pyplot as plt\n```\n\nHere, we use:\n * `numpy`: For numerical operations.\n * `torch`: For PyTorch, which is used for building and training neural networks.\n * `torch.nn`: For neural network modules in PyTorch.\n * `torchsummary`: For summarizing the model structure.\n * `kan`: Custom library containing the KAN model and dataset creation functions.\n * `matplotlib.pyplot`: For plotting and visualizations.\n\n\npython\n\n```\n# Define the dataset generation function\nf = lambda x: torch.exp(torch.sin(torch.pi * x[:, [0]]) + x[:, [1]] ** 2)\n```\n\nThis function includes both sinusoidal (`sin`) and exponential (`exp`) components. It takes a 2D input `x` and computes the output using the formula:\n![Dataset generation Function - Image by Author](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1ZdXj_WtOck-XWcJFm_h53Q-2.png)\nDataset generation Function - Image by Author\nLet's now fit a tensor of 100 points uniformly distributed between [-2, 2] to this function, to see what it looks like:\n![Plot generated by fitting 100 points uniformly distributed between \\[-2, 2\\] to Dataset generation function - Image by Author](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1ap4SLcVIX_cqpdHsEMfjTA-2.png)\nPlot generated by fitting 100 points uniformly distributed between [-2, 2] to Dataset generation function - Image by Author\npython\n\n```\n# Create the dataset\ndataset = create_dataset(f, n_var=2)\n```\n\n`create_dataset` generates a dataset based on the function `f`. The dataset includes input-output pairs that will be used for training and testing the neural networks.\nNow let's build a KAN model and train it on the dataset. We will start with a coarse grid (5 points) and gradually refine it (up to 100 points). This improves the model's accuracy by capturing finer details in the data.\npython\n\n```\ngrids = np.array([5, 10, 20, 50, 100])\ntrain_losses_kan = []\ntest_losses_kan = []\nsteps = 50\nk = 3\n\nfor i in range(grids.shape[0]):\n if i == 0:\n model = KAN(width=[2, 1, 1], grid=grids[i], k=k)\n else:\n model = KAN(width=[2, 1, 1], grid=grids[i], k=k).initialize_from_another_model(model, dataset['train_input'])\n results = model.train(dataset, opt=\"LBFGS\", steps=steps, stop_grid_update_step=30)\n train_losses_kan += results['train_loss']\n test_losses_kan += results['test_loss']\n\n print(f\"Train RMSE: {results['train_loss'][-1]:.8f} | Test RMSE: {results['test_loss'][-1]:.8f}\")\n```\n\nIn this example, we define an array called `grids` with values [5, 10, 20, 50, 100]. We iterate over these grids to fit models sequentially, meaning each new model is initialized using the previous one.\nFor each iteration, we define a model with k=3, where k is the order of the B-spline. We set the number of training steps (or epochs) to 50. The model's architecture consists of an input layer with 2 nodes, one hidden layer with 1 node, and an output layer with 1 node. We use the LFGBS optimizer for training.\nHere are the training and test losses during the training process:\n![KAN Train and Test Losses Plot - Image by Author](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1F4Ykngls_DGLdzfJH2Ap_Q-2.png)\nKAN Train and Test Losses Plot - Image by Author\nLet's now define and train a traditional MLP for comparison.\npython\n\n```\n# Define the MLP\nclass MLP(nn.Module):\n def __init__(self):\n super(MLP, self).__init__()\n self.layers = nn.Sequential(\n nn.Linear(dataset['train_input'].shape[1], 64),\n nn.ReLU(),\n nn.Linear(64, 64),\n nn.ReLU(),\n nn.Linear(64, 1)\n )\n def forward(self, x):\n return self.layers(x)\n\n# Instantiate the model\nmodel = MLP()\nsummary(model, input_size=(dataset['train_input'].shape[1],))\n```\n\nThe MLP has an input layer, two hidden layers with 64 neurons each, and an output layer. ReLU activation is used between the layers.\npython\n\n```\ncriterion = nn.MSELoss()\noptimizer = torch.optim.Adam(model.parameters(), lr=1e-2)\ntrain_loss_mlp = []\ntest_loss_mlp = []\n\nepochs = 250\nfor epoch in range(epochs):\n optimizer.zero_grad()\n output = model(dataset['train_input']).squeeze()\n loss = criterion(output, dataset['train_label'])\n loss.backward()\n optimizer.step()\n train_loss_mlp.append(loss.item()**0.5)\n\n # Test the model\n model.eval()\n with torch.no_grad():\n output = model(dataset['test_input']).squeeze()\n loss = criterion(output, dataset['test_label'])\n test_loss_mlp.append(loss.item()**0.5)\n\n print(f'Epoch {epoch+1}/{epochs}, Train Loss: {train_loss_mlp[-1]:.2f}, Test Loss: {test_loss_mlp[-1]:.2f}', end='r')\n```\n\nWe use mean squared error (MSE) loss and Adam optimizer, and train the model for 250 epochs, recording the training and testing losses.\nThis is what the train and test RMSE look like in MLP:\n![MLP Train and Test Losses Plot - Image by Author](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1uMCeVXYe6QRdYwbc1Lco_A-2.png)\nMLP Train and Test Losses Plot - Image by Author\nLet's put side to side the loss plots for a comparison:\n![KAN vs MLP training losses plot \\(left\\) and test losses plot \\(right\\) - Image by Author](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1-d00odoBn71sgEATWPZ06A-1.png)\nKAN vs MLP training losses plot (left) and test losses plot (right) - Image by Author\nThe plot shows that the KAN model achieves lower training RMSE than the MLP model, indicating better function-fitting capability. Similarly, the KAN model outperforms the MLP on the test set, demonstrating its superior generalization ability.\nThis example illustrates how KANs can more accurately fit complex functions than traditional MLPs, thanks to their flexible and adaptive structure. By refining the grid and employing learnable univariate functions on the edges, KANs capture intricate patterns in the data that MLPs may miss, leading to improved performance in function-fitting tasks.\nDoes this mean we should switch to KAN models permanently? Not necessarily.\nKANs showed great results in this example, but when I tested them on other scenarios with real data, MLPs often performed better. One thing you'll notice when working with KAN models is their sensitivity to hyperparameter optimization. Also, KANs have primarily been tested using spline functions, which work well for smoothly varying data like our example but might not perform as well in other situations.\nIn summary, KANs are definitely intriguing and have a lot of potential, but they need more study, especially regarding different datasets and the algorithm's inner workings, to really make them work effectively.\n## 5: Advantages of KANs\n### Accuracy\nOne of the standout advantages of Kolmogorov-Arnold Networks (KANs) is their ability to achieve higher accuracy with fewer parameters compared to traditional Multi-Layer Perceptrons (MLPs). This is primarily due to the learnable activation functions on the edges, which allow KANs to better capture complex patterns and relationships in the data.\nUnlike MLPs that use fixed activation functions at each node, KANs use univariate functions on the edges, making the network more flexible and capable of fine-tuning its learning process to the data.\nBecause KANs can adjust the functions between layers dynamically, they can achieve comparable or even superior accuracy with a smaller number of parameters. This efficiency is particularly beneficial for tasks with limited data or computational resources.\n### Interpretability\nKANs offer significant improvements in interpretability over traditional MLPs. This enhanced interpretability is crucial for applications where understanding the decision-making process is as important as the outcome.\nKANs can be simplified through techniques like sparsification and pruning, which remove unnecessary functions and parameters. These techniques not only improve interpretability but also enhance the network's performance by focusing on the most relevant components.\nFor some functions, it is possible to identify symbolic forms of the activation functions, making it easier to understand the mathematical transformations within the network.\n### Scalability\nKANs exhibit faster neural scaling laws compared to MLPs, meaning they improve more rapidly as the number of parameters increases.\nKANs benefit from more favorable scaling laws due to their ability to decompose complex functions into simpler, univariate functions. This allows them to achieve lower error rates with increasing model complexity more efficiently than MLPs.\nKANs can start with a coarser grid and extend it to finer grids during training, which helps in balancing computational efficiency and accuracy. This approach allows KANs to scale up more gracefully than MLPs, which often require complete retraining when increasing model size.\n## Conclusion\nKolmogorov-Arnold Networks (KANs) present a groundbreaking alternative to traditional Multi-Layer Perceptrons (MLPs), offering several key innovations that address the limitations of their predecessors. By leveraging learnable activation functions on the edges rather than fixed functions at the nodes, KANs introduce a new level of flexibility and adaptability. This structural change leads to:\n * **Enhanced Accuracy** : KANs achieve higher accuracy with fewer parameters, making them more efficient and effective for a wide range of tasks.\n * **Improved Interpretability** : The ability to visualize and simplify KANs aids in understanding the decision-making process, which is crucial for critical applications in healthcare, finance, and autonomous systems.\n * **Better Scalability** : KANs exhibit faster neural scaling laws, allowing them to handle increasing complexity more gracefully than MLPs.\n\n\nThe introduction of Kolmogorov-Arnold Networks marks an exciting development in the field of neural networks, opening up new possibilities for AI and machine learning.\n## References\n * Ziming Liu, et al**.** \"KAN: Kolmogorov-Arnold Networks\". \n\n\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](https://assets.insightmediagroup.io/media/1787657505387_wlyc0o.jpeg)\nWritten By\n## [Cristian Leo](https://towardsdatascience.com/author/cristianleo120/)\n[See all from Cristian Leo](https://towardsdatascience.com/author/cristianleo120/)\n[Data Science](https://towardsdatascience.com/tag/data-science/)[Machine Learning](https://towardsdatascience.com/tag/machine-learning/)[Mathematics](https://towardsdatascience.com/tag/mathematics/)[Programming](https://towardsdatascience.com/tag/programming/)[Thoughts And Theory](https://towardsdatascience.com/tag/thoughts-and-theory/)\nShare This Article\n * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fthe-math-behind-kan-kolmogorov-arnold-networks-7c12a164ba95%2F&title=The%20Math%20Behind%20KAN%20-%20Kolmogorov-Arnold%20Networks)\n * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fthe-math-behind-kan-kolmogorov-arnold-networks-7c12a164ba95%2F&title=The%20Math%20Behind%20KAN%20-%20Kolmogorov-Arnold%20Networks)\n * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fthe-math-behind-kan-kolmogorov-arnold-networks-7c12a164ba95%2F&text=The%20Math%20Behind%20KAN%20-%20Kolmogorov-Arnold%20Networks)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n * [![](https://towardsdatascience.com/images/fallback-image.jpg)](https://towardsdatascience.com/reinforcement-learning-with-sarsa-a-good-alternative-to-q-learning-algorithm-bf35b209e1c/)\n## [Reinforcement Learning with SARSA - A Good Alternative to Q-Learning Algorithm](https://towardsdatascience.com/reinforcement-learning-with-sarsa-a-good-alternative-to-q-learning-algorithm-bf35b209e1c/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nHow to teach an intelligent agent to play simple games using State-Action-Reward-State-Action (SARSA) algorithm\n[Saul Dobilas](https://towardsdatascience.com/author/solclover/)\nOctober 20, 202213 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/08/1wGP3m6o584HaUhzDplB7nw.png)](https://towardsdatascience.com/from-outdated-to-updated-2df4b43fcc38/)\n## [From Outdated to Updated](https://towardsdatascience.com/from-outdated-to-updated-2df4b43fcc38/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nHow to Create a Searchable Archive of PDFs with OCR, Python, & SQLite\n[Brandon Cosley](https://towardsdatascience.com/author/brandoncosley/)\nAugust 10, 20219 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/12/1tvOJmGHoli8iUDnqQfiDLQ-scaled.jpeg)](https://towardsdatascience.com/4-different-ways-of-creating-a-new-column-with-pyspark-1a6782fe764e/)\n## [4 Different Ways of Creating a New Column with PySpark](https://towardsdatascience.com/4-different-ways-of-creating-a-new-column-with-pyspark-1a6782fe764e/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nLearn how to create new columns in Spark data frames\n[Soner Yıldırım](https://towardsdatascience.com/author/sonery/)\nDecember 20, 20214 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/07/1vvferwlg__wkUAFC9o46zQ-scaled.jpeg)](https://towardsdatascience.com/data-democratisation-5-data-for-all-strategies-embraced-by-large-companies-e62fad52c250/)\n## [Data Democratisation: 5 'Data For All' Strategies Embraced by Large Companies](https://towardsdatascience.com/data-democratisation-5-data-for-all-strategies-embraced-by-large-companies-e62fad52c250/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nIt's about Data, Skills, Tools...and especially Culture!\n[Col Jung](https://towardsdatascience.com/author/col-jung/)\nJuly 19, 202316 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/11/00bE9hCODMumMbjUW-scaled.jpg)](https://towardsdatascience.com/boosting-algorithms-in-machine-learning-part-ii-gradient-boosting-c155ae505fe9/)\n## [Boosting Algorithms in Machine Learning, Part II: Gradient Boosting](https://towardsdatascience.com/boosting-algorithms-in-machine-learning-part-ii-gradient-boosting-c155ae505fe9/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nUncovering a simple yet powerful, award-winning machine learning algorithm\n[Gurjinder Kaur](https://towardsdatascience.com/author/gurjinderkaur95/)\nNovember 12, 20249 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2022/06/0TpQGt7MM_F5dFy6z-scaled.jpg)](https://towardsdatascience.com/linear-regression-what-is-the-sum-of-squares-3746db90a05d/)\n## [Linear Regression: What is the Sum of Squares?](https://towardsdatascience.com/linear-regression-what-is-the-sum-of-squares-3746db90a05d/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nA closer look at understanding why the Sum of Squares is important to Data Science\n[Mike Wolfe](https://towardsdatascience.com/author/miketechgame/)\nJune 14, 20225 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1TrPePlP9A9sm9o9vtNi5xw-scaled.jpeg)](https://towardsdatascience.com/a-hands-on-guide-to-sorting-dataframes-in-pandas-384996ca6bb8/)\n## [A hands-on guide to 'sorting' dataframes in Pandas](https://towardsdatascience.com/a-hands-on-guide-to-sorting-dataframes-in-pandas-384996ca6bb8/)\n[Programming](https://towardsdatascience.com/category/programming/)\nTen easy ways to sort data in pandas\n[Parul Pandey](https://towardsdatascience.com/author/pandeyparul/)\nMarch 22, 20217 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2019/07/1i-35MgrP_pUA7bLVFzH6kQ.png)](https://towardsdatascience.com/te-papa-tongarewa-museum-of-new-zealand-sentimental-footing-d062e41e430f/)\n## [Te Papa Tongarewa (Museum of New Zealand) Sentimental Footing](https://towardsdatascience.com/te-papa-tongarewa-museum-of-new-zealand-sentimental-footing-d062e41e430f/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nI paid a visit to this museum about four months ago on a tour of Wellington, the windiest capital of the world. The Maori name, 'Te Papa...\n[Herman Wandabwa](https://towardsdatascience.com/author/hermanwandabwa/)\nJuly 22, 20192 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/08/1H4S_zOCfcrVNuGYyG2JfCQ.jpeg)](https://towardsdatascience.com/bad-assumptions-the-downfall-of-even-experienced-data-scientists-8fe00726f849/)\n## [Bad Assumptions - The Downfall of Even Experienced Data Scientists](https://towardsdatascience.com/bad-assumptions-the-downfall-of-even-experienced-data-scientists-8fe00726f849/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nData can be deceptive, so be on your toes!\n[Michael Clayton](https://towardsdatascience.com/author/maclayton/)\nAugust 16, 202410 min read\n\n\n * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n * [X](https://x.com/TDataScience)\n * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n * [Threads](https://www.threads.net/@towardsdatascience)\n * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n#### Recommended Articles \nClose\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500) How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ \"How Relevance Models Foreshadowed Transformers for NLP\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n\n\n", "body_base_path": "attachments/8f333daf7214/pages/", "body_markdown_path": "attachments/8f333daf7214/pages/archived-body.md", "local_html": "attachments/8f333daf7214/pages/archived-body.html", "body_source": "prior_capture" }