{
  "source_file": "0930",
  "source_platform": "Hashnode",
  "author": "Wesley Kambale",
  "title_listed": "Machine Learning in SQL using BigQuery",
  "keywords": "",
  "domain": "数据处理",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://kambale.dev/ml-in-sql",
  "publish_date": "2023-05-09",
  "corpus_stage": "",
  "notes": "",
  "id": "93b0b0289002",
  "collected_at": "2026-10-02T14:11:38",
  "title": "Train ML Model in SQL with BigQuery",
  "body_markdown": "Machine Learning in SQL using BigQuery\nA beginner's guide to training machine learning models with SQL using Google's BigQuery ML\nUpdated\nJuly 18, 2023\n•\n4\nmin read\n•\nView as Markdown\nW\nWesley Kambale\nI'm a Machine Learning Engineer passionate about building production-ready ML systems for the African market.\nWith experience in TensorFlow, Keras, and Python-based workflows, I help teams bridge the gap between machine learning research and real-world deployment—especially on resource-constrained devices.\nI'm also a Google Developer Expert in AI. I regularly speak at tech conferences including PyCon Africa, DevFest Kampala, DevFest Nairobi and more and also write technical articles on AI/ML here.\nPart of series\nMachine Learning\nMachine learning is a rapidly growing field that has seen a lot of interest from businesses and organizations. With the growing amount of data that businesses collect, it has become essential to use machine learning to extract insights from that data. One of the best tools for this is BigQuery, a fully managed, cloud-native data warehouse that makes it easy to analyze large amounts of data quickly. In this article, we will explore how to perform machine learning with SQL using BigQuery.\nBut before we start, it is essential to note that machine learning with SQL is not the same as traditional machine learning methods such as regression, clustering, or classification. Instead, it focuses on using SQL to analyze and transform data into the format required for machine learning. With that said, let's get started.\nSetting up BigQuery\nBefore we start with machine learning, we need to set up BigQuery. If you haven't done this before, you can follow these simple steps:\nGo to the\nBigQuery console\nand sign up for a Google Cloud account if you haven't already.\nCreate a new project or select an existing one.\nClick on \"Create Dataset\" to create a new dataset to store your data.\nUpload your data to the dataset using the \"Create Table\" button.\nExplore the Data\nOnce you have uploaded your data, it is essential to understand it. In this tutorial, we will use the\nIris dataset\n, which is a popular dataset for classification tasks. It contains 150 instances of three classes, with each class having 50 instances.\nTo explore the data, we can use SQL queries to select and visualize the data. The following code snippet shows how to select the first five rows of the dataset:\nSELECT\n*\nFROM\n`<project_id>.<dataset_id>.<table-name>`\nLIMIT\n5\nThis will display the first five rows of the dataset. We can use the same query to select specific columns, such as the sepal length and sepal width:\nSELECT\nsepal_length, sepal_width\nFROM\n`<project_id>.<dataset_id>.iris`\nLIMIT\n5\nWe can also use SQL to calculate summary statistics of the data, such as the mean and standard deviation:\nSELECT\nAVG\n(sepal_length)\nAS\navg_sepal_length,\nSTDDEV\n(sepal_length)\nAS\nstddev_sepal_length\nFROM\n`<project_id>.<dataset_id>.iris`\nTransform the Data\nBefore we can perform machine learning on the data, we need to transform it into the required format. In the case of the Iris dataset, we need to convert the categorical target variable into a numerical value. We can do this using a SQL CASE statement:\nSELECT\nCASE\nclass\nWHEN\n'Iris-setosa'\nTHEN\n1\nWHEN\n'Iris-versicolor'\nTHEN\n2\nWHEN\n'Iris-virginica'\nTHEN\n3\nEND\nAS\ntarget,\n  sepal_length,\n  sepal_width,\n  petal_length,\n  petal_width\nFROM\n`<project_id>.<dataset_id>.iris`\nThis query will create a new column called\ntarget\nwith the numerical value of the\nclass\n.\nTrain a Machine Learning Model\nNow that we have transformed the data, we can train a machine learning model using SQL. In BigQuery, we can use the\nCREATE MODEL\nstatement to create a model. In this tutorial, we will use logistic regression to classify the Iris dataset\nTo train a logistic regression model, we can use the following SQL query:\nCREATE\nMODEL\n`<project_id>.<dataset_id>.iris_model`\nOPTIONS\n  (model_type=\n'logistic_reg'\n,\n  input_label_cols=[\n'target'\n],\n   max_iteration=\n50\n,\n   l1_reg=\n1\n,\n   l2_reg=\n0.1\n)\nAS\nSELECT\nCASE\nclass\nWHEN\n'Iris-setosa'\nTHEN\n1\nWHEN\n'Iris-versicolor'\nTHEN\n2\nWHEN\n'Iris-virginica'\nTHEN\n3\nEND\nAS\ntarget,\n  sepal_length,\n  sepal_width,\n  petal_length,\n  petal_width\nFROM\n`<project_id>.<dataset_id>.iris`\nThis query will create a new model called \"iris_model\" and train it using logistic regression. We can specify options such as the maximum number of iterations and regularization parameters.\nClick on\nGO TO MODEL\nbutton to see the training details of the model as shown below:\nEvaluate the Model\nOnce the model is trained, we can evaluate its performance using SQL. In BigQuery, we can use the\nML.EVALUATE\nfunction to evaluate the model. The following query shows how to evaluate the model on the training data:\nSELECT\n*\nFROM\nML.EVALUATE(\nMODEL\n`<project_id>.<dataset_id>.iris_model`\n,\n                (\nSELECT\nCASE\nspecies\nWHEN\n'Iris-setosa'\nTHEN\n1\nWHEN\n'Iris-versicolor'\nTHEN\n2\nWHEN\n'Iris-virginica'\nTHEN\n3\nEND\nAS\ntarget,\n                   sepal_length,\n                   sepal_width,\n                   petal_length,\n                   petal_width\nFROM\n`<project_id>.<dataset_id>.iris`\n))\nThis query will return metrics such as accuracy, precision, and recall.\nPrecision - 0.9534\nRecall - 0.9533\nAccuracy - 0.9533\nF1 score - 0.9533\nLog loss - 0.1231\nROC AUC - 0.9990\nMake Predictions\nFinally, we can use the trained model to make predictions on new data. In BigQuery, we can use the\nML.PREDICT\nfunction to make predictions. The following query shows how to make predictions on new data:\nSELECT\npredicted_target,\n  predicted_target_probs\nFROM\nML.PREDICT(\nMODEL\n`<project_id>.<dataset_id>.iris_model`\n,\n                (\nSELECT\nsepal_length,\n                   sepal_width,\n                   petal_length,\n                   petal_width\nFROM\n`<project_id>.<dataset_id>.new_data`\n))\nThis query will return the predicted target value and the probability of each class for each row in the new data.\nConclusion\nIn this article, we have explored how to perform machine learning with SQL using BigQuery. We started by setting up BigQuery and exploring the data using SQL queries. We then transformed the data into the required format, trained a logistic regression model, and evaluated its performance. Finally, we made predictions on new data using the trained model. With BigQuery, it is possible to perform powerful machine learning tasks without leaving the SQL environment.\n#\nbigquery\n#\nmachine-learning\n#\nsql\n#\ndatabases",
  "metadata": {
    "title": "Train ML Model in SQL with BigQuery",
    "description": "A beginner's guide to training machine learning models with SQL using Google's BigQuery ML",
    "keywords": null,
    "author": null,
    "og:title": "Train ML Model in SQL with BigQuery",
    "og:description": "A beginner's guide to training machine learning models with SQL using Google's BigQuery ML",
    "og:url": "https://kambale.dev/ml-in-sql",
    "og:site_name": "Wesley Kambale",
    "og:image": "https://cdn.hashnode.com/res/hashnode/image/upload/v1689689950698/2ff1a31e-4597-4f91-bd72-e4dc5f2e15f6.png",
    "og:image:width": "1200",
    "og:image:height": "630",
    "og:type": "article",
    "twitter:card": "summary_large_image",
    "twitter:creator": "https://twitter.com/WesKambale",
    "twitter:title": "Train ML Model in SQL with BigQuery",
    "twitter:description": "A beginner's guide to training machine learning models with SQL using Google's BigQuery ML",
    "twitter:image": "https://cdn.hashnode.com/res/hashnode/image/upload/v1689689950698/2ff1a31e-4597-4f91-bd72-e4dc5f2e15f6.png",
    "article:published_time": "2023-05-09T14:12:55.576Z",
    "article:modified_time": "2023-07-18T16:44:52.858Z",
    "article:author": "https://kambale.dev/@kambale"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/93b0b0289002/code/body-000.txt",
      "chars": 63,
      "language": "text"
    },
    {
      "file": "attachments/93b0b0289002/code/body-001.txt",
      "chars": 79,
      "language": "text"
    },
    {
      "file": "attachments/93b0b0289002/code/body-002.txt",
      "chars": 132,
      "language": "text"
    },
    {
      "file": "attachments/93b0b0289002/code/body-003.txt",
      "chars": 233,
      "language": "text"
    },
    {
      "file": "attachments/93b0b0289002/code/body-004.txt",
      "chars": 406,
      "language": "text"
    },
    {
      "file": "attachments/93b0b0289002/code/body-005.txt",
      "chars": 533,
      "language": "text"
    },
    {
      "file": "attachments/93b0b0289002/code/body-006.txt",
      "chars": 106,
      "language": "text"
    },
    {
      "file": "attachments/93b0b0289002/code/body-007.txt",
      "chars": 363,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/res/hashnode/image/upload/v1689689950698/2ff1a31e-4597-4f91-bd72-e4dc5f2e15f6.png",
      "bytes": 99431,
      "reused": true,
      "local_path": "attachments/93b0b0289002/images/000.png",
      "src": "https://cdn.hashnode.com/res/hashnode/image/upload/v1689689950698/2ff1a31e-4597-4f91-bd72-e4dc5f2e15f6.png",
      "alt": "Machine Learning in SQL using BigQuery",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/res/hashnode/image/upload/v1683638067551/fbc483ad-c6ed-4f56-8102-ae9bfb43f207.png",
      "bytes": 84619,
      "reused": true,
      "local_path": "attachments/93b0b0289002/images/002.png",
      "src": "https://cdn.hashnode.com/res/hashnode/image/upload/v1683638067551/fbc483ad-c6ed-4f56-8102-ae9bfb43f207.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/res/hashnode/image/upload/v1683638168695/802e84a8-bdae-47b4-916d-882bc0aaca10.png",
      "bytes": 88804,
      "reused": true,
      "local_path": "attachments/93b0b0289002/images/003.png",
      "src": "https://cdn.hashnode.com/res/hashnode/image/upload/v1683638168695/802e84a8-bdae-47b4-916d-882bc0aaca10.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/res/hashnode/image/upload/v1683638293875/b1e290cd-e424-41dd-970d-e5e24f3e521c.png",
      "bytes": 79122,
      "reused": true,
      "local_path": "attachments/93b0b0289002/images/004.png",
      "src": "https://cdn.hashnode.com/res/hashnode/image/upload/v1683638293875/b1e290cd-e424-41dd-970d-e5e24f3e521c.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/res/hashnode/image/upload/v1683638446153/43ee9b1e-d49d-4a64-9de6-347b42240a32.png",
      "bytes": 84327,
      "reused": true,
      "local_path": "attachments/93b0b0289002/images/005.png",
      "src": "https://cdn.hashnode.com/res/hashnode/image/upload/v1683638446153/43ee9b1e-d49d-4a64-9de6-347b42240a32.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/res/hashnode/image/upload/v1683638841668/6c2a0516-80fd-41c2-b4c9-8eeddc07584c.png",
      "bytes": 85318,
      "reused": true,
      "local_path": "attachments/93b0b0289002/images/006.png",
      "src": "https://cdn.hashnode.com/res/hashnode/image/upload/v1683638841668/6c2a0516-80fd-41c2-b4c9-8eeddc07584c.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/res/hashnode/image/upload/v1683640333130/2e8584ed-086a-4b8b-be86-d94fb7c677a6.png",
      "bytes": 87083,
      "reused": true,
      "local_path": "attachments/93b0b0289002/images/007.png",
      "src": "https://cdn.hashnode.com/res/hashnode/image/upload/v1683640333130/2e8584ed-086a-4b8b-be86-d94fb7c677a6.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/res/hashnode/image/upload/v1683640432723/38f4eb5d-def9-46f5-ad8a-4091cc514710.png",
      "bytes": 66850,
      "reused": true,
      "local_path": "attachments/93b0b0289002/images/008.png",
      "src": "https://cdn.hashnode.com/res/hashnode/image/upload/v1683640432723/38f4eb5d-def9-46f5-ad8a-4091cc514710.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/res/hashnode/image/upload/v1683640778845/f9100018-3589-43b3-aa79-6a8e2cf19367.png",
      "bytes": 93741,
      "reused": true,
      "local_path": "attachments/93b0b0289002/images/009.png",
      "src": "https://cdn.hashnode.com/res/hashnode/image/upload/v1683640778845/f9100018-3589-43b3-aa79-6a8e2cf19367.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/res/hashnode/image/upload/v1683641267740/193f81a1-70ff-4f4a-a92f-232b7bd4d8f4.png",
      "bytes": 79015,
      "reused": true,
      "local_path": "attachments/93b0b0289002/images/010.png",
      "src": "https://cdn.hashnode.com/res/hashnode/image/upload/v1683641267740/193f81a1-70ff-4f4a-a92f-232b7bd4d8f4.png",
      "alt": "",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://kambale.dev/ml-in-sql.md",
      "label": "View as Markdown",
      "type": "a"
    },
    {
      "href": "https://hashnode.com/@kambale",
      "label": "W",
      "type": "a"
    },
    {
      "href": "https://twitter.com/WesKambale",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://linkedin.com/in/WesKambale",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://kambale.dev/series/ml-series",
      "label": "Part of series Machine Learning",
      "type": "a"
    },
    {
      "href": "https://console.cloud.google.com/bigquery",
      "label": "BigQuery console",
      "type": "a"
    },
    {
      "href": "https://kambale.dev/tag/bigquery",
      "label": "# bigquery",
      "type": "a"
    },
    {
      "href": "https://kambale.dev/tag/machine-learning",
      "label": "# machine-learning",
      "type": "a"
    },
    {
      "href": "https://kambale.dev/tag/sql",
      "label": "# sql",
      "type": "a"
    },
    {
      "href": "https://kambale.dev/tag/databases",
      "label": "# databases",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:50:52Z",
  "source_url": "https://kambale.dev/ml-in-sql",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 6484,
  "expected_body_images": 10,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/93b0b0289002/pages/source.html",
  "local_html": "attachments/93b0b0289002/pages/article.html",
  "body_base_path": "attachments/93b0b0289002/pages/",
  "body_markdown_path": "attachments/93b0b0289002/pages/article.md"
}