{
  "source_file": "0930",
  "source_platform": "Hashnode",
  "author": "Wesley Kambale",
  "title_listed": "Fine-tuning BERT for text classification with KerasNLP",
  "keywords": "",
  "domain": "大模型算法",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://kambale.dev/fine-tuning-bert",
  "publish_date": "2024-05-11",
  "corpus_stage": "",
  "notes": "",
  "id": "1b6bc502fb50",
  "collected_at": "2026-10-02T14:11:26",
  "title": "Fine-tuning BERT for text classification with KerasNLP",
  "body_markdown": "Fine-tuning BERT for text classification with KerasNLP\nA guide to mastering text classification with BERT and KerasNLP\nPublished\nMay 11, 2024\n•\n5\nmin read\n•\nView as Markdown\nW\nWesley Kambale\nI'm a Machine Learning Engineer passionate about building production-ready ML systems for the African market.\nWith experience in TensorFlow, Keras, and Python-based workflows, I help teams bridge the gap between machine learning research and real-world deployment—especially on resource-constrained devices.\nI'm also a Google Developer Expert in AI. I regularly speak at tech conferences including PyCon Africa, DevFest Kampala, DevFest Nairobi and more and also write technical articles on AI/ML here.\nPart of series\nMachine Learning\nIntroduction\nText classification is a basic job in natural language processing (NLP) that is used in sentiment analysis, spam detection, and content categorization. Transformer-based models, like BERT (Bi-directional Encoder Representations from Transformers), have become popular recently because of their outstanding performance in different NLP tasks.\nIn this article, we'll explore how to implement text classification using BERT and the KerasNLP library, providing examples and code snippets to guide you through the process.\nUnderstanding BERT\nBERT, introduced by Google in 2018, is a pre-trained transformer-based model created for understanding natural language. Unlike traditional models that analyze text in one direction, BERT looks at context from both sides, which helps it capture complex relationships within sentences effectively.\nBERT Architecture\nBERT's architecture consists of layers of attention mechanisms and feedforward neural networks. It employs a transformer encoder stack, allowing it to learn contextualized representations of words. The model is pre-trained on large corpora, gaining a deep understanding of language nuances.\nTokenization with BERT\nBefore delving into text classification, it's crucial to understand tokenization, a process that breaks down text into smaller units, such as words or subwords. BERT utilizes WordPiece tokenization, which divides text into subword tokens, enhancing its ability to handle out-of-vocabulary words.\nSetting Up the Environment\nTo get started with BERT-based text classification, you need to set up your Python environment. Ensure you have the required libraries installed:\npip install tensorflow\npip install keras-nlp\npip install transformers\nThese packages include TensorFlow, KerasNLP, and the Hugging Face Transformers library, which provides pre-trained BERT models.\nImport the libraries\nfrom\nkeras_nlp\nimport\nload_bert_model\nimport\npandas\nas\npd\nfrom\nsklearn.model_selection\nimport\ntrain_test_split\nfrom\nkeras.layers\nimport\nInput, Dense\nfrom\nkeras.models\nimport\nModel\nfrom\nkeras.optimizers\nimport\nAdam\nfrom\nkeras_nlp\nimport\nTokenizer\nfrom\nkeras_nlp\nimport\nload_bert_finetuned_model\nLoading BERT Model with KerasNLP\nKerasNLP simplifies the process of working with BERT models in Keras. Let's load a pre-trained BERT model using KerasNLP:\nmodel_name =\n'bert-base-uncased'\nbert_model = load_bert_model(model_name)\nNote: You can choose other variants based on your requirements, such as multilingual models or models fine-tuned for specific tasks.\nText Classification\nNow, let's move on to text classification using BERT. For this example, we'll create a binary sentiment analysis model. Assume you have a dataset with labeled sentiments (positive or negative). First, load and preprocess the data\ndata = pd.read_csv(\n'sentiment_data.csv'\n)\n\ntrain_data, test_data = train_test_split(data, test_size=\n0.2\n, random_state=\n42\n)\n\ntokenizer = Tokenizer(model_name)\nX_train = tokenizer.tokenize(train_data[\n'text'\n].tolist())\nX_test = tokenizer.tokenize(test_data[\n'text'\n].tolist())\n\ny_train = train_data[\n'sentiment'\n].map({\n'negative'\n:\n0\n,\n'positive'\n:\n1\n}).values\ny_test = test_data[\n'sentiment'\n].map({\n'negative'\n:\n0\n,\n'positive'\n:\n1\n}).values\nIn this example, we assume that your dataset has a 'text' column containing the text data and a 'sentiment' column with labels ('negative' or 'positive'). Adjust the column names based on your dataset structure.\nBuilding the BERT Text Classification Model\nNow, let's build the BERT-based text classification model using Keras:\ninput_layer = Input(shape=(tokenizer.max_seq_length,), dtype=\n'int32'\n)\n\nbert_output = bert_model(input_layer)\n\noutput_layer = Dense(\n1\n, activation=\n'sigmoid'\n)(bert_output[\n'pooled_output'\n])\n\nmodel = Model(inputs=input_layer, outputs=output_layer)\n\nmodel.compile(optimizer=Adam(learning_rate=\n2e-5\n), loss=\n'binary_crossentropy'\n, metrics=[\n'accuracy'\n])\nThis code snippet creates a simple neural network for sentiment analysis. The BERT output is fed into a dense layer with a sigmoid activation function for binary classification. Adjust the architecture based on your specific task and requirements.\nTraining the BERT Text Classification Model\nNow, let's train the BERT text classification model using the prepared data:\nmodel.fit(X_train, y_train, epochs=\n3\n, batch_size=\n32\n, validation_split=\n0.1\n)\n\nloss, accuracy = model.evaluate(X_test, y_test)\nprint(\nf'Test Loss:\n{loss}\n, Test Accuracy:\n{accuracy}\n'\n)\nThis code snippet trains the model for three epochs with a batch size of 32 and validates on a 10% subset of the training data. After training, it evaluates the model on the test set, providing insights into its performance.\nFine-Tuning BERT for Specific Tasks\nWhile the above example demonstrates a basic BERT text classification model, fine-tuning allows you to adapt BERT to specific tasks or domains. Fine-tuning involves training the pre-trained BERT model on a task-specific dataset, enabling it to learn task-specific features.\nLoading a Fine-Tuned BERT Model\nAssuming you have a fine-tuned BERT model saved, you can load it using KerasNLP:\nfine_tuned_model_path =\n'path/to/fine_tuned_model'\nfine_tuned_model = load_bert_finetuned_model(fine_tuned_model_path)\nReplace 'path/to/fine_tuned_model' with the actual path to your fine-tuned BERT model.\nFine-Tuning BERT for Text Classification\nLet's explore how to fine-tune BERT for text classification using KerasNLP. Assume you have a task-specific dataset with text and corresponding labels:\ntask_data = pd.read_csv(\n'task_specific_data.csv'\n)\n\nX_task = tokenizer.tokenize(task_data[\n'text'\n].tolist())\n\ny_task = task_data[\n'label'\n].values\nNow, fine-tune the BERT model on your task-specific dataset:\nfine_tuned_model.fit(X_task, y_task, epochs=\n5\n, batch_size=\n16\n, validation_split=\n0.1\n)\n\nfine_tuned_model.save(\n'path/to/save/fine_tuned_model'\n)\nThis code snippet fine-tunes the BERT model on the task-specific dataset for five epochs with a batch size of 16, validating on a 10% subset. After fine-tuning, it saves the model for future use.\nConclusion\nIn this article, we explored text classification using BERT and KerasNLP. We explained the fundamentals of BERT, prepared the environment, loaded a pre-trained BERT model, and created a basic text classification model. Furthermore, we talked about fine-tuning BERT for particular tasks, offering code snippets and examples to help you along the way.\nImplementing text classification with BERT offers numerous opportunities for NLP applications. Whether you're focusing on sentiment analysis, spam detection, or any classification task, using BERT can greatly improve the accuracy and reliability of your models. As NLP progresses, keeping abreast of the newest developments and integrating them into your projects will keep you ahead in this dynamic and thrilling field.\n#\nfinetuning\n#\nbert\n#\nkerasnlp\n#\nkeras",
  "metadata": {
    "title": "Fine-tuning BERT for text classification with KerasNLP",
    "description": "In this article, we'll explore how to implement text classification using BERT and the KerasNLP library, providing examples and code snippets to guide you.",
    "keywords": null,
    "author": null,
    "og:title": "Fine-tuning BERT for text classification with KerasNLP",
    "og:description": "In this article, we'll explore how to implement text classification using BERT and the KerasNLP library, providing examples and code snippets to guide you.",
    "og:url": "https://kambale.dev/fine-tuning-bert",
    "og:site_name": "Wesley Kambale",
    "og:image": "https://cdn.hashnode.com/res/hashnode/image/upload/v1710174607858/d6b96b98-a153-4237-9cc7-a20a4c393264.png",
    "og:image:width": "1200",
    "og:image:height": "630",
    "og:type": "article",
    "twitter:card": "summary_large_image",
    "twitter:creator": "https://twitter.com/WesKambale",
    "twitter:title": "Fine-tuning BERT for text classification with KerasNLP",
    "twitter:description": "In this article, we'll explore how to implement text classification using BERT and the KerasNLP library, providing examples and code snippets to guide you.",
    "twitter:image": "https://cdn.hashnode.com/res/hashnode/image/upload/v1710174607858/d6b96b98-a153-4237-9cc7-a20a4c393264.png",
    "article:published_time": "2024-05-11T19:50:40.397Z",
    "article:author": "https://kambale.dev/@kambale"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/1b6bc502fb50/code/body-000.txt",
      "chars": 70,
      "language": "text"
    },
    {
      "file": "attachments/1b6bc502fb50/code/body-001.txt",
      "chars": 297,
      "language": "text"
    },
    {
      "file": "attachments/1b6bc502fb50/code/body-002.txt",
      "chars": 74,
      "language": "text"
    },
    {
      "file": "attachments/1b6bc502fb50/code/body-003.txt",
      "chars": 423,
      "language": "text"
    },
    {
      "file": "attachments/1b6bc502fb50/code/body-004.txt",
      "chars": 344,
      "language": "text"
    },
    {
      "file": "attachments/1b6bc502fb50/code/body-005.txt",
      "chars": 179,
      "language": "text"
    },
    {
      "file": "attachments/1b6bc502fb50/code/body-006.txt",
      "chars": 119,
      "language": "text"
    },
    {
      "file": "attachments/1b6bc502fb50/code/body-007.txt",
      "chars": 143,
      "language": "text"
    },
    {
      "file": "attachments/1b6bc502fb50/code/body-008.txt",
      "chars": 140,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/res/hashnode/image/upload/v1710174607858/d6b96b98-a153-4237-9cc7-a20a4c393264.png",
      "bytes": 178275,
      "reused": true,
      "local_path": "attachments/1b6bc502fb50/images/000.png",
      "src": "https://cdn.hashnode.com/res/hashnode/image/upload/v1710174607858/d6b96b98-a153-4237-9cc7-a20a4c393264.png",
      "alt": "Fine-tuning BERT for text classification with KerasNLP",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/res/hashnode/image/upload/v1715456920422/68ac176b-cff4-408f-92b4-0776ab115bab.png",
      "bytes": 163989,
      "reused": true,
      "local_path": "attachments/1b6bc502fb50/images/002.png",
      "src": "https://cdn.hashnode.com/res/hashnode/image/upload/v1715456920422/68ac176b-cff4-408f-92b4-0776ab115bab.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/res/hashnode/image/upload/v1710174138203/63b1bbb8-0bff-4cda-b9df-653e817d581a.jpeg",
      "bytes": 20058,
      "reused": true,
      "local_path": "attachments/1b6bc502fb50/images/003.jpeg",
      "src": "https://cdn.hashnode.com/res/hashnode/image/upload/v1710174138203/63b1bbb8-0bff-4cda-b9df-653e817d581a.jpeg",
      "alt": "BERT-enhanced tokenization. (c) Batuhan Gundogdu",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://kambale.dev/fine-tuning-bert.md",
      "label": "View as Markdown",
      "type": "a"
    },
    {
      "href": "https://hashnode.com/@kambale",
      "label": "W",
      "type": "a"
    },
    {
      "href": "https://twitter.com/WesKambale",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://linkedin.com/in/WesKambale",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://kambale.dev/series/ml-series",
      "label": "Part of series Machine Learning",
      "type": "a"
    },
    {
      "href": "https://kambale.dev/tag/finetuning",
      "label": "# finetuning",
      "type": "a"
    },
    {
      "href": "https://kambale.dev/tag/bert",
      "label": "# bert",
      "type": "a"
    },
    {
      "href": "https://kambale.dev/tag/kerasnlp",
      "label": "# kerasnlp",
      "type": "a"
    },
    {
      "href": "https://kambale.dev/tag/keras",
      "label": "# keras",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:50:49Z",
  "source_url": "https://kambale.dev/fine-tuning-bert",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 7610,
  "expected_body_images": 3,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/1b6bc502fb50/pages/source.html",
  "local_html": "attachments/1b6bc502fb50/pages/article.html",
  "body_base_path": "attachments/1b6bc502fb50/pages/",
  "body_markdown_path": "attachments/1b6bc502fb50/pages/article.md"
}