{
  "source_file": "0930",
  "source_platform": "Hashnode",
  "author": "Juan Carlos Olamendy",
  "title_listed": "ML in Real Life: Embeddings with Keras Examples - From Theory to Production-Ready Code",
  "keywords": "",
  "domain": "大模型算法",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://juancolamendy.hashnode.dev/ml-in-real-life-embeddings-with-keras-examples-from-theory-to-production-ready-code",
  "publish_date": "2025-08-11",
  "corpus_stage": "",
  "notes": "",
  "id": "1524856e5479",
  "collected_at": "2026-10-02T14:10:50",
  "title": "ML in Real Life: Embeddings with Keras Examples - From Theory to Production-Ready Code",
  "body_markdown": "ML in Real Life: Embeddings with Keras Examples - From Theory to Production-Ready Code\nPublished\nAugust 11, 2025\n•\n8\nmin read\n•\nView as Markdown\nJ\nJuan Carlos Olamendy\nSenior AI Engineer | Building & Telling Stories about AI/ML Systems | Software Engineer\nYou've spent three months trying to build a recommendation system, burned through your ML budget, and still can't figure out why your \"state-of-the-art\" embeddings perform worse than random suggestions.\nThis scenario plays out daily across tech companies worldwide.\nWhile embeddings power Netflix recommendations, Google Search, and ChatGPT's language understanding, most developers hit a wall when implementing them.\nTutorials overflow with buzzwords like \"dense representations\" and \"semantic similarity\" but lack the concrete Keras code that actually works.\nThe gap between theory and practice costs teams months of development time, thousands in wasted compute resources, and the crushing frustration of watching \"simple\" ML concepts remain mysteriously out of reach.\nThis guide eliminates that gap.\nYou'll master embeddings through first principles, see production-ready Keras implementations for text classification, recommenders, and tabular ML, and walk away with battle-tested code patterns you can deploy immediately. No abstractions, no toy examples—just practical knowledge that works.\nWhat Embeddings Actually Do\nEmbeddings solve a fundamental problem in machine learning: how do you feed discrete, symbolic data to algorithms that need continuous numbers?\nTraditional approaches use one-hot encoding, which creates massive sparse vectors where only one element equals 1.\nThis approach wastes memory, offers no similarity structure, and forces each category to learn independently.\nEmbeddings replace this with dense vectors where similar items cluster together in space.\nThe magic happens during training—your loss function sculpts the vector space so that semantically similar items end up geometrically close.\nThink of embeddings as a learned translation system. They convert discrete symbols (words, user IDs, product categories) into coordinates in a high-dimensional space.\nThe coordinates themselves mean nothing initially—they're just random numbers. But as your model trains, gradient descent pushes related items closer together and unrelated items further apart.\nWhy Embeddings Outperform Alternatives\nEmbeddings excel because they enable parameter sharing across similar items.\nWhen you use one-hot encoding with linear models, each category learns its own weight vector independently.\nIf \"iPhone 14\" and \"iPhone 15\" appear in your dataset, the model treats them as completely unrelated entities.\nEmbeddings let these similar items share statistical strength by placing them near each other in vector space.\nThe Keras Embedding Layer Explained\nKeras makes embeddings accessible through a single layer that hides complexity while exposing essential controls.\ntf.keras.layers.Embedding(\n    input_dim,\n# Vocabulary size (max index + 1)\noutput_dim,\n# Embedding dimension\nmask_zero=\nFalse\n,\n# Whether index 0 represents padding\nembeddings_initializer=\n'uniform'\n,\n    embeddings_regularizer=\nNone\n,\n    trainable=\nTrue\n)\nThe most common mistake happens with\ninput_dim\n—it must equal your maximum index plus one. If your user IDs range from 0 to 99999, set\ninput_dim=100000\n. Off-by-one errors here cause silent failures where the model learns garbage for out-of-bounds indices.\nThe\nmask_zero\nparameter deserves special attention for sequence tasks. When set to\nTrue\n, Keras treats index 0 as padding and propagates this mask to downstream layers. This prevents padding tokens from contributing to attention weights or pooling operations. Always reserve index 0 for padding when you enable masking.\nOutput dimension controls the expressiveness-efficiency tradeoff. Larger dimensions provide more capacity but consume more memory and computation.\nText Classification: Your First Embedding Implementation\nLet's build a complete text classification system that demonstrates core embedding concepts.\nimport\ntensorflow\nas\ntf\nfrom\ntensorflow.keras\nimport\nlayers, models\n# Sample data - in practice, load from your dataset\ntexts = tf.constant([\n\"i love this product amazing quality\"\n,\n\"terrible service very disappointed\"\n,\n\"excellent customer support helpful staff\"\n,\n\"poor quality broke immediately\"\n,\n\"fantastic experience highly recommend\"\n,\n\"worst purchase ever complete waste\"\n])\nlabels = tf.constant([\n1\n,\n0\n,\n1\n,\n0\n,\n1\n,\n0\n], dtype=tf.float32)\n# Configuration\nmax_tokens =\n10000\nsequence_length =\n20\nembedding_dim =\n64\n# Text preprocessing pipeline\ntext_vectorizer = layers.TextVectorization(\n    max_tokens=max_tokens,\n    output_mode=\n\"int\"\n,\n    output_sequence_length=sequence_length,\n    standardize=\n\"lower_and_strip_punctuation\"\n)\n# Fit the vectorizer on training data\ntext_vectorizer.adapt(texts.batch(\n8\n))\n# Model architecture\ndef\ncreate_text_classifier\n():\n# Input: raw strings\ntext_input = layers.Input(shape=(), dtype=tf.string, name=\n'text'\n)\n# Convert to integer sequences\nx = text_vectorizer(text_input)\n# Shape: (batch, sequence_length)\n# Embed tokens into dense vectors\nx = layers.Embedding(\n        input_dim=max_tokens,\n        output_dim=embedding_dim,\n        mask_zero=\nTrue\n,\n# Handle padding\nname=\n'token_embedding'\n)(x)\n# Shape: (batch, sequence_length, embedding_dim)\n# Pool across sequence dimension\nx = layers.GlobalAveragePooling1D()(x)\n# Shape: (batch, embedding_dim)\n# Classification head\nx = layers.Dense(\n128\n, activation=\n'relu'\n)(x)\n    x = layers.Dropout(\n0.2\n)(x)\n    output = layers.Dense(\n1\n, activation=\n'sigmoid'\n, name=\n'prediction'\n)(x)\nreturn\nmodels.Model(text_input, output)\n# Build and compile model\nmodel = create_text_classifier()\nmodel.compile(\n    optimizer=\n'adam'\n,\n    loss=\n'binary_crossentropy'\n,\n    metrics=[\n'accuracy'\n,\n'auc'\n]\n)\n# Train the model\nhistory = model.fit(\n    texts, labels,\n    batch_size=\n2\n,\n    epochs=\n10\n,\n    validation_split=\n0.2\n,\n    verbose=\n1\n)\n# Inspect learned embeddings\nembedding_layer = model.get_layer(\n'token_embedding'\n)\nvocab = text_vectorizer.get_vocabulary()\n# Get embeddings for specific tokens\ntoken_index = vocab.index(\n'excellent'\n)\ntoken_embedding = embedding_layer.get_weights()[\n0\n][token_index]\nprint(\nf\"Embedding for 'excellent':\n{token_embedding[:\n5\n]}\n...\"\n)\nThis example demonstrates several key concepts:\nThe\nTextVectorization\nlayer converts raw strings into integer sequences.\nThe\nEmbedding\nlayer transforms these integers into dense vectors.\nGlobalAveragePooling1D\naggregates sequence embeddings into fixed-size representations. The\nmask_zero=True\nparameter ensures padding tokens don't affect the average.\nNotice how the model accepts raw strings as input but processes integer sequences internally. You can modify tokenization logic without retraining the entire model.\nRecommender Systems: Two-Tower Architecture\nRecommender systems showcase embeddings' power for modeling user-item interactions with efficient serving through precomputed item vectors and fast retrieval.\nimport\ntensorflow\nas\ntf\nfrom\ntensorflow.keras\nimport\nlayers, Model\n# Toy interaction data\nuser_ids = tf.constant([\n10\n,\n42\n,\n7\n,\n99\n,\n5\n,\n5\n,\n10\n,\n12\n], dtype=tf.int32)\n# shape: (N,)\nitem_ids = tf.constant([\n1001\n,\n5\n,\n8\n,\n8\n,\n7\n,\n3\n,\n1001\n,\n5\n], dtype=tf.int32)\n# shape: (N,)\nlabels   = tf.constant([\n1\n,\n0\n,\n1\n,\n0\n,\n1\n,\n0\n,\n1\n,\n0\n], dtype=tf.float32)\n# click/purchase\nnum_users, num_items, emb_dim =\n100\n_000,\n50\n_000,\n64\n# Two-tower architecture\nuser_in = layers.Input(shape=(), dtype=tf.int32, name=\n\"user_id\"\n)\n# <-- pass scalars per row\nitem_in = layers.Input(shape=(), dtype=tf.int32, name=\n\"item_id\"\n)\n\nu = layers.Embedding(num_users, emb_dim, name=\n\"user_emb\"\n)(user_in)\n# (batch, emb_dim)\nv = layers.Embedding(num_items, emb_dim, name=\n\"item_emb\"\n)(item_in)\n# (batch, emb_dim)\nscore = layers.Dot(axes=\n1\n)([u, v])\n# (batch, 1)\nout = layers.Activation(\n\"sigmoid\"\n)(score)\n\nrecsys = Model([user_in, item_in], out)\nrecsys.compile(optimizer=\n\"adam\"\n, loss=\n\"binary_crossentropy\"\n, metrics=[\n\"AUC\"\n])\n# Train — you can pass a dict keyed by Input names or a list [user_ids, item_ids]\n# shapes: user_ids: (batch,), item_ids: (batch,), y: (batch,)\nrecsys.fit({\n\"user_id\"\n: user_ids,\n\"item_id\"\n: item_ids}, labels,\n           batch_size=\n4\n, epochs=\n5\n, validation_split=\n0.25\n)\n# Extract embeddings for similarity search\nuser_embeddings = recsys.get_layer(\n\"user_emb\"\n).get_weights()[\n0\n]\n# (num_users, emb_dim)\nitem_embeddings = recsys.get_layer(\n\"item_emb\"\n).get_weights()[\n0\n]\n# (num_items, emb_dim)\n# Find similar items via dot product\ndef\nfind_similar_items\n(\nitem_id, top_k=\n5\n):\nquery_vec = item_embeddings[item_id]\n# (emb_dim,)\nscores = item_embeddings @ query_vec\n# (num_items,)\ntop_indices = scores.argsort()[-top_k\n-1\n:\n-1\n][::\n-1\n]\n# exclude self\nreturn\ntop_indices, scores[top_indices]\n\nsimilar_items, similarity_scores = find_similar_items(item_id=\n5\n)\nprint(\nf\"Items similar to 5:\n{similar_items}\n\"\n)\nThe two-tower design separates user and item representations, enabling precomputation of item vectors for millisecond retrieval.\nDot product scoring makes the architecture compatible with approximate nearest neighbor libraries like Faiss.\nTabular Data: Mixing Categorical and Numeric Features\nReal-world datasets combine high-cardinality categoricals with numeric features. Embeddings handle the categoricals while standard preprocessing handles the numerics.\nimport\ntensorflow\nas\ntf\nimport\nnumpy\nas\nnp\nimport\npandas\nas\npd\nfrom\ntensorflow.keras\nimport\nlayers, Model\nfrom\nsklearn.preprocessing\nimport\nStandardScaler\n# Generate synthetic tabular data\nnp.random.seed(\n42\n)\nn_samples =\n10000\n# Categorical features\nn_cities, n_devices, n_categories, n_dim =\n500\n,\n2000\n,\n100\n,\n128\ndata = {\n'city_id'\n: np.random.randint(\n0\n, n_cities, n_samples),\n'device_id'\n: np.random.randint(\n0\n, n_devices, n_samples),\n'category_id'\n: np.random.randint(\n0\n, n_categories, n_samples),\n# Numeric features\n'age'\n: np.random.normal(\n35\n,\n12\n, n_samples).clip(\n18\n,\n80\n),\n'income'\n: np.random.lognormal(\n10.5\n,\n0.5\n, n_samples),\n'session_duration'\n: np.random.exponential(\n300\n, n_samples),\n'previous_purchases'\n: np.random.poisson(\n2\n, n_samples),\n'days_since_signup'\n: np.random.exponential(\n365\n, n_samples)\n}\n\ndf = pd.DataFrame(data)\n# Create target variable with realistic relationships\ndf[\n'target'\n] = (\n0.3\n* (df[\n'age'\n] >\n30\n) +\n0.2\n* (df[\n'income'\n] >\n50000\n) +\n0.1\n* (df[\n'session_duration'\n] >\n300\n) +\n0.2\n* (df[\n'previous_purchases'\n] >\n1\n) +\n0.1\n* (df[\n'category_id'\n] <\n20\n) +\n# Some categories are more valuable\nnp.random.normal(\n0\n,\n0.3\n, n_samples)\n).clip(\n0\n,\n1\n)\n# Preprocess numeric features\nnumeric_features = [\n'age'\n,\n'income'\n,\n'session_duration'\n,\n'previous_purchases'\n,\n'days_since_signup'\n]\nscaler = StandardScaler()\ndf[numeric_features] = scaler.fit_transform(df[numeric_features])\n# Split data\ntrain_size = int(\n0.8\n* len(df))\ntrain_df = df[:train_size]\nval_df = df[train_size:]\ndef\ncreate_tabular_model\n(\nn_cities, n_devices, n_categories, n_numeric_features\n):\n# Categorical inputs\ncity_input = layers.Input(shape=(), dtype=tf.int32, name=\n'city_id'\n)\n    device_input = layers.Input(shape=(), dtype=tf.int32, name=\n'device_id'\n)\n    category_input = layers.Input(shape=(), dtype=tf.int32, name=\n'category_id'\n)\n# Numeric inputs\nnumeric_input = layers.Input(shape=(n_numeric_features,), dtype=tf.float32, name=\n'numeric_features'\n)\n# Embed categorical features with different dimensions based on cardinality\ncity_embedding = layers.Embedding(\n        input_dim=n_cities,\n        output_dim=n_dim,\n        name=\n'city_embedding'\n)(city_input)\n\n    device_embedding = layers.Embedding(\n        input_dim=n_devices, \n        output_dim=n_dim,\n        name=\n'device_embedding'\n)(device_input)\n\n    category_embedding = layers.Embedding(\n        input_dim=n_categories,\n        output_dim=n_dim,\n        name=\n'category_embedding'\n)(category_input)\n# Concatenate all features\ncombined_features = layers.Concatenate(name=\n'feature_concat'\n)([\n        city_embedding,\n        device_embedding, \n        category_embedding,\n        numeric_input\n    ])\n# Deep neural network\nx = layers.Dense(\n256\n, activation=\n'relu'\n)(combined_features)\n    x = layers.BatchNormalization()(x)\n    x = layers.Dropout(\n0.3\n)(x)\n\n    x = layers.Dense(\n128\n, activation=\n'relu'\n)(x)\n    x = layers.BatchNormalization()(x)\n    x = layers.Dropout(\n0.3\n)(x)\n\n    x = layers.Dense(\n64\n, activation=\n'relu'\n)(x)\n    x = layers.Dropout(\n0.2\n)(x)\n# Output layer\noutput = layers.Dense(\n1\n, activation=\n'sigmoid'\n, name=\n'prediction'\n)(x)\nreturn\nModel([city_input, device_input, category_input, numeric_input], output)\n# Build model\nmodel = create_tabular_model(n_cities, n_devices, n_categories, len(numeric_features))\n\nmodel.compile(\n    optimizer=tf.keras.optimizers.Adam(learning_rate=\n0.001\n),\n    loss=\n'binary_crossentropy'\n,\n    metrics=[\n'auc'\n,\n'precision'\n,\n'recall'\n]\n)\n# Prepare training data\ndef\nprepare_inputs\n(\ndf\n):\nreturn\n{\n'city_id'\n: df[\n'city_id'\n].values,\n'device_id'\n: df[\n'device_id'\n].values,\n'category_id'\n: df[\n'category_id'\n].values,\n'numeric_features'\n: df[numeric_features].values\n    }\n\ntrain_inputs = prepare_inputs(train_df)\nval_inputs = prepare_inputs(val_df)\n# Train model\nhistory = model.fit(\n    train_inputs, train_df[\n'target'\n].values,\n    validation_data=(val_inputs, val_df[\n'target'\n].values),\n    batch_size=\n256\n,\n    epochs=\n50\n,\n)\nThis tabular example demonstrates several advanced techniques:\nAdaptive embedding dimensions based on cardinality prevent over-parameterization. L2 regularization controls embedding magnitude and prevents overfitting. Batch normalization and dropout improve generalization across mixed feature types. The analysis function provides insights into learned representations.\nThe key insight is treating categorical and numeric features as complementary information sources. Embeddings capture similarity structure in categoricals while standard preprocessing handles numerics. The concatenation layer fuses these representations for the downstream neural network.\nConclusion\nEmbeddings transform discrete symbols into continuous representations that capture learned similarity. This simple concept powers some of the most successful machine learning systems in production today.\nYou've seen how Keras makes embeddings accessible through a clean API that handles the complexity while exposing essential controls.\nThe examples demonstrate three fundamental patterns: text classification, collaborative filtering, and mixed tabular features. Each pattern adapts to different data types and business requirements while sharing core principles.\nThe key insights to remember:\nStart simple and iterate.\nBegin with basic architectures before adding complexity like attention or hierarchical embeddings. Monitor embedding quality throughout development to catch issues early. Focus on the loss function—it defines what similarity means in your vector space.\nHandle production concerns upfront.\nPlan for vocabulary changes, out-of-vocabulary tokens, and serving latency from the beginning. Implement proper monitoring and quality checks before deployment. Consider memory constraints and quantization strategies for large-scale systems.\nEmbeddings excel at capturing relationships but require careful tuning.\nDimension selection balances expressiveness with efficiency. Regularization prevents overfitting on sparse categorical features. Proper masking and sequence handling are crucial for temporal data.\nThe techniques in this guide provide a foundation for building robust embedding systems. Whether you're building recommenders, processing text, or analyzing tabular data, these patterns will serve you well.\nStart with the simplest approach that solves your problem, then add sophistication only when metrics demand it.\n#\nmachine-learning\n#\nartificial-intelligence\n#\nkeras\n#\nembeddings\n#\nrecommender-systems\n#\nnlp",
  "metadata": {
    "title": "ML in Real Life: Embeddings with Keras Examples - From Theory to Production-Ready Code",
    "description": "You've spent three months trying to build a recommendation system, burned through your ML budget, and still can't figure out why your \"state-of-the-art\" embeddings perform worse than random suggestions.\nThis scenario plays out daily across tech compa...",
    "keywords": null,
    "author": null,
    "og:title": "ML in Real Life: Embeddings with Keras Examples - From Theory to Production-Ready Code",
    "og:description": "You've spent three months trying to build a recommendation system, burned through your ML budget, and still can't figure out why your \"state-of-the-art\" embeddings perform worse than random suggestions.\nThis scenario plays out daily across tech compa...",
    "og:url": "https://juancolamendy.hashnode.dev/ml-in-real-life-embeddings-with-keras-examples-from-theory-to-production-ready-code",
    "og:site_name": "Juan C Olamendy",
    "og:image": "https://cdn.hashnode.com/res/hashnode/image/upload/v1754925681804/556c99e6-4f33-425e-a472-13f0d3f3d181.png",
    "og:image:width": "1200",
    "og:image:height": "630",
    "og:type": "article",
    "twitter:card": "summary_large_image",
    "twitter:creator": "https://twitter.com/juancolamendy",
    "twitter:title": "ML in Real Life: Embeddings with Keras Examples - From Theory to Production-Ready Code",
    "twitter:description": "You've spent three months trying to build a recommendation system, burned through your ML budget, and still can't figure out why your \"state-of-the-art\" embeddings perform worse than random suggestions.\nThis scenario plays out daily across tech compa...",
    "twitter:image": "https://cdn.hashnode.com/res/hashnode/image/upload/v1754925681804/556c99e6-4f33-425e-a472-13f0d3f3d181.png",
    "article:published_time": "2025-08-11T15:23:31.711Z",
    "article:author": "https://juancolamendy.hashnode.dev/@juancolamendy"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/1524856e5479/code/body-000.txt",
      "chars": 280,
      "language": "text"
    },
    {
      "file": "attachments/1524856e5479/code/body-001.txt",
      "chars": 2311,
      "language": "text"
    },
    {
      "file": "attachments/1524856e5479/code/body-002.txt",
      "chars": 2040,
      "language": "text"
    },
    {
      "file": "attachments/1524856e5479/code/body-003.txt",
      "chars": 4044,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/res/hashnode/image/upload/v1754925681804/556c99e6-4f33-425e-a472-13f0d3f3d181.png",
      "bytes": 1275730,
      "reused": true,
      "local_path": "attachments/1524856e5479/images/000.png",
      "src": "https://cdn.hashnode.com/res/hashnode/image/upload/v1754925681804/556c99e6-4f33-425e-a472-13f0d3f3d181.png",
      "alt": "ML in Real Life: Embeddings with Keras Examples - From Theory to Production-Ready Code",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://juancolamendy.hashnode.dev/ml-in-real-life-embeddings-with-keras-examples-from-theory-to-production-ready-code.md",
      "label": "View as Markdown",
      "type": "a"
    },
    {
      "href": "https://hashnode.com/@juancolamendy",
      "label": "J",
      "type": "a"
    },
    {
      "href": "https://twitter.com/juancolamendy",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://www.linkedin.com/in/juancolamendy/",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://juancolamendy.hashnode.dev/tag/machine-learning",
      "label": "# machine-learning",
      "type": "a"
    },
    {
      "href": "https://juancolamendy.hashnode.dev/tag/artificial-intelligence",
      "label": "# artificial-intelligence",
      "type": "a"
    },
    {
      "href": "https://juancolamendy.hashnode.dev/tag/keras",
      "label": "# keras",
      "type": "a"
    },
    {
      "href": "https://juancolamendy.hashnode.dev/tag/embeddings",
      "label": "# embeddings",
      "type": "a"
    },
    {
      "href": "https://juancolamendy.hashnode.dev/tag/recommender-systems",
      "label": "# recommender-systems",
      "type": "a"
    },
    {
      "href": "https://juancolamendy.hashnode.dev/tag/nlp",
      "label": "# nlp",
      "type": "a"
    }
  ],
  "retry_at": "2026-10-03T08:06:09",
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:50:36Z",
  "source_url": "https://juancolamendy.hashnode.dev/ml-in-real-life-embeddings-with-keras-examples-from-theory-to-production-ready-code",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 15939,
  "expected_body_images": 1,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/1524856e5479/pages/source.html",
  "local_html": "attachments/1524856e5479/pages/article.html",
  "body_base_path": "attachments/1524856e5479/pages/",
  "body_markdown_path": "attachments/1524856e5479/pages/article.md"
}