{
  "source_file": "0930",
  "source_platform": "Hashnode",
  "author": "Wesley Kambale",
  "title_listed": "Feeding the Beast: Data Pipelines with Grain",
  "keywords": "",
  "domain": "数据处理",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://kambale.dev/data-pipelines-with-grain",
  "publish_date": "2026-07-29",
  "corpus_stage": "",
  "notes": "",
  "id": "20348d1d5a54",
  "collected_at": "2026-10-02T14:10:59",
  "title": "Feeding the Beast: Data Pipelines with Grain",
  "body_markdown": "Feeding the Beast: Data Pipelines with Grain\nUpdated\nJuly 29, 2026\n•\n13\nmin read\n•\nView as Markdown\nW\nWesley Kambale\nI'm a Machine Learning Engineer passionate about building production-ready ML systems for the African market.\nWith experience in TensorFlow, Keras, and Python-based workflows, I help teams bridge the gap between machine learning research and real-world deployment—especially on resource-constrained devices.\nI'm also a Google Developer Expert in AI. I regularly speak at tech conferences including PyCon Africa, DevFest Kampala, DevFest Nairobi and more and also write technical articles on AI/ML here.\nPart of series\nJAX\nOver the last seven weeks we have\nbuilt models\n, written\ntraining loops\n, composed\noptimizers\n, bulletproofed our code with\nChex\n, and learned to debug inside\nJIT-compiled functions\n. Every one of those articles used dummy data or a small built-in dataset, generated on the fly with a few lines of NumPy or\ntfds.load()\n.\nReal projects do not work that way. Real datasets are millions of images sitting in cloud storage, gigabytes of audio clips streamed from disk, or CSV files too large to load into memory at once. And here is the uncomfortable truth: it does not matter how fast your\n@nnx.jit\ntraining step is if your data loader cannot keep up with it.\nThis is one of the most common and least glamorous bottlenecks in machine learning engineering. You buy an expensive GPU or rent TPU time, write a beautifully optimized training loop, and then watch your utilization graph sit at 20 percent because the accelerator is idle, waiting for the next batch of data to arrive from a slow Python loop.\nToday we fix that.\nWe are introducing\nGrain\n, a data loading library purpose built for JAX. Grain plays a role similar to\ntorch.utils.data.DataLoader\nin PyTorch, but its design accounts for the things JAX cares about most: determinism, explicit randomness, and staying out of the way of your compiled training step.\nBy the end of this article you will understand:\nWhy naive Python data loading becomes the bottleneck in JAX training\nThe three building blocks of a Grain pipeline: DataSource, Sampler, and Transformations\nHow to write deterministic and reproducible data augmentation\nHow to parallelize data loading across multiple CPU workers\nHow to shard data correctly for distributed training\nHow to wire a Grain pipeline into the training loop we built in Week 4\nLet's feed the beast.\nWhy data loading becomes the bottleneck\nPicture a training loop where the model itself runs in under a millisecond per batch, thanks to\n@nnx.jit\nand XLA compilation. Now picture a plain Python function that reads an image from disk, decodes it, resizes it, and normalizes it. That single operation, done in pure Python, can easily take several milliseconds.\nIf your data loading is a simple\nfor\nloop over a list of file paths, your training loop looks like this:\nfor i in range(num_steps):\n    # This line might take 5-10ms per image, done sequentially\n    batch = load_and_preprocess_batch(file_paths[i])\n    \n    # This line might take 0.5ms thanks to JIT\n    loss = train_step(model, optimizer, batch)\nThe GPU finishes its work almost instantly and then sits idle while the CPU slowly prepares the next batch. You paid for a race car and you are driving it in first gear.\nThe fix is not to make the model slower to match the data loader. The fix is to make data loading fast, parallel, and asynchronous, so that while the GPU is busy computing batch N, the CPU is already preparing batch N+1 in the background.\nThat is exactly what Grain is built to do.\nThe three building blocks of Grain\nA Grain pipeline is assembled from three components that work together: a\nDataSource\nthat knows how to read individual records, a\nSampler\nthat decides the order records are visited in, and a list of\nTransformations\nthat process each record or batch.\nimport grain.python as grain\n\ndata_loader = grain.DataLoader(\n    data_source=my_source,\n    sampler=my_sampler,\n    operations=my_transformations,\n    worker_count=4,\n)\nLet's build each piece.\nThe DataSource\nA\nDataSource\nis the simplest possible abstraction: something with a length and a way to fetch a single item by index. It does not know anything about batching, shuffling, or augmentation. Its only job is: given an index, return the raw record.\nimport grain.python as grain\n\nclass MyImageSource(grain.RandomAccessDataSource):\n    def __init__(self, file_paths, labels):\n        self._file_paths = file_paths\n        self._labels = labels\n    \n    def __len__(self):\n        return len(self._file_paths)\n    \n    def __getitem__(self, index):\n        # Return raw, unprocessed data. Keep this fast and simple.\n        return {\n            'path': self._file_paths[index],\n            'label': self._labels[index],\n        }\n\nsource = MyImageSource(file_paths=my_paths, labels=my_labels)\nNotice how minimal this is. It does not decode images or apply any transformations. That separation of concerns is intentional and it is what allows Grain to parallelize the expensive work later.\nThe Sampler\nThe\nSampler\ndecides which index to visit, in what order, for how many epochs, and with what random seed. This is where shuffling and reproducibility live.\nsampler = grain.IndexSampler(\n    num_records=len(source),\n    shuffle=True,\n    num_epochs=None,   # None means infinite, useful for continuous training\n    seed=42,\n)\nBecause the seed is explicit, two runs with the same seed visit records in exactly the same order. This matters enormously when you are debugging a training run or trying to reproduce a result from a paper or a colleague's experiment.\nTransformations\nTransformations are the actual processing steps applied to each record: decoding an image, normalizing pixel values, applying augmentation, or grouping records into batches.\nThere are two kinds of custom transforms in Grain, and the distinction matters.\nDeterministic transforms\ninherit from\nMapTransform\nand implement a\nmap\nmethod. Given the same input, they always produce the same output.\nimport numpy as np\n\nclass Normalize(grain.MapTransform):\n    def map(self, element):\n        element['image'] = element['image'].astype(np.float32) / 255.0\n        return element\nRandom transforms\ninherit from\nRandomMapTransform\nand implement\nrandom_map\n, which receives a\nnumpy.random.Generator\nas an argument. This is the critical detail: you must use the provided generator, not\nnp.random\ndirectly, or your augmentation will not be reproducible.\nclass RandomFlip(grain.RandomMapTransform):\n    def random_map(self, element, rng):\n        if rng.random() > 0.5:\n            element['image'] = np.fliplr(element['image'])\n        return element\nYou then assemble your transforms into an ordered list, finishing with\ngrain.Batch\nto group individual records into batches:\noperations = [\n    Normalize(),\n    RandomFlip(),\n    grain.Batch(batch_size=64, drop_remainder=True),\n]\nNotice\ndrop_remainder=True\n. If you have been following along since Week 6, you will remember exactly why this matters: an inconsistent final batch size forces JAX to recompile your JIT function. Dropping the incomplete final batch keeps every shape identical and avoids the recompilation trap entirely.\nPutting it together\nHere is a complete pipeline combining all three pieces:\nimport grain.python as grain\nimport numpy as np\n\n# DataSource: raw access to records\nclass MNISTSource(grain.RandomAccessDataSource):\n    def __init__(self, images, labels):\n        self._images = images\n        self._labels = labels\n    \n    def __len__(self):\n        return len(self._images)\n    \n    def __getitem__(self, index):\n        return {\n            'image': self._images[index],\n            'label': self._labels[index],\n        }\n\nsource = MNISTSource(images=train_images, labels=train_labels)\n\n# Sampler: order and reproducibility\nsampler = grain.IndexSampler(\n    num_records=len(source),\n    shuffle=True,\n    num_epochs=None,\n    seed=42,\n)\n\n# Transformations: processing pipeline\nclass Normalize(grain.MapTransform):\n    def map(self, element):\n        element['image'] = element['image'].astype(np.float32) / 255.0\n        return element\n\noperations = [\n    Normalize(),\n    grain.Batch(batch_size=64, drop_remainder=True),\n]\n\n# Assemble the DataLoader\ndata_loader = grain.DataLoader(\n    data_source=source,\n    sampler=sampler,\n    operations=operations,\n    worker_count=4,\n)\n\n# Use it as an iterator\ndata_iterator = iter(data_loader)\nfirst_batch = next(data_iterator)\n\nprint(f\"Batch image shape: {first_batch['image'].shape}\")\nprint(f\"Batch label shape: {first_batch['label'].shape}\")\nThis looks similar to a PyTorch\nDataLoader\n, and that is intentional. The difference is under the hood: Grain's sampler and random transforms are built around the same explicit, seedable randomness philosophy that JAX itself uses.\nParallelism: The worker_count parameter\nThe single most impactful performance knob in Grain is\nworker_count\n.\ndata_loader = grain.DataLoader(\n    data_source=source,\n    sampler=sampler,\n    operations=operations,\n    worker_count=0,   # Sequential, single process\n)\nWith\nworker_count=0\n, everything runs sequentially in your main Python process. This is useful for debugging, since you can set breakpoints inside your transforms and step through them normally, but it will not use your CPU efficiently.\ndata_loader = grain.DataLoader(\n    data_source=source,\n    sampler=sampler,\n    operations=operations,\n    worker_count=8,   # Parallel, using multiprocessing\n)\nWith\nworker_count > 0\n, Grain spins up multiple worker processes. Because these are separate processes rather than threads, they bypass Python's Global Interpreter Lock entirely, meaning your image decoding, resizing, and augmentation can genuinely run in parallel across CPU cores. Grain uses shared memory to transfer completed batches back to your main process efficiently, avoiding the overhead of pickling large arrays repeatedly.\nA practical rule of thumb: start with\nworker_count\nroughly equal to the number of CPU cores available, then benchmark. If your data source is already in memory (for example, a NumPy array you loaded once at the start), you may not need many workers at all, since there is no disk I/O to overlap. In that case:\ndata_loader = grain.DataLoader(\n    data_source=source,\n    sampler=sampler,\n    operations=operations,\n    worker_count=0,\n    read_options=grain.ReadOptions(num_threads=0),\n)\nDisabling thread prefetching when your dataset already lives in memory avoids unnecessary overhead from a prefetching mechanism you do not need.\nSharding for distributed training\nWhen you train across multiple JAX processes, whether that is multiple GPUs on one machine or multiple machines in a cluster, each process must see a distinct, non overlapping slice of the data. Otherwise you are wasting compute recomputing gradients on the same examples multiple times per step.\nGrain handles this with\nShardOptions\n, passed into your sampler:\ntry:\n    # Automatically detects jax.process_index() and jax.process_count()\n    shard_options = grain.ShardByJaxProcess(drop_remainder=True)\nexcept ImportError:\n    # Fallback for a single process setup\n    shard_options = grain.ShardOptions(\n        shard_index=0, shard_count=1, drop_remainder=True\n    )\n\nsampler = grain.IndexSampler(\n    num_records=len(source),\n    shard_options=shard_options,\n    shuffle=True,\n    num_epochs=None,\n    seed=42,\n)\nShardByJaxProcess\nis the recommended default. It reads directly from the JAX runtime to figure out how many processes are running and which one this particular process is, then automatically hands each process its own disjoint slice of the dataset. You do not need to manually calculate offsets or worry about duplicated examples across workers.\nWe will return to this in much more depth in Week 10, when we cover sharding and parallelism for the model itself. For now, the important thing to remember is that data sharding and model sharding are two separate concerns, and Grain handles the data side cleanly.\nRebuilding a training loop with Grain\nLet's take the training loop from\nweek 4\nand swap the dummy NumPy data generation for a real Grain pipeline.\nThe data pipeline\nimport grain.python as grain\nimport numpy as np\nimport jax\nimport jax.numpy as jnp\nfrom flax import nnx\nimport optax\n\nclass ArrayDataSource(grain.RandomAccessDataSource):\n    \"\"\"Wraps in-memory NumPy arrays as a Grain data source.\"\"\"\n    def __init__(self, images, labels):\n        self._images = images\n        self._labels = labels\n    \n    def __len__(self):\n        return len(self._images)\n    \n    def __getitem__(self, index):\n        return {\n            'image': self._images[index],\n            'label': self._labels[index],\n        }\n\n\nclass Normalize(grain.MapTransform):\n    def map(self, element):\n        element['image'] = element['image'].astype(np.float32) / 255.0\n        return element\n\n\ndef build_data_loader(images, labels, batch_size, seed):\n    source = ArrayDataSource(images, labels)\n    \n    sampler = grain.IndexSampler(\n        num_records=len(source),\n        shuffle=True,\n        num_epochs=None,\n        seed=seed,\n    )\n    \n    operations = [\n        Normalize(),\n        grain.Batch(batch_size=batch_size, drop_remainder=True),\n    ]\n    \n    return grain.DataLoader(\n        data_source=source,\n        sampler=sampler,\n        operations=operations,\n        worker_count=4,\n    )\nModel, optimizer, and training step\nThis should look familiar from week 4:\nclass ArchiveDigitizer(nnx.Module):\n    def __init__(self, hidden_dim: int, num_classes: int, rngs: nnx.Rngs):\n        self.linear1 = nnx.Linear(784, hidden_dim, rngs=rngs)\n        self.linear2 = nnx.Linear(hidden_dim, num_classes, rngs=rngs)\n    \n    def __call__(self, x):\n        x = self.linear1(x)\n        x = nnx.relu(x)\n        return self.linear2(x)\n\n\nrngs = nnx.Rngs(jax.random.PRNGKey(42))\nmodel = ArchiveDigitizer(hidden_dim=128, num_classes=10, rngs=rngs)\n\ntx = optax.adam(learning_rate=0.005)\noptimizer = nnx.Optimizer(model, tx=tx, wrt=nnx.Param)\n\n\n@nnx.jit\ndef train_step(model, optimizer, batch_images, batch_labels):\n    def loss_fn(m):\n        logits = m(batch_images)\n        loss = optax.softmax_cross_entropy_with_integer_labels(\n            logits=logits, labels=batch_labels\n        ).mean()\n        return loss\n    \n    loss_val, grads = nnx.value_and_grad(loss_fn)(model)\n    optimizer.update(model, grads)\n    return loss_val\nThe training loop, powered by Grain\n# Assume train_images, train_labels are loaded NumPy arrays, e.g. from MNIST\nbatch_size = 64\ndata_loader = build_data_loader(train_images, train_labels, batch_size, seed=42)\n\ndata_iterator = iter(data_loader)\nnum_steps = 2000\n\nprint(\"Starting training with Grain data pipeline...\")\n\nrunning_loss = 0.0\nfor step in range(num_steps):\n    batch = next(data_iterator)\n    \n    x_batch = jnp.array(batch['image']).reshape(batch_size, -1)\n    y_batch = jnp.array(batch['label'])\n    \n    loss = train_step(model, optimizer, x_batch, y_batch)\n    running_loss += loss.item()\n    \n    if step % 200 == 0 and step > 0:\n        avg_loss = running_loss / 200\n        print(f\"Step {step:4d} | Average Loss: {avg_loss:.4f}\")\n        running_loss = 0.0\n\nprint(\"Training complete.\")\nThe training step itself did not change at all from Week 4. Only the mechanism feeding it data changed, and that separation is exactly the point. Your model logic and your data pipeline should be independent, swappable pieces.\nReproducibility and checkpointing\nOne benefit of Grain's explicit, seeded design is that data pipelines are fully reproducible. Given the same seed and the same source, two runs will visit records in the identical order and apply the identical augmentations at each step.\nFor long running jobs, you also want to be able to resume training after an interruption without skipping or repeating data. Grain integrates with Orbax, the checkpointing library we will cover in Week 9, allowing you to save the exact position of your data iterator alongside your model weights and optimizer state. When training resumes, both the model and the data pipeline pick up exactly where they left off.\nCommon pitfalls\nA few mistakes come up often enough to call out directly.\nUsing np.random instead of the provided generator.\nInside a\nRandomMapTransform\n, always use the\nrng\nargument passed into\nrandom_map\n. Calling\nnp.random.random()\ndirectly breaks reproducibility, since that global state is not controlled by Grain's seeding.\nForgetting drop_remainder.\nAs covered in Week 6, an inconsistent final batch shape triggers expensive JIT recompilation. Set\ndrop_remainder=True\non\ngrain.Batch\nunless you have a specific reason not to.\nOverestimating worker_count.\nMore workers is not always better. If your\n__getitem__\nimplementation is already fast, for example reading from an in-memory array, spinning up eight worker processes adds multiprocessing overhead without any benefit. Benchmark before assuming.\nDoing heavy work in the DataSource instead of Transformations.\nKeep\n__getitem__\nminimal. Expensive operations like image decoding and augmentation belong in your transformation list, where Grain can distribute them across workers. A bloated\n__getitem__\ncannot be parallelized as effectively.\nExercises\nBuild a custom augmentation.\nWrite a\nRandomMapTransform\nthat randomly rotates an image by a small angle, using the provided\nrng\ngenerator. Verify that running the pipeline twice with the same seed produces identical rotations.\nBenchmark worker counts.\nRun the same data pipeline with\nworker_count\nset to 0, 2, 4, and 8. Time how long it takes to iterate through 500 batches at each setting. Plot the results.\nSimulate a distributed shard.\nManually construct two\nShardOptions\nobjects representing shard 0 of 2 and shard 1 of 2. Confirm that the two resulting samplers produce completely disjoint sets of indices across a full epoch.\nTrigger and fix a recompilation.\nRemove\ndrop_remainder=True\nfrom your\ngrain.Batch\ncall. Train for a few epochs and use\nchex.assert_max_traces\nfrom Week 6 to confirm that the final, undersized batch triggers a second compilation. Then fix it.\nQuick reference\nimport grain.python as grain\nimport numpy as np\n\n# DataSource\nclass MySource(grain.RandomAccessDataSource):\n    def __len__(self):\n        return len(self._data)\n    def __getitem__(self, index):\n        return self._data[index]\n\n# Sampler\nsampler = grain.IndexSampler(\n    num_records=len(source),\n    shuffle=True,\n    num_epochs=None,\n    seed=42,\n    shard_options=grain.ShardByJaxProcess(drop_remainder=True),\n)\n\n# Deterministic transform\nclass MyMap(grain.MapTransform):\n    def map(self, element):\n        return element\n\n# Random transform\nclass MyRandomMap(grain.RandomMapTransform):\n    def random_map(self, element, rng):\n        return element\n\n# Assembling the pipeline\ndata_loader = grain.DataLoader(\n    data_source=source,\n    sampler=sampler,\n    operations=[MyMap(), MyRandomMap(), grain.Batch(64, drop_remainder=True)],\n    worker_count=4,\n)\n\ndata_iterator = iter(data_loader)\nbatch = next(data_iterator)\nWhat's next\nOur training loop can now pull in real data at real speed. But every training run we have written so far lives and dies with the Python process running it. If your machine crashes, if you lose power, or if you simply need to stop and resume tomorrow, all of that progress vanishes.\nNext week we solve that problem with\nOrbax\n, JAX's checkpointing library. We will learn how to save not just model weights, but the entire training state: optimizer momentum, step count, and even the data iterator position from Grain, so a training run can be paused and resumed without losing a single batch of progress.\nNext week\n:\nNever lose your progress: Checkpointing with Orbax\n#\ndata-pipelines\n#\njax",
  "metadata": {
    "title": "Feeding the Beast: Data Pipelines with Grain",
    "description": "Over the last seven weeks we have built models, written training loops, composed optimizers, bulletproofed our code with Chex, and learned to debug inside JIT-compiled functions. Every one of those ar",
    "keywords": null,
    "author": null,
    "og:title": "Feeding the Beast: Data Pipelines with Grain",
    "og:description": "Over the last seven weeks we have built models, written training loops, composed optimizers, bulletproofed our code with Chex, and learned to debug inside JIT-compiled functions. Every one of those ar",
    "og:url": "https://kambale.dev/data-pipelines-with-grain",
    "og:site_name": "Wesley Kambale",
    "og:image": "https://cdn.hashnode.com/uploads/covers/61143e31119030192497a888/65c938bc-280a-4c5f-9561-935c68fad8df.png",
    "og:image:width": "1200",
    "og:image:height": "630",
    "og:type": "article",
    "twitter:card": "summary_large_image",
    "twitter:creator": "https://twitter.com/WesKambale",
    "twitter:title": "Feeding the Beast: Data Pipelines with Grain",
    "twitter:description": "Over the last seven weeks we have built models, written training loops, composed optimizers, bulletproofed our code with Chex, and learned to debug inside JIT-compiled functions. Every one of those ar",
    "twitter:image": "https://cdn.hashnode.com/uploads/covers/61143e31119030192497a888/65c938bc-280a-4c5f-9561-935c68fad8df.png",
    "article:published_time": "2026-07-29T01:04:35.736Z",
    "article:modified_time": "2026-07-29T01:04:35.736Z",
    "article:author": "https://kambale.dev/@kambale"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/20348d1d5a54/code/body-000.txt",
      "chars": 242,
      "language": "text"
    },
    {
      "file": "attachments/20348d1d5a54/code/body-001.txt",
      "chars": 170,
      "language": "text"
    },
    {
      "file": "attachments/20348d1d5a54/code/body-002.txt",
      "chars": 541,
      "language": "text"
    },
    {
      "file": "attachments/20348d1d5a54/code/body-003.txt",
      "chars": 169,
      "language": "text"
    },
    {
      "file": "attachments/20348d1d5a54/code/body-004.txt",
      "chars": 179,
      "language": "text"
    },
    {
      "file": "attachments/20348d1d5a54/code/body-005.txt",
      "chars": 197,
      "language": "text"
    },
    {
      "file": "attachments/20348d1d5a54/code/body-006.txt",
      "chars": 105,
      "language": "text"
    },
    {
      "file": "attachments/20348d1d5a54/code/body-007.txt",
      "chars": 1310,
      "language": "text"
    },
    {
      "file": "attachments/20348d1d5a54/code/body-008.txt",
      "chars": 157,
      "language": "text"
    },
    {
      "file": "attachments/20348d1d5a54/code/body-009.txt",
      "chars": 162,
      "language": "text"
    },
    {
      "file": "attachments/20348d1d5a54/code/body-010.txt",
      "chars": 177,
      "language": "text"
    },
    {
      "file": "attachments/20348d1d5a54/code/body-011.txt",
      "chars": 455,
      "language": "text"
    },
    {
      "file": "attachments/20348d1d5a54/code/body-012.txt",
      "chars": 1218,
      "language": "text"
    },
    {
      "file": "attachments/20348d1d5a54/code/body-013.txt",
      "chars": 952,
      "language": "text"
    },
    {
      "file": "attachments/20348d1d5a54/code/body-014.txt",
      "chars": 768,
      "language": "text"
    },
    {
      "file": "attachments/20348d1d5a54/code/body-015.txt",
      "chars": 924,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/uploads/covers/61143e31119030192497a888/65c938bc-280a-4c5f-9561-935c68fad8df.png",
      "bytes": 65936,
      "reused": true,
      "local_path": "attachments/20348d1d5a54/images/000.png",
      "src": "https://cdn.hashnode.com/uploads/covers/61143e31119030192497a888/65c938bc-280a-4c5f-9561-935c68fad8df.png",
      "alt": "Feeding the Beast: Data Pipelines with Grain",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://kambale.dev/data-pipelines-with-grain.md",
      "label": "View as Markdown",
      "type": "a"
    },
    {
      "href": "https://hashnode.com/@kambale",
      "label": "W",
      "type": "a"
    },
    {
      "href": "https://twitter.com/WesKambale",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://linkedin.com/in/WesKambale",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://kambale.dev/series/jax-series",
      "label": "Part of series JAX",
      "type": "a"
    },
    {
      "href": "https://kambale.dev/flax-nnx",
      "label": "built models",
      "type": "a"
    },
    {
      "href": "https://kambale.dev/training-loop-in-jax",
      "label": "training loops",
      "type": "a"
    },
    {
      "href": "https://kambale.dev/optax-optimizers",
      "label": "optimizers",
      "type": "a"
    },
    {
      "href": "https://kambale.dev/chex",
      "label": "Chex",
      "type": "a"
    },
    {
      "href": "https://kambale.dev/debugging-jax",
      "label": "JIT-compiled functions",
      "type": "a"
    },
    {
      "href": "https://kambale.dev/tag/data-pipelines",
      "label": "# data-pipelines",
      "type": "a"
    },
    {
      "href": "https://kambale.dev/tag/jax",
      "label": "# jax",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:50:46Z",
  "source_url": "https://kambale.dev/data-pipelines-with-grain",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 19782,
  "expected_body_images": 1,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/20348d1d5a54/pages/source.html",
  "local_html": "attachments/20348d1d5a54/pages/article.html",
  "body_base_path": "attachments/20348d1d5a54/pages/",
  "body_markdown_path": "attachments/20348d1d5a54/pages/article.md"
}