{
  "source_file": "0930",
  "source_platform": "Hashnode",
  "author": "Jeevika S",
  "title_listed": "Building an RL Traffic Agent for Indian Roads, Part 5: The Math, the Bugs, and Everything That Broke",
  "keywords": "",
  "domain": "大模型算法",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://local-minima.hashnode.dev/building-an-rl-traffic-agent-for-indian-roads-part-5-the-math-the-bugs-and-everything-that-broke",
  "publish_date": "2026-08-08",
  "corpus_stage": "",
  "notes": "",
  "id": "320ecc08b402",
  "collected_at": "2026-10-02T14:12:14",
  "title": "RL Traffic Agent, Part 5: The Math, the Bugs, the Fixes",
  "body_markdown": "Building an RL Traffic Agent for Indian Roads, Part 5: The Math, the Bugs, and Everything That Broke\nUpdated\nAugust 8, 2026\n•\n18\nmin read\n•\nView as Markdown\nJ\nJeevika S\nDocumenting research and projects across deep learning and reinforcement learning\nPart of series\nSelf-driving cars\nPart 4 was the reasoning that got me to an architecture on paper. This part is what happened once I actually tried to build it, which turned out to be a very different exercise. Most of Part 4 was \"what's the right idea.\" Most of this part is \"why doesn't the idea work yet.\"\nPhase 1: Turning LIDAR Into a Grid, for Real\nThe theory said: project raw LIDAR into a bird's-eye occupancy grid. The actual math behind that starts with converting an array index into a physical angle.\nWith 240 LIDAR rays sweeping a full circle, the angular step between rays is\nΔθ = 2π / 240 = π/120 radians\n, which works out to 1.5° per ray. Each ray's angle follows\nθᵢ = -i × π/120\n, negative because the sweep runs clockwise, something I'd confirmed empirically back in Part 4.\nFrom there it's just polar coordinates turning into cartesian ones. Each raw LIDAR value is normalized between 0 and 1, so the first step is denormalizing it back into an actual distance:\ndᵢ = lidarArray[i] × 50\n(50 meters being MetaDrive's max sensor range). Then standard polar-to-cartesian conversion:\nxᵢ = dᵢ × cos(θᵢ)\n,\nyᵢ = dᵢ × sin(θᵢ)\n.\nThen cartesian coordinates have to land somewhere on an actual grid. With a 50-meter physical range mapped onto 64 cells, resolution comes out to\n50 / 64 ≈ 0.78125 meters per cell\n. Center row and center column both sit at 32, the middle of the grid. Mapping x (forward/backward) to row and y (left/right) to column:\nrowᵢ = 32 − floor(xᵢ / 0.78125)\ncolᵢ = 32 + floor(yᵢ / 0.78125)\nClean enough on paper. It broke almost immediately once I actually ran it.\nThe Off-By-One That Wasn't Off By One\nRunning the projection function gave me row and column values of -28 and -15. Both negative. Both way outside the valid [0, 63] range a 64-cell grid can actually index into.\nMy first instinct was that this had to be some kind of indexing mistake, an off-by-one somewhere in the formula. It wasn't. The math was correct. The problem was the assumption baked into it: I'd centered the car symmetrically at row 32, column 32, giving it 32 cells of vision in every direction. But a car doesn't need to see equally far behind it as in front of it. Anything relevant enough to matter is almost always ahead. So a symmetric grid was wasting half its resolution looking at empty space behind the vehicle, and when something genuinely far ahead showed up, the math had nowhere left to put it.\nI looked at three ways to fix it. Keep the same 50-meter physical range but bump up the cell count, which fixes the resolution problem but costs hardware, going from 4,096 cells to 16,384 is a real jump for a CPU-only setup. Shrink the max range down to 25 meters instead, which keeps the grid the same size but trades away how far ahead the car can actually see, a real safety concern at any real driving speed. Or keep the 64×64 grid as-is and just expand the physical area it covers, out to 100×100 meters, which sounds reasonable until you realize resolution then drops to 1.5625 meters per cell, coarse enough that the car effectively loses the ability to thread between obstacles in traffic.\nNone of those were quite right on their own. The actual fix was to stop assuming the car needs to be centered at all. Recenter the origin and shift it down in the grid: 48 cells of forward vision (37.5 meters) and only 16 cells of rear vision (12.5 meters). Same 64×64 grid, same resolution, but the vision budget goes where it's actually needed.\nPhase 2: From Grid to Latent Vector\nWith a working grid, next came turning it into something the network could actually reason over.\nFirst snag: PyTorch's\nConv2d\nlayer doesn't accept a flat 2D matrix. It expects a channel dimension representing feature depth, so a raw\n(64, 64)\nmatrix has to become a\n(1, 64, 64)\ntensor, channels first, then height, then width. That channel dimension turned out to be useful for more than just satisfying PyTorch, though. Stacking the last four frames together, so the network can infer the speed of surrounding vehicles from how they move between frames rather than just where they sit in a single snapshot, gives a\n(4, 64, 64)\ntensor. And SAC doesn't train on one transition at a time anyway, it trains on random batches of past transitions, so the actual tensor shape flowing through the network is\n(batch_size, channels, height, width)\n, which in practice meant\n(256, 4, 64, 64)\n.\nFrom there, three convolutional layers do the downsampling. Each uses kernel size 3, stride 2, padding 1, which halves the spatial dimensions cleanly at every step: 64 → 32 → 16 → 8. Channel count grows the opposite direction as spatial size shrinks: 4 → 8 → 16 → 32. Thirty-two channels is enough, since the network only needs to pick out geometric gaps and the rough rectangular blob of a moving vehicle, not fine texture. By the third layer, the original 64×64 grid has been compressed down into an 8×8 feature map,\n(32, 8, 8)\n.\nThen came the part I had to think through more carefully: getting that feature map into shape for the attention layer. A 2D convolution outputs\n(batch, channels, height, width)\n. An attention layer wants\n(batch, sequence_length, embedding_dimension)\n. So the CNN's\n(32, 8, 8)\noutput effectively becomes 64 small \"tokens,\" each one representing a different patch of the original grid, like slicing the image into an 8×8 checkerboard of tiles. Feed that sequence through\nnn.MultiheadAttention\nand it evaluates the relationships between distant patches of the grid, letting the network reason about a vehicle three lanes over and gaps opening up somewhere else on the board at the same time. Flatten the attention output back down and you get a dense 1D vector of 2,048 dimensions,\ntorch.flatten(attn_output, start_dim=1)\n.\nOne more question came up here that I didn't want to skip past: does the attention layer actually need positional information? Without it, two options existed. Give the network explicit positional encoding, telling it exactly where each tile sits in the grid. Or let it work that out on its own through training, which keeps the code simpler but asks more of the model. I tested it. The shallow CNN genuinely could not work out spatial position on its own, no matter how much training data it got, there just wasn't enough depth in three conv layers for it to infer geometry implicitly. So positional encoding it was, giving the RL agent a representation that's not just rich, but actually spatially aware.\nThe Rolling Buffer, and the Bug That Taught Me Something\nStacking four frames sounds simple until you have to actually implement \"keep the last four, drop anything older\" cleanly. Python's\ncollections.deque\nwith\nmaxlen=4\nhandles that exact behavior, fixed size, push-new-drop-old, and it does it in O(1) time, which matters when this runs every single environment step.\nThat raised an immediate cold-start question, though: what happens at\nenv.reset()\n, before four real frames even exist yet? I went with repeating the very first frame four times, effectively telling the network \"the ego vehicle and all surrounding obstacles are currently stationary relative to my temporal window.\" Not perfectly accurate, but a reasonable assumption for a single instant in time.\nStacking order mattered too, entirely for my own sanity more than anything mathematically required: newest frame at the highest index, oldest at index 0, consistently, every time.\nThe wrapper architecture that ties all of this together works in five steps. The RL algorithm calls\nwrapper.step(action)\n. The wrapper calls the underlying simulator's\nenv.step(action)\n, retrieving the raw LIDAR array. The wrapper passes that raw array into a stateless\nproject_lidar_to_bev\nfunction and gets a 64×64 grid back. That grid gets appended to an internal deque capped at four frames. Finally the deque converts into a\n(4, 64, 64)\ntensor and gets handed back to the RL agent.\nI finally got\ntest_framestack.py\nrunning without crashing. And then immediately hit something worse than a crash: silent wrong output. The test reported\nnewest frame changed after step: False\n. It was supposed to say\nTrue\n.\nMy first two questions, before touching any code: if there were no obstacles anywhere near the car at reset time, what would the occupancy grid actually look like, mostly zeros, or full of ones? And if the car has barely moved after a single step at zero throttle, should I actually expect the surrounding environment to have changed meaningfully in that one step, or would it plausibly look identical to the frame before it, simply because nothing happened?\nThat second question was the real flaw in my test design, not a flaw in the code. Two things were true simultaneously, and I hadn't put them together. Mathematically: the projection function I'd built specifically filters out and drops any LIDAR return of 0.99 or higher, treating \"nothing detected\" as empty space. In a scene with no obstacles nearby, that gives you a full 64×64 matrix of absolute zero. And physically: with zero throttle applied, there's zero propulsion, so the ego vehicle's velocity stays at zero and its displacement across that single simulated step is precisely 0.0 meters. When the frame stack pushes that new grid into the deque, it's mathematically identical to the one before it, and the test correctly reported that the newest frame hadn't changed. Because it genuinely hadn't.\nSo the bug wasn't a bug. It was a test that couldn't have passed under those conditions no matter how correct the underlying code was. I fixed the test itself in three steps: added a debug print checking\ntorch.sum(obs)\n, since a sum of exactly zero confirms the empty-scene theory directly. Forced an actual physical state transition by changing the test action to\n[0.0, 1.0]\n, maximum throttle. And accounted for spatial resolution, since a single 0.1-second step at low starting speed might only move the car a few centimeters, nowhere near enough to shift an obstacle's projection into a genuinely different grid cell. To meaningfully test the rolling buffer at all, I needed a loop applying full throttle across ten to twenty steps, not one.\nAnd even after all that, the sum still came back zero. Which meant the actual bug wasn't in the frame stack logic at all, it was sitting one layer deeper, back in\nproject_lidar_to_bev\nitself. That's the same off-by-one-that-wasn't from Phase 1, the -28/-15 out-of-bounds row and column values. Two separate debugging threads, same root cause, and I only found the real one because the \"obvious\" symptom in the frame stack test forced me to question an assumption I hadn't actually verified yet.\nPhase 3: Teaching SAC to Speak in Steering Angles\nMetaDrive expects a continuous action between -1 and 1. SAC's actor, by construction, samples from an unbounded Gaussian distribution. Somewhere between those two facts, you need a function that squashes an unbounded real number into a strict range, and the standard choice is the hyperbolic tangent,\ntanh\n, in PyTorch.\nWorth being explicit about why this has to be\ntanh\nspecifically and not something simpler like clamping. Clamping isn't differentiable at the boundary, the gradient just dies the moment a value hits the clip. SAC needs gradients to flow cleanly through the entire action-generation step during training, so the squashing function has to be smooth everywhere, not just bounded.\ntanh\nfits:\na_squashed = tanh(a_raw)\n, and its derivative,\nd/dx tanh(x) = 1 − tanh²(x)\n, stays well-defined across the whole range, which is exactly what backpropagation needs to work through it.\nThe Change-of-Variables Trap\nHere's the part that isn't obvious until it bites you. Once you apply a nonlinear transform like\ntanh\nto a random variable, you fundamentally alter its probability density. The log-probability SAC needs for training isn't just the log-probability of the raw, pre-squash Gaussian sample, that number is now wrong, because the actual distribution of actions is the squashed one, not the raw one.\nFixing that means applying the change-of-variables formula from probability theory, sometimes called the Jacobian correction:\na_raw = μ + σ × ε\nCompute the normal log-probability of\na_raw\nunder the Gaussian.\na_squashed = tanh(a_raw)\nJacobian correction:\nlog_prob = normal_log_prob − log(1 − tanh²(a_raw) + ε)\nThat correction term increases the final log-probability, makes it less negative, and it took me a minute to convince myself why that direction made sense. The same amount of total probability mass, all of it, 1.0 worth, is now squeezed into a drastically smaller area once you've squashed the distribution through\ntanh\n. Squeeze the same mass into a smaller space and density has to go up. Subtracting that negative log-density term is what correctly accounts for that physical compression of probability mass. Get this step wrong and SAC's entropy term, the thing actively encouraging exploration, ends up computing exploration incentives against a distribution that doesn't actually match the agent's real behavior.\nOne implementation detail worth flagging on its own: that small epsilon inside the log has to be placed directly inside the log, not added somewhere else in the expression, specifically to protect the codebase from a floating-point failure when\n1 − tanh²(a_raw)\ngets close enough to zero that the log would otherwise blow up.\nWhere Exactly Does tanh Get Applied?\nI wanted the precise lifecycle of the tensor clear in my head before calling this piece of the actor done. The actor network evaluates the raw LIDAR state and outputs the parameters of a Gaussian distribution, mean and standard deviation. Using the reparameterization trick, it samples a standard normal noise vector,\nε ~ N(0,1)\n. That gets combined into the unbounded raw action,\na_raw = μ + σ·ε\n. Only at the very end of that sequence does the unbounded sample get squashed into the strict [-1, 1] range MetaDrive actually requires,\na = tanh(a_raw)\n.\nSteering and Throttle Aren't the Same Question\nThe actor outputs two independent values, steering and throttle, each with its own log-probability. If two events are genuinely independent, the probability of both happening together is\nP(A) × P(B)\n, and given log-probability values already, that becomes\nlog(P(A)) + log(P(B))\n. Which means combining the two dimensions correctly requires\n.sum(dim=-1)\n, not\n.mean()\n. Summing is the algebraically correct way to compute a joint log-probability across independent events. Averaging would quietly fabricate a probabilistic concept that doesn't actually exist, there's no meaningful \"average log-probability\" of two unrelated control decisions.\nAnd one more question that seemed worth settling explicitly: does a fully deterministic action, at evaluation time, still need to go through\ntanh\n? Yes, unconditionally. The squashing function isn't just a training-time trick, it's required to safely compress whatever the raw neural output is into a valid physical control signal, stochastic sample or not. The log-probability calculation, on the other hand, genuinely is only meaningful during training, so it's intentionally skipped at evaluation time rather than computed and thrown away.\nPhase 4: The Training Loop\nStrip away all the RL terminology and the training loop does exactly three things. It runs the agent through the environment, collecting experience. It stores that experience somewhere. And it periodically samples from that storage to actually update the network's weights.\nStoring Experience Without Bottlenecking the CPU\nFirst instinct was a Python list of tuples. Wrong instinct, that bottlenecks the CPU almost immediately once you're storing tens of thousands of transitions and sampling from them constantly. The fix was a custom replay buffer built on pre-allocated, fixed-size NumPy arrays instead:\nself.obs_actor_buffer = np.zeros((capacity, 4, 64, 64), dtype=np.float32)\n, with matching pre-allocated arrays for actions, rewards, and everything else. Sampling a batch just means drawing random indices,\nnp.random.randint(0, current_size, batch_size)\n, and indexing straight into those arrays.\nSoft Target Updates\nTarget networks stabilize training by moderating how fast Q-value estimates are allowed to shift, and the mechanism for that is Polyak averaging:\nθ_target = τ·θ_live + (1 − τ)·θ_target\n, where τ sits around 0.005. Each update absorbs only about half a percent of the live network's new values, retaining 99.5 percent of what the target network already knew. Slow, deliberate, and exactly what you want when the thing you're trying to stabilize is itself a moving target.\nAssembling What the Critic Actually Sees\nThe critic's full input vector breaks down into three segments:\n[0:19]\nis ego kinematics plus navigation, unchanged from the raw observation.\n[19:35]\nis the privileged sixteen-dimensional\nnum_others\nstate.\n[35:275]\nis the full 240-ray LIDAR sweep. That layout matters, since getting the slice indices wrong here means silently feeding the critic garbage instead of the ground-truth it's actually supposed to have.\nA Buffer That Was Too Big to Actually Exist\nI'd initially set replay buffer capacity to 100,000, which sounded reasonable until I actually computed the memory footprint.\ncapacity × 4 frames × 64 height × 64 width × 4 bytes\nworks out to roughly 6,553,660,000 bytes, about 6.10 GiB, just for the\nobs_actor_buffer\n. And the replay buffer also has to instantiate an identically-sized\nnext_obs_actor_buffer\nright alongside it, doubling that to roughly 12.20 GiB. Not remotely reasonable for a CPU-only setup with limited RAM. The fix was simply reducing capacity down to somewhere in the 25,000 to 50,000 range instead.\nTerminated Isn't the Same as Truncated\n\"Done\" isn't a single concept, it's actually two different signals wearing one name.\nterminated = True\nmeans the episode genuinely ended, the vehicle crashed, or it safely reached its destination.\ntruncated = True\nmeans something entirely different: the vehicle was driving perfectly safely the whole time, but an internal step-count clock simply ran out. Conflating those two would teach the agent that running out of time is somehow the same kind of failure as crashing into a wall, which is exactly backwards. The fix is separating the signals explicitly:\ndone_buffer = 1.0\nif terminated, and\n0.0\neven if truncated is true, since a truncated episode shouldn't zero out the Bellman target the way a genuine terminal state should.\nRedesigning the Frame Wrapper Around All of This\nOnce the asymmetric actor-critic design from Part 1 was actually being implemented, the wrapper needed a cleaner separation of responsibilities than it started with.\nThe wrapper's own job is narrow: the dictionary coming out of\nenv.step()\nhas to contain exactly two raw, unadulterated tensors, the\n(4, 64, 64)\nstacked bird's-eye grid, and the\n(16,)\nnum_others\nvector. Nothing processed, nothing latent, just the raw material.\nThe replay buffer's job is to respect that separation, too. It can never store latent vectors, only the raw grid in one array and the raw\nnum_others\nin a completely separate one. Otherwise you'd be baking a specific version of the CNN's weights into stored data that's supposed to outlive many rounds of training on that same CNN.\nThe actual assembly happens at update time, in four steps. Sample the raw batch, a grid tensor of shape\n(256, 4, 64, 64)\nand a\nnum_others\ntensor of shape\n(256, 16)\n. Run a dynamic forward pass, pushing the grid batch through the Phase 2 CNN-and-attention module to get a latent batch of shape\n(256, 2048)\n. Route to the actor: since its state representation relies entirely on spatial geometry,\nobs_actor = latent_batch\n, full stop. And assemble the critic's privileged input by concatenating the latent with the ground-truth vehicle data:\nobs_critic = torch.cat([latent_batch, num_others_batch], dim=1)\n.\nWhy Any of This Actually Matters\nTwo reasons this structure earns its complexity, and neither is just theoretical.\nAlgorithmic stability and variance reduction. Continuous control RL runs into a real scarcity of high-quality samples, which is exactly why experience replay exists, to store and reuse transitions instead of learning from each one once and discarding it. If the vehicle executes a long, sweeping left turn across fifty consecutive steps, and you perform fifty consecutive gradient updates purely on that stretch, the network forgets the generalized knowledge it previously built up about ordinary straight-line driving. Collecting a batch of varied experience before updating lets the replay buffer feed the network a wider, more representative slice of the actual driving problem, not just whatever's happening in this exact moment.\nComputational efficiency and hardware constraints. Decoupling data collection from the actual network update lets the system balance its workload deliberately instead of doing both at once on every single step. In practice that means tracking a global step counter, using it to dictate the warmup-phase logic, and checking it against a modulo operator to decide exactly when an update actually fires.\nThat's Phase 4, and with it, the full pipeline: raw LIDAR in, a trained continuous-control policy out. What's left for a possible Part 6 is the actual training run itself, whatever that ends up looking like once it's been running long enough to say something real about it.\n#\nreinforcement-learning\n#\npytorch\n#\nmachine-learning\n#\npython",
  "metadata": {
    "title": "RL Traffic Agent, Part 5: The Math, the Bugs, the Fixes",
    "description": "Turning theory into code — LIDAR-to-grid math, tanh squashing, the change-of-variables trap, and bugs that looked broken but weren't.",
    "keywords": null,
    "author": null,
    "og:title": "RL Traffic Agent, Part 5: The Math, the Bugs, the Fixes",
    "og:description": "Turning theory into code — LIDAR-to-grid math, tanh squashing, the change-of-variables trap, and bugs that looked broken but weren't.",
    "og:url": "https://local-minima.hashnode.dev/building-an-rl-traffic-agent-for-indian-roads-part-5-the-math-the-bugs-and-everything-that-broke",
    "og:site_name": "Local Minima",
    "og:image": "https://local-minima.hashnode.dev/api/og/post?og=eyJhdXRob3IiOiJKZWV2aWthJTIwUyIsImlzRGVmYXVsdE1vZGVEYXJrIjpmYWxzZSwicmVhZFRpbWUiOjE4LCJjb21tZW50cyI6MCwicmVhY3Rpb25zIjowLCJkb21haW4iOiJsb2NhbC1taW5pbWEuaGFzaG5vZGUuZGV2IiwidGl0bGUiOiJCdWlsZGluZyUyMGFuJTIwUkwlMjBUcmFmZmljJTIwQWdlbnQlMjBmb3IlMjBJbmRpYW4lMjBSb2FkcyUyQyUyMFBhcnQlMjA1JTNBJTIwVGhlJTIwTWF0aCUyQyUyMHRoZSUyMEJ1Z3MlMkMlMjBhbmQlMjBFdmVyeXRoaW5nJTIwVGhhdCUyMEJyb2tlIiwicGhvdG8iOiJodHRwczovL2xoMy5nb29nbGV1c2VyY29udGVudC5jb20vYS9BQ2c4b2NLTlhCQUdIdlNmWW1VUlg1dzQzenNTZXQ0dWpxMjl2MjB6VU4yX2o4RlhaTTdqeXc9czk2LWMiLCJsb2dvIjoiaHR0cHM6Ly9jZG4uaGFzaG5vZGUuY29tL3Jlcy9oYXNobm9kZS9pbWFnZS91cGxvYWQvdjE1OTM2ODAyODI4OTYva05DN0U4SVI0LnBuZyIsImZhdmljb24iOiJodHRwczovL2Nkbi5oYXNobm9kZS5jb20vcmVzL2hhc2hub2RlL2ltYWdlL3VwbG9hZC92MTU5MzY4MDI4Mjg5Ni9rTkM3RThJUjQucG5nIiwicHVibGlzaGVkQXQiOiIyMDI2LTA4LTA4VDA0OjM4OjA3LjQyM1oiLCJ1cGRhdGVkQXQiOiIyMDI2LTA4LTA4VDA0OjM4OjA3LjQyM1oifQ==",
    "og:image:width": "1200",
    "og:image:height": "630",
    "og:type": "article",
    "twitter:card": "summary_large_image",
    "twitter:title": "RL Traffic Agent, Part 5: The Math, the Bugs, the Fixes",
    "twitter:description": "Turning theory into code — LIDAR-to-grid math, tanh squashing, the change-of-variables trap, and bugs that looked broken but weren't.",
    "twitter:image": "https://local-minima.hashnode.dev/api/og/post?og=eyJhdXRob3IiOiJKZWV2aWthJTIwUyIsImlzRGVmYXVsdE1vZGVEYXJrIjpmYWxzZSwicmVhZFRpbWUiOjE4LCJjb21tZW50cyI6MCwicmVhY3Rpb25zIjowLCJkb21haW4iOiJsb2NhbC1taW5pbWEuaGFzaG5vZGUuZGV2IiwidGl0bGUiOiJCdWlsZGluZyUyMGFuJTIwUkwlMjBUcmFmZmljJTIwQWdlbnQlMjBmb3IlMjBJbmRpYW4lMjBSb2FkcyUyQyUyMFBhcnQlMjA1JTNBJTIwVGhlJTIwTWF0aCUyQyUyMHRoZSUyMEJ1Z3MlMkMlMjBhbmQlMjBFdmVyeXRoaW5nJTIwVGhhdCUyMEJyb2tlIiwicGhvdG8iOiJodHRwczovL2xoMy5nb29nbGV1c2VyY29udGVudC5jb20vYS9BQ2c4b2NLTlhCQUdIdlNmWW1VUlg1dzQzenNTZXQ0dWpxMjl2MjB6VU4yX2o4RlhaTTdqeXc9czk2LWMiLCJsb2dvIjoiaHR0cHM6Ly9jZG4uaGFzaG5vZGUuY29tL3Jlcy9oYXNobm9kZS9pbWFnZS91cGxvYWQvdjE1OTM2ODAyODI4OTYva05DN0U4SVI0LnBuZyIsImZhdmljb24iOiJodHRwczovL2Nkbi5oYXNobm9kZS5jb20vcmVzL2hhc2hub2RlL2ltYWdlL3VwbG9hZC92MTU5MzY4MDI4Mjg5Ni9rTkM3RThJUjQucG5nIiwicHVibGlzaGVkQXQiOiIyMDI2LTA4LTA4VDA0OjM4OjA3LjQyM1oiLCJ1cGRhdGVkQXQiOiIyMDI2LTA4LTA4VDA0OjM4OjA3LjQyM1oifQ==",
    "article:published_time": "2026-08-08T04:38:07.423Z",
    "article:modified_time": "2026-08-08T04:38:07.423Z",
    "article:author": "https://local-minima.hashnode.dev/@jeevikasrinath"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://local-minima.hashnode.dev/building-an-rl-traffic-agent-for-indian-roads-part-5-the-math-the-bugs-and-everything-that-broke.md",
      "label": "View as Markdown",
      "type": "a"
    },
    {
      "href": "https://hashnode.com/@jeevikasrinath",
      "label": "J",
      "type": "a"
    },
    {
      "href": "https://www.linkedin.com/in/jeevika-s-82ab66325/",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://local-minima.hashnode.dev/series/self-driving-rl",
      "label": "Part of series Self-driving cars",
      "type": "a"
    },
    {
      "href": "https://local-minima.hashnode.dev/tag/reinforcement-learning",
      "label": "# reinforcement-learning",
      "type": "a"
    },
    {
      "href": "https://local-minima.hashnode.dev/tag/pytorch",
      "label": "# pytorch",
      "type": "a"
    },
    {
      "href": "https://local-minima.hashnode.dev/tag/machine-learning",
      "label": "# machine-learning",
      "type": "a"
    },
    {
      "href": "https://local-minima.hashnode.dev/tag/python",
      "label": "# python",
      "type": "a"
    }
  ],
  "retry_at": "2026-10-03T08:10:10",
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:51:02Z",
  "source_url": "https://local-minima.hashnode.dev/building-an-rl-traffic-agent-for-indian-roads-part-5-the-math-the-bugs-and-everything-that-broke",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 21659,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/320ecc08b402/pages/source.html",
  "local_html": "attachments/320ecc08b402/pages/article.html",
  "body_base_path": "attachments/320ecc08b402/pages/",
  "body_markdown_path": "attachments/320ecc08b402/pages/article.md"
}