{
  "source_file": "0930",
  "source_platform": "Ghost",
  "author": "Avi Chawla / Daily Dose of Data Science",
  "title_listed": "How To Build An RL Environment",
  "keywords": "",
  "domain": "大模型算法",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://www.dailydoseofds.com/p/how-to-build-an-rl-environment/",
  "publish_date": "2026-07-14",
  "corpus_stage": "",
  "notes": "",
  "id": "8140a8bb210c",
  "collected_at": "2026-10-02T13:46:34",
  "title": "How to build an RL environment?",
  "body_markdown": "Jul 14, 2026\nReinforcement Learning\nHow to build an RL environment?\n...explained step-by-step with code.\nAvi Chawla\n👉\nAndrej Karpathy summarized the entire history of LLM training in three nouns:\ntext\nconversations\nand environments\nPretraining ran on internet text, supervised fine-tuning ran on curated conversations, and the current era of reinforcement learning runs on environments.\nOpenAI’s o1 proved that framing by training on math and coding problems with verifiable answers, and DeepSeek-R1 published the recipe openly.\nThe industry now treats environments as the scarce resource. Anthropic reportedly discussed spending over $1 billion on them in a single year.\nMeanwhile, someone is giving them away for free through a Hugging Face-style hub hosting 2,500+ open-source environments.\nAnd today, we’ll use their framework to build one of our own from the ground up.\nBefore we build anything, let’s understand what an environment actually is. Every RL setup is a loop with four moving parts, and they map onto anything the model interacts with:\nState → what the model sees right now\nAction → the choice it makes from that state\nReward → a number that says how good that choice was\nEnvironment → the thing that holds the state, accepts the action, and returns the reward\nThe hard part of this loop was always the reward, which traditionally required training a separate model on human preferences.\nDeepSeek-R1 replaced that entire model with a plain Python function through GRPO (Group Relative Policy Optimization). Score several answers to the same prompt and push the model toward the ones that beat the group average.\nHere is an illustration of how GRPO works:\nThat leaves the environment as the barrier still standing. A reward function can score a final state, but something has to present the task, accept the model’s actions, enforce the rules, and produce that final state in the first place.\nFrontier labs guard this part as a proprietary asset, which is why few practitioners have ever seen a working environment from the inside.\nThat team giving environments away is Prime Intellect, and their library,\nVerifiers\n, is the framework we’ll build with (100% open-source).\nThe design is model-agnostic, the rewards are fully verifiable, and the same skeleton works for any turn-based task you want to train on.\nHow to read this issue\nThe goal here is simple. We’re building a complete RL environment, and we’re covering it in a way that stays easy to digest even if this is your first one.\nEvery idea comes with the code that implements it, kept short enough to read in the flow of the article. The complete, runnable version is shared at the end.\nAnd the game we’ll build around is just the working example. By the time you finish, you’ll be able to take this exact structure and adapt it to your own use case.\nThe Othello RL env: Why a board game\nOthello is a two-player game on an 8x8 grid. You place a disc so it traps opponent discs in a straight line, every trapped disc flips to your color, and whoever owns more discs when the board fills up wins.\nThat flipping rule is why a single move can swing the score hard, and why corners matter so much (a corner disc can never be flipped back).\nEvery RL part has a home here. The board is the state, the disc placement is the action, the final result feeds the reward, and the game engine that validates moves, flips discs, and plays the other side is the environment.\nThe LLM plays Black and a built-in engine plays White. After each Black move, White responds, the updated board goes back to the model, and this repeats until the game ends.\nTech stack\nThree tools make this work, each handling one layer:\nVerifiers\n: the RL framework. It defines the environment, runs the turn loop, and handles evals.\nLightning AI\n: an OpenAI-compatible inference API, so the same code calls hosted models like Claude or DeepSeek without provider-specific rewrites.\nvLLM\n: serves open-weight models locally behind that same OpenAI-compatible endpoint.\nThe shared interface is what makes the environment model-agnostic. A local Ministral-3B and a hosted GPT-4.1 swap in one line, and you change the model name, nothing else.\nThe game loop\nHere’s what the model sees on each turn:\nThe board, the score, and the list of valid moves are the entire state. Everything the model knows about the game comes from this text.\nIt must respond with a <think> section followed by a\n<move>\ntag, which forces it to reason through the position before committing to a move.\nThe environment validates the move against the valid list, applies it to the board, lets White respond, and sends the updated board back. An invalid move gets an error message and a retry, with a penalty applied to the reward.\nAll of that lives in one method.\nOthelloEnv\ninherits from\nMultiTurnEnv\nin Verifiers, which handles the loop, turn tracking, and termination, and the base class calls your env_response every time the model sends a move:\nclass OthelloEnv(MultiTurnEnv):\n    def env_response(self, model_output, state):\n        move = parse_move(model_output)\n\n        if not is_valid(move, state.board):\n            state.penalty += INVALID_MOVE_PENALTY\n            return error_message(move), state          # same turn, retry\n\n        state.board = apply_move(state.board, move, player=\"black\")\n\n        if not game_over(state.board):\n            white_move = opponent_engine(state.board, state.difficulty)\n            state.board = apply_move(state.board, white_move, player=\"white\")\n\n        return render_board(state.board), state\nThis is the shape of the logic; the real version also handles board parsing and edge cases that the snippet skips. I’ll share the full working code later once we understand the setup end-to-end.\nThe built-in opponent engine\nThe built-in engine plays white and has two modes:\nRandom\npicks any legal move. Useful for early training, since the model can win just by making reasonable decisions.\nMinimax\nsimulates future moves, scores each resulting position by corner control, board position, and available moves, and picks the move with the best worst-case outcome. Depth-3 means it looks three moves ahead, enough to set traps and avoid obvious blunders.\nA randomness parameter controls how often White ignores its strategy and picks randomly instead. Lower randomness means more consistent, punishing play.\nWhite’s moves are generated from the current board state and a fixed game seed, so the same position always produces the same response. Determinism lets you compare different models under identical conditions.\nHere’s how the code for the same looks like:\ndef opponent_engine(board, randomness, depth):\n    if random.random() < randomness:               # random mode\n        return random.choice(legal_moves(board))\n    best_move = None                                 # minimax mode: try every move,\n    best_score = float(\"-inf\")                       # keep the one with the best score\n    for move in legal_moves(board):\n        score = minimax(apply_move(board, move), depth - 1, my_turn=False)\n        if score > best_score:\n            best_move, best_score = move, score\n    return best_move\nThe reward functions\nAt game end, four signals combine into a single reward:\nEach one is a function that reads something off the final state and returns a number, and the win signal is the simplest:\ndef win_reward_func(state):\n    result = state.get(\"result\")\n    if result == \"black\":   # the model's color\n        return 1.0\n    if result == \"draw\":\n        return 0.5\n    return 0.0              # loss, or game never finished\nThe other three follow the same shape, and all four combine into one score:\ndef total_reward(state):\n    return (\n        win_loss_score(state)\n        + piece_advantage(state)\n        + format_compliance(state)\n        - invalid_move_penalty(state)\n    )\nThe reason for four signals instead of a single win/loss bit is resolution. Early in training, most games are losses, and to a pure win/loss reward, they all look identical, so the model has nothing to optimize.\nPiece advantage separates a close loss from a blowout, giving the model a gradient before it starts winning.\nFormat compliance carries a low weight, so clean formatting never outweighs good play.\nThe invalid move penalty is capped, so one broken game can’t drown out everything the model did right.\nEvery score comes directly from the game state and rules, with no judge model or LLM evaluator involved, so the reward is fully deterministic and reproducible.\nWiring it together\nThe environment generates games across different starting positions and opponent difficulties, the model plays each one through the loop above, and at game end the four rewards are computed and combined. During evaluation, rewards, token usage, and turn counts are aggregated across all games into a results table.\nOne function wires everything up, and it’s what the prime eval command calls behind the scenes:\ndef load_environment(min_random_move_prob, max_random_move_prob, parse_think):\n    dataset = generate_games(min_random_move_prob, max_random_move_prob)\n    parser = XMLParser(fields=[\"think\", \"move\"])\n    rubric = Rubric(funcs=[...], weights=[1.0, 1.0, 0.2, 1.0])\n    return OthelloEnv(dataset, parser, rubric)\nWhat the numbers show\nRunning an evaluation is one command, with the opponent settings passed as arguments:\nprime eval run othello -m openai/gpt-4.1 -n 100 \\\n  -a '{\"min_random_move_prob\": 0.0, \"max_random_move_prob\": 0.0, \"minimax_depth\": 3}'\nSwap the model name to test something else, whether it’s hosted through Lightning AI or running on a local vLLM server. Here’s how two models fared against two opponents:\nFrom evaluation to training\nEvaluation tells you where the model struggles, and training fixes it in three stages. The same environment supports all of them:\nData generation\n: have your strongest model play a few hundred games and save the results. The same eval command writes straight to a dataset:\nprime eval run othello -m openai/gpt-4.1 -n 200 \\\n  --save-to-hf-hub --hf-hub-dataset-name your-username/othello-data\nFilter it down to wins and draws before training on it, so you’re not teaching the model your strongest player’s mistakes alongside its format.\nSupervised fine-tuning\n: teach format and valid moves first. The Ministral-3B turn counts make the point directly, since unreliable formatting and illegal moves are noise RL can’t train through.\nRL training\n: this is where strategy improves. Multiple games are played from the same starting position, each is scored with the same reward functions from evaluation, and the model is updated toward the higher-scoring rollouts.\nStripped to its core, that’s the GRPO loop from the top of the article:\nfor prompt in batch:\n    rollouts = [play_game(model, prompt) for _ in range(group_size)]\n    rewards = [total_reward(r.final_state) for r in rollouts]\n    advantage = rewards - mean(rewards)     # relative to the group\n    update_model(model, rollouts, advantage)\nEach rollout is scored against the average of its own group, so a move that wins 6 of 10 games from a given position is rewarded more than the weaker attempts beside it.\nAdapting this to your own task\nOnce you remove the Othello specifics and the same MultiTurnEnv gives you a skeleton that fits any turn-based task:\nclass TaskEnv(MultiTurnEnv):\n    def env_response(self, model_output, state):\n        action = parse_action(model_output)          # your task's syntax\n        if not is_valid(action, state):\n            return error_message(action), penalize(state)\n        state = apply_action(state, action)          # your task's rules\n        if not task_complete(state):\n            state = environment_step(state)          # tool, API call, or opponent\n        return render_state(state), state            # your task's display\nThis isn’t specific to games. A coding agent swaps\napply_action\nfor running a test suite against generated code, a support agent swaps it for checking whether a tool call retrieved the right record, and a research task swaps it for verifying a claim against a cited source.\nAdapting it comes down to four swaps:\nTask logic\n: your domain’s rules for what counts as a valid action and how the state changes\nResponse engine\n: whatever the model reacts to, from a rule-based simulator to a live API to another model\nReward functions\n: keep the pattern (outcome signal, partial credit, format, penalty) and replace the domain logic\nState rendering\n: whatever the model needs to see, whether that’s a file diff, a conversation transcript, or a tool’s response\nThe structure underneath stays the same regardless of domain: parse, validate, apply, respond, score. The rubric is the design, and if you get the components right, the training signal takes care of itself.\nTry it yourself\nAll the code, setup instructions, and ready-to-use GPUs to reproduce these results are in the Lightning AI Studio template:\n​\nBuild a Custom RL Environment →\n​\n​\nCheck out Lightning AI Inference →\n​\nGood day!\nPublished on Jul 14, 2026\nComments\nShare\nCopy link\nShare to X\nShare to Facebook\nShare to Linkedin\nCopied\nPrevious\nAgentic RL: Environments, Trajectories, and the Training\nNext\nWhy Small Models Alone Don't Reduce Inference Costs",
  "metadata": {
    "title": "How to build an RL environment?",
    "description": "...explained step-by-step with code.",
    "keywords": null,
    "author": null,
    "og:site_name": "Daily Dose of Data Science",
    "og:type": "article",
    "og:title": "How to build an RL environment?",
    "og:description": "...explained step-by-step with code.",
    "og:url": "https://www.dailydoseofds.com/p/how-to-build-an-rl-environment/",
    "og:image": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/07/ef043ae9-92eb-450e-8132-5638bd3d16b6_1199x650.png",
    "og:image:width": "1199",
    "og:image:height": "650",
    "twitter:card": "summary_large_image",
    "twitter:title": "How to build an RL environment?",
    "twitter:description": "...explained step-by-step with code.",
    "twitter:url": "https://www.dailydoseofds.com/p/how-to-build-an-rl-environment/",
    "twitter:image": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/07/ef043ae9-92eb-450e-8132-5638bd3d16b6_1199x650.png",
    "twitter:label1": "Written by",
    "twitter:data1": "Avi Chawla",
    "twitter:label2": "Filed under",
    "twitter:data2": "Reinforcement Learning",
    "twitter:site": "@_avichawla",
    "article:published_time": "2026-07-14T15:51:19.000Z",
    "article:modified_time": "2026-07-14T15:51:19.000Z",
    "article:tag": "Reinforcement Learning"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/8140a8bb210c/code/body-000.txt",
      "chars": 597,
      "language": "text"
    },
    {
      "file": "attachments/8140a8bb210c/code/body-001.txt",
      "chars": 544,
      "language": "text"
    },
    {
      "file": "attachments/8140a8bb210c/code/body-002.txt",
      "chars": 230,
      "language": "text"
    },
    {
      "file": "attachments/8140a8bb210c/code/body-003.txt",
      "chars": 179,
      "language": "text"
    },
    {
      "file": "attachments/8140a8bb210c/code/body-004.txt",
      "chars": 310,
      "language": "text"
    },
    {
      "file": "attachments/8140a8bb210c/code/body-005.txt",
      "chars": 135,
      "language": "text"
    },
    {
      "file": "attachments/8140a8bb210c/code/body-006.txt",
      "chars": 117,
      "language": "text"
    },
    {
      "file": "attachments/8140a8bb210c/code/body-007.txt",
      "chars": 264,
      "language": "text"
    },
    {
      "file": "attachments/8140a8bb210c/code/body-008.txt",
      "chars": 517,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/size/w400/2026/03/avi-google.jpg",
      "bytes": 11344,
      "reused": true,
      "local_path": "attachments/8140a8bb210c/images/000.jpg",
      "src": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/size/w400/2026/03/avi-google.jpg",
      "alt": "Avi Chawla",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!DXZZ!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fef043ae9-92eb-450e-8132-5638bd3d16b6_1199x650.png",
      "bytes": 161031,
      "reused": true,
      "local_path": "attachments/8140a8bb210c/images/001.png",
      "src": "https://substackcdn.com/image/fetch/$s_!DXZZ!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fef043ae9-92eb-450e-8132-5638bd3d16b6_1199x650.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!SzKd!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc489fef2-ee3f-4421-8f1d-d8ccd4eea29a_1080x1029.gif",
      "bytes": 625265,
      "reused": true,
      "local_path": "attachments/8140a8bb210c/images/002.gif",
      "src": "https://substackcdn.com/image/fetch/$s_!SzKd!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc489fef2-ee3f-4421-8f1d-d8ccd4eea29a_1080x1029.gif",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!Y0Ky!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0bbf007b-d75f-41b7-aa21-b6f34e3bdaba_596x335.png",
      "bytes": 100146,
      "reused": true,
      "local_path": "attachments/8140a8bb210c/images/003.png",
      "src": "https://substackcdn.com/image/fetch/$s_!Y0Ky!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0bbf007b-d75f-41b7-aa21-b6f34e3bdaba_596x335.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!bjNW!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc13f3328-1e9b-484a-be35-945706c63a40_1200x569.png",
      "bytes": 165153,
      "reused": true,
      "local_path": "attachments/8140a8bb210c/images/004.png",
      "src": "https://substackcdn.com/image/fetch/$s_!bjNW!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc13f3328-1e9b-484a-be35-945706c63a40_1200x569.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!h4b8!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb2427bdd-f9a8-463f-a070-a316e31c6fde_892x538.png",
      "bytes": 10532,
      "reused": true,
      "local_path": "attachments/8140a8bb210c/images/005.png",
      "src": "https://substackcdn.com/image/fetch/$s_!h4b8!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb2427bdd-f9a8-463f-a070-a316e31c6fde_892x538.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!mcsI!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F23ec7e12-4c04-40ef-bff1-47cba968c800_1200x655.png",
      "bytes": 237436,
      "reused": true,
      "local_path": "attachments/8140a8bb210c/images/006.png",
      "src": "https://substackcdn.com/image/fetch/$s_!mcsI!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F23ec7e12-4c04-40ef-bff1-47cba968c800_1200x655.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!EbzK!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc9a25df1-ed2f-4ef4-9fe2-4ad62e4ef273_1200x380.png",
      "bytes": 116760,
      "reused": true,
      "local_path": "attachments/8140a8bb210c/images/007.png",
      "src": "https://substackcdn.com/image/fetch/$s_!EbzK!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc9a25df1-ed2f-4ef4-9fe2-4ad62e4ef273_1200x380.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!zy6V!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F146d2e45-0ec4-44a8-8f08-e00e07253473_1199x420.png",
      "bytes": 99759,
      "reused": true,
      "local_path": "attachments/8140a8bb210c/images/008.png",
      "src": "https://substackcdn.com/image/fetch/$s_!zy6V!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F146d2e45-0ec4-44a8-8f08-e00e07253473_1199x420.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!pC0h!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F65beaeef-e2a2-4c35-bf41-45ca2a68a5f5_1024x559.png",
      "bytes": 145104,
      "reused": true,
      "local_path": "attachments/8140a8bb210c/images/009.png",
      "src": "https://substackcdn.com/image/fetch/$s_!pC0h!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F65beaeef-e2a2-4c35-bf41-45ca2a68a5f5_1024x559.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!Cki6!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F61f39fe1-eba6-4e71-a222-88c5bfe8f8d7_1200x668.png",
      "bytes": 116778,
      "reused": true,
      "local_path": "attachments/8140a8bb210c/images/010.png",
      "src": "https://substackcdn.com/image/fetch/$s_!Cki6!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F61f39fe1-eba6-4e71-a222-88c5bfe8f8d7_1200x668.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!G9sd!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F75d86b5f-fd22-4935-9164-ec2a622cf9a6_1200x548.png",
      "bytes": 180159,
      "reused": true,
      "local_path": "attachments/8140a8bb210c/images/011.png",
      "src": "https://substackcdn.com/image/fetch/$s_!G9sd!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F75d86b5f-fd22-4935-9164-ec2a622cf9a6_1200x548.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!5Wil!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0debc0af-fd97-42c0-8cf3-a7a6c4b43a34_1200x748.png",
      "bytes": 161302,
      "reused": true,
      "local_path": "attachments/8140a8bb210c/images/012.png",
      "src": "https://substackcdn.com/image/fetch/$s_!5Wil!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0debc0af-fd97-42c0-8cf3-a7a6c4b43a34_1200x748.png",
      "alt": "",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://github.com/PrimeIntellect-ai/verifiers?ref=dailydoseofds.com",
      "label": "Verifiers",
      "type": "a"
    },
    {
      "href": "https://lightning.ai/lightning-ai/templates/build-a-custom-rl-environment?utm_campaign=akshay&utm_medium=newsletter&ref=dailydoseofds.com",
      "label": "​",
      "type": "a"
    },
    {
      "href": "https://lightning.ai/lightning-ai/models?utm_campaign=akshay&utm_medium=newsletter&ref=dailydoseofds.com",
      "label": "​",
      "type": "a"
    },
    {
      "href": "https://x.com/intent/tweet?url=https%3A%2F%2Fwww.dailydoseofds.com%2Fp%2Fhow-to-build-an-rl-environment%2F&text=How%20to%20build%20an%20RL%20environment%3F",
      "label": "Share to X",
      "type": "a"
    },
    {
      "href": "https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fwww.dailydoseofds.com%2Fp%2Fhow-to-build-an-rl-environment%2F",
      "label": "Share to Facebook",
      "type": "a"
    },
    {
      "href": "https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fwww.dailydoseofds.com%2Fp%2Fhow-to-build-an-rl-environment%2F",
      "label": "Share to Linkedin",
      "type": "a"
    },
    {
      "href": "https://www.dailydoseofds.com/rl-course-part-12/",
      "label": "Previous Agentic RL: Environments, Trajectories, and the Training",
      "type": "a"
    },
    {
      "href": "https://www.dailydoseofds.com/p/why-small-models-alone-dont-reduce-inference-costs/",
      "label": "Next Why Small Models Alone Don't Reduce Inference Costs",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:47:57Z",
  "source_url": "https://www.dailydoseofds.com/p/how-to-build-an-rl-environment/",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 13245,
  "expected_body_images": 13,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/8140a8bb210c/pages/source.html",
  "local_html": "attachments/8140a8bb210c/pages/article.html",
  "body_base_path": "attachments/8140a8bb210c/pages/",
  "body_markdown_path": "attachments/8140a8bb210c/pages/article.md"
}