{
  "source_file": "0930",
  "source_platform": "Substack",
  "author": "SwirlAI Newsletter Aurimas Griciūnas",
  "title_listed": "Data Engineering for Machine Learning: The Complete Guide",
  "keywords": "数据工程,ML,管道",
  "domain": "数据处理",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://www.newsletter.swirlai.com/p/data-engineering-for-machine-learning",
  "publish_date": "2024-03",
  "corpus_stage": "数据处理",
  "notes": "ML数据工程完整指南",
  "id": "d14dddf584d7",
  "collected_at": "2026-10-01T20:46:53",
  "title": "SwirlAI Newsletter | Aurimas Griciūnas | Substack",
  "body_markdown": "[![SwirlAI Newsletter](https://substackcdn.com/image/fetch/$s_!JA65!,w_40,h_40,c_fill,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1382ba7c-34db-4b33-bbb2-8e23af4b6f7a_1280x1280.png)](https://www.newsletter.swirlai.com/)\n# [SwirlAI Newsletter](https://www.newsletter.swirlai.com/)\nSubscribeSign in\n[Home](https://www.newsletter.swirlai.com/)\n[AI Engineering Bootcamp](https://swrlai.com/ai-bootcamp)\n[AI Engineer's Handbook](https://swrlai.com/4l80OR8)\n[YouTube](https://www.youtube.com/@swirlai)\n[Table of Contents](https://www.newsletter.swirlai.com/p/table-of-contents)\n[Sponsoring SwirlAI](https://www.newsletter.swirlai.com/p/sponsorships)\n[Archive](https://www.newsletter.swirlai.com/archive)\n[About](https://www.newsletter.swirlai.com/about)\n## Page not found\nLatestTopDiscussions\n[Stop Monitoring AI Systems Like Web Services](https://www.newsletter.swirlai.com/p/stop-monitoring-ai-systems-like-web)\n[Five questions every AI system has to answer, and the metrics that answer them.](https://www.newsletter.swirlai.com/p/stop-monitoring-ai-systems-like-web)\nJun 14 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n28\n1\n![](../images/013.png)\n### May 2026\n[Agentic AI Flywheels](https://www.newsletter.swirlai.com/p/agentic-ai-flywheels)\n[The production loop after your agent ships, and the eval set that grows with it.](https://www.newsletter.swirlai.com/p/agentic-ai-flywheels)\nMay 27 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n33\n1\n3\n![](../images/031.png)\n### March 2026\n[State of Context Engineering in 2026](https://www.newsletter.swirlai.com/p/state-of-context-engineering-in-2026)\n[Five patterns for managing what goes into the context window, and when.](https://www.newsletter.swirlai.com/p/state-of-context-engineering-in-2026)\nMar 22 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n74\n4\n7\n![](../images/body-45d2d5c775998d7822e2.png)\n[New: SwirlAI on YouTube + Context Engineering Workshop](https://www.newsletter.swirlai.com/p/new-swirlai-on-youtube-context-engineering)\n[Hands-on AI engineering content, now in video form, plus a live workshop on context engineering this Friday](https://www.newsletter.swirlai.com/p/new-swirlai-on-youtube-context-engineering)\nMar 17 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n13\n1\n![](../images/body-a43ba08a5db928829d38.png)\n[Agent Skills: Progressive Disclosure as a System Design Pattern](https://www.newsletter.swirlai.com/p/agent-skills-progressive-disclosure)\n[A simple file format, powered by an architectural shift in how agents manage context.](https://www.newsletter.swirlai.com/p/agent-skills-progressive-disclosure)\nMar 11 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n29\n2\n![](../images/body-e80cad635bbb0365829e.png)\n### September 2025\n[Learning AI Engineering in 2025](https://www.newsletter.swirlai.com/p/learning-ai-engineering-in-2025)\n[My Reflections and Tips.](https://www.newsletter.swirlai.com/p/learning-ai-engineering-in-2025)\nSep 6, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n26\n1\n1\n![](../images/body-d3bb8ca39ea2629034b5.png)\n### August 2025\n[Breaking Down Context Engineering](https://www.newsletter.swirlai.com/p/breaking-down-context-engineering)\n[All you ned to know about the challenges surrounding the practice.](https://www.newsletter.swirlai.com/p/breaking-down-context-engineering)\nAug 30, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n59\n7\n![](../images/body-6184cfdb8b550bfea297.png)\n### June 2025\n[Enterprise Agentic AI Hierarchy of Needs](https://www.newsletter.swirlai.com/p/enterprise-agentic-ai-hierarchy-of)\n[The crucial layers of infrastructure that make up a production grade Agentic AI system.](https://www.newsletter.swirlai.com/p/enterprise-agentic-ai-hierarchy-of)\nJun 18, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n62\n1\n8\n![](../images/body-d7c6d6c640e628f0ebcb.png)\n[Breaking into AI Engineering in 2025.](https://www.newsletter.swirlai.com/p/breaking-into-ai-engineering-in-2025)\n[A roadmap that will help you up-skill or re-skill into an AI Engineer role.](https://www.newsletter.swirlai.com/p/breaking-into-ai-engineering-in-2025)\nJun 4, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n288\n23\n47\n![](../images/body-6c519eb71cae19ac3534.png)\n### May 2025\n[Evaluation Driven Development for Agentic Systems.](https://www.newsletter.swirlai.com/p/evaluation-driven-development-for)\n[My step-by-step approach for building and evolving Agentic Systems that work.](https://www.newsletter.swirlai.com/p/evaluation-driven-development-for)\nMay 22, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n63\n2\n3\n![](../images/body-191433bbcab73a24e4e7.png)\n[Announcing the End-to-End AI Engineering Bootcamp.](https://www.newsletter.swirlai.com/p/announcing-the-end-to-end-ai-engineering)\n[Also join me this Thursday in a free Lightning Lesson on how to break into AI Engineering.](https://www.newsletter.swirlai.com/p/announcing-the-end-to-end-ai-engineering)\nMay 13, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n17\n5\n1\n![](../images/body-d0a1f4024a9ff091240a.png)\n[Distributing FlashAttention: Solving memory Bottlenecks of Context Windows](https://www.newsletter.swirlai.com/p/distributing-flashattention-solving)\n[Let's explore FlashAttention, types of distributed LLM training and how to fuse both concepts together.](https://www.newsletter.swirlai.com/p/distributing-flashattention-solving)\nMay 11, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n21\n2\n![](../images/body-34a00f4c6d22ea8e3a42.png)\n### April 2025\n[MCP vs. A2A: Friends or Foes?](https://www.newsletter.swirlai.com/p/mcp-vs-a2a-friends-or-foes)\n[Could MCP fade into irrelevance in long term?](https://www.newsletter.swirlai.com/p/mcp-vs-a2a-friends-or-foes)\nApr 13, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n150\n7\n18\n![](../images/body-3024d4727fe940ff919c.png)\n[The evolution of Modern RAG Architectures.](https://www.newsletter.swirlai.com/p/the-evolution-of-modern-rag-architectures)\n[Learn how to choose the best RAG architecture for your business case.](https://www.newsletter.swirlai.com/p/the-evolution-of-modern-rag-architectures)\nApr 7, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n123\n2\n26\n![](../images/body-63a94d31578d248750e4.png)\n### March 2025\n[Everything you need to know about MCP.](https://www.newsletter.swirlai.com/p/everything-you-need-to-know-about)\n[And why it is important for how we shape and build Agentic Systems.](https://www.newsletter.swirlai.com/p/everything-you-need-to-know-about)\nMar 15, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n113\n7\n20\n![](../images/body-0fa18dd60bf29e9b8104.png)\n[Building Deep Research Agent from scratch](https://www.newsletter.swirlai.com/p/building-deep-research-agent-from)\n[Let's build a Deep Research Agent to help you in your day to day work powered by DeepSeek R1 from scratch.](https://www.newsletter.swirlai.com/p/building-deep-research-agent-from)\nMar 11, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n116\n6\n14\n![](../images/body-9bbc4a1dfaa5f72f5ede.png)\n### February 2025\n[Simple way to explain Memory in AI Agents.](https://www.newsletter.swirlai.com/p/simple-way-to-explain-memory-in-ai)\n[Win a NVIDIA RTX 4080 SUPER GPU!](https://www.newsletter.swirlai.com/p/simple-way-to-explain-memory-in-ai)\nFeb 26, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n23\n1\n3\n![](../images/body-99947ca7aef816a1c8c1.png)\n[Data Pipelines in Machine Learning Systems.](https://www.newsletter.swirlai.com/p/data-pipelines-in-machine-learning)\n[A hands-on tutorial of real time web data ingestion pipeline followed by Apache Spark based ETL.](https://www.newsletter.swirlai.com/p/data-pipelines-in-machine-learning)\nFeb 24, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n25\n2\n![](../images/body-8096a5d0120f28f2c368.png)\n### January 2025\n[Building AI Agents from scratch - Part 2: Reflection and Working Memory](https://www.newsletter.swirlai.com/p/building-ai-agents-from-scratch-part-8ca)\n[Let's implement AI Agent from scratch without using any framework. Today we implement the reflection pattern coupled with simple implementation of…](https://www.newsletter.swirlai.com/p/building-ai-agents-from-scratch-part-8ca)\nJan 4, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n79\n6\n16\n![](../images/body-959a10ac1f7a7f677619.png)\n### December 2024\n[Building AI Agents from scratch - Part 1: Tool use](https://www.newsletter.swirlai.com/p/building-ai-agents-from-scratch-part)\n[Let's implement AI Agent from scratch without using any framework. Today we implement the tool use capability.](https://www.newsletter.swirlai.com/p/building-ai-agents-from-scratch-part)\nDec 21, 2024 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n316\n21\n40\n![](../images/body-f29d3dafe240369d3c33.png)\n[AI Clouds and their role in the AI era](https://www.newsletter.swirlai.com/p/ai-clouds-and-their-role-in-the-ai)\n[And a hands-on project: Mistral-7B powered chatbot on Nebius AI Cloud](https://www.newsletter.swirlai.com/p/ai-clouds-and-their-role-in-the-ai)\nDec 6, 2024 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n19\n7\n1\n![](../images/body-059618ce8aa29e670fff.png)\n### November 2024\n[What is AI Engineering?](https://www.newsletter.swirlai.com/p/what-is-ai-engineering)\n[And what you need to break into the role.](https://www.newsletter.swirlai.com/p/what-is-ai-engineering)\nNov 30, 2024 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n114\n24\n![](../images/body-c44c3c3e4eebfb1607ef.png)\n### October 2024\n[Memory in Agent Systems](https://www.newsletter.swirlai.com/p/memory-in-agent-systems)\n[In this article I outline my thoughts on implementation of memory in GenAI systems.](https://www.newsletter.swirlai.com/p/memory-in-agent-systems)\nOct 30, 2024 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n85\n2\n14\n![](../images/body-f2a0184f89505e920dd7.png)\n[Observability in LLMOps pipeline - Different Levels of Scale](https://www.newsletter.swirlai.com/p/observability-in-llmops-pipeline)\n[SwirlAI newsletter is back! In this piece I outline my observations about the scale of infrastructure needed specifically for observing your processes…](https://www.newsletter.swirlai.com/p/observability-in-llmops-pipeline)\nOct 21, 2024 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n62\n11\n![](../images/body-8cdad077e42af741b668.png)\n© 2026 Aurimas Griciūnas · [Privacy](https://substack.com/privacy) ∙ [Terms](https://substack.com/tos) ∙ [Collection notice](https://substack.com/ccpa#personal-data-collected)\n[ Start your Substack](https://substack.com/signup?utm_source=substack&utm_medium=web&utm_content=footer)[Get the app](https://substack.com/app/app-store-redirect?utm_campaign=app-marketing&utm_content=web-footer-button)\n[Substack](https://substack.com) is the home for great culture\n",
  "metadata": {
    "title": "SwirlAI Newsletter | Aurimas Griciūnas | Substack",
    "description": "Learn about end-to-end Data Systems and stay up to date with what is happening in the Data World. Click to read SwirlAI Newsletter, by Aurimas Griciūnas, a Substack publication with tens of thousands of subscribers.",
    "keywords": null,
    "author": "Aurimas Griciūnas",
    "og:type": "article",
    "og:title": "SwirlAI Newsletter | Aurimas Griciūnas | Substack",
    "og:description": "Learn about end-to-end Data Systems and stay up to date with what is happening in the Data World. Click to read SwirlAI Newsletter, by Aurimas Griciūnas, a Substack publication with tens of thousands of subscribers.",
    "og:image": "https://substackcdn.com/image/fetch/$s_!JTbn!,f_auto,q_auto:best,fl_progressive:steep/https%3A%2F%2Fswirlai.substack.com%2Ftwitter%2Fsubscribe-card.jpg%3Fv%3D1486187235%26version%3D9",
    "og:url": "https://www.newsletter.swirlai.com/data-engineering-for-machine-learning",
    "twitter:title": "SwirlAI Newsletter | Aurimas Griciūnas | Substack",
    "twitter:description": "Learn about end-to-end Data Systems and stay up to date with what is happening in the Data World. Click to read SwirlAI Newsletter, by Aurimas Griciūnas, a Substack publication with tens of thousands of subscribers.",
    "twitter:image": "https://substackcdn.com/image/fetch/$s_!JTbn!,f_auto,q_auto:best,fl_progressive:steep/https%3A%2F%2Fswirlai.substack.com%2Ftwitter%2Fsubscribe-card.jpg%3Fv%3D1486187235%26version%3D9",
    "twitter:card": "summary_large_image"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 302,
    "error": ""
  },
  "code_blocks": [],
  "images": [
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!VROF!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3d630c9d-6b9b-4c4f-b845-41e7a243b384_2218x1232.png",
      "bytes": 6853,
      "reused": true,
      "local_path": "attachments/d14dddf584d7/images/013.png",
      "src": "https://substackcdn.com/image/fetch/$s_!VROF!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3d630c9d-6b9b-4c4f-b845-41e7a243b384_2218x1232.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!kU75!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7ce8817-27d2-42f3-a413-fc9c70c0c497_6876x4679.png",
      "bytes": 7565,
      "reused": true,
      "local_path": "attachments/d14dddf584d7/images/031.png",
      "src": "https://substackcdn.com/image/fetch/$s_!kU75!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7ce8817-27d2-42f3-a413-fc9c70c0c497_6876x4679.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!82Y5!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd47b96f2-27c5-4b8b-abb5-12d3494b04f7_5738x3255.png",
      "bytes": 10142,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-45d2d5c775998d7822e2.png",
      "src": "https://substackcdn.com/image/fetch/$s_!82Y5!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd47b96f2-27c5-4b8b-abb5-12d3494b04f7_5738x3255.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!PbOw!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F07f7b5b3-00f2-443a-9d9c-fe852717d794_1506x1222.png",
      "bytes": 7948,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-a43ba08a5db928829d38.png",
      "src": "https://substackcdn.com/image/fetch/$s_!PbOw!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F07f7b5b3-00f2-443a-9d9c-fe852717d794_1506x1222.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!ufaN!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff8b195c7-bc33-4bee-be52-984301bfdd43_4157x2209.png",
      "bytes": 8775,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-e80cad635bbb0365829e.png",
      "src": "https://substackcdn.com/image/fetch/$s_!ufaN!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff8b195c7-bc33-4bee-be52-984301bfdd43_4157x2209.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!0pmF!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F18ccd2cf-e975-4671-879e-32b58dd349dd_1368x994.png",
      "bytes": 5077,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-d3bb8ca39ea2629034b5.png",
      "src": "https://substackcdn.com/image/fetch/$s_!0pmF!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F18ccd2cf-e975-4671-879e-32b58dd349dd_1368x994.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!pyEk!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ff8da46-fa9f-4589-89ac-46d5ed749787_836x568.png",
      "bytes": 9183,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-6184cfdb8b550bfea297.png",
      "src": "https://substackcdn.com/image/fetch/$s_!pyEk!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ff8da46-fa9f-4589-89ac-46d5ed749787_836x568.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!eB52!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6bfb6908-6268-4528-9e79-7f3f2cbc15c6_3757x3153.png",
      "bytes": 8820,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-d7c6d6c640e628f0ebcb.png",
      "src": "https://substackcdn.com/image/fetch/$s_!eB52!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6bfb6908-6268-4528-9e79-7f3f2cbc15c6_3757x3153.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!aix4!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdb68af97-9e4f-43e7-8c1d-37a79c90e939_7992x6613.png",
      "bytes": 6743,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-6c519eb71cae19ac3534.png",
      "src": "https://substackcdn.com/image/fetch/$s_!aix4!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdb68af97-9e4f-43e7-8c1d-37a79c90e939_7992x6613.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!7a7z!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4aa1ed38-f60a-4673-a181-c2e31c38551b_4218x3628.png",
      "bytes": 9303,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-191433bbcab73a24e4e7.png",
      "src": "https://substackcdn.com/image/fetch/$s_!7a7z!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4aa1ed38-f60a-4673-a181-c2e31c38551b_4218x3628.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!I7oH!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd245b06d-3f86-401a-ba4a-4e187d5d5dbf_1139x861.png",
      "bytes": 9113,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-d0a1f4024a9ff091240a.png",
      "src": "https://substackcdn.com/image/fetch/$s_!I7oH!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd245b06d-3f86-401a-ba4a-4e187d5d5dbf_1139x861.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!rN5a!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf39a917-1397-4352-a252-438f08fe5671_1380x1222.png",
      "bytes": 7479,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-34a00f4c6d22ea8e3a42.png",
      "src": "https://substackcdn.com/image/fetch/$s_!rN5a!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf39a917-1397-4352-a252-438f08fe5671_1380x1222.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!IdfD!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F65e21ee4-ed01-4ff3-967c-e1a57ccbdd41_3037x2606.png",
      "bytes": 8012,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-3024d4727fe940ff919c.png",
      "src": "https://substackcdn.com/image/fetch/$s_!IdfD!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F65e21ee4-ed01-4ff3-967c-e1a57ccbdd41_3037x2606.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!ltYL!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7430fbad-21da-4918-88cc-3d593254f310_2789x2392.png",
      "bytes": 8874,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-63a94d31578d248750e4.png",
      "src": "https://substackcdn.com/image/fetch/$s_!ltYL!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7430fbad-21da-4918-88cc-3d593254f310_2789x2392.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!9E4r!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9c348e1b-a175-4c65-8ea6-d773f957488e_1934x1554.png",
      "bytes": 7869,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-0fa18dd60bf29e9b8104.png",
      "src": "https://substackcdn.com/image/fetch/$s_!9E4r!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9c348e1b-a175-4c65-8ea6-d773f957488e_1934x1554.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!eUR5!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd70c1604-8ab3-4476-8ef6-f8f8f5a4eadd_3040x2289.png",
      "bytes": 7417,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-9bbc4a1dfaa5f72f5ede.png",
      "src": "https://substackcdn.com/image/fetch/$s_!eUR5!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd70c1604-8ab3-4476-8ef6-f8f8f5a4eadd_3040x2289.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!Spqw!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb4797f9b-8172-4885-ad37-16901f87227e_3614x2582.png",
      "bytes": 9543,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-99947ca7aef816a1c8c1.png",
      "src": "https://substackcdn.com/image/fetch/$s_!Spqw!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb4797f9b-8172-4885-ad37-16901f87227e_3614x2582.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!ZPXC!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F27e4bd56-08f7-4478-b666-e1e47ee8b49a_4248x2213.png",
      "bytes": 11171,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-8096a5d0120f28f2c368.png",
      "src": "https://substackcdn.com/image/fetch/$s_!ZPXC!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F27e4bd56-08f7-4478-b666-e1e47ee8b49a_4248x2213.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!yR_I!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F50268cdf-10a7-48f2-bc57-e7e31e5516f1_2882x2349.png",
      "bytes": 10030,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-959a10ac1f7a7f677619.png",
      "src": "https://substackcdn.com/image/fetch/$s_!yR_I!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F50268cdf-10a7-48f2-bc57-e7e31e5516f1_2882x2349.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!fyUm!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1144abe7-1fb8-4190-b32d-6e59647c858b_2974x2388.png",
      "bytes": 7553,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-f29d3dafe240369d3c33.png",
      "src": "https://substackcdn.com/image/fetch/$s_!fyUm!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1144abe7-1fb8-4190-b32d-6e59647c858b_2974x2388.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!m37i!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2f758def-d71b-4ae7-9548-cd6cc58bde0a_2970x2243.png",
      "bytes": 6475,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-059618ce8aa29e670fff.png",
      "src": "https://substackcdn.com/image/fetch/$s_!m37i!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2f758def-d71b-4ae7-9548-cd6cc58bde0a_2970x2243.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!ptD1!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdf15f15b-3871-489b-83f4-7c8c38df9f6f_2709x2402.png",
      "bytes": 4588,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-c44c3c3e4eebfb1607ef.png",
      "src": "https://substackcdn.com/image/fetch/$s_!ptD1!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdf15f15b-3871-489b-83f4-7c8c38df9f6f_2709x2402.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!TLtB!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc7650705-54b4-49a3-91a4-aad0c4093c4b_2926x2198.png",
      "bytes": 7778,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-f2a0184f89505e920dd7.png",
      "src": "https://substackcdn.com/image/fetch/$s_!TLtB!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc7650705-54b4-49a3-91a4-aad0c4093c4b_2926x2198.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!JRsT!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ffaed7241-c0f0-492d-b546-56cd44d7319e_2926x2192.png",
      "bytes": 10709,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/d14dddf584d7/images/body-8cdad077e42af741b668.png",
      "src": "https://substackcdn.com/image/fetch/$s_!JRsT!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ffaed7241-c0f0-492d-b546-56cd44d7319e_2926x2192.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    }
  ],
  "audio": [],
  "videos": [
    {
      "url": "https://www.youtube.com/@swirlai",
      "kind": "channel_reference",
      "status": "reference_only"
    }
  ],
  "links": [
    {
      "type": "internal",
      "href": "https://www.newsletter.swirlai.com/"
    },
    {
      "type": "internal",
      "href": "https://www.newsletter.swirlai.com/p/table-of-contents"
    },
    {
      "type": "internal",
      "href": "https://www.newsletter.swirlai.com/p/sponsorships"
    },
    {
      "type": "internal",
      "href": "https://www.newsletter.swirlai.com/archive"
    },
    {
      "type": "internal",
      "href": "https://www.newsletter.swirlai.com/about"
    },
    {
      "type": "internal",
      "href": "https://www.newsletter.swirlai.com/p/stop-monitoring-ai-systems-like-web"
    },
    {
      "type": "internal",
      "href": "https://www.newsletter.swirlai.com/p/agentic-ai-flywheels"
    },
    {
      "type": "internal",
      "href": "https://www.newsletter.swirlai.com/p/state-of-context-engineering-in-2026"
    },
    {
      "type": "internal",
      "href": "https://www.newsletter.swirlai.com/p/new-swirlai-on-youtube-context-engineering"
    },
    {
      "type": "internal",
      "href": "https://www.newsletter.swirlai.com/p/agent-skills-progressive-disclosure"
    },
    {
      "type": "internal",
      "href": "https://www.newsletter.swirlai.com/p/learning-ai-engineering-in-2025"
    },
    {
      "type": "internal",
      "href": "https://www.newsletter.swirlai.com/p/breaking-down-context-engineering"
    },
    {
      "type": "internal",
      "href": "https://www.newsletter.swirlai.com/p/enterprise-agentic-ai-hierarchy-of"
    },
    {
      "type": "internal",
      "href": "https://www.newsletter.swirlai.com/p/breaking-into-ai-engineering-in-2025"
    },
    {
      "type": "internal",
      "href": "https://www.newsletter.swirlai.com/p/evaluation-driven-development-for"
    },
    {
      "type": "internal",
      "href": "https://www.newsletter.swirlai.com/p/announcing-the-end-to-end-ai-engineering"
    },
    {
      "type": "internal",
      "href": "https://www.newsletter.swirlai.com/p/distributing-flashattention-solving"
    },
    {
      "type": "internal",
      "href": "https://www.newsletter.swirlai.com/p/mcp-vs-a2a-friends-or-foes"
    },
    {
      "type": "internal",
      "href": "https://www.newsletter.swirlai.com/p/the-evolution-of-modern-rag-architectures"
    },
    {
      "type": "internal",
      "href": "https://www.newsletter.swirlai.com/p/everything-you-need-to-know-about"
    },
    {
      "type": "external",
      "href": "https://swrlai.com/ai-bootcamp"
    },
    {
      "type": "external",
      "href": "https://swrlai.com/4l80OR8"
    },
    {
      "type": "external",
      "href": "https://www.youtube.com/@swirlai"
    },
    {
      "type": "external",
      "href": "https://substack.com/@swirlai"
    },
    {
      "type": "external",
      "href": "https://substack.com/privacy"
    },
    {
      "type": "external",
      "href": "https://substack.com/tos"
    },
    {
      "type": "external",
      "href": "https://substack.com/ccpa"
    },
    {
      "type": "external",
      "href": "https://substack.com/signup"
    },
    {
      "type": "external",
      "href": "https://substack.com/app/app-store-redirect"
    },
    {
      "type": "external",
      "href": "https://substack.com"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-20T17:33:59Z",
  "source_url": "https://www.newsletter.swirlai.com/p/data-engineering-for-machine-learning",
  "delivery_status": "unverified",
  "discovery_status": "http_404",
  "missing_reason": "http_404",
  "repair_error": "",
  "actual_video_files": [
    {
      "local_path": "attachments/d14dddf584d7/videos/3WsEgLtwsGs.mp4",
      "bytes": 79063100,
      "valid": true,
      "height": 1080,
      "width": 1920,
      "duration": "2245.941000",
      "status": "ok"
    },
    {
      "local_path": "attachments/d14dddf584d7/videos/IcbzksbMhuM.mp4",
      "bytes": 80361926,
      "valid": true,
      "height": 1080,
      "width": 1920,
      "duration": "1501.841000",
      "status": "ok"
    },
    {
      "local_path": "attachments/d14dddf584d7/videos/bZHaCAFAUCs.mp4",
      "bytes": 81349064,
      "valid": true,
      "height": 1080,
      "width": 1920,
      "duration": "1616.521000",
      "status": "ok"
    },
    {
      "local_path": "attachments/d14dddf584d7/videos/cGOGA7GFecI.mp4",
      "bytes": 97810511,
      "valid": true,
      "height": 1080,
      "width": 1920,
      "duration": "1929.541000",
      "status": "ok"
    }
  ],
  "expected_saved_body_images": 24,
  "image_discovery": "saved_markdown",
  "source_body_markdown": "[![SwirlAI Newsletter](https://substackcdn.com/image/fetch/$s_!JA65!,w_40,h_40,c_fill,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1382ba7c-34db-4b33-bbb2-8e23af4b6f7a_1280x1280.png)](https://www.newsletter.swirlai.com/)\n# [SwirlAI Newsletter](https://www.newsletter.swirlai.com/)\nSubscribeSign in\n[Home](https://www.newsletter.swirlai.com/)\n[AI Engineering Bootcamp](https://swrlai.com/ai-bootcamp)\n[AI Engineer's Handbook](https://swrlai.com/4l80OR8)\n[YouTube](https://www.youtube.com/@swirlai)\n[Table of Contents](https://www.newsletter.swirlai.com/p/table-of-contents)\n[Sponsoring SwirlAI](https://www.newsletter.swirlai.com/p/sponsorships)\n[Archive](https://www.newsletter.swirlai.com/archive)\n[About](https://www.newsletter.swirlai.com/about)\n## Page not found\nLatestTopDiscussions\n[Stop Monitoring AI Systems Like Web Services](https://www.newsletter.swirlai.com/p/stop-monitoring-ai-systems-like-web)\n[Five questions every AI system has to answer, and the metrics that answer them.](https://www.newsletter.swirlai.com/p/stop-monitoring-ai-systems-like-web)\nJun 14 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n28\n1\n![](https://substackcdn.com/image/fetch/$s_!VROF!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3d630c9d-6b9b-4c4f-b845-41e7a243b384_2218x1232.png)\n### May 2026\n[Agentic AI Flywheels](https://www.newsletter.swirlai.com/p/agentic-ai-flywheels)\n[The production loop after your agent ships, and the eval set that grows with it.](https://www.newsletter.swirlai.com/p/agentic-ai-flywheels)\nMay 27 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n33\n1\n3\n![](https://substackcdn.com/image/fetch/$s_!kU75!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7ce8817-27d2-42f3-a413-fc9c70c0c497_6876x4679.png)\n### March 2026\n[State of Context Engineering in 2026](https://www.newsletter.swirlai.com/p/state-of-context-engineering-in-2026)\n[Five patterns for managing what goes into the context window, and when.](https://www.newsletter.swirlai.com/p/state-of-context-engineering-in-2026)\nMar 22 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n74\n4\n7\n![](https://substackcdn.com/image/fetch/$s_!82Y5!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd47b96f2-27c5-4b8b-abb5-12d3494b04f7_5738x3255.png)\n[New: SwirlAI on YouTube + Context Engineering Workshop](https://www.newsletter.swirlai.com/p/new-swirlai-on-youtube-context-engineering)\n[Hands-on AI engineering content, now in video form, plus a live workshop on context engineering this Friday](https://www.newsletter.swirlai.com/p/new-swirlai-on-youtube-context-engineering)\nMar 17 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n13\n1\n![](https://substackcdn.com/image/fetch/$s_!PbOw!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F07f7b5b3-00f2-443a-9d9c-fe852717d794_1506x1222.png)\n[Agent Skills: Progressive Disclosure as a System Design Pattern](https://www.newsletter.swirlai.com/p/agent-skills-progressive-disclosure)\n[A simple file format, powered by an architectural shift in how agents manage context.](https://www.newsletter.swirlai.com/p/agent-skills-progressive-disclosure)\nMar 11 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n29\n2\n![](https://substackcdn.com/image/fetch/$s_!ufaN!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff8b195c7-bc33-4bee-be52-984301bfdd43_4157x2209.png)\n### September 2025\n[Learning AI Engineering in 2025](https://www.newsletter.swirlai.com/p/learning-ai-engineering-in-2025)\n[My Reflections and Tips.](https://www.newsletter.swirlai.com/p/learning-ai-engineering-in-2025)\nSep 6, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n26\n1\n1\n![](https://substackcdn.com/image/fetch/$s_!0pmF!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F18ccd2cf-e975-4671-879e-32b58dd349dd_1368x994.png)\n### August 2025\n[Breaking Down Context Engineering](https://www.newsletter.swirlai.com/p/breaking-down-context-engineering)\n[All you ned to know about the challenges surrounding the practice.](https://www.newsletter.swirlai.com/p/breaking-down-context-engineering)\nAug 30, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n59\n7\n![](https://substackcdn.com/image/fetch/$s_!pyEk!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ff8da46-fa9f-4589-89ac-46d5ed749787_836x568.png)\n### June 2025\n[Enterprise Agentic AI Hierarchy of Needs](https://www.newsletter.swirlai.com/p/enterprise-agentic-ai-hierarchy-of)\n[The crucial layers of infrastructure that make up a production grade Agentic AI system.](https://www.newsletter.swirlai.com/p/enterprise-agentic-ai-hierarchy-of)\nJun 18, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n62\n1\n8\n![](https://substackcdn.com/image/fetch/$s_!eB52!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6bfb6908-6268-4528-9e79-7f3f2cbc15c6_3757x3153.png)\n[Breaking into AI Engineering in 2025.](https://www.newsletter.swirlai.com/p/breaking-into-ai-engineering-in-2025)\n[A roadmap that will help you up-skill or re-skill into an AI Engineer role.](https://www.newsletter.swirlai.com/p/breaking-into-ai-engineering-in-2025)\nJun 4, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n288\n23\n47\n![](https://substackcdn.com/image/fetch/$s_!aix4!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdb68af97-9e4f-43e7-8c1d-37a79c90e939_7992x6613.png)\n### May 2025\n[Evaluation Driven Development for Agentic Systems.](https://www.newsletter.swirlai.com/p/evaluation-driven-development-for)\n[My step-by-step approach for building and evolving Agentic Systems that work.](https://www.newsletter.swirlai.com/p/evaluation-driven-development-for)\nMay 22, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n63\n2\n3\n![](https://substackcdn.com/image/fetch/$s_!7a7z!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4aa1ed38-f60a-4673-a181-c2e31c38551b_4218x3628.png)\n[Announcing the End-to-End AI Engineering Bootcamp.](https://www.newsletter.swirlai.com/p/announcing-the-end-to-end-ai-engineering)\n[Also join me this Thursday in a free Lightning Lesson on how to break into AI Engineering.](https://www.newsletter.swirlai.com/p/announcing-the-end-to-end-ai-engineering)\nMay 13, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n17\n5\n1\n![](https://substackcdn.com/image/fetch/$s_!I7oH!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd245b06d-3f86-401a-ba4a-4e187d5d5dbf_1139x861.png)\n[Distributing FlashAttention: Solving memory Bottlenecks of Context Windows](https://www.newsletter.swirlai.com/p/distributing-flashattention-solving)\n[Let's explore FlashAttention, types of distributed LLM training and how to fuse both concepts together.](https://www.newsletter.swirlai.com/p/distributing-flashattention-solving)\nMay 11, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n21\n2\n![](https://substackcdn.com/image/fetch/$s_!rN5a!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf39a917-1397-4352-a252-438f08fe5671_1380x1222.png)\n### April 2025\n[MCP vs. A2A: Friends or Foes?](https://www.newsletter.swirlai.com/p/mcp-vs-a2a-friends-or-foes)\n[Could MCP fade into irrelevance in long term?](https://www.newsletter.swirlai.com/p/mcp-vs-a2a-friends-or-foes)\nApr 13, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n150\n7\n18\n![](https://substackcdn.com/image/fetch/$s_!IdfD!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F65e21ee4-ed01-4ff3-967c-e1a57ccbdd41_3037x2606.png)\n[The evolution of Modern RAG Architectures.](https://www.newsletter.swirlai.com/p/the-evolution-of-modern-rag-architectures)\n[Learn how to choose the best RAG architecture for your business case.](https://www.newsletter.swirlai.com/p/the-evolution-of-modern-rag-architectures)\nApr 7, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n123\n2\n26\n![](https://substackcdn.com/image/fetch/$s_!ltYL!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7430fbad-21da-4918-88cc-3d593254f310_2789x2392.png)\n### March 2025\n[Everything you need to know about MCP.](https://www.newsletter.swirlai.com/p/everything-you-need-to-know-about)\n[And why it is important for how we shape and build Agentic Systems.](https://www.newsletter.swirlai.com/p/everything-you-need-to-know-about)\nMar 15, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n113\n7\n20\n![](https://substackcdn.com/image/fetch/$s_!9E4r!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9c348e1b-a175-4c65-8ea6-d773f957488e_1934x1554.png)\n[Building Deep Research Agent from scratch](https://www.newsletter.swirlai.com/p/building-deep-research-agent-from)\n[Let's build a Deep Research Agent to help you in your day to day work powered by DeepSeek R1 from scratch.](https://www.newsletter.swirlai.com/p/building-deep-research-agent-from)\nMar 11, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n116\n6\n14\n![](https://substackcdn.com/image/fetch/$s_!eUR5!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd70c1604-8ab3-4476-8ef6-f8f8f5a4eadd_3040x2289.png)\n### February 2025\n[Simple way to explain Memory in AI Agents.](https://www.newsletter.swirlai.com/p/simple-way-to-explain-memory-in-ai)\n[Win a NVIDIA RTX 4080 SUPER GPU!](https://www.newsletter.swirlai.com/p/simple-way-to-explain-memory-in-ai)\nFeb 26, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n23\n1\n3\n![](https://substackcdn.com/image/fetch/$s_!Spqw!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb4797f9b-8172-4885-ad37-16901f87227e_3614x2582.png)\n[Data Pipelines in Machine Learning Systems.](https://www.newsletter.swirlai.com/p/data-pipelines-in-machine-learning)\n[A hands-on tutorial of real time web data ingestion pipeline followed by Apache Spark based ETL.](https://www.newsletter.swirlai.com/p/data-pipelines-in-machine-learning)\nFeb 24, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n25\n2\n![](https://substackcdn.com/image/fetch/$s_!ZPXC!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F27e4bd56-08f7-4478-b666-e1e47ee8b49a_4248x2213.png)\n### January 2025\n[Building AI Agents from scratch - Part 2: Reflection and Working Memory](https://www.newsletter.swirlai.com/p/building-ai-agents-from-scratch-part-8ca)\n[Let's implement AI Agent from scratch without using any framework. Today we implement the reflection pattern coupled with simple implementation of…](https://www.newsletter.swirlai.com/p/building-ai-agents-from-scratch-part-8ca)\nJan 4, 2025 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n79\n6\n16\n![](https://substackcdn.com/image/fetch/$s_!yR_I!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F50268cdf-10a7-48f2-bc57-e7e31e5516f1_2882x2349.png)\n### December 2024\n[Building AI Agents from scratch - Part 1: Tool use](https://www.newsletter.swirlai.com/p/building-ai-agents-from-scratch-part)\n[Let's implement AI Agent from scratch without using any framework. Today we implement the tool use capability.](https://www.newsletter.swirlai.com/p/building-ai-agents-from-scratch-part)\nDec 21, 2024 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n316\n21\n40\n![](https://substackcdn.com/image/fetch/$s_!fyUm!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1144abe7-1fb8-4190-b32d-6e59647c858b_2974x2388.png)\n[AI Clouds and their role in the AI era](https://www.newsletter.swirlai.com/p/ai-clouds-and-their-role-in-the-ai)\n[And a hands-on project: Mistral-7B powered chatbot on Nebius AI Cloud](https://www.newsletter.swirlai.com/p/ai-clouds-and-their-role-in-the-ai)\nDec 6, 2024 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n19\n7\n1\n![](https://substackcdn.com/image/fetch/$s_!m37i!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2f758def-d71b-4ae7-9548-cd6cc58bde0a_2970x2243.png)\n### November 2024\n[What is AI Engineering?](https://www.newsletter.swirlai.com/p/what-is-ai-engineering)\n[And what you need to break into the role.](https://www.newsletter.swirlai.com/p/what-is-ai-engineering)\nNov 30, 2024 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n114\n24\n![](https://substackcdn.com/image/fetch/$s_!ptD1!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdf15f15b-3871-489b-83f4-7c8c38df9f6f_2709x2402.png)\n### October 2024\n[Memory in Agent Systems](https://www.newsletter.swirlai.com/p/memory-in-agent-systems)\n[In this article I outline my thoughts on implementation of memory in GenAI systems.](https://www.newsletter.swirlai.com/p/memory-in-agent-systems)\nOct 30, 2024 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n85\n2\n14\n![](https://substackcdn.com/image/fetch/$s_!TLtB!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc7650705-54b4-49a3-91a4-aad0c4093c4b_2926x2198.png)\n[Observability in LLMOps pipeline - Different Levels of Scale](https://www.newsletter.swirlai.com/p/observability-in-llmops-pipeline)\n[SwirlAI newsletter is back! In this piece I outline my observations about the scale of infrastructure needed specifically for observing your processes…](https://www.newsletter.swirlai.com/p/observability-in-llmops-pipeline)\nOct 21, 2024 • [Aurimas Griciūnas](https://substack.com/@swirlai)\n62\n11\n![](https://substackcdn.com/image/fetch/$s_!JRsT!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ffaed7241-c0f0-492d-b546-56cd44d7319e_2926x2192.png)\n© 2026 Aurimas Griciūnas · [Privacy](https://substack.com/privacy) ∙ [Terms](https://substack.com/tos) ∙ [Collection notice](https://substack.com/ccpa#personal-data-collected)\n[ Start your Substack](https://substack.com/signup?utm_source=substack&utm_medium=web&utm_content=footer)[Get the app](https://substack.com/app/app-store-redirect?utm_campaign=app-marketing&utm_content=web-footer-button)\n[Substack](https://substack.com) is the home for great culture\n",
  "body_base_path": "attachments/d14dddf584d7/pages/",
  "body_markdown_path": "attachments/d14dddf584d7/pages/archived-body.md",
  "local_html": "attachments/d14dddf584d7/pages/archived-body.html",
  "body_source": "prior_capture"
}