{ "source_file": "0930", "source_platform": "Substack", "author": "Nathan Lambert / Interconnects", "title_listed": "Open-Source AI & Open Models Reading List", "keywords": "开源AI,开放模型,RLHF,对齐", "domain": "大模型算法", "content_type": "资源列表", "is_paid_flag": "否", "url": "https://www.interconnects.ai/p/open-source-ai-reading-list", "publish_date": "2024-08", "corpus_stage": "SFT", "notes": "开源AI模型全链路阅读清单，含训练和对齐", "id": "3149e64f7f8b", "collected_at": "2026-10-01T19:50:31", "title": "Open-Source AI & Open Models Reading List", "body_markdown": "Hey all! I’ve been prepping for some public-audience and policy-facing writing on open models, so I figured I would share my research materials. There’s lots of wonderful stuff in here.\nThis is my list of the best writing on open models in the last few years. If someone decides they want to get up to speed on the area, reading this will be a comprehensive overview of the state of affairs. Please comment pieces to consider adding below, and I’ll update this over time.\nList last updated: 15 Sep. 2026\nShare\nFoundation\nWhat open models are, why people release them, how they relate to business strategy, and what the risks are.\nOn open source AI strategy, a walkthrough of how open-source software has been used by businesses and early signs of what that means for AI —\nFrom Open Source Software to Open Source Strategy\n, Bill Gurley (May 2026).\nOne of the clearest articulations is Mark Zuckerberg’s comments around Llama 3’s release as to why Meta releases open models —\nOpen Source AI is the Path Forward\n, Mark Zuckerberg (Jul. 2024)\nWhy you should view open models on a gradient, rather than binary open/closed, based on factors such as licenses, cost of running the model, data access, etc. —\nThe Gradient of Generative AI Release: Methods and Considerations\n, Irene Solaiman (Feb. 2023).\nThe role open models will play in the economy of the future, as a complement to strong closed models. Why open models will be used to create custom agentic workflows in enterprises across the world –\nWhat comes next with open models\n, Nathan Lambert / Interconnects (Mar. 2026)\nA position on how open models will capture value by providing a complementary tool to large swaths of the existing economy, drawing on the history of IP and current debates on open vs. closed models (e.g. distillation) —\nSome Simple Economics of Open versus Closed AI\n, Christian Catalini (Aug. 2026)\nWhy open models will constantly be behind closed models in performance —\nOpen models in perpetual catch-up\n, Nathan Lambert / Interconnects (Feb. 2026)\nWhere adoption differs for open and closed models —\nOpen and closed models are on different exponentials\n, Nathan Lambert / Interconnects\n(Jun. 2026)\nA clear articulation on how to balance releasing powerful open-weight models while taking safety seriously —\nA Safe Path to Open Weights\n, Thinking Machines Lab (Jul. 2026).\nEarly paper on marginal risks that showed text-focused LLMs very marginally increased documented potential risks of models —\nOn the Societal Impact of Open Foundation Models\n, Sayash Kapoor, Rishi Bommasani et al. (Feb. 2024).\nClosed models safety guardrails are regularly bypassed causing a plethora of real AI-risks before hypothetical risks of open weight models have emerged —\nThe Myth of unsafe Open Source AI\n, Florian Brand (Jun. 2026).\nThe mass reduction in open data, which is a crucial factor that has hampered truly open AI research —\nConsent in Crisis: The Rapid Decline of the AI Data Commons\n, Shayne Longpre et al. (Jul. 2024).\nRecent examples on how strong Chinese models impact the AI ecosystem —\nKimi K3: The open-weights escalation\n, Nathan Lambert / Interconnects (Jul. 2026) /\nGLM-5.2 is the step change for open agents\n, Nathan Lambert / Interconnects (Jun. 2026).\nA summary of the story of open models in 2025:\nNathan Lambert on China’s AI Ecosystem and the Open Model Gap | The Curve 2025\n, Golden Gate Institute for AI (Nov. 2025).\n[\nOptional\n] Latest data on open model adoption:\nA general summary on US vs. China model adoption —\nThe ATOM Report\n(Apr. 2026),\nThe latest data on model downloads, derivatives, and research adoption by region —\nInterconnects Adoption Dashboard\n, and\nThe most important models to know about in the ecosystem —\nInterconnects Artifacts Hub\nInterconnects AI is a reader-supported publication. Consider becoming a subscriber.\nSubscribe\nUS-China Competition\nWho is leading in open models, how this has changed over time, how China maintains its leading position, and relevant history.\nWhy the U.S. needs to invest in open models for fundamental R&D / innovation in the face of growing competition from China –\nThe ATOM Project\n, Nathan Lambert (Aug. 2025)\nThe lens as to why open models help spur research innovation and beneficial outcomes for AI —\nWhy I build open language models\n, Nathan Lambert / Interconnects (Oct. 2024)\nWhy open models foster education, innovation and competition, three core American values —\nBanning Open Source AI Would Be A Mistake\n, Nathan Lambert & Kevin Xu\n(Jun. 2026)\nWhy the recent “vibe regulation” / vague federal oversight mechanisms set us up for a clash and-or ban of frontier open models in the near future —\n6 months to live for open models\n, Nathan Lambert / Interconnects (Jul. 2026)\n[\nOptional\n] Fully open language model technical reports to illustrate the start of the art in understanding:\nPythia\n(EleutherAI, 2023),\nOlmo\n(2024),\nOlmo 2\n(2024),\nOlmo 3\n(2025)\nChinese open-source history leading up to AI —\nChinese Open Source: A Definitive History\n, Kevin Xu (Mar. 2026).\nChina’s structural advantages in open-source —\nChina’s Structural Advantage in Open Source AI\n, Kevin Xu (Jun. 2025).\nHow Chinese labs themselves discuss building models, and how the Chinese industry differs from the U.S. —\nNotes from inside China’s AI labs\n, Nathan Lambert / Interconnects (May 2026).\nWhy Chinese labs are so good at keeping up with American competition (e.g. American open weight labs struggle to compete with Chinese labs on fair performance comparisons) —\nGLM-5.3: How Chinese labs keep stride with the frontier\n, Nathan Lambert / Interconnects (Aug. 2026).\nProminent uses of Chinese models by Western companies have prompted meaningful regulatory attention (\nmore\ndiscussion\n)\nLawmakers have probed the following companies over using Chinese models: DoorDash (\nCNBC\n, Jul. 31 2026), Airbnb (\nBloomberg\n, Apr. 29 2026;\nSemafor\n, Apr. 29 2026), Anysphere / Cursor (\nBloomberg\n, Apr. 29 2026;\nSemafor\n, Apr. 29 2026), Apple (\nReuters\n, May 17 2025),\nHarvey\n(Aug. 2026)\nOther western companies have very publicly shifted the models they use from American, closed labs to Chinese open models to save costs. Examples include Perplexity prominently and rapidly adopted DeepSeek R1 (\nForbes\n, Jan. 28 2025) and Thomson Reuters building on Qwen to move off Claude (\nBusiness Insider\n, Aug. 24 2026)\nLeave a comment\nTechnical Details\nWhat is distillation and how much does it help Chinese labs, how do open models impact frontier AI risks like cybersecurity, and how far are open models behind the closed frontier?\nThe open-closed model gap has reduced in recent years, and is now at roughly 4-6 months. The leading open models have all come from Chinese labs since ~2024.\nSemiAnalysis article which ran independent evaluations, concluding that open models have been getting closer to the closer frontier of performance over time —\nAre Open Models Catching Up?\n, SemiAnalysis (Aug. 2026)\nOpen models are on the Pareto\ncost\nfrontier, while not at the absolute performance frontier. E.g. DeepSeek V4 Flash, see\nevaluation and cost on Artificial Analysis\n.\nData sources from\nEpoch AI\nand\nArtificial Analysis\n(and\nU.S. v China\n, related) showing the open-closed gap over time.\nAn independent analysis of the open-closed gap across a mix of public and private evaluations —\nHow far behind are open models?\n, Håvard Tveit Ihle (May 2026)\nE.g. in 2025, the product lead of\nZ.ai\nsaid with respect to their release time “Get it out fast. We open source it within a few hours.” —\nThe Z.ai Playbook\n, ChinaTalk (Nov. 21, 2025)\nCyber, risks & open models (I plan to develop this further)\nWhy we cannot effectively ban open models as used by bad actors for cyber capabilities (they will always have access) —\nThe OpenAI/Huggingface incident; how we should manage the imminent arrival of autonomous hacking too cheap to meter\n, Joshua Saxe (Jul. 2026)\nWhat the government should do to observe, orient, decide, and act with respect to emerging cyber threats (versus blocking models based on in-house capability assessments) —\nWe urgently need a coherent national AI cybersecurity policy\n, Joshua Saxe (Aug. 2026)\nWhy you cannot expect to control access to AI at a certain threshold (e.g. open weight models) and need to prepare society to tackle risks downstream of available intelligence —\nNonproliferation is the wrong approach to AI misuse\n, Helen Toner (Apr. 2025)\nDistillation – the process of training on output tokens from another model – is the single most eventful debate around open models in 2026.\nFor basic background, see a\ntextbook chapter\non synthetic data & distillation generally, from\nReinforcement Learning from Human Feedback\n(post-training textbook published in 2026)\nHow distillation helps the Chinese labs, but doesn’t take away from their innovation —\nHow much does distillation really matter for Chinese LLMs?\n, Nathan Lambert / Interconnects (Feb. 2026)\nA very transparent documentation of how Chinese company use Anthropic’s products and circumvent the terms of service or intended use. The report details at-scale usage of Anthropic’s products by banned parties, as a mix of technical distillation (mentioned via SFT data) and extensive routing of Claude into their products and services without telling users —\nDetecting and countering misuse of AI: September 2026\n.\nA recent paper that showed that the frontier labs had implementations in their APIs that made systematic extraction of reasoning traces (the crucial part of modern training) through clever tricks. Recent distillation paper, my writing on it —\nStealing Reasoning Traces from Proprietary LLM APIs\n,\nPanfilov, Schmotz, Shumailov et. al 2026 (more on\nX\n).\nAnthropic confirmed\nthis technique was used by Chinese labs.\nWhy the political panic over distillation, claiming that distillation is the only reason Chinese models are close to the frontier, is not grounded in the evidence —\nThe distillation panic\n, Nathan Lambert / Interconnects (May 2026)\nHow labs can use distillation to improve models in an era of scaling RL environments across agentic behaviors —\nHow distillation is used today and what performance uplift it gives to open models\n, Nathan Lambert (Jul. 2026)\n[\nOptional\n] More history: In 2024, I wrote\nFrontiers in synthetic data\nwhere the key points were that synthetic data, primarily in “distilling” models by training with SFT on outputs from a stronger model, was the dominant form of distillation. Frontier labs had been shifting the logit-based, knowledge distillation, confirmed earliest in Gemini and continuing to this day. In early 2025, there was substantial debate on if DeepSeek-R1 was distilled from OpenAI’s o1 model. There is no clear evidence suggesting that they did, and in Apr. of 2025 I\nwrote confidently\nthat DeepSeek did\nnot\ndistill. At the time of R1, it is more possible than I gave it credit to that DeepSeek did distill some o1 traces to make it easier for them to train their R1 model – based on the above reasoning trace extraction methods. This does not take away from the innovation of it, but it’s worth being realistic and is a way that distillation could accelerate China closing the gap to American labs.", "metadata": { "title": "Open-Source AI & Open Models Reading List", "description": "How to get up to speed on open models and their implications.", "keywords": null, "author": "Nathan Lambert", "og:type": "article", "og:title": "Open-Source AI & Open Models Reading List", "og:description": "How to get up to speed on open models and their implications.", "og:image": "https://substackcdn.com/image/fetch/$s_!pdpS!,w_1200,h_675,c_fill,f_jpg,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F53e3cead-b3c3-4fba-8503-c5a599d5fb01_3182x1790.png", "og:url": "https://www.interconnects.ai/p/open-source-ai-reading-list", "twitter:title": "Open-Source AI & Open Models Reading List", "twitter:description": "How to get up to speed on open models and their implications.", "twitter:image": "https://substackcdn.com/image/fetch/$s_!WUfA!,f_auto,q_auto:best,fl_progressive:steep/https%3A%2F%2Frobotic.substack.com%2Fapi%2Fv1%2Fpost_preview%2F215211011%2Ftwitter.jpg%3Fversion%3D4", "twitter:card": "summary_large_image", "article:modified_time": "2026-09-15T19:43:24.686Z" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 200, "error": "" }, "code_blocks": [], "images": [], "audio": [], "videos": [ { "url": "https://www.youtube.com/watch?v=VpYU4VOicI0", "status": "ok", "local_path": "attachments/3149e64f7f8b/videos/VpYU4VOicI0.mp4", "bytes": 340912428, "valid": true, "height": 1080, "width": 1920, "duration": "2907.361000" } ], "links": [ { "href": "https://www.interconnects.ai/p/open-source-ai-reading-list?utm_source=substack&utm_medium=email&utm_content=share&action=share", "label": "Share", "type": "a" }, { "href": "https://p3institute.substack.com/p/from-open-source-software-to-open", "label": "From Open Source Software to Open Source Strategy", "type": "a" }, { "href": "https://about.fb.com/news/2024/07/open-source-ai-is-the-path-forward/", "label": "Open Source AI is the Path Forward", "type": "a" }, { "href": "https://arxiv.org/abs/2302.04844", "label": "The Gradient of Generative AI Release: Methods and Considerations", "type": "a" }, { "href": "https://www.interconnects.ai/p/the-next-phase-of-open-models", "label": "What comes next with open models", "type": "a" }, { "href": "https://www.a16z.news/p/some-simple-economics-of-open-versus", "label": "", "type": "a" }, { "href": "https://www.interconnects.ai/p/open-models-in-perpetual-catch-up", "label": "Open models in perpetual catch-up", "type": "a" }, { "href": "https://www.interconnects.ai/p/open-and-closed-models-are-on-different", "label": "Open and closed models are on different exponentials", "type": "a" }, { "href": "https://thinkingmachines.ai/blog/a-safe-path-to-open-weights/", "label": "", "type": "a" }, { "href": "https://arxiv.org/abs/2403.07918", "label": "", "type": "a" }, { "href": "https://florianbrand.com/posts/open-model-safety", "label": "The Myth of unsafe Open Source AI", "type": "a" }, { "href": "https://arxiv.org/abs/2407.14933", "label": "", "type": "a" }, { "href": "https://www.interconnects.ai/p/kimi-k3-the-open-weights-escalation", "label": "", "type": "a" }, { "href": "https://www.interconnects.ai/p/glm-52-is-the-step-change-for-open", "label": "", "type": "a" }, { "href": "https://www.youtube.com/watch?v=VpYU4VOicI0", "label": "Nathan Lambert on China’s AI Ecosystem and the Open Model Gap | The Curve 2025", "type": "a" }, { "href": "https://arxiv.org/abs/2604.07190", "label": "The ATOM Report", "type": "a" }, { "href": "https://dashboard.interconnects.ai/", "label": "Interconnects Adoption Dashboard", "type": "a" }, { "href": "https://artifactshub.ai/", "label": "Interconnects Artifacts Hub", "type": "a" }, { "href": "http://atomproject.ai/", "label": "The ATOM Project", "type": "a" }, { "href": "https://www.interconnects.ai/p/why-i-build-open-language-models", "label": "Why I build open language models", "type": "a" }, { "href": "https://www.interconnects.ai/p/banning-open-source-ai-would-be-a", "label": "Banning Open Source AI Would Be A Mistake", "type": "a" }, { "href": "https://www.interconnects.ai/p/6-months-to-live-for-open-models", "label": "6 months to live for open models", "type": "a" }, { "href": "https://arxiv.org/abs/2304.01373", "label": "Pythia", "type": "a" }, { "href": "https://arxiv.org/abs/2402.00838", "label": "Olmo", "type": "a" }, { "href": "https://arxiv.org/abs/2501.00656", "label": "Olmo 2", "type": "a" }, { "href": "https://arxiv.org/abs/2512.13961", "label": "Olmo 3", "type": "a" }, { "href": "https://interconnect.substack.com/p/chinese-open-source-a-definitive", "label": "", "type": "a" }, { "href": "https://interconnect.substack.com/p/chinas-structural-advantage-in-open", "label": "", "type": "a" }, { "href": "https://www.interconnects.ai/p/notes-from-inside-chinas-ai-labs", "label": "", "type": "a" }, { "href": "https://www.interconnects.ai/p/glm-53-how-chinese-labs-keep-stride", "label": "", "type": "a" }, { "href": "https://x.com/CharlesRollet1/status/2091929539433890088", "label": "more", "type": "a" }, { "href": "https://x.com/kyleichan/status/2092065399701471396?s=46", "label": "discussion", "type": "a" }, { "href": "https://www.cnbc.com/2026/07/31/us-lawmakers-doordash-chinese-ai-models.html", "label": "CNBC", "type": "a" }, { "href": "https://www.bloomberg.com/news/articles/2026-04-29/us-house-probes-airbnb-anysphere-s-use-of-chinese-ai-models", "label": "Bloomberg", "type": "a" }, { "href": "https://www.semafor.com/article/04/29/2026/house-committee-probes-cursor-parent-airbnb-over-chinese-ai", "label": "Semafor", "type": "a" }, { "href": "https://www.reuters.com/world/china/trump-administration-is-concerned-by-deal-put-alibabas-ai-iphones-nyt-reports-2025-05-17/", "label": "Reuters", "type": "a" }, { "href": "https://www.harvey.ai/blog/post-training-update-harvey-tenet", "label": "Harvey", "type": "a" }, { "href": "https://www.forbes.com/sites/luisromero/2025/01/28/deepseek-now-in-perplexitys-ai-search-us-ai-dominance-challenged/", "label": "Forbes", "type": "a" }, { "href": "https://www.businessinsider.com/thomson-reuters-builds-ai-model-rely-less-on-anthropic-2026-8", "label": "Business Insider", "type": "a" }, { "href": "https://www.interconnects.ai/p/open-source-ai-reading-list/comments", "label": "Leave a comment", "type": "a" }, { "href": "https://newsletter.semianalysis.com/p/are-open-models-catching-up", "label": "Are Open Models Catching Up?", "type": "a" }, { "href": "https://artificialanalysis.ai/articles/deepseek-v4-flash-0731-scores-50-on-the-artificial-analysis-intelligence-index-10-points-above-previous-deepseek-v4-flash?utm_source=chatgpt.com", "label": "evaluation and cost on Artificial Analysis", "type": "a" }, { "href": "https://epoch.ai/benchmarks/eci?view=graph&tab=release-date&subset-view=graph&subset-tab=Software+engineering&showFrontierTrend=true&colorCategorization=Accessibility", "label": "Epoch AI", "type": "a" }, { "href": "https://artificialanalysis.ai/trends", "label": "Artificial Analysis", "type": "a" }, { "href": "https://artificialanalysis.ai/trends?country=us%2Ccn", "label": "U.S. v China", "type": "a" }, { "href": "https://www.lesswrong.com/posts/rJcCrXyEsJKmmDpWG/how-far-behind-are-open-models", "label": "How far behind are open models?", "type": "a" }, { "href": "http://z.ai/", "label": "Z.ai", "type": "a" }, { "href": "https://www.chinatalk.media/p/the-zai-playbook", "label": "", "type": "a" }, { "href": "https://joshuasaxe181906.substack.com/p/the-openaihuggingface-incident-how", "label": "The OpenAI/Huggingface incident; how we should manage the imminent arrival of autonomous hacking too cheap to meter", "type": "a" }, { "href": "https://joshuasaxe181906.substack.com/p/we-urgently-need-a-coherent-national", "label": "We urgently need a coherent national AI cybersecurity policy", "type": "a" }, { "href": "https://helentoner.substack.com/p/nonproliferation-is-the-wrong-approach", "label": "Nonproliferation is the wrong approach to AI misuse", "type": "a" }, { "href": "https://rlhfbook.com/c/12-synthetic-data", "label": "textbook chapter", "type": "a" }, { "href": "https://www.interconnects.ai/p/how-much-does-distillation-really", "label": "How much does distillation really matter for Chinese LLMs?", "type": "a" }, { "href": "https://www.anthropic.com/threat-intelligence-report-september-2026", "label": "Detecting and countering misuse of AI: September 2026", "type": "a" }, { "href": "https://arxiv.org/abs/2608.09867", "label": "Stealing Reasoning Traces from Proprietary LLM APIs", "type": "a" }, { "href": "https://x.com/kotekjedi_ml/status/2087147042888114428", "label": "X", "type": "a" }, { "href": "https://www.interconnects.ai/p/the-distillation-panic", "label": "The distillation panic", "type": "a" }, { "href": "https://natolambert.substack.com/p/how-distillation-is-used-today-and", "label": "How distillation is used today and what performance uplift it gives to open models", "type": "a" }, { "href": "https://www.interconnects.ai/p/frontiers-in-synthetic-data", "label": "Frontiers in synthetic data", "type": "a" }, { "href": "https://www.interconnects.ai/i/156272283/did-deepseek-distill-openais-o1-model-hint-no", "label": "wrote confidently", "type": "a" } ], "repair_version": 2, "repaired_at": "2026-10-14T10:47:01Z", "source_url": "https://www.interconnects.ai/p/open-source-ai-reading-list", "papers": [ { "status": "ok", "url": "https://arxiv.org/pdf/2302.04844", "bytes": 1483735, "reused": false, "local_path": "papers/repair/73d6aefca4567a9ba3dd.pdf" }, { "status": "ok", "url": "https://arxiv.org/pdf/2403.07918", "bytes": 976594, "reused": false, "local_path": "papers/repair/afeea6f602f1e4033da7.pdf" }, { "status": "ok", "url": "https://arxiv.org/pdf/2407.14933", "bytes": 853000, "reused": false, "local_path": "papers/repair/c021765e8185118438b5.pdf" }, { "status": "ok", "url": "https://arxiv.org/pdf/2604.07190", "bytes": 703680, "reused": false, "local_path": "papers/repair/ff226a759cf21e64c8d2.pdf" }, { "status": "ok", "url": "https://arxiv.org/pdf/2304.01373", "bytes": 1091147, "reused": false, "local_path": "papers/repair/3c082b260d9eb0b3488d.pdf" }, { "status": "ok", "url": "https://arxiv.org/pdf/2402.00838", "bytes": 562224, "reused": false, "local_path": "papers/repair/8fd8880ac3f82766be24.pdf" }, { "status": "ok", "url": "https://arxiv.org/pdf/2501.00656", "bytes": 4213966, "reused": false, "local_path": "papers/repair/c3650555ef74a19c70cd.pdf" }, { "status": "ok", "url": "https://arxiv.org/pdf/2512.13961", "bytes": 6817890, "reused": false, "local_path": "papers/repair/1c43811b8e35d22b44d2.pdf" }, { "status": "ok", "url": "https://arxiv.org/pdf/2608.09867", "bytes": 3849013, "reused": true, "local_path": "papers/repair/3fe09f86593e43731015.pdf" } ], "actual_video_files": [ { "local_path": "attachments/3149e64f7f8b/videos/VpYU4VOicI0.mp4", "bytes": 340912428, "valid": true, "height": 1080, "width": 1920, "duration": "2907.361000", "status": "ok" } ], "body_chars": 11168, "expected_body_images": 0, "discovery_status": "ok", "delivery_status": "ok", "missing_reason": "", "raw_html": "attachments/3149e64f7f8b/pages/source.html", "local_html": "attachments/3149e64f7f8b/pages/article.html", "body_base_path": "attachments/3149e64f7f8b/pages/", "body_markdown_path": "attachments/3149e64f7f8b/pages/article.md" }