{ "source_file": "0930", "source_platform": "Medium", "author": "Dario Radečić", "title_listed": "LLMs + Pandas: How I Use Generative AI to Generate Pandas DataFrame Summaries", "keywords": "LLM,Pandas,数据摘要", "domain": "数据处理", "content_type": "深度技术解析", "is_paid_flag": "部分免费/会员墙", "url": "https://towardsdatascience.com/llms-pandas-how-i-use-generative-ai-to-generate-pandas-dataframe-summaries-2/", "publish_date": "2025-06", "corpus_stage": "预训练/SFT", "notes": "", "id": "11a72b21491d", "collected_at": "2026-10-01T20:54:40", "title": "LLMs + Pandas: How I Use Generative AI to Generate Pandas DataFrame Summaries | Towards Data Science", "body_markdown": "Toggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/)\n# LLMs + Pandas: How I Use Generative AI to Generate Pandas DataFrame Summaries\nLocal Large Language Models can convert massive DataFrames to presentable Markdown reports — here's how.\n[Dario Radečić](https://towardsdatascience.com/author/radecicdario/)\nJune 2, 202510 min read\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/06/headway-5QgIuuBxKwM-unsplash-scaled-1.jpg)\nPhoto by [Headway](https://unsplash.com/@headwayio?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash) on [Unsplash](https://unsplash.com/photos/black-smartphone-near-person-5QgIuuBxKwM?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash)\nIf you're drowning in massive datasets and are looking for quick insights without too much manual grind, you've come to the right place.\nIn 2025, datasets often contain millions of rows and hundreds of columns, which makes manual analysis next to impossible. Local Large Language Models can transform your raw DataFrame statistics into polished, readable reports in seconds — minutes at worst. This approach eliminates the tedious process of analyzing data by hand and writing executive reports, especially if the data structure doesn't change.\nPandas handles the heavy lifting of data extraction while LLMs convert your technical outputs into presentable reports. You'll still need to write functions that pull key statistics from your datasets, but it's a one-time effort.\nThis guide assumes you have Ollama installed locally. If you don't, you can still use third-party LLM vendors, but I won't explain how to connect to their APIs.\nLearn this step by step with the interactive [AI Engineer](https://roadmap.sh/ai-engineer) roadmap.\n### Table of contents:\n * Dataset Introduction and Exploration\n * The Boring Part: Extracting Summary Statistics\n * The Cool Part: Working with LLMs\n * What You Could Improve\n\n\n## Dataset Introduction and Exploration\nFor this guide, I'm using the [MBA admissions dataset from Kaggle](https://www.kaggle.com/datasets/taweilo/mba-admission-dataset). Download it if you want to follow along.\nThe dataset is licensed under the **Apache 2.0 license** , which means you can use it freely for both personal and commercial projects.\nTo get started, you'll need a few Python libraries installed on your system.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/06/1-1.png)\nImage 1 - Required Python libraries and versions (image by author)\nOnce you have everything installed, import the necessary libraries in a new script or a notebook:\npython\n\n```\nimport pandas as pd\nfrom langchain_ollama import ChatOllama\nfrom typing import Literal\n```\n\n### Dataset loading and preprocessing\nStart by loading the dataset with Pandas. This snippet loads the CSV file, prints basic information about the dataset shape, and shows how many missing values exist in each column:\npython\n\n```\ndf = pd.read_csv(\"data/MBA.csv\")\n\n# Basic dataset info\nprint(f\"Dataset shape: {df.shape}\\n\")\nprint(\"Missing value stats:\")\nprint(df.isnull().sum())\nprint(\"-\" * 25)\ndf.sample(5)\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/06/2-1024x560.png)\nImage 2 - Basic dataset statistics (image by author)\nSince data cleaning isn't the main focus of this article, I'll keep the preprocessing minimal. The dataset only has a couple of missing values that need attention:\npython\n\n```\ndf[\"race\"] = df[\"race\"].fillna(\"Unknown\")\ndf[\"admission\"] = df[\"admission\"].fillna(\"Deny\")\n```\n\nThat's it! Let's see how to go from this to a meaningful report next.\n## The Boring Part: Extracting Summary Statistics\nEven with all the advances in AI capability and availability, you probably don't want to send your entire dataset to an LLM provider. There are a couple of good reasons why.\nIt could **consume way too many tokens** , which translates directly to higher costs. Processing large datasets can **take a long time** , especially when you're running models locally on your own hardware. You might also be dealing with **sensitive data** that shouldn't leave your organization.\nSome manual work is still the way to go.\nThis approach requires you to write a function that extracts key elements and statistics from your Pandas DataFrame. You'll have to write this function from scratch for different datasets, but the core idea transfers easily between projects.\nThe `get_summary_context_message()` function takes in a DataFrame and returns a formatted multi-line string with a detailed summary. Here's what it includes:\n * Total application count and gender distribution\n * International vs domestic applicant breakdown\n * GPA and GMAT score quartile statistics\n * Admission rates by academic major (sorted by rate)\n * Admission rates by work industry (top 8 industries)\n * Work experience analysis with categorical breakdowns\n * Key insights highlighting top-performing categories\n\n\nHere's the complete source code for the function:\npython\n\n```\ndef get_summary_context_message(df: pd.DataFrame) -> str:\n \"\"\"\n Generate a comprehensive summary report of MBA admissions dataset statistics.\n \n This function analyzes MBA application data to provide detailed statistics on\n applicant demographics, academic performance, professional backgrounds, and\n admission rates across various categories. The summary includes gender and\n international status distributions, GPA and GMAT score statistics, admission\n rates by academic major and work industry, and work experience impact analysis.\n \n Parameters\n ----------\n df : pd.DataFrame\n DataFrame containing MBA admissions data with the following expected columns:\n - 'gender', 'international', 'gpa', 'gmat', 'major', 'work_industry', 'work_exp', 'admission'\n \n Returns\n -------\n str\n A formatted multi-line string containing comprehensive MBA admissions\n statistics.\n \"\"\"\n # Basic application statistics\n total_applications = len(df)\n\n # Gender distribution\n gender_counts = df[\"gender\"].value_counts()\n male_count = gender_counts.get(\"Male\", 0)\n female_count = gender_counts.get(\"Female\", 0)\n\n # International status\n international_count = (\n df[\"international\"].sum()\n if df[\"international\"].dtype == bool\n else (df[\"international\"] == True).sum()\n )\n\n # GPA statistics\n gpa_data = df[\"gpa\"].dropna()\n gpa_avg = gpa_data.mean()\n gpa_25th = gpa_data.quantile(0.25)\n gpa_50th = gpa_data.quantile(0.50)\n gpa_75th = gpa_data.quantile(0.75)\n\n # GMAT statistics\n gmat_data = df[\"gmat\"].dropna()\n gmat_avg = gmat_data.mean()\n gmat_25th = gmat_data.quantile(0.25)\n gmat_50th = gmat_data.quantile(0.50)\n gmat_75th = gmat_data.quantile(0.75)\n\n # Major analysis - admission rates by major\n major_stats = []\n for major in df[\"major\"].unique():\n major_data = df[df[\"major\"] == major]\n admitted = len(major_data[major_data[\"admission\"] == \"Admit\"])\n total = len(major_data)\n rate = (admitted / total) * 100\n major_stats.append((major, admitted, total, rate))\n\n # Sort by admission rate (descending)\n major_stats.sort(key=lambda x: x[3], reverse=True)\n\n # Work industry analysis - admission rates by industry\n industry_stats = []\n for industry in df[\"work_industry\"].unique():\n if pd.isna(industry):\n continue\n industry_data = df[df[\"work_industry\"] == industry]\n admitted = len(industry_data[industry_data[\"admission\"] == \"Admit\"])\n total = len(industry_data)\n rate = (admitted / total) * 100\n industry_stats.append((industry, admitted, total, rate))\n\n # Sort by admission rate (descending)\n industry_stats.sort(key=lambda x: x[3], reverse=True)\n\n # Work experience analysis\n work_exp_data = df[\"work_exp\"].dropna()\n avg_work_exp_all = work_exp_data.mean()\n\n # Work experience for admitted students\n admitted_students = df[df[\"admission\"] == \"Admit\"]\n admitted_work_exp = admitted_students[\"work_exp\"].dropna()\n avg_work_exp_admitted = admitted_work_exp.mean()\n\n # Work experience ranges analysis\n def categorize_work_exp(exp):\n if pd.isna(exp):\n return \"Unknown\"\n elif exp < 2:\n return \"0-1 years\"\n elif exp < 4:\n return \"2-3 years\"\n elif exp < 6:\n return \"4-5 years\"\n elif exp < 8:\n return \"6-7 years\"\n else:\n return \"8+ years\"\n\n df[\"work_exp_category\"] = df[\"work_exp\"].apply(categorize_work_exp)\n work_exp_category_stats = []\n\n for category in [\"0-1 years\", \"2-3 years\", \"4-5 years\", \"6-7 years\", \"8+ years\"]:\n category_data = df[df[\"work_exp_category\"] == category]\n if len(category_data) > 0:\n admitted = len(category_data[category_data[\"admission\"] == \"Admit\"])\n total = len(category_data)\n rate = (admitted / total) * 100\n work_exp_category_stats.append((category, admitted, total, rate))\n\n # Build the summary message\n summary = f\"\"\"MBA Admissions Dataset Summary (2025)\n \nTotal Applications: {total_applications:,} people applied to the MBA program.\n\nGender Distribution:\n- Male applicants: {male_count:,} ({male_count/total_applications*100:.1f}%)\n- Female applicants: {female_count:,} ({female_count/total_applications*100:.1f}%)\n\nInternational Status:\n- International applicants: {international_count:,} ({international_count/total_applications*100:.1f}%)\n- Domestic applicants: {total_applications-international_count:,} ({(total_applications-international_count)/total_applications*100:.1f}%)\n\nAcademic Performance Statistics:\n\nGPA Statistics:\n- Average GPA: {gpa_avg:.2f}\n- 25th percentile: {gpa_25th:.2f}\n- 50th percentile (median): {gpa_50th:.2f}\n- 75th percentile: {gpa_75th:.2f}\n\nGMAT Statistics:\n- Average GMAT: {gmat_avg:.0f}\n- 25th percentile: {gmat_25th:.0f}\n- 50th percentile (median): {gmat_50th:.0f}\n- 75th percentile: {gmat_75th:.0f}\n\nMajor Analysis - Admission Rates by Academic Background:\"\"\"\n\n for major, admitted, total, rate in major_stats:\n summary += (\n f\"\\n- {major}: {admitted}/{total} admitted ({rate:.1f}% admission rate)\"\n )\n\n summary += (\n \"\\n\\nWork Industry Analysis - Admission Rates by Professional Background:\"\n )\n\n # Show top 8 industries by admission rate\n for industry, admitted, total, rate in industry_stats[:8]:\n summary += (\n f\"\\n- {industry}: {admitted}/{total} admitted ({rate:.1f}% admission rate)\"\n )\n\n summary += \"\\n\\nWork Experience Impact on Admissions:\\n\\nOverall Work Experience Comparison:\"\n summary += (\n f\"\\n- Average work experience (all applicants): {avg_work_exp_all:.1f} years\"\n )\n summary += f\"\\n- Average work experience (admitted students): {avg_work_exp_admitted:.1f} years\"\n\n summary += \"\\n\\nAdmission Rates by Work Experience Range:\"\n for category, admitted, total, rate in work_exp_category_stats:\n summary += (\n f\"\\n- {category}: {admitted}/{total} admitted ({rate:.1f}% admission rate)\"\n )\n\n # Key insights\n best_major = major_stats[0]\n best_industry = industry_stats[0]\n\n summary += \"\\n\\nKey Insights:\"\n summary += (\n f\"\\n- Highest admission rate by major: {best_major[0]} at {best_major[3]:.1f}%\"\n )\n summary += f\"\\n- Highest admission rate by industry: {best_industry[0]} at {best_industry[3]:.1f}%\"\n\n if avg_work_exp_admitted > avg_work_exp_all:\n summary += f\"\\n- Admitted students have slightly more work experience on average ({avg_work_exp_admitted:.1f} vs {avg_work_exp_all:.1f} years)\"\n else:\n summary += \"\\n- Work experience shows minimal difference between admitted and all applicants\"\n\n return summary\n```\n\nOnce you've defined the function, simply call it and print the results:\npython\n\n```\nprint(get_summary_context_message(df))\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/06/3-1024x908.png)\nImage 3 - Extracted findings and statistics from the dataset (image by author)\nNow let's move on to the fun part.\n## The Cool Part: Working with LLMs\nThis is where things get interesting and your manual data extraction work pays off.\n### Python helper function for working with LLMs\nIf you have decent hardware, I strongly recommend using local LLMs for simple tasks like this. I use [Ollama](https://ollama.com) and the latest version of the **Mistral model** for the actual LLM processing.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/06/4-1024x569.png)\nImage 4 - Available Ollama models (image by author)\nIf you want to use something like ChatGPT through OpenAI API, you can still do that. You'll just need to modify the function below to set up your API key and return the appropriate instance from Langchain.\nRegardless of the option you choose, a call to `get_llm()` with a test message shouldn't return an error:\npython\n\n```\ndef get_llm(model_name: str = \"mistral:latest\") -> ChatOllama:\n \"\"\"\n Create and configure a ChatOllama instance for local LLM inference.\n \n This function initializes a ChatOllama client configured to connect to a\n local Ollama server. The client is set up with deterministic output\n (temperature=0) for consistent responses across multiple calls with the\n same input.\n \n Parameters\n ----------\n model_name : str, optional\n The name of the Ollama model to use for chat completions.\n Must be a valid model name that is available on the local Ollama\n installation. Default is \"mistral:latest\".\n \n Returns\n -------\n ChatOllama\n A configured ChatOllama instance ready for chat completions.\n \"\"\"\n return ChatOllama(\n model=model_name, base_url=\"http://localhost:11434\", temperature=0\n )\n\n\nprint(get_llm().invoke(\"test\").content)\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/06/5-1024x199.png)\nImage 5 - LLM test message (image by author)\n### Summarization prompt\nThis is where you can get creative and write ultra-specific instructions for your LLM. I've decided to keep things light for demonstration purposes, but feel free to experiment here.\nThere isn't a single right or wrong prompt.\nWhatever you do, make sure to include the format arguments using curly brackets - these values will be filled dynamically later:\npython\n\n```\nSUMMARIZE_DATAFRAME_PROMPT = \"\"\"\nYou are an expert data analyst and data summarizer. Your task is to take in complex datasets\nand return user-friendly descriptions and findings.\n\nYou were given this dataset:\n- Name: {dataset_name}\n- Source: {dataset_source}\n\nThis dataset was analyzed in a pipeline before it was given to you.\nThese are the findings returned by the analysis pipeline:\n\n\n{context}\n\n\nBased on these findings, write a detailed report in {report_format} format.\nGive the report a meaningful title and separate findings into sections with headings and subheadings.\nOutput only the report in {report_format} and nothing else.\n\nReport:\n\"\"\"\n```\n\n### Summarization Python function\nWith the prompt and the `get_llm()` functions declared, the only thing left is to connect the dots. The `get_report_summary()` function takes in arguments that will fill the format placeholders in the prompt, then invokes the LLM with that prompt to generate a report.\nYou can choose between Markdown or HTML formats:\npython\n\n```\ndef get_report_summary(\n dataset: pd.DataFrame,\n dataset_name: str,\n dataset_source: str,\n report_format: Literal[\"markdown\", \"html\"] = \"markdown\",\n) -> str:\n \"\"\"\n Generate an AI-powered summary report from a pandas DataFrame.\n \n This function analyzes a dataset and generates a comprehensive summary report\n using a large language model (LLM). It first extracts statistical context\n from the dataset, then uses an LLM to create a human-readable report in the\n specified format.\n \n Parameters\n ----------\n dataset : pd.DataFrame\n The pandas DataFrame to analyze and summarize.\n dataset_name : str\n A descriptive name for the dataset that will be included in the\n generated report for context and identification.\n dataset_source : str\n Information about the source or origin of the dataset.\n report_format : {\"markdown\", \"html\"}, optional\n The desired output format for the generated report. Options are:\n - \"markdown\" : Generate report in Markdown format (default)\n - \"html\" : Generate report in HTML format\n \n Returns\n -------\n str\n A formatted summary report.\n \n \"\"\"\n context_message = get_summary_context_message(df=dataset)\n prompt = SUMMARIZE_DATAFRAME_PROMPT.format(\n dataset_name=dataset_name,\n dataset_source=dataset_source,\n context=context_message,\n report_format=report_format,\n )\n return get_llm().invoke(input=prompt).content\n```\n\nUsing the function is straightforward - just pass in the dataset, its name, and source. The report format defaults to Markdown:\npython\n\n```\nmd_report = get_report_summary(\n dataset=df, \n dataset_name=\"MBA Admissions (2025)\",\n dataset_source=\"https://www.kaggle.com/datasets/taweilo/mba-admission-dataset\"\n)\nprint(md_report)\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/06/6-987x1024.png)\nImage 6 - Final report in Markdown format (image by author)\nThe HTML report is just as detailed, but could use some styling. Maybe you could ask the LLM to handle that as well!\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/06/7-882x1024.png)\nImage 7 - Final report in HTML format (image by author)\n## What You Could Improve\nI could have easily turned this into a 30-minute read by optimizing every detail of the pipeline, but I kept it simple for demonstration purposes. You don't have to (and shouldn't) stop here though.\nHere are the things you can improve to make this pipeline even more powerful:\n * **Write a function that saves the report (Markdown or HTML) directly to disk**. This way you can automate the entire process and generate reports on a schedule without manual intervention.\n * In the prompt, **ask the LLM to add CSS styling to the HTML report** to make it look more presentable. You could even provide your company's brand colors and fonts to maintain consistency across all your data reports.\n * **Expand the prompt to follow more specific instructions**. You might want reports that focus on specific business metrics, follow a particular template, or include recommendations based on the findings.\n * Expand the `get_llm()` function so it can **connect both to Ollama and other vendors** like OpenAI, Anthropic, or Google. This gives you flexibility to switch between local and cloud-based models depending on your needs.\n * Do literally anything in the get_summary_context_message() function since it serves as the foundation for all **context data provided to the LLM**. This is where you can get creative with feature engineering, statistical analysis, and data insights that matter to your specific use case.\n\n\nI hope this minimal example has set you on the right track to automate your own data reporting workflows.\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](https://assets.insightmediagroup.io/media/1787657095692_93hujy.png)\nWritten By\n## [Dario Radečić](https://towardsdatascience.com/author/radecicdario/)\nhttps://writingfordevs.substack.com\n[See all from Dario Radečić](https://towardsdatascience.com/author/radecicdario/)\n[Data Science](https://towardsdatascience.com/tag/data-science/)[Generative Ai](https://towardsdatascience.com/tag/generative-ai/)[Llm](https://towardsdatascience.com/tag/llm/)[Pandas](https://towardsdatascience.com/tag/pandas/)[Python](https://towardsdatascience.com/tag/python/)\nShare This Article\n * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fllms-pandas-how-i-use-generative-ai-to-generate-pandas-dataframe-summaries-2%2F&title=LLMs%20%2B%20Pandas%3A%20How%20I%20Use%20Generative%20AI%20to%20Generate%20Pandas%20DataFrame%20Summaries)\n * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fllms-pandas-how-i-use-generative-ai-to-generate-pandas-dataframe-summaries-2%2F&title=LLMs%20%2B%20Pandas%3A%20How%20I%20Use%20Generative%20AI%20to%20Generate%20Pandas%20DataFrame%20Summaries)\n * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fllms-pandas-how-i-use-generative-ai-to-generate-pandas-dataframe-summaries-2%2F&text=LLMs%20%2B%20Pandas%3A%20How%20I%20Use%20Generative%20AI%20to%20Generate%20Pandas%20DataFrame%20Summaries)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/01/0JGfltYm9_6YMqQaZ-scaled.jpg)](https://towardsdatascience.com/reducing-model-training-cost-with-active-learning-255b40a2aa98/)\n## [Reducing Model Training Cost With Active Learning](https://towardsdatascience.com/reducing-model-training-cost-with-active-learning-255b40a2aa98/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nA complete case study with a Gaussian Process model.\n[Shuai Guo](https://towardsdatascience.com/author/shuaiguo/)\nJanuary 6, 202111 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/12vsWx2qQzV4CBKIrsDtWow.jpeg)](https://towardsdatascience.com/top-10-matrix-operations-in-numpy-with-examples-d761448cb7a8/)\n## [Top 10 Matrix Operations in Numpy with Examples](https://towardsdatascience.com/top-10-matrix-operations-in-numpy-with-examples-d761448cb7a8/)\n[](https://towardsdatascience.com/llms-pandas-how-i-use-generative-ai-to-generate-pandas-dataframe-summaries-2/)\nPerform Linear Algebra with Python\n[Rukshan Pramoditha](https://towardsdatascience.com/author/rukshanpramoditha/)\nMarch 24, 20217 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/10/0NYDgzZWGw6hzMeVI-scaled.jpg)](https://towardsdatascience.com/beginner-tutorial-connect-gpt-models-with-company-data-in-microsoft-azure-81177929da18/)\n## [Beginner Tutorial: Connect GPT Models with Company Data in Microsoft Azure](https://towardsdatascience.com/beginner-tutorial-connect-gpt-models-with-company-data-in-microsoft-azure-81177929da18/)\n[Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/)\nUsing OpenAI Studio, Cognitive Search and Storage Accounts\n[Eirik Berge](https://towardsdatascience.com/author/ebbeberge/)\nOctober 15, 20239 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2026/06/compare_flour_being_sifted_5947516_v3_card.jpg)](https://towardsdatascience.com/anchor-detection-for-rag-parallel-detectors-then-one-llm-call-at-the-end/)\n## [Finding the Right Anchor Is Where RAG Stops Hallucinating: Keyword, Embedding, and TOC Signals in Parallel](https://towardsdatascience.com/anchor-detection-for-rag-parallel-detectors-then-one-llm-call-at-the-end/)\n[Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/)\nEnterprise Document Intelligence [Vol.1 #7B] - Retrieval is filtering on structured tables: keywords first, TOC second, embeddings last\n[Angela and Kezhan Shi](https://towardsdatascience.com/author/angela.shi/)\nJune 24, 202629 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2026/05/Gemini_Generated_Image_uoh7tkuoh7tkuoh7-1.jpg)](https://towardsdatascience.com/the-must-know-topics-for-an-llm-engineer/)\n## [The Must-Know Topics for an LLM Engineer](https://towardsdatascience.com/the-must-know-topics-for-an-llm-engineer/)\n[Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/)\nFrom tokenisation to evaluation : how modern language models actually work in practice\n[Aliaksei Mikhailiuk](https://towardsdatascience.com/author/mikhailiuk/)\nMay 9, 202624 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/12/1Y0n6pABOINiptYbRrNKnng.png)](https://towardsdatascience.com/epic-crossover-between-alphafold-3-and-gpt-4os-knowledge-of-protein-data-bank-entries-ec7b6dd589e0/)\n## [Epic \"Crossover\" Between AlphaFold 3 and GPT-4o's Knowledge of Protein Data Bank Entries](https://towardsdatascience.com/epic-crossover-between-alphafold-3-and-gpt-4os-knowledge-of-protein-data-bank-entries-ec7b6dd589e0/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nExploring how GPT-4o's knowledge of the Protein Data Bank coupled to systems like AlphaFold 3 could allow for new ways to search and study...\n[Luciano Abriata](https://towardsdatascience.com/author/lucianosphere/)\nDecember 17, 202413 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2026/06/compare_old_and_rusty_36758467_v3_card.jpg)](https://towardsdatascience.com/question-parsing-in-rag-structure-before-you-search/)\n## [Parse the question before you search: the missing step in most RAG pipelines](https://towardsdatascience.com/question-parsing-in-rag-structure-before-you-search/)\n[Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/)\nEnterprise Document Intelligence [Vol.1 #6a] - Why a user question deserves the same parsing as the document, and how it splits into a retrieval brief and a generation brief before either runs\n[Angela and Kezhan Shi](https://towardsdatascience.com/author/angela.shi/)\nJune 16, 202611 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/01/1la6c-wr0NS9F6fpPB5Q7qA-scaled.jpeg)](https://towardsdatascience.com/one-big-lesson-on-creating-a-software-library-723c36180941/)\n## [One Big Lesson on Creating a Software Library](https://towardsdatascience.com/one-big-lesson-on-creating-a-software-library-723c36180941/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nTales of an almost 2-year-old data science software library\n[Rose Day](https://towardsdatascience.com/author/rjday/)\nJanuary 18, 20215 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/06/ChatGPT-Image-11-juin-2025-09_16_53.png)](https://towardsdatascience.com/boost-your-llm-outputdesign-smarter-prompts-real-tricks-from-an-ai-engineers-toolbox/)\n## [Design Smarter Prompts and Boost Your LLM Output: Real Tricks from an AI Engineer’s Toolbox](https://towardsdatascience.com/boost-your-llm-outputdesign-smarter-prompts-real-tricks-from-an-ai-engineers-toolbox/)\n[Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/)\nNot just what you ask, but how you ask it. Practical techniques for prompt engineering that deliver\n[Ugo Pradère](https://towardsdatascience.com/author/ugopradere/)\nJune 12, 20258 min read\n\n\n * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n * [X](https://x.com/TDataScience)\n * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n * [Threads](https://www.threads.net/@towardsdatascience)\n * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n#### Recommended Articles \nClose\nword word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word\n \nmmMwWLliI0fiflO&1 \nmmMwWLliI0fiflO&1 \nmmMwWLliI0fiflO&1 \nmmMwWLliI0fiflO&1 \nmmMwWLliI0fiflO&1 \nmmMwWLliI0fiflO&1 \nmmMwWLliI0fiflO&1\n", "metadata": { "title": "LLMs + Pandas: How I Use Generative AI to Generate Pandas DataFrame Summaries | Towards Data Science", "description": "Local Large Language Models can convert massive DataFrames to presentable Markdown reports — here's how.", "keywords": null, "author": "Dario Radečić", "og:locale": "en_US", "og:type": "article", "og:title": "LLMs + Pandas: How I Use Generative AI to Generate Pandas DataFrame Summaries", "og:description": "Local Large Language Models can convert massive DataFrames to presentable Markdown reports — here's how.", "og:url": "https://towardsdatascience.com/llms-pandas-how-i-use-generative-ai-to-generate-pandas-dataframe-summaries-2/", "og:site_name": "Towards Data Science", "og:image": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/06/headway-5QgIuuBxKwM-unsplash-scaled-1.jpg", "og:image:width": "2560", "og:image:height": "1707", "og:image:type": "image/jpeg", "twitter:card": "summary_large_image", "twitter:site": "@TDataScience", "twitter:creator": "@TDataScience", "twitter:label1": "Written by", "twitter:data1": "Dario Radečić", "twitter:label2": "Est. reading time", "twitter:data2": "10 min read", "article:published_time": "2025-06-02T18:23:18.000Z", "article:modified_time": "2025-06-02T18:23:18.000Z", "article:author": "https://towardsdatascience.com/author/radecicdario/" }, "paywall": { "listed_paid": true, "detected": false, "marker": "", "public_only": true, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 200, "error": "", "retried": true }, "code_blocks": [ { "language": "txt", "file": "attachments/11a72b21491d/code/block_000.txt", "chars": 87 }, { "language": "txt", "file": "attachments/11a72b21491d/code/block_001.txt", "chars": 177 }, { "language": "txt", "file": "attachments/11a72b21491d/code/block_002.txt", "chars": 91 }, { "language": "txt", "file": "attachments/11a72b21491d/code/block_003.txt", "chars": 6840 }, { "language": "txt", "file": "attachments/11a72b21491d/code/block_004.txt", "chars": 39 }, { "language": "txt", "file": "attachments/11a72b21491d/code/block_005.txt", "chars": 912 }, { "language": "txt", "file": "attachments/11a72b21491d/code/block_006.txt", "chars": 669 }, { "language": "txt", "file": "attachments/11a72b21491d/code/block_007.txt", "chars": 1507 }, { "language": "txt", "file": "attachments/11a72b21491d/code/block_008.txt", "chars": 193 } ], "images": [ { "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/06/headway-5QgIuuBxKwM-unsplash-scaled-1.jpg", "alt": "", "local_path": "attachments/11a72b21491d/images/000.jpg", "bytes": 397029, "downloaded": true }, { "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/06/1-1.png", "alt": "", "local_path": "attachments/11a72b21491d/images/001.png", "bytes": 10983, "downloaded": true }, { "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/06/2-1024x560.png", "alt": "", "local_path": "attachments/11a72b21491d/images/002.png", "bytes": 90139, "downloaded": true }, { "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/01/0JGfltYm9_6YMqQaZ-scaled.jpg", "alt": "", "local_path": "attachments/11a72b21491d/images/003.jpg", "bytes": 294890, "downloaded": true }, { "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/12vsWx2qQzV4CBKIrsDtWow.jpeg", "alt": "", "local_path": "attachments/11a72b21491d/images/004.jpeg", "bytes": 616529, "downloaded": true }, { "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2023/10/0NYDgzZWGw6hzMeVI-scaled.jpg", "alt": "", "local_path": "attachments/11a72b21491d/images/005.jpg", "bytes": 403809, "downloaded": true }, { "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2026/06/compare_flour_being_sifted_5947516_v3_card.jpg", "alt": "", "local_path": "attachments/11a72b21491d/images/006.jpg", "bytes": 117877, "downloaded": true }, { "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2026/05/Gemini_Generated_Image_uoh7tkuoh7tkuoh7-1.jpg", "alt": "", "local_path": "attachments/11a72b21491d/images/007.jpg", "bytes": 612971, "downloaded": true }, { "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2024/12/1Y0n6pABOINiptYbRrNKnng.png", "alt": "", "local_path": "attachments/11a72b21491d/images/008.png", "bytes": 211339, "downloaded": true }, { "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2026/06/compare_old_and_rusty_36758467_v3_card.jpg", "alt": "", "local_path": "attachments/11a72b21491d/images/009.jpg", "bytes": 124798, "downloaded": true }, { "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/01/1la6c-wr0NS9F6fpPB5Q7qA-scaled.jpeg", "alt": "", "local_path": "attachments/11a72b21491d/images/010.jpeg", "bytes": 835457, "downloaded": true }, { "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/06/ChatGPT-Image-11-juin-2025-09_16_53.png", "alt": "", "local_path": "attachments/11a72b21491d/images/011.png", "bytes": 3086467, "downloaded": true } ], "audio": [], "videos": [], "links": [], "retry_at": "2026-10-03T07:58:06", "repair_version": 2, "repaired_at": "2026-10-14T11:11:48Z", "source_url": "https://towardsdatascience.com/llms-pandas-how-i-use-generative-ai-to-generate-pandas-dataframe-summaries-2/", "delivery_status": "excluded_paid", "discovery_status": "excluded_paid", "missing_reason": "按原约定排除付费/会员全文" }