{
  "source_file": "0930",
  "source_platform": "Medium",
  "author": "Federico Trotta",
  "title_listed": "Unleashing the Power of Prompt Engineering for Data Scientists",
  "keywords": "Prompt工程,数据科学",
  "domain": "大模型算法",
  "content_type": "深度技术解析",
  "is_paid_flag": "部分免费/会员墙",
  "url": "https://towardsdatascience.com/unleashing-the-power-of-prompt-engineering-for-data-scientists-16b6d1f2bf85/",
  "publish_date": "2023-06",
  "corpus_stage": "预训练/SFT",
  "notes": "",
  "id": "accb21218fdf",
  "collected_at": "2026-10-02T00:40:13",
  "title": "Unleashing the Power of Prompt Engineering for Data Scientists | Towards Data Science",
  "body_markdown": "![Revisit consent button](https://cdn-cookieyes.com/assets/images/revisit.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](https://cdn-cookieyes.com/assets/images/close.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n  * Cookie\nBCTempID\n  * Duration\n10 minutes\n  * Description\nNo description available.\n\n\n  * Cookie\n__cf_bm\n  * Duration\n1 hour\n  * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n  * Cookie\nAWSALBCORS\n  * Duration\n7 days\n  * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n  * Cookie\n_cfuvid\n  * Duration\nsession\n  * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n  * Cookie\nli_gc\n  * Duration\n6 months\n  * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n  * Cookie\n__hssrc\n  * Duration\nsession\n  * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n  * Cookie\n__hssc\n  * Duration\n1 hour\n  * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n  * Cookie\nwpEmojiSettingsSupports\n  * Duration\nsession\n  * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n  * Cookie\nBCSessionID\n  * Duration\n1 year 1 month 4 days\n  * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n  * Cookie\n_octo\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nlogged_in\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\n__Secure-YEC\n  * Duration\npast\n  * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n  * Cookie\n__eoi\n  * Duration\n6 months\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nAWSALBTGCORS\n  * Duration\n7 days\n  * Description\nNo description available.\n\n\n  * Cookie\nlogin-status-p\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nAWSALBTG\n  * Duration\n7 days\n  * Description\nNo description available.\n\n\n  * Cookie\ncsrf_token\n  * Duration\nsession\n  * Description\nNo description available.\n\n\n  * Cookie\ntoken_v2\n  * Duration\n1 day\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nD\n  * Duration\n1 year\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nPHPSESSID\n  * Duration\nsession\n  * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n  * Cookie\nVISITOR_PRIVACY_METADATA\n  * Duration\n6 months\n  * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n  * Cookie\ncookietest\n  * Duration\nsession\n  * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n  * Cookie\n__Host-airtable-session\n  * Duration\n1 year\n  * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n  * Cookie\n__Host-airtable-session.sig\n  * Duration\n1 year\n  * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n  * Cookie\nm\n  * Duration\n1 year 1 month 4 days\n  * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n  * Cookie\nBIGipServer*\n  * Duration\nsession\n  * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n  * Cookie\n__cfruid\n  * Duration\nsession\n  * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n  * Cookie\n_GRECAPTCHA\n  * Duration\n6 months\n  * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n  * Cookie\n__Secure-YNID\n  * Duration\n6 months\n  * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n  * Cookie\ncookieyes-consent\n  * Duration\n1 year\n  * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n  * Cookie\nlidc\n  * Duration\n1 day\n  * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n  * Cookie\nbrw\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nbrwConsent\n  * Duration\n5 minutes\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nWMF-Uniq\n  * Duration\n1 year\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nloom_anon_comment\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nloom_referral_video\n  * Duration\nsession\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nVISITOR_INFO1_LIVE\n  * Duration\n6 months\n  * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n  * Cookie\nyt-remote-connected-devices\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n  * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n  * Duration\nNever Expires\n  * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n  * Cookie\nyt-remote-device-id\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n  * Cookie\nyt-remote-session-name\n  * Duration\nsession\n  * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n  * Cookie\nyt-remote-fast-check-period\n  * Duration\nsession\n  * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n  * Cookie\nyt-remote-session-app\n  * Duration\nsession\n  * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n  * Cookie\nyt-remote-cast-available\n  * Duration\nsession\n  * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n  * Cookie\nyt-remote-cast-installed\n  * Duration\nsession\n  * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n  * Cookie\ncp_session\n  * Duration\n3 months\n  * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n  * Cookie\nloid\n  * Duration\n1 year 1 month 4 days\n  * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n  * Cookie\n__hstc\n  * Duration\n6 months\n  * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n  * Cookie\nhubspotutk\n  * Duration\n6 months\n  * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n  * Cookie\n_ga\n  * Duration\n1 year 1 month 4 days\n  * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n  * Cookie\n_ga_*\n  * Duration\n1 year 1 month 4 days\n  * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n  * Cookie\n__Host-psifi.analyticsTrace\n  * Duration\n6 hours\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\n__Host-psifi.analyticsTraceV2\n  * Duration\n6 hours\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\n_gh_sess\n  * Duration\nsession\n  * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n  * Cookie\nYSC\n  * Duration\nsession\n  * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n  * Cookie\najs_anonymous_id\n  * Duration\n1 year\n  * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n  * Cookie\nvuid\n  * Duration\n1 year 1 month 4 days\n  * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n  * Cookie\nAWSALB\n  * Duration\n7 days\n  * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n  * Cookie\nacq\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nacq.sig\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nptc\n  * Duration\n2 years\n  * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n  * Cookie\nmuc_ads\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n  * Cookie\nguest_id_marketing\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n  * Cookie\nguest_id_ads\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n  * Cookie\npersonalization_id\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n  * Cookie\nguest_id\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n  * Cookie\nbcookie\n  * Duration\n1 year\n  * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n  * Cookie\n__Secure-ROLLOUT_TOKEN\n  * Duration\n6 months\n  * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n  * Cookie\nyt.innertube::nextId\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n  * Cookie\nyt.innertube::requests\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n  * Cookie\nsession_tracker\n  * Duration\nsession\n  * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n  * Cookie\nedgebucket\n  * Duration\nsession\n  * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n  * Cookie\ndid\n  * Duration\n1 year\n  * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\n# Unleashing the Power of Prompt Engineering for Data Scientists\nHow and why write effective prompts if you work with data\n[Federico Trotta](https://towardsdatascience.com/author/federicotrotta/)\nJune 7, 202319 min read\n![Image by Gerd Altmann on Pixabay](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/06/1YVJFwLNW6MZaDSgS-w4vtw.jpeg)\nImage by [Gerd Altmann](https://pixabay.com/it/users/geralt-9301/?utm_source=link-attribution&utm_medium=referral&utm_campaign=image&utm_content=4126485) on [Pixabay](https://pixabay.com/it//?utm_source=link-attribution&utm_medium=referral&utm_campaign=image&utm_content=4126485)\nThanks to GPT models, prompt engineering is becoming a field of interest in Data Science. In the beginning, we've seen a lot of curious people across the globe testing ChatGPT to trick it. Then, while this trend has (finally!) finished, the movement of people using it to automate boring stuff or to be helped with general tasks, is steadily increased.\nDevelopers and Data Scientists are the ones that are having great benefits from using prompt systems like ChatGPT. So, in this article, we'll see an overview of prompt engineering and how to write efficient prompts for Data Scientists.\nI know you're benefiting a lot from ChatGPT, don't you?! But the truth is that sometimes we're not just able to obtain what we'd like from it as Data Scientists. So, let's see how we can improve our prompt skills with some easy precautions.\nThis is what you'll read in this article:\ntext\n\n```\nTable of Contents:\n\nThe importance of prompt engineering today\nHow prompt engineering can affect Data Scientists\nExamples of effective prompts for Data Scientists\n```\n\n* * *\nLearn this step by step with the interactive [AI Engineer](https://roadmap.sh/ai-engineer), [AI and Data Scientist](https://roadmap.sh/ai-data-scientist), [Computer Science](https://roadmap.sh/computer-science) and [Machine Learning](https://roadmap.sh/machine-learning) roadmaps.\n## The importance of prompt engineering today\nThe word of the last 150 years could be \"automation\". In fact, the world has evolved from making stuff by hand to assembly lines. And while crafts are still (highly) valuable, \"mass production\" has become a word coupled with \"automation\".\nThe mechanization and automation of the work are increasing and this has permeated different fields, not only the ones directly involved in producing goods like, for example, manufacturing or agriculture.\nIf we take a look at a Software, for example, the first thing that we should see is automation. When I was learning Python about three years ago a mentor, reviewing my first project, told me: \"Federico, developing software means automating stuff!\". And, if you're asking, then yes: my first project was a mess (as with everything we do for the first time!).\nAnyway, here's the truth: human beings have evolved with a clear goal: automating stuff. This can be related to automating boring stuff or \"hard work\". It doesn't matter; the point is the direction towards automation.\nIn this scenario, prompt engineering is just the latest tool that can help us automate stuff. On a \"code sight,\" this means automating the automatable: software development is automation at its core, and using prompt engineering means pushing automation even harder.\nThe truth, in fact, is that even in software development there are boring tasks, even if, for example, we've created classes that can be imported (but need to be modified a little bit).\nThink of that for a moment: as a Data Scientist, how many prototypes per week do you develop? And how much time do you have to develop them?\nYou create a prototype and then what? The specifications of the projects change, the customer changes ideas, your boss is not satisfied...well, you named it.\nSo, why should we spend a lot of effort on low-value but time-consuming tasks rather than automate them? Here are the core concepts of prompt engineering, to me.\nSo, let's see how prompt engineering can affect Data scientists, and then let's see how we can create useful and efficient prompts.\n## How prompt engineering can affect Data Scientists\nEvery new technology has some pros and some cons, and so does prompt engineering. First, let's see the pros and then the cons.\n### Prompt Engineering for Data Scientists: pros\n  1. **Faster learning**. If you're a beginner in Data Science (and in Software Development, in general) you'll find very beneficial a tool like ChatGPT because it's like having a senior developer available 24/7. Anyway, it shouldn't be trusted as an oracle, primarily because it still makes some errors. I've written a dedicated article on how to effectively start coding in the era of ChatGPT [here](https://medium.com/towards-data-science/how-to-effectively-start-coding-in-the-era-of-chatgpt-cfc5151e1c42) if you're interested.\n  2. **Faster prototypes**. In my opinion, one of the most important parts of the job of a Data Scientist is prototyping. In fact, very often we need to give a fast answer based on data (often, with little available data. Very often, with dirty data). So, a prototype is what can give the sense of the answer a customer needs, giving us time for: a) ask/get for more data, b) ask/get for more specifications, c) clean data, and d) make necessary research.\n  3. **Faster debugging and error management**. We have to be honest: debugging and error management in software development is more a curse than a delight. This is also true when we develop software for Machine/Deep Learning algorithms. ChatGPT is a great tool for debugging and error management: with the right prompt, it can find errors and bugs in a matter of seconds, making us save a lot of time and effort. Just a quick reminder: since ChatGPT (and similar tools) work in the cloud and also they may use our prompts to train their algorithms, remember to not write code with sensible information because they can get you in trouble in cases of data breaches.\n  4. **Faster research**. An important part of the job of a Data Scientist is making research. We absolutely need to research a lot of stuff to solve our problems like: info on particular libraries and their usages, info related to the domain knowledge of the problem we're facing, etc...Well, a good prompt is generally useful to let us grab the info we need. The only thing to remember is that we always need to verify the correctness of the output by deepening it on the internet or on books. Especially with code, is always important to read the documentation: otherwise, the risk is to copy and paste the code without actually understanding it.\n\n\n### Prompt Engineering for Data Scientists: cons\n  1. **Possibility to lose your job**. Yes, we have to say it: AI tools can make us lose our job. It seems a contradiction: the necessity for data professionals in the market is increasing in these months, but tools like ChatGPT may substitute us. Well, let's say the truth: this possibility is far away at the time because AI tools need the supervision of an expert, as we've also discussed in the pros. Sure, you can ask for some code and for some data analysis, but if you don't know how to use it what do you do with the code? So, yes: prompt engineering may lead to job loss for some data professionals, but it's a matter of years not months.\n  2. **Possibility to forgot how to code**. This is an actual problem. If we rely too much on prompt engineering rather than writing code by ourselves, we can forget how to code. You know: coding is a matter of practicing and it needs everyday practice. Sure, is like going on a bicycle: you'll never forget how to do so. But, you know: relying too much on the prompts rather than on writing code can atrophy your muscles because you've become too much comfortable. So, use tools like ChatGPT but don't rely only on those: strive to write code as much as you can. Because I know you love to code, so don't leave it too much to machines.\n  3. **Possibility to not learn new things**. The beauty of working in IT, especially in the data field, is that new topics and technologies are born barely every day. This is one of the most important reasons [why I changed my career to work in IT](https://betterhumans.pub/from-gears-to-code-how-i-successfully-transitioned-from-mechanical-engineering-to-tech-44909e31cbb0): because I love to always learn new things, and I'd like that this is recognized as a good thing (yes, there are fields/companies where self-improvement is not considered a good thing). But if you rely only on the answer you get from your prompts and copy and paste the code (or the info you get) you won't learn new things. On the code sight, at least you need to ask for explanations, then read the documentation. And then, if possible, try the code in different scenarios so that you can get comfortable with the code you received. This is also the same approach we need to have when we ask for general information: always deepen them by reading other sources (as it always should be: but let's just keep this in mind!).\n  4. **Possibility to implement \"wordy code\".** The fact that a piece of code just works is not a good reason to leave it as is, as you know. Code quality is also important for those who will read that snippet in the future (maybe, you). Sometimes, due to different factors (often, to not correct prompts), tools like ChatGPT can give us \"wordy code\" with unnecessary parts. And here comes the importance of a professional: don't blindly copy and paste it. If you copy and paste it, after you verified it works, strive to improve it as much as you can.\n\n\n## Examples of effective prompts for Data Scientists\nNow, with these pros and cons in mind, let's see some effective prompts we can create in ChatGPT to improve and speed up our jobs as Data Scientists.\n### Example n°1: prototype\nSuppose we are studying a binary classification example. We want to compare three ML models using learning curves, but we don't actually remember the code to use. Also, our repositories are a little bit messy and we don't want to waste much time.\nTo speed up our job in such situations, we can ask ChatGPT a prompt like the following: \"_Create, in Python, a binary classification problem with synthetic data. Standardize the data using the standard scaler, then split the data into the train and the test set. Then, fit the train set with a random forest, a KNN, and a SVM classifiers. Then, plot ROC curve to compare the models._ \" And these are the results:\n> Certainly! I can help you create a binary classification problem using synthetic data in Python and apply the standard scaler for data standardization. Then, we can split the data into train and test sets. Finally, we'll fit the train set with three classifiers: Random Forest, K-Nearest Neighbors (KNN), and Support Vector Machine (SVM), and plot the ROC curve to compare the models.\npython\n\n```\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom sklearn.datasets import make_classification\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.preprocessing import StandardScaler\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.svm import SVC\nfrom sklearn.metrics import roc_curve, auc\n\n# Generate synthetic data for binary classification\nX, y = make_classification(n_samples=1000, n_features=20, n_informative=10,\n        n_redundant=10, random_state=42)\n\n# Standardize the data using StandardScaler\nscaler = StandardScaler()\nX = scaler.fit_transform(X)\n\n# Split the data into train and test sets\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,\n                                    random_state=42)\n\n# Fit the train set with classifiers\nrf_classifier = RandomForestClassifier(random_state=42)\nknn_classifier = KNeighborsClassifier()\nsvm_classifier = SVC(probability=True, random_state=42)\n\nrf_classifier.fit(X_train, y_train)\nknn_classifier.fit(X_train, y_train)\nsvm_classifier.fit(X_train, y_train)\n\n# Generate predictions for the test set\nrf_probs = rf_classifier.predict_proba(X_test)[:, 1]\nknn_probs = knn_classifier.predict_proba(X_test)[:, 1]\nsvm_probs = svm_classifier.predict_proba(X_test)[:, 1]\n\n# Calculate false positive rate, true positive rate,\n# and area under the curve for ROC curve\nrf_fpr, rf_tpr, _ = roc_curve(y_test, rf_probs)\nrf_auc = auc(rf_fpr, rf_tpr)\n\nknn_fpr, knn_tpr, _ = roc_curve(y_test, knn_probs)\nknn_auc = auc(knn_fpr, knn_tpr)\n\nsvm_fpr, svm_tpr, _ = roc_curve(y_test, svm_probs)\nsvm_auc = auc(svm_fpr, svm_tpr)\n\n# Plot the ROC curve\nplt.figure()\nplt.plot(rf_fpr, rf_tpr, label=f'Random Forest (AUC = {rf_auc:.2f})')\nplt.plot(knn_fpr, knn_tpr, label=f'KNN (AUC = {knn_auc:.2f})')\nplt.plot(svm_fpr, svm_tpr, label=f'SVM (AUC = {svm_auc:.2f})')\nplt.plot([0, 1], [0, 1], 'k--')\nplt.xlabel('False Positive Rate')\nplt.ylabel('True Positive Rate')\nplt.title('Receiver Operating Characteristic')\nplt.legend(loc='lower right')\nplt.show()\n```\n\n> The provided code generates a synthetic binary classification problem using scikit-learn's `make_classification` function. It then applies the `StandardScaler` to standardize the data and splits it into a train set and a test set.\n> Three classifiers are then trained on the train set: Random Forest, K-Nearest Neighbors (KNN), and Support Vector Machine (SVM). For evaluation, the classifiers generate probabilities for the test set using the `predict_proba` method.\n> The next step involves plotting the Receiver Operating Characteristic (ROC) curve to compare the performance of the classifiers. The False Positive Rate (FPR) and True Positive Rate (TPR) are calculated using the `roc_curve` function, and the Area Under the Curve (AUC) is computed using the `auc` function.\n> Finally, the ROC curve is plotted using matplotlib, with each classifier's AUC displayed in the legend. The curve represents the trade-off between the true positive rate and the false positive rate for each classifier, allowing visual comparison of their performance.\n> By examining the ROC curve and comparing the AUC values, you can assess the relative performance of the Random Forest, KNN, and SVM classifiers for the given binary classification problem.\nNow, if we copy and paste this code into a Jupyter Notebook we get:\n![The ROC curve resulting from the above code. Image by Federico Trotta.](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/06/18G6z-8pwcoQQ4RNK7WCJ-A.png)\nThe ROC curve resulting from the above code. Image by Federico Trotta.\nWe get what we wanted, so let's analyze the prompt:\n  1. **It's specific**. I specified: a) the language (Python), b) the type of problem (binary classification), c) the process to follow (data creation, data normalization, data split), and d) the ML models to use.\n  2. **It has everything needed**. There are no extra words that may create confusion. The prompt goes straight to the point.\n  3. **It's short**. It's only 3 lines. Longer prompts may lead to incorrect outputs or can lead the model to add information we may not need. If you have to insert \"long prompts\" my advice is to subdivide the \"big task\" into smaller tasks and input different short prompts that can be sequential. To be clear, the above prompt could be subdivided into sequential chats like so: a) \"_create a binary classification dataset with Python, normalize the data, and split them into the train and the test set_ \", b) \"_fit the train test with random forest, SVM, and KNN classifiers. Then, print the ROC curve_ \"\n\n\n### Example n°2: research\nSuppose we know that polynomial regression generally overfits, but we want a prove because we've never seen this. The prompt could be something like that: \" _I know that polynomial regression models generally overfit: can you show me something, creating some data in Python_?\"\nAnd here's what we get:\n> Certainly! I can generate some random data and demonstrate how polynomial regression can lead to overfitting. Here's an example in Python:\npython\n\n```\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom sklearn.linear_model import LinearRegression\nfrom sklearn.preprocessing import PolynomialFeatures\n\n# Generate random data\nnp.random.seed(0)\nX = np.linspace(-3, 3, 100)\ny = 0.5 * X**2 + X + np.random.normal(0, 1, 100)\n\n# Reshape the input data\nX = X.reshape(-1, 1)\n\n# Plot the original data\nplt.scatter(X, y, color='b', label='Data')\n\n# Fit polynomial regression models of different degrees\ndegrees = [1, 4, 15]\ncolors = ['r', 'g', 'm']\nfor degree, color in zip(degrees, colors):\n    # Create polynomial features\n    polynomial_features = PolynomialFeatures(degree=degree)\n    X_poly = polynomial_features.fit_transform(X)\n\n    # Fit the polynomial regression model\n    model = LinearRegression()\n    model.fit(X_poly, y)\n\n    # Predict the values\n    y_pred = model.predict(X_poly)\n\n    # Plot the fitted curve\n    plt.plot(X, y_pred, color=color, linewidth=2, label=f'Degree {degree}')\n\n# Add labels and title to the plot\nplt.xlabel('X')\nplt.ylabel('y')\nplt.title('Polynomial Regression - Overfitting Example')\nplt.legend(loc='upper left')\n\n# Display the plot\nplt.show()\n```\n\n> This code generates random data points with a quadratic relationship and adds some noise to the y values. It then fits polynomial regression models of different degrees (1, 4, and 15 in this example) to the data and plots the resulting curves.\n> You will observe that as the degree of the polynomial increases, the model starts to fit the noise in the data rather than the underlying trend. This is an example of overfitting. The linear model (degree 1) is too simple to capture the underlying quadratic relationship, while the high-degree polynomial (degree 15) tries to fit every data point precisely, resulting in a highly oscillating curve.\n> It's important to note that the example I provided is a simple illustration of overfitting in polynomial regression. In practice, you would typically use more realistic and complex datasets to evaluate and address overfitting issues.\nNow, let's see what we get in a Jupyter Notebook:\n![A plot showing the difference in fitting the data created with the above code. Image by Federico Trotta.](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/06/1y6zPtlKHaVOlCMzTW0lxsw.png)\nA plot showing the difference in fitting the data created with the above code. Image by Federico Trotta.\nSo, this plot shows that, given some quadratic distributed data:\n  * A linear model can't grab the path.\n  * The 4-degree and the 15-degree polynomials \"grab the path too well\".\n\n\nNow, tell me: does it means that the two polynomials overfit?\nStop reading for a second a think about that.\nWell, yes and no, but primarily no. In fact, as we know, overfitting occurs when:\n  1. We define metrics to evaluate our data.\n  2. The metrics are calculated both on the train and test sets and we find big numerical differences between them.\n\n\nSo, this prompt is not a satisfying one and we have to ask for more. For example, we could ask: \"_Good, so use the same data and the same polynomials. Then, normalize the data, split the dataset into train and test sets, and calculate R² for both sets_ \".\nAnd we get:\n> Certainly! Here's an updated version of the code that normalizes the data, splits it into train and test sets, and calculates the R² score for both sets:\npython\n\n```\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom sklearn.linear_model import LinearRegression\nfrom sklearn.preprocessing import PolynomialFeatures\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import r2_score\n\n# Generate random data\nnp.random.seed(0)\nX = np.linspace(-3, 3, 100)\ny = 0.5 * X**2 + X + np.random.normal(0, 1, 100)\n\n# Reshape the input data\nX = X.reshape(-1, 1)\n\n# Normalize the data\nX_normalized = (X - np.mean(X)) / np.std(X)\ny_normalized = (y - np.mean(y)) / np.std(y)\n\n# Split the data into train and test sets\nX_train, X_test, y_train, y_test = train_test_split(\n    X_normalized, y_normalized, test_size=0.2, random_state=0\n)\n\n# Plot the original data\nplt.scatter(X_normalized, y_normalized, color='b', label='Data')\n\n# Fit polynomial regression models of different degrees\ndegrees = [1, 4, 15]\ncolors = ['r', 'g', 'm']\nfor degree, color in zip(degrees, colors):\n    # Create polynomial features\n    polynomial_features = PolynomialFeatures(degree=degree)\n    X_poly_train = polynomial_features.fit_transform(X_train)\n    X_poly_test = polynomial_features.transform(X_test)\n\n    # Fit the polynomial regression model\n    model = LinearRegression()\n    model.fit(X_poly_train, y_train)\n\n    # Predict the values for train and test sets\n    y_pred_train = model.predict(X_poly_train)\n    y_pred_test = model.predict(X_poly_test)\n\n    # Calculate R² scores\n    r2_train = r2_score(y_train, y_pred_train)\n    r2_test = r2_score(y_test, y_pred_test)\n\n    # Plot the fitted curve\n    plt.plot(X_normalized, model.predict(polynomial_features.transform(X_normalized)), color=color, linewidth=2,\n             label=f'Degree {degree} (Train R²={r2_train:.2f}, Test R²={r2_test:.2f})')\n\n# Add labels and title to the plot\nplt.xlabel('X')\nplt.ylabel('y')\nplt.title('Polynomial Regression - Overfitting Example (Normalized)')\nplt.legend(loc='upper left')\n\n# Display the plot\nplt.show()\n```\n\n> In this updated code, the data is first normalized by subtracting the mean and dividing by the standard deviation. Then, the data is split into a train set (80% of the data) and a test set (20% of the data) using `train_test_split()` from scikit-learn.\n> The polynomial regression models are fitted using the normalized training data, and the R² scores are calculated for both the train and test sets using `r2_score()` from scikit-learn. The R² scores are then displayed in the legend of each fitted curve.\n> By normalizing the data and calculating R² scores on both the train and test sets, you can evaluate the performance of the polynomial regression models and assess whether they overfit or generalize well to unseen data.\nIn this case, we get:\n![A plot showing the difference in fitting the data created with the above code. Image by Federico Trotta.](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/06/1P-PL9gznZvRH7QXUUPY9qQ.png)\nA plot showing the difference in fitting the data created with the above code. Image by Federico Trotta.\nSo, what did we get? Well...We can't see great signs of overfitting because R² slightly increases for the 4-degree polynomial on the test set while it slightly decreases for the 15-degree polynomial.\nHere the mistake is that we've continued the chat we started. Let's remember that LLM are neural networks and they have the tendency to overfit. So, sometimes, if you just continue prompting you get the same answer (or something very similar) even if it returns you some \"apologies for the mistake\".\nAlso, I don't want any plot. So, in this case, we'd better create another chat. Let's try to be more specific now:\" _Create a 2-degree data set in Python. Normalize the data, and split it into the train and the test set. Fit the data with: linear regression, 4-degree polynomial, and 15-degree polynomial models. Print r-squared for each model for both train and test sets._ \"\nAnd we get:\n> To create a 2-degrees dataset in Python, normalize it, and split it into train and test sets, you can use the `numpy` and `sklearn` libraries. Here's an example of how you can achieve this:\npython\n\n```\nimport numpy as np\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.preprocessing import PolynomialFeatures\nfrom sklearn.linear_model import LinearRegression\nfrom sklearn.metrics import r2_score\n\n# Create the dataset\nX = np.random.rand(100, 1)  # Independent variable\ny = 3 * X**2 + np.random.randn(100, 1)  # Dependent variable (quadratic relationship)\n\n# Normalize the data\nX_normalized = (X - np.mean(X)) / np.std(X)\ny_normalized = (y - np.mean(y)) / np.std(y)\n\n# Split the data into train and test sets\nX_train, X_test, y_train, y_test = train_test_split(X_normalized, y_normalized, test_size=0.2, random_state=42)\n\n# Fit the linear regression model\nlinear_regression = LinearRegression()\nlinear_regression.fit(X_train, y_train)\n\n# Fit the 4-degree polynomial regression model\npoly_features = PolynomialFeatures(degree=4)\nX_poly_train = poly_features.fit_transform(X_train)\nX_poly_test = poly_features.transform(X_test)\npoly_regression = LinearRegression()\npoly_regression.fit(X_poly_train, y_train)\n\n# Fit the 15-degree polynomial regression model\npoly_features = PolynomialFeatures(degree=15)\nX_poly_train = poly_features.fit_transform(X_train)\nX_poly_test = poly_features.transform(X_test)\npoly_regression_15 = LinearRegression()\npoly_regression_15.fit(X_poly_train, y_train)\n\n# Calculate R-squared for train and test sets\nlinear_train_r2 = linear_regression.score(X_train, y_train)\nlinear_test_r2 = linear_regression.score(X_test, y_test)\n\npoly_train_r2 = poly_regression.score(X_poly_train, y_train)\npoly_test_r2 = poly_regression.score(X_poly_test, y_test)\n\npoly_15_train_r2 = poly_regression_15.score(X_poly_train, y_train)\npoly_15_test_r2 = poly_regression_15.score(X_poly_test, y_test)\n\n# Print the results\nprint(\"Linear Regression R-squared (Train):\", linear_train_r2)\nprint(\"Linear Regression R-squared (Test):\", linear_test_r2)\n\nprint(\"4-Degree Polynomial Regression R-squared (Train):\", poly_train_r2)\nprint(\"4-Degree Polynomial Regression R-squared (Test):\", poly_test_r2)\n\nprint(\"15-Degree Polynomial Regression R-squared (Train):\", poly_15_train_r2)\nprint(\"15-Degree Polynomial Regression R-squared (Test):\", poly_15_test_r2)\n```\n\n> In this example, we generate a random dataset with a quadratic relationship between the independent variable `X` and the dependent variable `y`. The data is then normalized using z-score normalization. Next, the dataset is split into a train set (80%) and a test set (20%) using `train_test_split` from the `sklearn.model_selection` module.\n> We then fit three different models: linear regression, 4-degree polynomial regression, and 15-degree polynomial regression. For polynomial regression, we use the `PolynomialFeatures` class from `sklearn.preprocessing` to create polynomial features up to the desired degree. The models are fitted using the normalized training data.\n> Finally, we calculate the R-squared value for both the train and test sets for each model using the `score` method of the respective models. The R-squared value represents the proportion of the variance in the dependent variable that can be explained by\nGood, so we get what we wanted and with some explanations. Now, if we try it we get:\ntext\n\n```\nValueError: X has 16 features, but LinearRegression\nis expecting 5 features as input.\n```\n\nOps! As we said, it can happen to get some errors. So, in these cases, we can simply continue the chat like so: \"_I get: ValueError: X has 16 features, but LinearRegression is expecting 5 features as input._ \"\nAnd now it returns:\n> Apologies for the error. It seems that there was an issue with the polynomial features transformation. To fix the problem, we need to ensure that we apply the polynomial transformation on the normalized `X` data, rather than the original `X` data. Here's the corrected code:\npython\n\n```\nimport numpy as np\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.preprocessing import PolynomialFeatures\nfrom sklearn.linear_model import LinearRegression\nfrom sklearn.metrics import r2_score\n\n# Create the dataset\nX = np.random.rand(100, 1)  # Independent variable\ny = 3 * X**2 + np.random.randn(100, 1)  # Dependent variable (quadratic relationship)\n\n# Normalize the data\nX_normalized = (X - np.mean(X)) / np.std(X)\ny_normalized = (y - np.mean(y)) / np.std(y)\n\n# Split the data into train and test sets\nX_train, X_test, y_train, y_test = train_test_split(X_normalized, y_normalized, test_size=0.2, random_state=42)\n\n# Fit the linear regression model\nlinear_regression = LinearRegression()\nlinear_regression.fit(X_train, y_train)\n\n# Fit the 4-degree polynomial regression model\npoly_features = PolynomialFeatures(degree=4)\nX_poly_train = poly_features.fit_transform(X_train)\nX_poly_test = poly_features.transform(X_test)\npoly_regression = LinearRegression()\npoly_regression.fit(X_poly_train, y_train)\n\n# Fit the 15-degree polynomial regression model\npoly_features_15 = PolynomialFeatures(degree=15)\nX_poly_train_15 = poly_features_15.fit_transform(X_train)\nX_poly_test_15 = poly_features_15.transform(X_test)\npoly_regression_15 = LinearRegression()\npoly_regression_15.fit(X_poly_train_15, y_train)\n\n# Calculate R-squared for train and test sets\nlinear_train_r2 = linear_regression.score(X_train, y_train)\nlinear_test_r2 = linear_regression.score(X_test, y_test)\n\npoly_train_r2 = poly_regression.score(X_poly_train, y_train)\npoly_test_r2 = poly_regression.score(X_poly_test, y_test)\n\npoly_15_train_r2 = poly_regression_15.score(X_poly_train_15, y_train)\npoly_15_test_r2 = poly_regression_15.score(X_poly_test_15, y_test)\n\n# Print the results\nprint(\"Linear Regression R-squared (Train):\", linear_train_r2)\nprint(\"Linear Regression R-squared (Test):\", linear_test_r2)\n\nprint(\"4-Degree Polynomial Regression R-squared (Train):\", poly_train_r2)\nprint(\"4-Degree Polynomial Regression R-squared (Test):\", poly_test_r2)\n\nprint(\"15-Degree Polynomial Regression R-squared (Train):\", poly_15_train_r2)\nprint(\"15-Degree Polynomial Regression R-squared (Test):\", poly_15_test_r2)\n```\n\n> Now the code should work correctly, applying the polynomial transformation on the normalized data and fitting the models accordingly.\nNow, if we test it we get:\ntext\n\n```\nLinear Regression R-squared (Train): 0.5287297254229243\nLinear Regression R-squared (Test): 0.4627146369316367\n\n4-Degree Polynomial Regression R-squared (Train): 0.5895312356444804\n4-Degree Polynomial Regression R-squared (Test): 0.4601670183046247\n\n15-Degree Polynomial Regression R-squared (Train): 0.6256705123172766\n15-Degree Polynomial Regression R-squared (Test): 0.26972190687190367\n```\n\nSo, here we can see some slight overfitting of the 4-degree model and a clear overfitting of the 15-degree model. In fact, the higher the degree of the polynomial the higher the probability it has to overfit the data. Let's think about that: we have a 2-degree data distribution and we're trying to fit it with a 15-degree polynomial. So, it's easy that the 15-degree polynomial can grab every small modification in the data, leading to overfitting.\n* * *\n### Conclusions\nIn this article, we've seen the importance of prompt engineering today, how it can affect Data Scientists, and some prompt examples for Data Scientists.\nRemember that a prompt should be short and clear and that we always have to verify the answers. But, as we've shown, the benefits are huge.\n* * *\n**FREE PYTHON EBOOK:**\nStarted learning Python Data Science but struggling with it? [_**Subscribe to my newsletter and get my free ebook: this will give you the right learning path to follow to learn Python for Data Science with hands-on experience.**_](https://federico-trotta.ck.page/a3970f33f4)\nEnjoyed the story? Become a Medium member for 5$/month [through my referral link](https://medium.com/@federicotrotta/membership): I'll earn a small commission to no additional fee to you:\n> [**Join Medium with my referral link - Federico Trotta**](https://federicotrotta.medium.com/membership)\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](https://assets.insightmediagroup.io/media/1787657183787_vf5bj8.png)\nWritten By\n## [Federico Trotta](https://towardsdatascience.com/author/federicotrotta/)\n[See all from Federico Trotta](https://towardsdatascience.com/author/federicotrotta/)\n[Artificial Intelligence](https://towardsdatascience.com/tag/artificial-intelligence/)[Data Science](https://towardsdatascience.com/tag/data-science/)[Federico Trotta](https://towardsdatascience.com/tag/federico-trotta/)[Machine Learning](https://towardsdatascience.com/tag/machine-learning/)[Technology](https://towardsdatascience.com/tag/technology/)\nShare This Article\n  * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Funleashing-the-power-of-prompt-engineering-for-data-scientists-16b6d1f2bf85%2F&title=Unleashing%20the%20Power%20of%20Prompt%20Engineering%20for%20Data%20Scientists)\n  * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Funleashing-the-power-of-prompt-engineering-for-data-scientists-16b6d1f2bf85%2F&title=Unleashing%20the%20Power%20of%20Prompt%20Engineering%20for%20Data%20Scientists)\n  * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Funleashing-the-power-of-prompt-engineering-for-data-scientists-16b6d1f2bf85%2F&text=Unleashing%20the%20Power%20of%20Prompt%20Engineering%20for%20Data%20Scientists)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/0xJRPATLYmdtYpIkz-scaled.jpg)](https://towardsdatascience.com/ternary-operators-in-python-49c685183c50/)\n## [Ternary Operators in Python](https://towardsdatascience.com/ternary-operators-in-python-49c685183c50/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nImprove your Python code with ternary operators!\n[Luay Matalka](https://towardsdatascience.com/author/lmatalka90/)\nFebruary 12, 20213 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1En6dg0dOK7Nwst9Mdp-VPw.jpeg)](https://towardsdatascience.com/statistically-confirm-your-benchmark-case-study-comparing-pandas-and-polars-with-1-million-rows-0ea04d7b61f2/)\n## [Statistically Confirm Your -Comparing Pandas and Polars with 1 Million Rows of Data](https://towardsdatascience.com/statistically-confirm-your-benchmark-case-study-comparing-pandas-and-polars-with-1-million-rows-0ea04d7b61f2/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nUsing the Independent samples t-test and Welch's t-test to compare scores in benchmarking.\n[Boriharn K](https://towardsdatascience.com/author/borih.k/)\nJune 18, 202413 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/09/javier-allegue-barros-C7B-ExXpOIE-unsplash-scaled-1.jpg)](https://towardsdatascience.com/the-art-of-asking-good-questions/)\n## [The Art of Asking Good Questions](https://towardsdatascience.com/the-art-of-asking-good-questions/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nAs a data scientist, are you driving product decisions? Or just supporting them? The right questions can turn AI from a threat into your career’s best ally. Here’s how to start asking them.\n[Greg Rafferty](https://towardsdatascience.com/author/raffg/)\nSeptember 23, 20256 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2018/07/18LbEDR_t-E2TuwZZqrLFJw.gif)](https://towardsdatascience.com/better-internal-audits-with-artificial-intelligence-53b6a2ec7878/)\n## [Better Internal Audits with Artificial Intelligence](https://towardsdatascience.com/better-internal-audits-with-artificial-intelligence-53b6a2ec7878/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nAuditing internal reports is a very manual process. We saw an opportunity to use artificial intelligence to improve that process. This...\n[Daniel Shapiro, PhD](https://towardsdatascience.com/author/lemaysolutions/)\nJuly 20, 20189 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/09/0bitzulmdqGJoaiL--scaled.jpg)](https://towardsdatascience.com/tips-tricks-and-tools-of-the-trade-7-popular-posts-you-should-read-6853d9fefa09/)\n## [Tips, Tricks, and Tools of the Trade: 7 Popular Posts You Should Read](https://towardsdatascience.com/tips-tricks-and-tools-of-the-trade-7-popular-posts-you-should-read-6853d9fefa09/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nSome of our most-read recent articles focus on making your data science journey easier and smoother.\n[TDS Editors](https://towardsdatascience.com/author/towardsdatascience/)\nSeptember 15, 20213 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/01/1zIo1BYl3MhPf0ss5vFdhTw-scaled.jpeg)](https://towardsdatascience.com/python-memory-and-objects-e7bec4a2845/)\n## [Python, Memory, and Objects](https://towardsdatascience.com/python-memory-and-objects-e7bec4a2845/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nThe basics of memory management for data scientists\n[Naser Tamimi](https://towardsdatascience.com/author/tamimi-naser/)\nJanuary 4, 20218 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/07/1YhkSrhbZGj-x2OhKPHspKA.jpeg)](https://towardsdatascience.com/why-we-should-always-consider-regularization-3767062709e8/)\n## [Why Regularization Works](https://towardsdatascience.com/why-we-should-always-consider-regularization-3767062709e8/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nThe intuition behind the Regularization\n[Suyash Gupta](https://towardsdatascience.com/author/suyag10/)\nJuly 27, 20203 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/12/0cEmk0ipm-DAVSaq0-scaled.jpg)](https://towardsdatascience.com/a-simple-guide-to-command-line-arguments-with-argparse-6824c30ab1c3/)\n## [A Simple Guide To Command Line Arguments With ArgParse](https://towardsdatascience.com/a-simple-guide-to-command-line-arguments-with-argparse-6824c30ab1c3/)\n[Programming](https://towardsdatascience.com/category/programming/)\nYour go-to guide to get argparse quickly up and running\n[Sam Starkman](https://towardsdatascience.com/author/sam-starkman/)\nDecember 3, 20208 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1Ca5JIV121PXJal8fJmfLmw.png)](https://towardsdatascience.com/customizing-pandas-profiling-summaries-b16714d0dac9/)\n## [Customizing Pandas-Profiling Summaries](https://towardsdatascience.com/customizing-pandas-profiling-summaries-b16714d0dac9/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nIf you've previously used pandas-profiling, you might have observed that column summaries are unique to the data types of each feature in...\n[Ian Eaves](https://towardsdatascience.com/author/ianeaves/)\nMarch 5, 20214 min read\n\n\n  * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n  * [X](https://x.com/TDataScience)\n  * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n  * [Threads](https://www.threads.net/@towardsdatascience)\n  * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n####  Recommended Articles \nClose\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607631/image?etag=1781543525510&width=500&height=500) Critical Mistakes Companies Make When Integrating AI/ML into Their Processes ](https://towardsdatascience.com/critical-mistakes-companies-make-when-integrating-ai-ml-into-their-processes/ \"Critical Mistakes Companies Make When Integrating AI/ML into Their Processes\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500) How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ \"How Relevance Models Foreshadowed Transformers for NLP\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n\n\n",
  "metadata": {
    "title": "Unleashing the Power of Prompt Engineering for Data Scientists | Towards Data Science",
    "description": "How and why write effective prompts if you work with data",
    "keywords": null,
    "author": "Federico Trotta",
    "og:locale": "en_US",
    "og:type": "article",
    "og:title": "Unleashing the Power of Prompt Engineering for Data Scientists",
    "og:description": "How and why write effective prompts if you work with data",
    "og:url": "https://towardsdatascience.com/unleashing-the-power-of-prompt-engineering-for-data-scientists-16b6d1f2bf85/",
    "og:site_name": "Towards Data Science",
    "og:image": "https://assets.insightmediagroup.io/media/wp-content/uploads/2023/06/1YVJFwLNW6MZaDSgS-w4vtw.jpeg",
    "og:image:width": "1280",
    "og:image:height": "853",
    "og:image:type": "image/jpeg",
    "twitter:card": "summary_large_image",
    "twitter:site": "@TDataScience",
    "twitter:creator": "@TDataScience",
    "twitter:label1": "Written by",
    "twitter:data1": "Federico Trotta",
    "twitter:label2": "Est. reading time",
    "twitter:data2": "19 min read",
    "article:published_time": "2023-06-07T14:00:11.000Z",
    "article:modified_time": "2023-06-07T14:00:11.000Z",
    "article:author": "https://towardsdatascience.com/author/federicotrotta/"
  },
  "paywall": {
    "listed_paid": true,
    "detected": false,
    "marker": "",
    "public_only": true,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "language": "txt",
      "file": "attachments/accb21218fdf/code/block_000.txt",
      "chars": 163
    },
    {
      "language": "txt",
      "file": "attachments/accb21218fdf/code/block_001.txt",
      "chars": 2077
    },
    {
      "language": "txt",
      "file": "attachments/accb21218fdf/code/block_002.txt",
      "chars": 1128
    },
    {
      "language": "txt",
      "file": "attachments/accb21218fdf/code/block_003.txt",
      "chars": 1927
    },
    {
      "language": "txt",
      "file": "attachments/accb21218fdf/code/block_004.txt",
      "chars": 2163
    },
    {
      "language": "txt",
      "file": "attachments/accb21218fdf/code/block_005.txt",
      "chars": 86
    },
    {
      "language": "txt",
      "file": "attachments/accb21218fdf/code/block_006.txt",
      "chars": 2187
    },
    {
      "language": "txt",
      "file": "attachments/accb21218fdf/code/block_007.txt",
      "chars": 390
    }
  ],
  "images": [
    {
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2023/06/1YVJFwLNW6MZaDSgS-w4vtw.jpeg",
      "alt": "Image by Gerd Altmann on Pixabay",
      "local_path": "attachments/accb21218fdf/images/000.jpeg",
      "bytes": 101326,
      "downloaded": true
    },
    {
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2023/06/18G6z-8pwcoQQ4RNK7WCJ-A.png",
      "alt": "The ROC curve resulting from the above code. Image by Federico Trotta.",
      "local_path": "attachments/accb21218fdf/images/001.png",
      "bytes": 36687,
      "downloaded": true
    },
    {
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2023/06/1y6zPtlKHaVOlCMzTW0lxsw.png",
      "alt": "A plot showing the difference in fitting the data created with the above code. Image by Federico Trotta.",
      "local_path": "attachments/accb21218fdf/images/002.png",
      "bytes": 52628,
      "downloaded": true
    },
    {
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2023/06/1P-PL9gznZvRH7QXUUPY9qQ.png",
      "alt": "A plot showing the difference in fitting the data created with the above code. Image by Federico Trotta.",
      "local_path": "attachments/accb21218fdf/images/003.png",
      "bytes": 62605,
      "downloaded": true
    },
    {
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/0xJRPATLYmdtYpIkz-scaled.jpg",
      "alt": "",
      "local_path": "attachments/accb21218fdf/images/004.jpg",
      "bytes": 306040,
      "downloaded": true
    },
    {
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1En6dg0dOK7Nwst9Mdp-VPw.jpeg",
      "alt": "",
      "local_path": "attachments/accb21218fdf/images/005.jpeg",
      "bytes": 295250,
      "downloaded": true
    },
    {
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/09/javier-allegue-barros-C7B-ExXpOIE-unsplash-scaled-1.jpg",
      "alt": "",
      "local_path": "attachments/accb21218fdf/images/006.jpg",
      "bytes": 142967,
      "downloaded": true
    },
    {
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2018/07/18LbEDR_t-E2TuwZZqrLFJw.gif",
      "alt": "",
      "local_path": "attachments/accb21218fdf/images/007.gif",
      "bytes": 70175,
      "downloaded": true
    },
    {
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/09/0bitzulmdqGJoaiL--scaled.jpg",
      "alt": "",
      "local_path": "attachments/accb21218fdf/images/008.jpg",
      "bytes": 323064,
      "downloaded": true
    },
    {
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/01/1zIo1BYl3MhPf0ss5vFdhTw-scaled.jpeg",
      "alt": "",
      "local_path": "attachments/accb21218fdf/images/009.jpeg",
      "bytes": 130939,
      "downloaded": true
    },
    {
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2020/07/1YhkSrhbZGj-x2OhKPHspKA.jpeg",
      "alt": "",
      "local_path": "attachments/accb21218fdf/images/010.jpeg",
      "bytes": 222530,
      "downloaded": true
    },
    {
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2020/12/0cEmk0ipm-DAVSaq0-scaled.jpg",
      "alt": "",
      "local_path": "attachments/accb21218fdf/images/011.jpg",
      "bytes": 226684,
      "downloaded": true
    },
    {
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1Ca5JIV121PXJal8fJmfLmw.png",
      "alt": "",
      "local_path": "attachments/accb21218fdf/images/012.png",
      "bytes": 35227,
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [
    {
      "src": "https://www.youtube.com/c/TowardsDataScience",
      "quality": "unavailable",
      "local_path": "",
      "note": "无1080p且无720p，未交付更低清晰度"
    }
  ],
  "links": [
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/latest/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/editors-pick/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/the-variable/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/shipai/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/submissions/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/category/artificial-intelligence/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/author/federicotrotta/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/artificial-intelligence/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/data-science/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/federico-trotta/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/machine-learning/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/technology/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/ternary-operators-in-python-49c685183c50/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/author/lmatalka90/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/statistically-confirm-your-benchmark-case-study-comparing-pandas-and-polars-with-1-million-rows-0ea04d7b61f2/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/category/data-science/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/author/borih.k/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/the-art-of-asking-good-questions/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/author/raffg/"
    },
    {
      "type": "external",
      "href": "https://write.insightmediagroup.io"
    },
    {
      "type": "external",
      "href": "https://www.linkedin.com/company/towards-data-science/"
    },
    {
      "type": "external",
      "href": "https://x.com/TDataScience"
    },
    {
      "type": "external",
      "href": "https://pixabay.com/it/users/geralt-9301/"
    },
    {
      "type": "external",
      "href": "https://pixabay.com/it//"
    },
    {
      "type": "external",
      "href": "https://roadmap.sh/ai-engineer"
    },
    {
      "type": "external",
      "href": "https://roadmap.sh/ai-data-scientist"
    },
    {
      "type": "external",
      "href": "https://roadmap.sh/computer-science"
    },
    {
      "type": "external",
      "href": "https://roadmap.sh/machine-learning"
    },
    {
      "type": "external",
      "href": "https://medium.com/towards-data-science/how-to-effectively-start-coding-in-the-era-of-chatgpt-cfc5151e1c42"
    },
    {
      "type": "external",
      "href": "https://betterhumans.pub/from-gears-to-code-how-i-successfully-transitioned-from-mechanical-engineering-to-tech-44909e31cbb0"
    },
    {
      "type": "external",
      "href": "https://federico-trotta.ck.page/a3970f33f4"
    },
    {
      "type": "external",
      "href": "https://medium.com/@federicotrotta/membership"
    },
    {
      "type": "external",
      "href": "https://federicotrotta.medium.com/membership"
    },
    {
      "type": "external",
      "href": "https://www.facebook.com/sharer/sharer.php?title=Unleashing+the+Power+of+Prompt+Engineering+for+Data+Scientists&u=https%3A%2F%2Ftowardsdatascience.com%2Funleashing-the-power-of-prompt-engineering-for-data-scientists-16b6d1f2bf85%2F"
    },
    {
      "type": "external",
      "href": "https://www.linkedin.com/shareArticle?mini=true&title=Unleashing+the+Power+of+Prompt+Engineering+for+Data+Scientists&url=https%3A%2F%2Ftowardsdatascience.com%2Funleashing-the-power-of-prompt-engineering-for-data-scientists-16b6d1f2bf85%2F"
    },
    {
      "type": "external",
      "href": "https://x.com/share?text=Unleashing+the+Power+of+Prompt+Engineering+for+Data+Scientists&url=https%3A%2F%2Ftowardsdatascience.com%2Funleashing-the-power-of-prompt-engineering-for-data-scientists-16b6d1f2bf85%2F"
    },
    {
      "type": "external",
      "href": "https://www.youtube.com/c/TowardsDataScience"
    },
    {
      "type": "external",
      "href": "https://www.threads.net/@towardsdatascience"
    },
    {
      "type": "external",
      "href": "https://bsky.app/profile/towardsdatascience.com"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T11:11:49Z",
  "source_url": "https://towardsdatascience.com/unleashing-the-power-of-prompt-engineering-for-data-scientists-16b6d1f2bf85/",
  "delivery_status": "excluded_paid",
  "discovery_status": "excluded_paid",
  "missing_reason": "按原约定排除付费/会员全文"
}