保留正文与补采附件。源页面当前未重新核验；具体限制见交付清单。

Revisit consent button We value your privacy We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies. Customise Reject All Accept All Customise Consent Preferences We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below. The cookies that are categorised as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more NecessaryAlways Active Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

Functional Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.

Analytics Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

Performance Performance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.

Advertisement Advertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.

Uncategorised Other uncategorised cookies are those that are being analysed and have not been classified into a category as yet. No cookies to display. Reject All Save My Preferences Accept All Toggle SearchToggle Mobile Navigation Publish AI, ML & data-science insights to a global community of data professionals. Sign inSubmit an Article LatestEditor's PicksNewsletterShipAI

Write For TDS LinkedInXToggle Search Artificial Intelligence

Generative AI Myths, Busted: An Engineer’s Quick Guide

A super simple and quick guide to how generative AI works, the myths around it, and why it won’t replace engineers anytime soon. Amy Ma September 23, 202510 min read I wrote this article initially for my team, as a number of our engineers felt uneasy utilizing AI in their daily job. They asked some large and reasonable questions: Where do these models originate? Any concern on data breaches? And the hardest one (took me a bit of time to figure it out myself) - am I making AI replace me by using it? As a team member who is actually quite passionate about AI, I get those worries. That's why I've put together this short, handy manual:to debunk some myths and provide a helpful playbook for engineers to utilize AI. To me, success for AI isn't measured by how many fancy tools it can spit out. That’s not the point. The point is: does it actually make us more productive? How effectively does it make humans work smarter and even lazier? And since engineers are among the most important consumers of this technology, I believe it's worth clearing things up.

So ... What is Generative AI, and What is an LLM?

When people discuss tools like ChatGPT, Claude, or Amazon Q, what they’re really referring to is a language model. This is the core technology that powers much of generative AI. It’s trained to understand written language and respond by generating new text. When these models are scaled up to include billions, or even trillions, of parameters, they become what we call Large Language Models , or LLMs. LLMs are just one kind of generative AI, specifically designed to work with text and code. Let’s start with the basics. Learn this step by step with the interactive AI Engineer roadmap.

Tokens, not just words

In language models, the smallest unit of understanding isn’t always a full word, it’s a token. Think of tokens like LEGO bricks: small pieces that come together to build bigger structures. A token might be a complete word (like “castle”), part of a word (“cast”), or even just a single letter or character. For example:

The sentence “I love programming” might be split into tokens:[I] [love] [pro] [gramming]. This process is called tokenization. It’s how a model learns those materials to build its vocabulary. These vocabularies can be huge: GPT-4’s has around 100,000 tokens. Claude’s is even larger.

How Language Models Learn (differently)

There are two main learning approaches for language models:

  1. Masked models : These models learn by hiding a word and trying to predict what should go in the blank..

“Tony Stark is also known as ___ Man.”
The model learns to fill in the blank. It figures out that “Tony Stark” provides the strongest clue, while “is” and “also” don’t a add much value.

  1. Autoregressive models : These models predict the next word , one word at a time.

Start with: “Elsa walked into the castle…”
The model continues: “…and the doors slammed shut behind her.” In this way it builds the story, word by word. Both of these approaches give the model a superpower: the ability to generate text. A model that can generate open-ended outputs is called generative, hence the term generative AI.

What Does “Open-Ended Output” Mean?

Back to our Marvel example:

“Tony Stark is also known as ___ Man.” Most people (and most models) would say “Iron.” But a model could also come up with “Powerful,” “Funny,” or even “Family.” That’s open-ended output: the model isn’t tied to just one “correct” answer. It makes predictions based on probabilities. Sometimes it nails it. Other times? Not so much. This is why AI can feel both magical and unpredictable.

The big leap: from LM to LLM

Scaling up turns a language model into a large language model. It is similar to studying the whole dictionary to score high on the GRE. Given more data and more parameters, the model can capture more nuanced patterns. Parameters are similar to knobs on a soundboard — the more knobs you have, the more complicated the music gets. GPT-4, for example, has been estimated to have 1.76 trillion parameters. Obviously, that requires enormous quantities of data. And we can't label data forever. The thing that changed everything was self-supervised learning. Instead of relying on humans to label every sentence, the model is able to learn by itself by covering up words and predicting them back out. It's like taking fill-in-the-blanks questions at superhuman speed. And that's why LLMs shine with code: programming languages are a trainer's ideal domain. Code is highly formal, syntactically rigorous, and avoids the messy vagaries of natural language. The same action in Python will behave in the same manner each and every time, as opposed to the English sentence, which can hold multiple different meanings depending upon tone, culture, or situation. That precision is what makes the model not need trillions of lines of code in order to "get it." Even with smaller data sets of code, the patterns are consistent enough that the model can generalize well and produce surprisingly robust results.

Not only LLMs

Let it be noted before we go further that an LLM is just one type of generative AI. It just happens that it's the most common and most widely used today, but not the whole story. Current models are typicallyLMMs (Large Multimodal Models). They don't just work with text. They also comprehend and generate through images, audio, or even video. That's why you see models that can read a block of code, describe an image, and then translate it to plain English, in one go. In the past several years, there has also been another term in the conversation:foundation models. They are very large models trained on general-purpose data with broad scope. Think of them as the "ground floor" or "vanilla flavor in ice cream world". Once trained, they can subsequently be fine-tuned or stretched to apply to more specialized tasks , like creating software documentation, powering a company chatbot, or reviewing contracts. "Foundation" is the term used to describe them in opposition to the smaller, specialty models developed above them. ···

Common Myths (If you don’t care about definitions and details)

Myth 1: “AI understands like we do.”

Fact: AI doesn’t “understand” in the human sense, it predicts (with probabilities). With previous text (the context) it makes an educated guess about what comes next. Good predictions can simulate understanding, but under the hood probabilities all the way down. Nevertheless, some experiments are intriguing. One blog showed how different models seem to play “personalities.” OpenAI’s O3 turned out to be a master of deception in a diplomacy game, while Claude Opus 4 just wanted everyone to get along.

Myth 2: “AI just copies training data.”

Fact: It doesn't store training examples about like a clipboard. It stores patterns into weights (numerical dials of the model) that it tweaks during training. When you query it, the model doesn't copy-paste but instead creates new sequences. That being said, data privacy remains an issue. If private data was used for training, it can still pop out indirectly, or be read by humans during reinforcement.

Myth 3: “Prompts are hacks.”

Fact: Prompts aren't hacks, they're steering wheels. A prompt sets up context so the model knows which tokens to weight more heavily. If you still remember our previous example:

“Tony Stark is also known as ___ Man.” Here, "Tony Stark" carries the greatest weight, and capitalized "Man" helps the model to zero in on "Iron." The filler words ("is also known as") weigh relatively less. An effective prompt does the same: it highlights the important cues so the model can reach a correct inference. That is no hack but a guidance.

Myth 4: “AI always gives the same answer.”

Fact: Nope. Generative AI relies on probabilities, not preprogrammed scripts. That means that the same question will have different answers depending on randomness, temperature settings, or even slight difference in your prompt. Every now and then that's fantastic (brainstorming), and sometimes it's confusing and frustrating (inconsistent answers).

Myth 5: “Bigger models are always better.”

Fact: Size helps, but it’s not everything. That's to say, big is not always better. An enormous model such as GPT-4 may be amazing, but a smaller specialized model may outdo it on a specific task. Agility and specialization can sometimes beat force..

Myth 6: “AI can replace engineers.”

Fact: Generative AI is a fantastic programming assistant, thanks to its disposition towards systematic, rule-governed programming languages. But it still doesn't qualify it as a lead engineer. Rather, it's akin to a speedy intern, who are wonderful at creating rough drafts, boilerplate, and even great snippets, but in need of review and oversight. In short, Generative AI will not replace engineers, but it will change what engineering is. The skill set will be different. Instead of spending most of their time typing out every line of code, engineers are able to focus more on system architecture, design, and project oversight. Generative AI can write functions but cannot weigh trade-offs, understand business goals, or manage complexity at scale. That's where there needs to be a human at the helm.

Myth 7: “AI is just too dumb when it makes things up.”

Fact: What make AI seems to be very stupid is usually a hallucination. Remember when we told you that a model gives open-ended responses based on probabilities? That's where it begins. If the model doesn't have context, or if the fact you're asking about is an unusual one, it still has to fill in the blank and make you happy. So it anticipates something that sounds good, though it may be mistaken. Another common reason is the context window. A model has only a chance to "see" a number of tokens at once like a short-term memory buffer. If your question depends on information outside of that window, it simply gets lost. The model then guesses and fills in the blank with it, and it could be completely off. Hallucinations do not mean the AI is malfunctioning, they're a consequence of a system designed to generate freely and never follow a script. ···

Practical Playbook: Working with AI as a Software Engineer

Think of AI as a (human) team member. Tell it what you need in plain language. Don't mind grammar, and don't try to be funny. Be direct and keep going. Ask as many "stupid" questions as you want along the way. The loop is simple: ask, review (results), refine (requests), repeat.

1) Context first

Tell the model who this is for, what the goal is, and what constraints matter. Mini prompt Example : markup

You are helping our payments team. Goal: add idempotency to the charge endpoint.Constraints: Python 3.11, FastAPI, Postgres, existing retry logic.Output: explain first, then give code in ```python``` fences.

2) Declare the task

Define steps or acceptance criteria clearly, so "done" is well-defined. Mini prompt Example : markup

Make a step list for the refactor. Include pre-checks, code changes, tests, and a final rollout check.Mark each step with [owner], [est], and [risk].

3) Constrain the output

Ask for a shape so you don't get a wall of text. Mini prompt Example : markup

# Example 1Return JSON with keys: rationale, risks, code, tests. No extra prose.# Example 2Generate a SQL query inside sql … fences.

4) Inspect and iterate

Treat the first draft as a sketch. Point to the piece that is wrong. Ask for a correction. Loop again.

5) Keep a living log file

Ask the model to save each large step and change into a file so you can reuse context later. Mini prompt Example: markup

Start or update a file named AI_LOG.md.Append a dated entry with sections: Context, Decision, Commands, Snippets, Open Questions.Only add new content. Keep older entries.

6) Work within the context window

Models only "see" a brief snippet of recent text. Compress chat history when the thread grows too long.
A. Do it with a prompt. You can use the following compression prompt and paste it when the chat is heavy: markup

Summarize all prior messages into a compact brief I can reuse.Format:- Facts we agreed on- Constraints and conventions- Decisions and their reasons- Open questions- Next actionsKeep it under 300 tokens. Preserve file names, APIs, and versions exactly.

B. Use the tool when available. Some of the services that employ AI also have built-in tools to help. For example, Amazon Q has a /compact command, which condenses the chat history into a shorter one. It even warns you when your conversation is getting close to the limit and suggests compacting. (Related AWS Documentation)
C. Re-seed the next turn. Paste the brief back in after compaction as the new context. Then your next request. ···

Wrapping Up

Generative AI is not out to replace engineers, but it's revolutionizing what engineering is. Properly leveraged, it's less of a competitor and more of an assistant to help speed up the drudge so that we (engineers) can focus on design, systems, and the hard decisions. The engineers that'll thrive will be those who can guide it, push back on it, and make it a team player. Because, ultimately, even Iron Man had to fly the suit. ··· Enjoyed this? Read more posts from me on TDS — or subscribe to my newsletter for more. Thank you!

Register Now to Continue Reading

Unlock Towards Data Science with a Free Account

All articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more. Register for freeSign in to your account

Written By

Amy Ma

See all from Amy Ma Generative Ailarge language modelLlmPrompt EngineeringSoftwar Engineering Share This Article

Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program. Write for TDS

Related Articles

Generative AI and Civic Institutions

Artificial Intelligence Should human obsolescence be our goal? Stephanie Kirmer March 3, 202510 min read

Feature Selection Using Genetic Algorithms

Artificial Intelligence Learn how GA work by applying it to the feature optimization process Javier Martínez Ojeda May 29, 20228 min read

Graph RAG - A conceptual introduction

Graph RAG answers the big questions where text embeddings won't help you. Jakob Pörschmann August 22, 20249 min read

AI Agents from Scratch: Iterations & Chains

Artificial Intelligence From Zero to Hero using only Python & Ollama (no GPU, no APIKEY) Mauro Di Pietro March 26, 20259 min read

Thriving As A Remote Data Scientist

Artificial Intelligence My Experience of Working Remotely Kurtis Pykes December 10, 20205 min read

5 Use Cases of Pandas loc and iloc Methods

Artificial Intelligence Make them more useful. Soner Yıldırım March 22, 20214 min read

Create Your Own Stable Diffusion UI on AWS in Minutes

Deep Learning Deploy a text-to-image web app with just one command Heiko Hotz January 3, 20238 min read

Democratising Machine learning with H2O

Artificial Intelligence It is important to make AI accessible to everyone for the sake of social and economic stability. Parul Pandey April 20, 20199 min read

Data Analysis at Its Finest

Artificial Intelligence An example that will make you like R better Soner Yıldırım March 25, 20213 min read

Your home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals. Subscribe to Our NewsletterWrite For TDS•About•Advertise•Privacy Policy•Terms of Use © Insight Media Group, LLC 2026•The New Stack•roadmap.sh•Creator Network

Recommended Articles

Close