{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "Priyesh Dave",
  "title_listed": "Day 20: LSTM Networks Explained",
  "keywords": "",
  "domain": "文本处理",
  "content_type": "技术教程",
  "is_paid_flag": "否",
  "url": "https://dev.to/priyeshdave6/day-20-lstm-networks-explained-3fi2",
  "publish_date": "2026-09-18",
  "corpus_stage": "",
  "notes": "",
  "id": "73d6ea364d6b",
  "collected_at": "2026-10-02T13:49:28",
  "title": "Day 20: LSTM Networks Explained - DEV Community",
  "body_markdown": "Sequences in Language, and Why Simple RNNs Struggle\nLanguage is made of sequences. Every sentence is a list of words. Each word is a sequence of characters. Each item in the sequence depends on what came before it—and sometimes, what comes after.\nTake this sentence:\n“I grew up in France, so I speak fluent ___.”\nTo predict the missing word (“French”), you need to remember “France” earlier in the sentence. This is called a\nlong-term dependency\n: you have to use information from much earlier in the sequence to make a correct decision later.\nRecurrent neural networks\n(RNNs) were invented to handle sequence data. An RNN takes one input at a time, updating its memory at each step. In theory, this lets it “remember” what happened earlier in the sequence.\nBut in practice, simple RNNs have trouble holding on to information as the sequence gets longer. Earlier information fades away. The technical name for this is the\nvanishing gradient problem\n: when training, error signals sent backwards through the RNN shrink rapidly, so the model forgets things from many steps ago. Plain RNNs work well only for short-term dependencies.\nThe LSTM Cell: A New Type of RNN Building Block\nLong Short-Term Memory networks\n(LSTMs) were designed to fix this memory problem.\nAn LSTM is a special kind of RNN block. Inside each LSTM “cell” are components called\ngates\n. You can think of these gates as switches or valves. They decide what information to keep, what to update, and what to forget as the network reads a sequence.\nEach LSTM cell keeps its own memory, called the\ncell state\n. This is like a conveyor belt running through the network, carrying important data from one step to the next.\nEach LSTM cell has four main parts:\nCell state:\nThe main memory, able to carry information forward for many steps.\nInput gate:\nDecides how much of the current input to write into memory.\nForget gate:\nDecides how much of the previous memory to erase.\nOutput gate:\nDecides how much of the memory to use to make an output.\nEach gate is a small neural network: some weights and biases, trained to do the job. The gates look at the current input and what the cell remembers from the previous step (the hidden state), and then decide what to do.\nHow LSTMs Handle Long-Range Dependencies\nPicture the cell state as a highway running through the whole sequence. The gates are like traffic lights. The highway lets important information travel far. The lights decide who gets to stay, who merges in, and who exits.\nForget gate:\nLooks at what's happening now and what the model remembers. It decides which parts of the memory are no longer useful and removes them from the cell state.\nInput gate:\nLets new, relevant information onto the highway, updating the cell state with what's important right now.\nOutput gate:\nSelects what information leaves the memory and becomes visible as the “output” for this step.\nBecause the cell state can carry information untouched through many sequence steps, and the gates only make precise, targeted changes, LSTMs can remember key facts far back in the sequence—even hundreds of steps ago.\nThis makes LSTMs good at problems where you need long memory, like predicting the next word in a sentence, machine translation, or anything with important patterns spread out over time.\nA Minimal LSTM Example on Text\nHere’s a toy example. We'll use an LSTM to predict the next character in a sequence using Keras (a popular Python deep learning library).\nWe’ll use the sequence “hel” and try to predict the next letter, “l”.\nimport\nnumpy\nas\nnp\nfrom\ntensorflow.keras.models\nimport\nSequential\nfrom\ntensorflow.keras.layers\nimport\nLSTM\n,\nDense\n# Map characters to integers: {'h':0, 'e':1, 'l':2, 'o':3}\nchar_to_int\n=\n{\n'\nh\n'\n:\n0\n,\n'\ne\n'\n:\n1\n,\n'\nl\n'\n:\n2\n,\n'\no\n'\n:\n3\n}\nint_to_char\n=\n{\ni\n:\nc\nfor\nc\n,\ni\nin\nchar_to_int\n.\nitems\n()}\n# Input sequence: \"h\", \"e\", \"l\"\nX\n=\nnp\n.\narray\n([[\n[\n0\n],\n[\n1\n],\n[\n2\n]\n]])\n# shape (samples, timesteps, features)\n# Next char to predict: \"l\"\ny\n=\nnp\n.\narray\n([\n2\n])\n# One-hot encode output\ny\n=\nnp\n.\neye\n(\n4\n)[\ny\n]\n# Build model: 3 timesteps, 1 feature per step\nmodel\n=\nSequential\n([\nLSTM\n(\n8\n,\ninput_shape\n=\n(\n3\n,\n1\n)),\nDense\n(\n4\n,\nactivation\n=\n'\nsoftmax\n'\n)\n])\nmodel\n.\ncompile\n(\nloss\n=\n'\ncategorical_crossentropy\n'\n,\noptimizer\n=\n'\nadam\n'\n)\n# Train briefly (just to show it runs)\nmodel\n.\nfit\n(\nX\n,\ny\n,\nepochs\n=\n100\n,\nverbose\n=\n0\n)\n# Predict what comes after \"hel\"\npred\n=\nmodel\n.\npredict\n(\nX\n)\nprint\n(\n\"\nPredicted next char:\n\"\n,\nint_to_char\n[\nnp\n.\nargmax\n(\npred\n[\n0\n])])\nEnter fullscreen mode\nExit fullscreen mode\nInputs:\nThe input is a sequence of three characters, each mapped to an integer, then turned into a column vector.\nOutputs:\nThe model picks the most likely next character out of four possible characters.\nWhat's happening:\nAt each step (“h”, then “e”, then “l”), the LSTM updates its memory with its gates.\nAfter the last letter, the LSTM predicts the next character.\nThis example is small for teaching. In practice, you’d use much larger inputs and vocabularies. But the pattern is the same: a sequence goes in, the LSTM processes it step by step, and then predicts the next item.\nWhen to Use LSTMs (and When Not To)\nLSTMs are best when your data comes in order and things far apart in the sequence matter. Examples:\nPredicting the next word or character in text\nTranslating one language to another, sequence by sequence\nSpeech recognition, music, or any task where order and timing are important\nLSTMs solved the memory loss in plain RNNs and pushed the field forward. But they are not the top choice for every sequence problem today. For much longer sequences, or for tasks that need to be trained very quickly and in parallel, the\nTransformer\nmodel works better. Transformers use\nattention\ninstead of recurrence. They remember details from anywhere in the sequence and train faster.\nFor sequences up to a few hundred steps, LSTMs remain practical and easy to use with existing tools. For much longer text or very large context windows, newer models are a better fit.\nKey Takeaways\nLanguage involves sequences where later predictions depend on earlier context.\nSimple RNNs suffer from the vanishing gradient problem and cannot remember long-term dependencies well.\nLSTM networks use gates to control memory, allowing them to retain important information across many sequence steps.\nGates in LSTMs include the input gate, forget gate, output gate, and the cell state.\nLSTMs outperform plain RNNs on tasks requiring long-context memory, such as language modeling and translation.\nTry It Yourself\nRun the provided minimal LSTM example using Keras to predict the next character in a short sequence (e.g., predicting 'l' after 'hel'). Then, change the input sequence to something else (like 'heo') and observe how the output prediction changes. Reflect on how the model responds to different input contexts.\nFurther Resources\n🎥\nLSTM Networks: Explained Step by Step!\n🎥\nLong Short-Term Memory - LSTM Models with TensorFlow (MLCon conference talk)\n📄\nUnderstanding LSTM Networks (Christopher Olah’s blog)\n📄\nLSTM Models: A Complete Guide to Long Short‑Term Memory Networks (DataCamp)\nComing up on Day 21:\nGRU Networks Explained",
  "metadata": {
    "title": "Day 20: LSTM Networks Explained - DEV Community",
    "description": "Sequences in Language, and Why Simple RNNs Struggle   Language is made of sequences. Every... Tagged with ai, nlp, genai, 100daysofcode.",
    "keywords": "ai, nlp, genai, 100daysofcode, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/priyeshdave6/day-20-lstm-networks-explained-3fi2",
    "og:title": "Day 20: LSTM Networks Explained",
    "og:description": "Sequences in Language, and Why Simple RNNs Struggle   Language is made of sequences. Every...",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9ticvk2cekshfugvkzwm.png",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@",
    "twitter:title": "Day 20: LSTM Networks Explained",
    "twitter:description": "Sequences in Language, and Why Simple RNNs Struggle   Language is made of sequences. Every...",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9ticvk2cekshfugvkzwm.png"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/73d6ea364d6b/code/body-000.txt",
      "chars": 869,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [
    {
      "url": "https://www.youtube.com/watch?v=P_TZN8kRObQ",
      "status": "ok",
      "local_path": "attachments/73d6ea364d6b/videos/P_TZN8kRObQ.mp4",
      "bytes": 910882101,
      "valid": true,
      "height": 1080,
      "width": 1920,
      "duration": "2087.001000"
    }
  ],
  "links": [
    {
      "href": "https://www.youtube.com/watch?v=P_TZN8kRObQ",
      "label": "LSTM Networks: Explained Step by Step!",
      "type": "a"
    },
    {
      "href": "https://www.classcentral.com/course/youtube-long-short-term-memory-lstm-models-with-tensorflow-ml-conference-session-sahil-dua-241059",
      "label": "Long Short-Term Memory - LSTM Models with TensorFlow (MLCon conference talk)",
      "type": "a"
    },
    {
      "href": "https://colah.github.io/posts/2015-08-Understanding-LSTMs/",
      "label": "Understanding LSTM Networks (Christopher Olah’s blog)",
      "type": "a"
    },
    {
      "href": "https://www.datacamp.com/tutorial/lstm-models",
      "label": "LSTM Models: A Complete Guide to Long Short‑Term Memory Networks (DataCamp)",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:48:05Z",
  "source_url": "https://dev.to/priyeshdave6/day-20-lstm-networks-explained-3fi2",
  "papers": [],
  "actual_video_files": [
    {
      "local_path": "attachments/73d6ea364d6b/videos/P_TZN8kRObQ.mp4",
      "bytes": 910882101,
      "valid": true,
      "height": 1080,
      "width": 1920,
      "duration": "2087.001000",
      "status": "ok"
    }
  ],
  "body_chars": 7152,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/73d6ea364d6b/pages/source.html",
  "local_html": "attachments/73d6ea364d6b/pages/article.html",
  "body_base_path": "attachments/73d6ea364d6b/pages/",
  "body_markdown_path": "attachments/73d6ea364d6b/pages/article.md"
}