{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "Priyesh Dave",
  "title_listed": "Day 18: Recurrent Neural Networks (RNNs) Explained",
  "keywords": "",
  "domain": "文本处理",
  "content_type": "技术教程",
  "is_paid_flag": "否",
  "url": "https://dev.to/priyeshdave6/day-18-recurrent-neural-networks-rnns-explained-502f",
  "publish_date": "2026-09-15",
  "corpus_stage": "",
  "notes": "",
  "id": "f8e034a5eedf",
  "collected_at": "2026-10-02T13:49:37",
  "title": "Day 18: Recurrent Neural Networks (RNNs) Explained - DEV Community",
  "body_markdown": "Previously, on Day 17:\nExplained the structure and function of neural networks, including neurons, layers, weights, biases, and activation functions, and how neural networks are trained to solve natural language processing tasks.\nThe Challenge with Sequences in NLP\nHuman language is structured as a sequence. The meaning of a sentence depends on the order of its words:\n“I ate breakfast before running” is not the same as “I ran before eating breakfast.”\nFeedforward neural networks, which we've seen so far, treat each input as independent from the others. These models can classify an image or predict a single label, but they have no way to remember earlier inputs. If you process a sentence word by word, a feedforward network processes each word as if it's unrelated to what came before.\nThis becomes a serious limitation for tasks like machine translation, speech recognition, or conversational context. In language, the order of words and the context from earlier words are essential. We need models that can remember what has already happened and use that memory to interpret each new word.\nIntroducing Recurrent Neural Networks (RNNs)\nA Recurrent Neural Network (RNN) is designed for sequential data. RNNs process data one element at a time—think one word at a time—while carrying a short-term memory, called the\nhidden state\n, from one step to the next.\nUnlike feedforward networks, each step in an RNN passes information forward through this hidden state. The network can \"remember\" what it has seen and use that memory when processing new inputs.\nThis memory makes RNNs a much better fit for sequences like sentences, where the meaning of a word depends on what came before.\nHow RNNs Work: Unrolling in Time\nImagine reading a sentence one word at a time from left to right. You don't reset your memory at each word; you remember what you’ve read so far. RNNs are built with this same idea.\nTo make training possible, we\nunroll\nthe RNN in time:\nFor an input sequence of five words, imagine five copies of the RNN cell—one for each word in the sequence.\nEach cell shares the same parameters (weights), but receives its own input and the hidden state from the previous step.\nAt each step, the RNN cell:\nTakes in the current input (e.g., the current word’s encoding).\nReceives the summary, or\nhidden state\n, from the preceding step.\nUpdates its hidden state to reflect both the current input and past context.\nOptionally, produces an output (such as a predicted word or tag).\nThis chained process lets information—the hidden state—move forward through the sequence and capture context up to the current position.\nThe RNN Cell: What Happens at Each Step\nInside an RNN cell, the computation is similar to a layer in a regular neural network, but with one key difference: it takes two inputs at each timestep—\nThe current input (such as the vector representing the current word),\nAnd the previous hidden state (the memory of everything seen so far).\nThe core update at each time step ( t ) is:\n[\nh_t = \\tanh(W_{xh} x_t + W_{hh} h_{t-1} + b_h)\n]\nWhere:\n( x_t ) is the current input at time ( t )\n( h_{t-1} ) is the hidden state from the previous timestep\n( W_{xh} ) is the weight matrix for input-to-hidden connections\n( W_{hh} ) is the weight matrix for hidden-to-hidden (memory) connections\n( b_h ) is a bias term\n( \\tanh ) is an activation function, producing outputs between -1 and 1\nThe new hidden state ( h_t ) carries information from both the new input and everything summarized up to the previous step.\nRNNs in Action: A Simple Code Example\nHere's a minimal PyTorch example showing how an RNN processes a sequence. This code:\nDefines a single-layer RNN cell,\nFeeds in a sequence of numbers (1 to 5),\nShows the hidden state after each input.\nimport\ntorch\nimport\ntorch.nn\nas\nnn\n# Example sequence: batch size 1, sequence length 5, input size 1\nsequence\n=\ntorch\n.\ntensor\n([[[\n1.0\n]],\n[[\n2.0\n]],\n[[\n3.0\n]],\n[[\n4.0\n]],\n[[\n5.0\n]]])\ninput_size\n=\n1\n# Input dimension at each step\nhidden_size\n=\n2\n# Hidden state size\nrnn\n=\nnn\n.\nRNN\n(\ninput_size\n,\nhidden_size\n,\nbatch_first\n=\nFalse\n)\n# Initialize hidden state: (num_layers, batch_size, hidden_size)\nh_t\n=\ntorch\n.\nzeros\n(\n1\n,\n1\n,\nhidden_size\n)\nprint\n(\n\"\nStep | Input | Hidden State\n\"\n)\nprint\n(\n\"\n-------------------------\n\"\n)\nfor\ni\n,\nx_t\nin\nenumerate\n(\nsequence\n):\nout\n,\nh_t\n=\nrnn\n(\nx_t\n.\nunsqueeze\n(\n0\n),\nh_t\n)\n# Add batch/time axes\nprint\n(\nf\n\"\n{\ni\n+\n1\n}\n|\n{\nx_t\n.\nitem\n()\n}\n|\n{\nh_t\n.\nsqueeze\n(\n0\n).\ndetach\n().\nnumpy\n()\n}\n\"\n)\nEnter fullscreen mode\nExit fullscreen mode\nAt each step, the RNN's hidden state reflects not just the current number, but also what it's seen so far. This is what gives the RNN its memory of the sequence.\nRNN Limitations and the Path Forward\nBasic, or \"vanilla,\" RNNs can remember context for short sequences, but they struggle with longer ones.\nThe challenge comes from training. As the RNN updates its weights, important signals from early in the sequence can quickly shrink to near zero (vanish) or grow out of control (explode). This is called the\nvanishing or exploding gradient problem\n. In practice, vanilla RNNs often forget information from more than a few steps back.\nTo fix this, researchers developed enhanced RNNs, such as\nLSTMs\n(Long Short-Term Memory networks) and\nGRUs\n(Gated Recurrent Units). These are specially designed to remember important information for much longer, allowing models to capture context even when it appears many words back in a sentence or conversation.\nUnderstanding basic RNNs gives you the foundation to appreciate why these more advanced architectures are needed—and how they power real-world NLP systems, where meaning can depend on something far earlier in the sequence.\nKey Takeaways\nLanguage understanding requires remembering word order and prior context.\nRNNs keep a hidden state to maintain memory across sequence steps.\nAt each step, an RNN combines current input with previous hidden state.\nVanilla RNNs struggle to remember information in long sequences due to vanishing/exploding gradients.\nEnhanced architectures like LSTMs and GRUs help RNNs capture longer-range dependencies.\nTry It Yourself\nChoose a short word (such as 'cat') and represent each character as a simple number (e.g., c=1, a=2, t=3). Using the update rule h_t = tanh(0.5 * x_t + 0.8 * h_{t-1}) with h_0 = 0, manually compute and record the hidden state for each character in order. Write down the value of h after processing each character to observe how the hidden state accumulates information.\nFurther Resources\n🎥\nRecurrent Neural Networks (RNNs), Clearly Explained!!! – StatQuest with Josh Starmer\n🎥\nRecurrent Neural Networks - EXPLAINED! – CodeEmporium\n📘\nSequence Models and Long Short‑Term Memory Networks — PyTorch Tutorials\n📄\nRecurrent Neural Networks and Long Short‑Term Memory Networks: Tutorial and Survey\n📄\nLLMs Part 2: Building a Vanilla RNN\nComing up on Day 19:\nThe Vanishing Gradient Problem",
  "metadata": {
    "title": "Day 18: Recurrent Neural Networks (RNNs) Explained - DEV Community",
    "description": "Previously, on Day 17: Explained the structure and function of neural networks, including neurons,... Tagged with ai, nlp, genai, 100daysofcode.",
    "keywords": "ai, nlp, genai, 100daysofcode, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/priyeshdave6/day-18-recurrent-neural-networks-rnns-explained-502f",
    "og:title": "Day 18: Recurrent Neural Networks (RNNs) Explained",
    "og:description": "Previously, on Day 17: Explained the structure and function of neural networks, including neurons,...",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F8fk5y56od4kaumz4bfft.png",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@",
    "twitter:title": "Day 18: Recurrent Neural Networks (RNNs) Explained",
    "twitter:description": "Previously, on Day 17: Explained the structure and function of neural networks, including neurons,...",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F8fk5y56od4kaumz4bfft.png"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/f8e034a5eedf/code/body-000.txt",
      "chars": 675,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [
    {
      "url": "https://www.youtube.com/watch?v=AsNTP8Kwu80",
      "src": "https://www.youtube.com/watch?v=AsNTP8Kwu80",
      "status": "ok",
      "kind": "single_video",
      "local_path": "attachments/f8e034a5eedf/videos/repair-AsNTP8Kwu80.mp4",
      "bytes": 34009427,
      "valid": true,
      "height": 1080,
      "width": 1920,
      "duration": "996.841000",
      "reused": true,
      "reused_from": "xinxing-zimeiti-corpus/attachments/c771a5c0a9d9/videos/AsNTP8Kwu80.mp4"
    },
    {
      "url": "https://www.youtube.com/watch?v=yZv_yRgOvMg",
      "status": "ok",
      "local_path": "attachments/f8e034a5eedf/videos/yZv_yRgOvMg.mp4",
      "bytes": 29298642,
      "valid": true,
      "height": 720,
      "width": 1280,
      "duration": "1024.501000"
    }
  ],
  "links": [
    {
      "href": "https://www.youtube.com/watch?v=AsNTP8Kwu80",
      "label": "Recurrent Neural Networks (RNNs), Clearly Explained!!! – StatQuest with Josh Starmer",
      "type": "a"
    },
    {
      "href": "https://www.youtube.com/watch?v=yZv_yRgOvMg",
      "label": "Recurrent Neural Networks - EXPLAINED! – CodeEmporium",
      "type": "a"
    },
    {
      "href": "https://docs.pytorch.org/tutorials/beginner/nlp/sequence_models_tutorial.html",
      "label": "Sequence Models and Long Short‑Term Memory Networks — PyTorch Tutorials",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/abs/2304.11461",
      "label": "Recurrent Neural Networks and Long Short‑Term Memory Networks: Tutorial and Survey",
      "type": "a"
    },
    {
      "href": "https://olliegreen.info/posts/7/llms-part-2-building-a-vanilla-rnn/",
      "label": "LLMs Part 2: Building a Vanilla RNN",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:48:07Z",
  "source_url": "https://dev.to/priyeshdave6/day-18-recurrent-neural-networks-rnns-explained-502f",
  "papers": [
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2304.11461",
      "bytes": 884700,
      "reused": false,
      "local_path": "papers/repair/68fcebf27532149e9036.pdf"
    }
  ],
  "actual_video_files": [
    {
      "local_path": "attachments/f8e034a5eedf/videos/yZv_yRgOvMg.mp4",
      "bytes": 29298642,
      "valid": true,
      "height": 720,
      "width": 1280,
      "duration": "1024.501000",
      "status": "ok"
    }
  ],
  "body_chars": 6908,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/f8e034a5eedf/pages/source.html",
  "local_html": "attachments/f8e034a5eedf/pages/article.html",
  "body_base_path": "attachments/f8e034a5eedf/pages/",
  "body_markdown_path": "attachments/f8e034a5eedf/pages/article.md"
}