{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "Priyesh Dave",
  "title_listed": "Day 21: GRU Networks Explained",
  "keywords": "",
  "domain": "文本处理",
  "content_type": "技术教程",
  "is_paid_flag": "否",
  "url": "https://dev.to/priyeshdave6/day-21-gru-networks-explained-4hnc",
  "publish_date": "2026-09-20",
  "corpus_stage": "",
  "notes": "",
  "id": "deb2458cd420",
  "collected_at": "2026-10-02T13:49:24",
  "title": "Day 21: GRU Networks Explained - DEV Community",
  "body_markdown": "Sequence data shows up everywhere in natural language processing (NLP). Sentences are sequences of words. Time-stamped data—like sensor readings or stock prices—are sequences of numbers. To predict the next word in a sentence, translate languages, or detect sentiment, a model needs to “remember” what came earlier in the sequence. Ordinary neural networks, which treat each input independently, fail at this. They have no memory.\nA typical fully connected (or\ndense\n) neural network expects input in a fixed shape—like a flattened image vector or a list of features. There’s no built-in idea of “earlier” or “later.” For sequences, this means losing the chain of context. In language, for example, “The cat sat on the...” could end with “mat,” “roof,” or “sofa.” The correct word depends on every word that came before.\nWhat is a Recurrent Neural Network (RNN)?\nA recurrent neural network (RNN) handles sequences directly. Imagine reading a sentence one word at a time. With each new word, you update your understanding, carrying forward what you’ve seen so far. An RNN does the same by passing information forward through the sequence, one time step at a time.\nAt each step, an RNN takes the current input and mixes it with its “memory” from the previous step (called the\nhidden state\n). It then produces a new hidden state to pass to the next time step.\nThe hidden state works like a running notebook. Each new word or data point lets you update, erase, or reinforce what you’ve written. This lets RNNs, at least in theory, learn patterns that depend on previous elements of the sequence.\nThe Vanishing Gradient Problem in Vanilla RNNs\nVanilla RNNs (the basic kind) have a big limitation: they struggle to learn relationships between distant parts of a sequence. If something from early in the data matters much later, the RNN often can't connect the dots.\nThis happens because of the\nvanishing gradient problem\n. When we train RNNs using a process called backpropagation through time, the “signal” that updates their parameters—the gradient—must pass backward through every time step. If the sequence is long, these gradients get multiplied by numbers slightly less than one, over and over. The effect: the gradient quickly shrinks toward zero, like a message fading in a long game of telephone. The network can’t learn connections that span many steps.\nPicture a row of dominoes where each only gives a slight nudge. If you have a lot of dominoes, the force barely reaches the last one. Similarly, the signal in an RNN dies out, and the network “forgets” by the time it reaches a distant step.\nGated Recurrent Unit (GRU): Smarter Memory with Gates\nResearchers developed new RNN variants specifically to tackle these memory issues. The Gated Recurrent Unit (GRU) is a popular choice. It adds internal mechanisms, called\ngates\n, to control how information moves through the network.\nGates are like smart switches. At each time step, they decide which information to keep, which to update, and what to forget. This helps the RNN keep important facts from earlier and ignore irrelevant data.\nWith this setup, GRUs let models carry forward important information across many time steps. They work much better for tasks where context really matters.\nHow a GRU Works: The Gates\nA standard GRU cell uses two gates: the\nupdate gate\nand the\nreset gate\n.\nUpdate gate\n: Decides how much of the previous hidden state (memory) to keep. When the update gate is close to one, nearly all previous memory is kept. When it's close to zero, the cell mostly listens to the new input and forgets the past.\nReset gate\n: Decides how much of the previous memory to ignore before combining with the new input. If the reset gate is zero, the cell forgets the past for that step. If it's one, it considers both the new input and the memory in full.\nHere’s what happens inside a GRU cell at each time step:\nThe current input (like a word vector) and previous hidden state are fed into both gates.\nEach gate uses a small neural net (with a sigmoid activation squashing outputs between 0 and 1) to decide its value.\nThe reset gate determines how much memory to blend with the new input to create a “candidate” new memory.\nThe update gate scales how much of that candidate memory replaces the old hidden state versus just passing the old memory forward.\nThe result is a new hidden state, carried to the next time step.\nGates give the network fine control: it can “remember” relevant dependencies for as many steps as it needs, and “forget” what's unimportant.\nVisualizing the GRU Cell\nPicture a GRU cell as a box with two sliders—one for each gate.\nThe previous hidden state and the current input come in.\nThe reset gate filters the previous memory. This filtered memory and the input are used to propose a new “candidate” memory.\nThe update gate's value sets how much of the old memory versus the candidate memory should form the next hidden state.\nThe output hidden state is passed to the next time step.\nIf you sketched it, you’d see arrows for input and previous hidden state feeding into two “gate” blocks (update and reset), then combining to create a candidate hidden state, which merges with prior hidden state based on the update gate’s position.\nVanilla RNNs overwrite their memory at every step. The GRU’s careful switches keep crucial context alive and ignore distractions.\nGRU in Action: Code Example\nHere's a minimal GRU in PyTorch. It runs a sequence of numbers through a single GRU cell so you can watch the hidden state evolve:\nimport\ntorch\nimport\ntorch.nn\nas\nnn\n# Make a GRU: input size 1, hidden size 1, single layer\ngru\n=\nnn\n.\nGRU\n(\ninput_size\n=\n1\n,\nhidden_size\n=\n1\n,\nbatch_first\n=\nTrue\n)\n# Sequence: batch size 1, sequence length 5, 1 feature per step\ninput_seq\n=\ntorch\n.\ntensor\n([[[\n1.0\n],\n[\n2.0\n],\n[\n3.0\n],\n[\n4.0\n],\n[\n5.0\n]]])\n# shape (1, 5, 1)\n# Initial hidden state: zeros\nh0\n=\ntorch\n.\nzeros\n(\n1\n,\n1\n,\n1\n)\n# Forward pass\noutput_seq\n,\nhn\n=\ngru\n(\ninput_seq\n,\nh0\n)\nprint\n(\n\"\nOutput at each timestep:\n\"\n)\nprint\n(\noutput_seq\n)\nprint\n(\n\"\nFinal hidden state:\n\"\n)\nprint\n(\nhn\n)\nEnter fullscreen mode\nExit fullscreen mode\nTry changing the input sequence—or initial hidden state—and see how the hidden state tracks the “memory” of what’s happened so far.\nWhen and Why to Use GRUs\nGRUs are a common choice for sequence problems where you need memory, but want something faster and simpler than an LSTM (another variant, covered next). In actual NLP tasks, GRUs excel at:\nSentiment analysis (positive/negative sentence meaning)\nLanguage modeling (predicting the next word)\nNamed entity recognition (finding names, places, dates)\nGRUs have fewer gates than LSTMs, so they train faster and use fewer parameters. They’re strong when your sequences are of moderate length, or you want fast experimentation.\nOn the toughest tasks with really long-range dependencies, LSTMs can sometimes outperform GRUs. But for many practical problems, GRUs offer a great combination of speed and effectiveness. As a starting point for sequence models, they’re a strong default—much better than vanilla RNNs for real tasks.\nYou’ll learn about LSTMs next. For now, think of GRUs as a smart, efficient upgrade for learning from sequences—especially where basic RNNs would forget important context.\nKey Takeaways\nOrdinary neural networks cannot capture dependencies across sequence steps due to lack of memory.\nThe vanishing gradient problem prevents vanilla RNNs from learning long-range dependencies.\nGRUs use update and reset gates to control what information to keep or forget from previous steps.\nGRUs are more efficient and train faster than LSTMs while effectively modeling moderate-length sequences.\nPyTorch provides an easy way to experiment with GRUs using simple code on numeric sequences.\nTry It Yourself\nUsing PyTorch, define a simple GRU layer with one input and hidden unit. Pass a sequence like [[0.1], [0.2], [0.3], [0.4]] through the GRU, printing the output at each timestep. Change one value in the input sequence and observe how the GRU's outputs change, illustrating how it retains memory of earlier inputs.\nFurther Resources\n🎥\nGRUs Explained Simply | Gated Recurrent Units in Deep Learning (AI for Beginners)\n📘\ntf.keras.layers.GRU  |  TensorFlow v2.16.1\n📄\nRNNs, LSTMs & GRUs – Complete NLP Series Part 7\n📄\nSection 10.3: GRU Networks | Building Temporal AI\n📄\nGated Recurrent Units (GRU) – Stanford CS224n lecture slides\nComing up on Day 22:\nSequence-to-Sequence Models",
  "metadata": {
    "title": "Day 21: GRU Networks Explained - DEV Community",
    "description": "Sequence data shows up everywhere in natural language processing (NLP). Sentences are sequences of... Tagged with ai, nlp, genai, 100daysofcode.",
    "keywords": "ai, nlp, genai, 100daysofcode, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/priyeshdave6/day-21-gru-networks-explained-4hnc",
    "og:title": "Day 21: GRU Networks Explained",
    "og:description": "Sequence data shows up everywhere in natural language processing (NLP). Sentences are sequences of...",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F1x4xij69parzeg2enhvo.png",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@",
    "twitter:title": "Day 21: GRU Networks Explained",
    "twitter:description": "Sequence data shows up everywhere in natural language processing (NLP). Sentences are sequences of...",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F1x4xij69parzeg2enhvo.png"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/deb2458cd420/code/body-000.txt",
      "chars": 501,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [
    {
      "url": "https://www.youtube.com/watch?v=t7Q5TvWhLls",
      "status": "ok",
      "local_path": "attachments/deb2458cd420/videos/t7Q5TvWhLls.mp4",
      "bytes": 52994310,
      "valid": true,
      "height": 1080,
      "width": 1920,
      "duration": "136.101000"
    }
  ],
  "links": [
    {
      "href": "https://www.youtube.com/watch?v=t7Q5TvWhLls",
      "label": "GRUs Explained Simply | Gated Recurrent Units in Deep Learning (AI for Beginners)",
      "type": "a"
    },
    {
      "href": "https://tensorflow.google.cn/api_docs/python/tf/keras/layers/GRU",
      "label": "tf.keras.layers.GRU  |  TensorFlow v2.16.1",
      "type": "a"
    },
    {
      "href": "https://www.wasilzafar.com/pages/series/nlp/nlp-rnn-lstm-gru.html",
      "label": "RNNs, LSTMs & GRUs – Complete NLP Series Part 7",
      "type": "a"
    },
    {
      "href": "https://temporalbook.icsgen-ai.org/part-3-temporal-deep-learning/module-10-recurrent-neural-networks/section-10.3.html",
      "label": "Section 10.3: GRU Networks | Building Temporal AI",
      "type": "a"
    },
    {
      "href": "https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/slides/cs224n-2020-lecture07-fancy-rnn.pdf",
      "label": "Gated Recurrent Units (GRU) – Stanford CS224n lecture slides",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:48:05Z",
  "source_url": "https://dev.to/priyeshdave6/day-21-gru-networks-explained-4hnc",
  "papers": [
    {
      "status": "ok",
      "url": "https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/slides/cs224n-2020-lecture07-fancy-rnn.pdf",
      "bytes": 3351533,
      "reused": false,
      "local_path": "papers/repair/49953d52554a56d8c48d.pdf"
    }
  ],
  "actual_video_files": [
    {
      "local_path": "attachments/deb2458cd420/videos/t7Q5TvWhLls.mp4",
      "bytes": 52994310,
      "valid": true,
      "height": 1080,
      "width": 1920,
      "duration": "136.101000",
      "status": "ok"
    }
  ],
  "body_chars": 8450,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/deb2458cd420/pages/source.html",
  "local_html": "attachments/deb2458cd420/pages/article.html",
  "body_base_path": "attachments/deb2458cd420/pages/",
  "body_markdown_path": "attachments/deb2458cd420/pages/article.md"
}