When you sit down to solve a complex puzzle or plan three moves ahead in chess, do you narrate every synaptic firing to yourself in full, grammatically correct English sentences? Of course not. Human cognition operates across continuous, multi-dimensional mental representations—spatial, relational, intuitive. We only convert our thoughts into human language when we need to speak to someone else.
Yet the dominant paradigm in AI reasoning today (from DeepSeek-R1 to OpenAI o1) forces neural networks to do the exact opposite. We train models to emit thousands of discrete text tokens enclosed in
<think>...</think>
blocks:
“Wait, let me think... Actually, let me reconsider... Hmm, what if I try...”
During reinforcement learning (RL) post-training, this imposes an astronomical
token tax
. Models burn 80% to 90% of their generation time emitting sprawling English prose that no human will ever read. In multi-turn engineering workflows (such as multi-step SQL querying, hardware synthesis, or complex network routing), verbose text reasoning exhausts context limits, balloons the KV-cache, and stretches training runs from hours into days. Worse: in our experiments,
up to a third of all model rollouts got chopped off mid-sentence by token limits
, scoring zero reward and poisoning gradient updates.
Latent-GRPO
asks the obvious question:
Why should an artificial neural network think in human words?
By replacing discrete vocabulary tokens with continuous recurrent thought vectors in embedding space, models can reason internally across continuous manifold dimensions.
Start here
Understand why discrete Chain-of-Thought creates massive rollout bottlenecks, how continuous thought recurrence works in embedding space, how to solve the RL exploration dilemma with tractable action densities, and how two-pass gradient replay enables exact backpropagation.
Embedding:
A dense vector of real numbers representing a concept in continuous semantic space.
Hidden state:
The internal activation vector produced by intermediate transformer layers.
Latent thought step:
A recurrent model cycle where continuous vectors loop internally without emitting vocabulary logits.
GRPO:
Group Relative Policy Optimization: policy gradients computed relative to a group of sampled attempts.
Two-pass replay:
Collecting rollouts in eval mode with active KV cache, followed by an exact gradient forward pass.
1. The Token Tax: How Textual Chain-of-Thought Kills Rollouts
In discrete Chain-of-Thought (CoT), reasoning is forced through an artificial bottleneck: the model’s vocabulary projection layer. For a modern foundation model like
Qwen3.6-27B
:
The Autoregressive Penalty:
Every single intermediate thought word requires a full forward pass through 64 transformer layers, a massive projection across 152,064 vocabulary logits, and an addition to the active KV-cache state.
Context Window Choke:
In agent tasks, environment observations and schema definitions already consume 1,500 to 3,000 tokens. Generating another 3,000 tokens of internal soliloquy per turn blows right past sequence limits.
The Truncation Disaster:
In our
gft-studio
multi-hop routing runs with discrete CoT,
19% to 34% of all generated trajectories hit the sequence ceiling mid-sentence
. In reinforcement learning, a truncated completion fails its verifier and receives a reward of exactly 0. Your trainer ends up penalizing a model not because its reasoning was wrong, but because it didn’t finish talking before the bell rang.
Continuous latent thinking eliminates this bottleneck: instead of projecting hidden states onto discrete words, the model feeds its internal recurrent representations directly back into itself in continuous space (
d
= 5,120):
The short version of the same two paths. Left, the state is squeezed through one token. Right, the full vector is passed on.
h
t
=
f
θ
(
c
o
n
t
e
x
t
,
z
<
t
)
,
z
t
=
P
ϕ
(
h
t
)
h_t=f_\theta(\mathrm{context},z_{<t}),\qquad z_t=P_\phi(h_t)
h
t
​
=
f
θ
​
(
context
,
z
<
t
​
)
,
z
t
​
=
P
ϕ
​
(
h
t
​
)
Here
f
θ
represents the transformer backbone,
z
<t
are past latent thought vectors, and
P
φ
is a projection layer mapping the last hidden state back to the model’s input embedding space.
Example numbers, not a measured forward pass. The sizes are the real ones: hidden state 5,120, vocabulary 152,064. Softmax turns logits into probabilities, and one sampled token is all the next step receives.
Same drawing as a recurrent net, unrolled. The projector writes the hidden state back into the input embedding of the next step. Every box is the same transformer.
2. The Core Dilemma: Why Continuous Policy Gradients Are Hard
If continuous reasoning is so obviously superior, why isn’t everyone doing it? The obstacle is not deep learning architecture—it is the foundational mathematics of policy gradients.
In
Group Relative Policy Optimization (GRPO)
, the trainer samples a group of
G
candidate trajectories {o₁, o₂, ..., o_G} from an old policy π
θ_old
for a given prompt
q
. The environment computes scalar task rewards
r
i
, which are normalized across the group into advantages:
A
i
=
r
i
−
mean
⁡
(
r
)
std
⁡
(
r
)
+
1
0
−
8
A_i = \frac{r_i - \operatorname{mean}(r)}{\operatorname{std}(r) + 10^{-8}}
A
i
​
=
std
(
r
)
+
1
0
−
8
r
i
​
−
mean
(
r
)
​
The policy network is then optimized using the clipped surrogate objective:
L
G
R
P
O
(
θ
)
=
E
[
1
G
∑
i
=
1
G
∑
t
=
1
T
min
⁡
(
π
θ
(
a
t
∣
s
t
)
π
θ
o
l
d
(
a
t
∣
s
t
)
A
i
,
clip
⁡
(
π
θ
(
a
t
∣
s
t
)
π
θ
o
l
d
(
a
t
∣
s
t
)
,
1
−
ϵ
,
1
+
ϵ
)
A
i
)
]
\mathcal{L}_{\mathrm{GRPO}}(\theta) = \mathbb{E}\left[ \frac{1}{G}\sum_{i=1}^G \sum_{t=1}^T \min\left( \frac{\pi_\theta(a_t\mid s_t)}{\pi_{\theta_{\mathrm{old}}}(a_t\mid s_t)} A_i,\; \operatorname{clip}\left(\frac{\pi_\theta(a_t\mid s_t)}{\pi_{\theta_{\mathrm{old}}}(a_t\mid s_t)},\, 1-\epsilon,\, 1+\epsilon\right) A_i \right) \right]
L
GRPO
​
(
θ
)
=
E
[
G
1
​
i
=
1
∑
G
​
t
=
1
∑
T
​
min
(
π
θ
old
​
​
(
a
t
​
∣
s
t
​
)
π
θ
​
(
a
t
​
∣
s
t
​
)
​
A
i
​
,
clip
(
π
θ
old
​
​
(
a
t
​
∣
s
t
​
)
π
θ
​
(
a
t
​
∣
s
t
​
)
​
,
1
−
ϵ
,
1
+
ϵ
)
A
i
​
)
]
This formulation relies on two absolute requirements:
Stochastic Exploration:
The policy must sample distinct candidate rollouts
o
i
≠ o
j
so that the advantage
A
i
can tell winners from losers.
Tractable Action Densities:
The trainer must be able to compute the exact log-probability log π
θ
(a
t
| s
t
) of the sampled action under both current and old weights to form the importance sampling ratio π
θ
/ π
old
.
3. The Research Landscape: How to Add Noise to Thought
Every major research paper in continuous reasoning over the last two years is essentially an answer to one question:
Where do we inject randomness into the thoughts, and how do we calculate the probability density of that noise?
Approach Family
How It Works
Is It Stochastic?
How Gradients Flow
Key Papers
A. Hidden-State Recurrence
Last-layer hidden state
h
t
loops back as input
Deterministic (no noise)
Answer-only GRPO or Monte Carlo surrogates
Coconut (2024), CODI (2025), Switch (2026), SLPO (2026)
B. Soft Tokens (Vocab Mixture)
Weighted combination of top-
K
word embeddings with Gumbel noise
Yes (Gumbel perturbations)
Gumbel log-density or soft-label surrogates
SofT-GRPO (2025), Latent-SFT (2025), LEPO (2026), Latent-GRPO (2026)
C. Parameterized Latent Head
Explicitly samples a vector from a continuous distribution (Gaussian)
Yes (reparameterization trick)
Exact analytical Gaussian log-density
CoLaR (NeurIPS 2025), Soft Latent Thinking (2026)
D. Hybrid Discrete-Latent
Gated blend of a sampled discrete token and projected state
Yes (from discrete token)
Token categorical density
HRPO (NeurIPS 2025)
3.1 SofT-GRPO: Treating Gumbel Perturbations as the Action
In SofT-GRPO (Zheng & Lee, arXiv:2511.06411), the model forms “soft tokens” by perturbing top-
K
logits with Gumbel noise:
g′
i
= log
p
i
+ ε
i
. The input embedding is the softmax-weighted superposition of those top embeddings.
The breakthrough insight:
the action is the noise vector ε
t
itself
, not the continuous embedding. This allows computing exact closed-form importance sampling updates through continuous thought steps.
3.2 Latent-GRPO: Stabilizing the Mixture
Building on soft tokens, Latent-GRPO (Deng et al., 2026) adds three practical guardrails:
Dead-Sample Masking:
Trajectories that fail to finish within the turn budget are immediately excised from group variance calculations.
One-Sided Perturbations (ξ > 0):
Ensures that noise boosts promising paths without accidentally degrading probabilities on valuable tokens.
Winner-Take-All Reinforcement:
When multiple rollouts in a group solve the problem, only the top-scoring trajectory’s initial latent vector receives positive gradient reinforcement, avoiding muddy representation drift.
3.3 CoLaR: Direct Gaussian Latent Heads
Instead of constraining thoughts to the vocabulary hull, CoLaR (Tan et al., NeurIPS 2025) equips the transformer with an MLP head that predicts mean and variance (μ
t
, σ
t
) directly in representation space:
z
=
μ
θ
(
s
)
+
σ
ϵ
,
ϵ
∼
N
(
0
,
I
)
z=\mu_\theta(s)+\sigma\epsilon,\qquad \epsilon\sim\mathcal N(0,I)
z
=
μ
θ
​
(
s
)
+
σ
ϵ
,
ϵ
∼
N
(
0
,
I
)
Sampling
z
t
= μ
t
+ σ
t
⊙ ε yields a clean, closed-form analytical Gaussian log-density:
log
⁡
π
θ
(
z
∣
s
)
=
−
1
2
∑
j
[
(
z
j
−
μ
θ
,
j
(
s
)
σ
j
)
2
+
2
log
⁡
σ
j
+
log
⁡
(
2
π
)
]
\log \pi_\theta(z\mid s)=-\frac{1}{2}\sum_j\left[\left(\frac{z_j-\mu_{\theta,j}(s)}{\sigma_j}\right)^2+2\log\sigma_j+\log(2\pi)\right]
lo
g
π
θ
​
(
z
∣
s
)
=
−
2
1
​
j
∑
​
[
(
σ
j
​
z
j
​
−
μ
θ
,
j
​
(
s
)
​
)
2
+
2
lo
g
σ
j
​
+
lo
g
(
2
π
)
]
4. Systems Engineering: The Two-Pass Gradient Replay
In our research platform (
gft-studio
), we implemented this progression step-by-step:
L0 (Deterministic Loop, Answer-Only GRPO):
A simple residual
ThoughtProjector
feeds the last hidden state back:
e
t
+1
=
e
t
+ Projector(
h
t
). The model takes 8 deep thought steps silently. GRPO gradients only optimize the subsequent discrete action tokens.
L1 (Dynamic Thought Budgets):
The model learns boundary tokens allowing it to decide dynamically whether a problem needs 2 latent cycles or 12.
L2b (Gaussian Latent Policy):
The projector predicts Gaussian parameters, allowing full policy gradient backpropagation through the internal thought cycles.
The Engineering Wall: Two-Pass Gradient Replay
Here is a practical systems trap that cost us days of debugging:
gradient checkpointing deletes KV caches inside checkpointed transformer layers
. If you try to extend an existing cached sequence during backward passes in PyTorch, you get violent tensor shape mismatch errors between cached keys and attention masks.
We solved this with a
Two-Pass Replay Architecture
:
Pass 1 (Thought Generation in Eval Mode):
Recompute the exact continuous thought vectors without gradients using an active KV cache (mirroring rollout inference).
Pass 2 (Unified Embedding Forward):
Stitch the tensors into a single unified sequence
[prompt_embeds, thought_embeds, answer_embeds]
and run a standard forward pass with gradient checkpointing, computing exact loss and gradients across all answer tokens in one shot.
5. Benchmarks: Discrete CoT vs. Latent Thinking on Qwen3.6-27B
We pitted discrete Chain-of-Thought against continuous latent thinking in a controlled 4-arm study on a complex
multi-hop-routing-gym
using
Qwen3.6-27B
with QLoRA on NVIDIA H100 and H200 clusters:
Experimental Arm
Rollout Engine & Silicon
Completed Steps
Avg Generation Time / Step
Avg Total Step Time
Avg Emitted Tokens
1. Discrete CoT (Visible <think>)
Hugging Face / 1x H100
5 / 50 (Aborted: Too Slow)
1,878.4 s (~31.3 min)
1,932.7 s
~4,131 tokens
2. No-Think Baseline (Direct Action)
Hugging Face / 1x H100
50 / 50 (SUCCESS)
690.6 s (~11.5 min)
722.8 s
~680 tokens
3. Discrete CoT (Visible <think>)
vLLM + Graphs / 1x H100
50 / 50 (Parity Failed)
612.4 s (~10.2 min)
660.7 s
~4,050 tokens
4. Latent Thinking (L2b Continuous Loop)
vLLM + Graphs / 1x H200
50 / 50 (SUCCESS; Parity Passed)
132.7 s (~2.2 min)
171.2 s
~688 tokens
What the Data Actually Tells Us
The 14x Speedup Dissected:
In our initial runs, the Latent L2b arm finished rollouts in 132.7 seconds compared to 1,878.4 seconds for Hugging Face Discrete CoT—a staggering 14.2x speedup. To be completely honest, that jump comes from two combined factors: moving from H100 to H200 memory bandwidth (4.8 TB/s) and compressing sequence length from ~4,131 chatty tokens down to ~688 action tokens.
Continuous Thoughts Consume Zero Token Budget:
This is the true superpower. While the discrete CoT arm repeatedly truncated its second reasoning cycle and forfeited its reward, the Latent arm executed 8 deep recurrent thought cycles without consuming a single token from the 1,024-token sequence ceiling.
Total Training Wallclock:
Over the full 50-step run, Latent L2b finished all training in
2.38 hours of total cluster time
, compared to a projected 26+ hours for discrete CoT.
6. The Road Ahead: Pure Latent Cognition
Teaching language models to reason by making them write endless stream-of-consciousness essays was a brilliant stepping stone. But as agents tackle multi-step tools, deep code refactors, and complex simulations, forced textualization becomes an unacceptable bottleneck.
The future of agent post-training isn’t models that talk faster. It is models that think silently, deeply, and continuously in latent space—and only speak when they have something to say.
Applying this to your own models?
We post-train open models on teams' own tasks with SFT and GRPO, and hand back the weights.
LLM post-training with g factor
.
Originally published at
g-ftech.com
.