DLoop: Looped Speculative Decoding Cuts Target-Model Passes

NAVER AI Lab has proposed DLoop, a change to speculative decoding that lets a single verification pass cover several rounds of drafting instead of just one. In a preprint posted to arXiv on October 6, 2026, the team reports wall-clock speedups 5 to 41 percent higher than the draft models it builds on — EAGLE-3, DFlash, Domino, DSpark, and the multi-token prediction (MTP) modules shipped with Qwen3.5-9B and Gemma-4-12B — while leaving the target model’s output distribution unchanged. The method adds no parameters and no extra modules; it only changes how many drafting stages precede a verification.

Advanced

The Problem: A Verification After Every Draft

Speculative decoding pairs a target model with a lightweight draft model. The draft model proposes a block of k tokens, and the target model verifies them in a single forward pass, accepting them under a rule that preserves its output distribution. Because verification requires a full target-model forward pass, it dominates decoding time — and the standard pipeline runs a verification after every drafting stage, even when the draft model could clearly have kept going.

That assumption has aged poorly. As draft models improve, a large share of drafting stages are now accepted in full. Across four drafters and three task categories, the paper measures full acceptance rates ranging from roughly 10 percent to more than 55 percent, with DSpark the most accurate of the four. Earlier work tried to fix this with adaptive draft-length methods, which decide mid-decoding how many draft tokens precede a verification. The authors argue these only help autoregressive draft models: a parallel drafter produces its whole block in one forward pass, so stopping early does not save any compute, and drafting further would require target-model hidden states that do not exist yet for unverified tokens.

How DLoop Works

DLoop keeps a full drafting stage as the minimum unit and then decides whether another stage follows. After each stage it sums the log probabilities the draft model assigned to the block’s tokens. Because every log probability is non-positive, one uncertain position drags the score down — matching the acceptance rule, which discards everything after the first rejection. If the summed score stays above a threshold g, the loop drafts again; otherwise the target model verifies all accumulated tokens at once. Since the log probabilities are already computed during drafting, the gate costs only a summation and a comparison. The threshold is the loop’s sole hyperparameter; the authors sweep it over {−0.25, −0.5, −0.75, −1.0} and settle on −0.5 for all settings except EAGLE-3 with Qwen3-8B, where they use −0.75.

The second component, loop-aware training, addresses a subtlety: additional drafting stages have no target-model hidden states to condition on, because the intervening tokens have not been verified. DLoop substitutes the draft model’s own final-layer hidden states, and then trains the drafter to expect them. This extends the unrolling trick from EAGLE-3 — which exposes the drafter to its own predicted features within one drafting stage — across the drafting stages a single verification covers. The training loss combines the first and last stages in the loop, and the intermediate stages supply hidden states without contributing to the loss. Training starts from each drafter’s publicly released weights and runs for one epoch on 1.3 million prompts from the PerfectBlend dataset. The verification rule itself is untouched, so decoding stays lossless.

Results

Averaged over eight benchmarks in math, code, and chat, DLoop raises the acceptance length by 12 to 102 percent and wall-clock speedup by 5 to 41 percent over the released draft models. DSpark gains the most: with Qwen3-4B it moves from an average acceptance length of 5.47 and 3.77× speedup to 8.59 and 4.76×, and with Qwen3-8B from 5.58 / 3.81× to 9.08 / 4.82×. EAGLE-3 with Qwen3-4B rises from 6.92 / 3.35× to 9.63 / 3.77×, DFlash from 6.10 / 4.19× to 7.21 / 4.67×, and Domino from 7.08 / 4.38× to 7.90 / 4.58×. The MTP modules gain most in relative terms: Qwen3.5-9B’s goes from 3.32 / 1.55× to 6.70 / 2.18×, a doubling of acceptance length.

Total decode time falls by 4.9 to 24.3 percent across the four drafters, with 34 to 71 percent of generated tokens coming from loops that cover more than one drafting stage. The number of verifications drops by 7.6 to 34.1 percent, and average time per output token falls from 4.60–6.42 ms at one stage to 0.68–2.97 ms at five or more. Against adaptive draft-length baselines, no method beat the base DSpark, and on EAGLE-3 the seven compared methods improved speedup by at most 0.14×. DLoop also composes with tree verification, adding 7 to 25 percent on top of DARTree for Domino and DSpark. All runs used NVIDIA H100 80GB GPUs in bf16 precision.

What This Means

The appeal of DLoop is that it is orthogonal to the drafter. It does not retrain an architecture, add a prediction head, or change the acceptance rule — it sits on top of released weights and turns “draft, verify, draft, verify” into “draft until unsure, then verify once.” That makes it composable with the tree-verification and parallel-drafting lines of work that have driven most of the recent gains, rather than competing with them.

The caveat is where it does not pay. The gains track full-acceptance rates, so the smallest improvements land in chat, where drafters are least reliable — EAGLE-3 with Qwen3-4B moves only from 2.38× to 2.47×. DLoop also spends extra draft-model passes to save target-model passes, a trade that only works because verification is the expensive half of the loop; on hardware where the target model is small or the drafter is unusually heavy, the arithmetic could invert. For local and edge inference, where a single large target model is the bottleneck, the direction is attractive. The code is not yet available — the project’s GitHub repository notes it is under internal review — so the numbers remain the authors’ own for now.

Related Coverage

• JetSpec: Causal Parallel Tree Drafting Hits 9.64x Faster LLM Inference — a tree-based drafting method that also leaves model outputs unchanged.
• DFlash: Block Diffusion Delivers 6x Faster LLM Inference — the parallel drafter DLoop builds on and beats in its own benchmarks.
• Gemma 4 Gets Multi-Token Prediction Drafters — the MTP family that DLoop retrains for its loop.
• Luce DFlash Brings 2x Speculative Decoding to Qwen3.6-27B on a Single RTX 3090 — a consumer-GPU port of the same drafting approach.
• NInfer: 700 tok/s on One RTX 5090 — a single-GPU inference engine that also leans on speculative decoding.

This post was drafted with AI assistance and reviewed by RITS staff.

Sources

• DLoop: Looped Speculative Decoding — arXiv:2610.07659 (Geonmo Gu, Byeongho Heo, HeeJae Jun, Yoohoon Kang, Sangmin Lee, Sangdoo Yun, Dongyoon Han; NAVER AI Lab, NAVER AI Search Platform, Korea University; submitted October 6, 2026)
• DLoop: Looped Speculative Decoding — Hugging Face Papers
• naver-ai/DLoop — official repository (code forthcoming)