AdvancedAI

Tencent’s DSpine Drafter Lifts Qwen3.8-27B Throughput 3.19x

Tencent published a speculative-decoding draft model for Qwen3.8-27B on Hugging Face on November 7, 2026, and says it lifts end-to-end agent throughput to 3.19 times autoregressive decoding on SWE-bench Pro replays. The checkpoint, tencent/Qwen3.8-27B-DSpine-b16, is a five-layer drafter of about 2.03 billion parameters with block size 16 and an 8,192-token sliding window, released under Apache 2.0. Its method, called DSpine, changes where a parallel drafter gets its conditioning: instead of waiting until after the backbone, it passes each position’s predicted feature to its immediate successor after every layer.

Architecture diagram: predecessor features are injected into successor positions at every layer of the draft backbone, followed by predecessor-conditioned decoding
DSpine’s adjacent causal injection passes predecessor features to successors at every layer. Image credit: Tencent DSpine project page

The problem with drafting in parallel

Speculative decoding speeds up a large model by having a small one guess several tokens ahead, then checking those guesses in a single pass of the large model. The guesses are only useful if they are accepted, so the metric that matters is acceptance length: how many tokens survive verification. A parallel drafter produces a whole block of candidates in one backbone pass, which is fast, but it also means every candidate is chosen before its predecessors are known. Independent selection produces continuations that do not fit together, and the accepted prefix gets shorter.

Existing methods handle this in two ways. DFlash, the drafter family this work builds on, does not use the predecessor at all. Domino and DSpark add a lightweight head after the last layer that conditions each token on the one before it. DSpine’s authors probed the DFlash drafter and found two things: early positions in a block already form recoverable predictions in shallow layers, and a correct adjacent predecessor helps a successor far more when it arrives earlier. In their measurements, supplying correct prefix features from an early layer raised consecutive correct tokens by 6.7 percent, and moving the same number of injections earlier lifted that gain from 1.2 to 4.3 percent. Swapping only the predecessor for its draft prediction lowered successor accuracy by 31.8 points.

How DSpine works

After every layer, each position receives a gated message from its immediate predecessor only. Because attention is block-causal, that dependency chain unfolds over network depth while every layer still updates all positions in parallel. The message is not the predecessor’s hidden state, which mixes prediction with context; it is the prediction itself, expressed in a fixed transfer space built from the target model’s output embeddings. An undecided predecessor sends a predicted feature, and a decoded token enters the same pathway as its embedding, so one route carries either.

Once a token is decoded, its embedding replaces the predicted feature and the last write is redone, with no separate module. A transition cache scores all candidate predecessor-successor pairs in one parallel pass, turning selection into a lookup. Layer-wise supervision aligns each layer’s predicted feature by cosine similarity with the correct token’s embedding, which pushes useful features to form in shallow layers. The accounting matters for a latency technique: the paper reports at most 0.15 milliseconds per round for injection, and 0.36 milliseconds for the post-backbone head against 0.97 for Domino and 0.88 for DSpark.

What Tencent measured

The cleanest comparison is on Qwen3-8B, where every drafter was retrained with the same five-layer, block-16, six-epoch budget and then evaluated in SGLang on seven math, code and chat benchmarks:

Drafter (Qwen3-8B) Mean acceptance, T=0 Mean acceptance, T=1
DFlash 5.92 4.94
Domino 7.05 5.59
DSpark 7.44 6.65
DFlash2 7.55 6.53
DSpine 7.83 6.98

For the 27B target, the released checkpoint is compared against the official DFlash2 drafter, not against a matched retraining, and the two use different block sizes: 16 for DSpine and 8 for DFlash2. Acceptance length rises on every general benchmark, from 5.69 to 7.20 on GSM8K at temperature 0 and from 5.49 to 6.49 on MATH-500, and on every agent suite Tencent reports.

Agent replay (Qwen3.8-27B, xhigh) Autoregressive DFlash2 DSpine
SWE-bench Pro 248.17 tok/s 711.60 (2.87×) 791.30 (3.19×)
τ²-bench 340.39 tok/s 871.98 (2.56×) 978.39 (2.87×)
WorkBuddy Office 215.05 tok/s 662.34 (3.08×) 763.83 (3.55×)

Those throughput figures come from replaying 20 recorded conversations per subset, with every tool-decision prefix replayed in order, prefix caching on, and up to 32,768 new tokens per request. At the lower reasoning setting, Tencent’s best row is 805.53 tokens per second on WorkBuddy Office, 3.84 times autoregressive. The best results in both tables are Tencent’s own.

The fine print

Three costs are worth stating alongside the speedups. First, this is a per-model drafter: 2.03 billion additional parameters in 4.05 GB of BF16 weights, trained for one Qwen checkpoint and useless for another. Second, it depends on a specific serving stack. The launch command requires SGLang 0.5.22 with Tencent’s plugin, FlashAttention 3, DSpark’s dynamic verification and ReplaySSM, the last of which replays the target model’s SSM state so that verification does not invalidate it. The README and project page also disagree on the shared training set used for the Qwen3-8B comparison, naming Open-PerfectBlend in one place and ShareGPT in the other. Third, the throughput numbers describe agent workloads with long generations and prefix caching, not ordinary chat serving; a short prompt gives a drafter far less to work with.

None of that undercuts the reported direction. On the matched-budget Qwen3-8B comparison, DSpine’s gain over DFlash2 comes from training a better drafter rather than a larger one, which is the comparison that matters when the technique is judged as a technique.

What This Means

Speculative decoding has been drifting from a general-purpose trick toward a per-model component. EAGLE-4 in October argued the opposite way: one drafter that works at every batch size without retuning. DSpine goes the other way, spending 2 billion parameters and a stack of verification dependencies to get acceptance length closer to the block size its drafter was trained for. For anyone serving a single model at scale, that trade can pay, and Apache 2.0 makes it easy to test. For anyone serving many models, it is another artifact to track per checkpoint.

This post was drafted with AI assistance and reviewed by RITS staff.

Sources

Related stories

eagle

Eagle Ships EAGLE-4: One Drafter for Every Batch Size

Eagle released EAGLE-4 on October 20, 2026, the fourth generation of its speculative decoding family, built around a single claim: one trained drafter can serve a lone request and a fully loaded server without being retrained or hand-tuned. The release ships code, a paper, and plugins for three inference engines, and reports gains that hold […]

benchmark

DLoop: Looped Speculative Decoding Cuts Target-Model Passes

NAVER AI Lab has proposed DLoop, a change to speculative decoding that lets a single verification pass cover several rounds of drafting instead of just one. In a preprint posted to arXiv on October 6, 2026, the team reports wall-clock speedups 5 to 41 percent higher than the draft models it builds on — EAGLE-3, […]

AI

DFlash: Block Diffusion Delivers 6x Faster LLM Inference

DFlash is a new speculative decoding framework that uses block diffusion models to generate draft tokens in parallel rather than sequentially, achieving over 6× lossless acceleration on large language models — up to 2.5× faster than the previous state-of-the-art method EAGLE-3. The paper was published in February 2026 by Jian Chen, Yesheng Liang, and Zhijian […]

alibaba

Luce DFlash Brings 2x Speculative Decoding to Qwen3.6-27B on a Single RTX 3090

Luce DFlash now runs Qwen3.6-27B on a single RTX 3090, delivering up to 2× throughput over autoregressive decoding. The Lucebox Hub project, which hand-tunes LLM inference for specific consumer GPUs, ported its DFlash speculative decoding stack to Qwen3.6-27B in late April 2026. Because Qwen3.6-27B reuses the Qwen3.5 architecture string with identical layer and head dimensions, […]

Never miss a RITS update

Subscribe to the RSS feed for AI, emerging technology, and digital scholarship news curated for the NYU Shanghai community, or get in touch to suggest a story.