AdvancedAI
Tencent’s DSpine Drafter Lifts Qwen3.8-27B Throughput 3.19x
- inference
- llm
- open source
- qwen
- speculative decoding
Tencent published a speculative-decoding draft model for Qwen3.8-27B on Hugging Face on November 7, 2026, and says it lifts end-to-end agent throughput to 3.19 times autoregressive decoding on SWE-bench Pro replays. The checkpoint, tencent/Qwen3.8-27B-DSpine-b16, is a five-layer drafter of about 2.03 billion parameters with block size 16 and an 8,192-token sliding window, released under Apache 2.0. Its method, called DSpine, changes where a parallel drafter gets its conditioning: instead of waiting until after the backbone, it passes each position’s predicted feature to its immediate successor after every layer.
The problem with drafting in parallel
Speculative decoding speeds up a large model by having a small one guess several tokens ahead, then checking those guesses in a single pass of the large model. The guesses are only useful if they are accepted, so the metric that matters is acceptance length: how many tokens survive verification. A parallel drafter produces a whole block of candidates in one backbone pass, which is fast, but it also means every candidate is chosen before its predecessors are known. Independent selection produces continuations that do not fit together, and the accepted prefix gets shorter.
Existing methods handle this in two ways. DFlash, the drafter family this work builds on, does not use the predecessor at all. Domino and DSpark add a lightweight head after the last layer that conditions each token on the one before it. DSpine’s authors probed the DFlash drafter and found two things: early positions in a block already form recoverable predictions in shallow layers, and a correct adjacent predecessor helps a successor far more when it arrives earlier. In their measurements, supplying correct prefix features from an early layer raised consecutive correct tokens by 6.7 percent, and moving the same number of injections earlier lifted that gain from 1.2 to 4.3 percent. Swapping only the predecessor for its draft prediction lowered successor accuracy by 31.8 points.
How DSpine works
After every layer, each position receives a gated message from its immediate predecessor only. Because attention is block-causal, that dependency chain unfolds over network depth while every layer still updates all positions in parallel. The message is not the predecessor’s hidden state, which mixes prediction with context; it is the prediction itself, expressed in a fixed transfer space built from the target model’s output embeddings. An undecided predecessor sends a predicted feature, and a decoded token enters the same pathway as its embedding, so one route carries either.
Once a token is decoded, its embedding replaces the predicted feature and the last write is redone, with no separate module. A transition cache scores all candidate predecessor-successor pairs in one parallel pass, turning selection into a lookup. Layer-wise supervision aligns each layer’s predicted feature by cosine similarity with the correct token’s embedding, which pushes useful features to form in shallow layers. The accounting matters for a latency technique: the paper reports at most 0.15 milliseconds per round for injection, and 0.36 milliseconds for the post-backbone head against 0.97 for Domino and 0.88 for DSpark.
What Tencent measured
The cleanest comparison is on Qwen3-8B, where every drafter was retrained with the same five-layer, block-16, six-epoch budget and then evaluated in SGLang on seven math, code and chat benchmarks:
| Drafter (Qwen3-8B) | Mean acceptance, T=0 | Mean acceptance, T=1 |
|---|---|---|
| DFlash | 5.92 | 4.94 |
| Domino | 7.05 | 5.59 |
| DSpark | 7.44 | 6.65 |
| DFlash2 | 7.55 | 6.53 |
| DSpine | 7.83 | 6.98 |
For the 27B target, the released checkpoint is compared against the official DFlash2 drafter, not against a matched retraining, and the two use different block sizes: 16 for DSpine and 8 for DFlash2. Acceptance length rises on every general benchmark, from 5.69 to 7.20 on GSM8K at temperature 0 and from 5.49 to 6.49 on MATH-500, and on every agent suite Tencent reports.
| Agent replay (Qwen3.8-27B, xhigh) | Autoregressive | DFlash2 | DSpine |
|---|---|---|---|
| SWE-bench Pro | 248.17 tok/s | 711.60 (2.87×) | 791.30 (3.19×) |
| τ²-bench | 340.39 tok/s | 871.98 (2.56×) | 978.39 (2.87×) |
| WorkBuddy Office | 215.05 tok/s | 662.34 (3.08×) | 763.83 (3.55×) |
Those throughput figures come from replaying 20 recorded conversations per subset, with every tool-decision prefix replayed in order, prefix caching on, and up to 32,768 new tokens per request. At the lower reasoning setting, Tencent’s best row is 805.53 tokens per second on WorkBuddy Office, 3.84 times autoregressive. The best results in both tables are Tencent’s own.
The fine print
Three costs are worth stating alongside the speedups. First, this is a per-model drafter: 2.03 billion additional parameters in 4.05 GB of BF16 weights, trained for one Qwen checkpoint and useless for another. Second, it depends on a specific serving stack. The launch command requires SGLang 0.5.22 with Tencent’s plugin, FlashAttention 3, DSpark’s dynamic verification and ReplaySSM, the last of which replays the target model’s SSM state so that verification does not invalidate it. The README and project page also disagree on the shared training set used for the Qwen3-8B comparison, naming Open-PerfectBlend in one place and ShareGPT in the other. Third, the throughput numbers describe agent workloads with long generations and prefix caching, not ordinary chat serving; a short prompt gives a drafter far less to work with.
None of that undercuts the reported direction. On the matched-budget Qwen3-8B comparison, DSpine’s gain over DFlash2 comes from training a better drafter rather than a larger one, which is the comparison that matters when the technique is judged as a technique.
What This Means
Speculative decoding has been drifting from a general-purpose trick toward a per-model component. EAGLE-4 in October argued the opposite way: one drafter that works at every batch size without retuning. DSpine goes the other way, spending 2 billion parameters and a stack of verification dependencies to get acceptance length closer to the block size its drafter was trained for. For anyone serving a single model at scale, that trade can pay, and Apache 2.0 makes it easy to test. For anyone serving many models, it is another artifact to track per checkpoint.
This post was drafted with AI assistance and reviewed by RITS staff.
Sources
- tencent/Qwen3.8-27B-DSpine-b16 on Hugging Face — model card, evaluation tables and launch command
- Tencent/DSpine on GitHub — inference code and Qwen3-8B comparison table
- DSpine project page — method, ablation numbers and serving results
- Zhang et al., “Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding,” arXiv:2609.36173, September 28, 2026






