Qwen3.8-Flash-Next: Alibaba Previews the Qwen4 Architecture

Alibaba’s Qwen team has scheduled an open-weight release of Qwen3.8-Flash-Next for 23:00 on August 26, 2026 (UTC+08:00), describing it on ModelScope as “a multimodal MoE model built on the next-generation Qwen4 architecture.” The weights were not yet public at the time of writing. What has circulated instead is a specification block that appeared briefly on the model’s ModelScope page and was then removed — and if it is accurate, the interesting number in it is not the 125 billion parameters but the 51 billion sitting outside the transformer entirely.

Advanced

What Alibaba Has Actually Confirmed

Very little. The ModelScope teaser carries a countdown, the tagline “Onward to the Next-Gen — Lightning-Fast,” a chalkboard illustration with a partially turned page revealing a “4,” and the one-line description above. The matching Hugging Face repository is listed as an upcoming release for the same date. No parameter count, no context length, no benchmark table, and — notably — no license.

Everything more specific comes from a spec block that community members captured before it was pulled. As reported by Decrypt and reconstructed from those screenshots, it listed a 125B-parameter main model with roughly 6B parameters active per token, an additional 51B of N-gram embeddings, and two named architectural changes: GDN hybrid layers and something called Qwen Sparse Attention (QSA). There is no public technical description of QSA yet. Treat all of it as unverified until the model card lands.

The Hybrid Part Is Familiar

The “GDN hybrid” half is a continuation, not a surprise. Qwen3-Next, released in September 2025, interleaved Gated DeltaNet blocks with conventional gated attention — three linear-attention layers for every full-attention layer across 48 layers, with 512 routed experts plus one shared expert and 10 experts firing per token. The point is that Gated DeltaNet carries a fixed-size recurrent state instead of a KV cache that grows with sequence length, so per-token decode cost stays close to constant on long contexts.

That ratio has held as the family scaled. Community inspection of the Qwen3.8-2.4T-A95B checkpoint found 69 GDN layers against 23 attention layers — the same three-to-one pattern at twenty times the size. A Flash-Next built on the same principle is the expected move.

The N-Gram Table Is the New Part

The 51B figure is what makes the leaked spec worth attention. N-gram embeddings are a trainable lookup memory rather than a computed layer: the model hashes the last few tokens, retrieves a handful of vectors from a large table, and mixes them into the hidden state. Retrieval costs almost no FLOPs, so the parameters are effectively free at inference time in arithmetic terms — but not in memory terms. Every one of them has to be resident somewhere.

The idea has recent research behind it. Scaling Embeddings Outperforms Scaling Experts in Language Models (arXiv, February 2026) argues that spending a parameter budget on hashed n-gram embedding tables beats spending it on additional MoE experts, which inverts the assumption that has driven sparse-model design for the past three years.

For anyone planning to run this locally, that distinction is the whole ballgame. Practitioners on NVIDIA’s DGX Spark forum worked the arithmetic at 4-bit: roughly 58.2 GiB for the 125B main weights, 23.7 GiB for the n-gram table, about 82 GiB total — before accounting for the possibility that lookup tables tolerate aggressive quantization badly and ship at higher precision. As one participant put it: “Each token uses a tiny fraction of the lookup table, but all 51B parameters still need to be stored somewhere — in VRAM, RAM, or via a dedicated offload.” Throughput guesses in the thread ran to 50–60 tokens per second for a comparable 6–7B-active architecture, with consensus leaning toward two Spark units rather than one.

What This Means

Shipping an architecture preview before the flagship is a deliberate sequencing choice. A 125B-A6B model is small enough for vLLM, SGLang and llama.cpp to add GDN and QSA support against a real checkpoint, so that Qwen4 proper arrives into runtimes that already handle it — the same day-zero pattern Alibaba has been building toward with its Qwen3.8 launches. The trade is that a preview invites comparison it was not designed to win. The geometric-mean heuristic that circulates for MoE models, √(125 × 6) ≈ 27B, puts Flash-Next’s effective quality near Qwen3.8-27B, a model already released with published scores — while running considerably faster.

The open question is the license. Reuters reported on August 7, 2026 that Alibaba intends to introduce revenue-sharing terms for large commercial users of its next open-weight model, following Moonshot, which requires a commercial agreement from anyone serving Kimi K3 above $20 million in annual revenue. Whether those terms attach to Flash-Next or wait for Qwen4 will say more about the direction of Chinese open-weight releases than any benchmark in the model card.

Related Coverage

• Qwen3.8-27B: Frontier Agentic Scores on a Single Consumer GPU — the dense model Flash-Next is expected to approximate
• Qwen3.8-2.4T-A95B: Alibaba Open-Weights Its Max-Tier Flagship — the checkpoint whose 69:23 GDN-to-attention split community analysis measured
• Qwen3-Coder-Next: Alibaba’s Ultra-Sparse 80B Coding Agent — the previous “-Next” branded architecture experiment
• Kimi K3 Open Weights Ship: 2.8T Parameters, 1.4 TB to Run — the revenue-share licensing precedent

This post was drafted with AI assistance and reviewed by RITS staff.

Sources

• Qwen3.8-Flash-Next teaser page — ModelScope
• Qwen/Qwen3.8-Flash-Next — Hugging Face, listed as an upcoming release
• Alibaba to Release Qwen 3.8-Flash-Next as a Preview of What Qwen 4 Will Offer — Decrypt, August 25, 2026
• Qwen3.8-Flash-Next: Qwen4 Architecture Preview, What We Know — OrcaRouter
• vLLM Now Supports Qwen3-Next: Hybrid Architecture with Extreme Efficiency — vLLM, September 11, 2025
• New Open Source Qwen3-Next Models Preview Hybrid MoE Architecture — NVIDIA Technical Blog
• Scaling Embeddings Outperforms Scaling Experts in Language Models — arXiv:2601.21204, February 2026
• Qwen3.8-Flash-Next — NVIDIA DGX Spark / GB10 developer forum thread
• Alibaba Reportedly Plans Revenue-Sharing Terms for Next Qwen Model — TechNode, August 7, 2026