Meta Opens Muse Spark Weights: 816B MoE, 1M Context

On October 15, 2026, Meta Superintelligence Labs released the weights for Muse Spark Open — an 816-billion-parameter sparse mixture-of-experts model that activates 42 billion parameters per token, with a 1,048,576-token context window. Meta calls it “the most capable US open-weight model to date,” and the release is as much a hardware story as a benchmark one: Meta says the checkpoint runs on two Apple M5 Ultra Mac Studios linked with MLX.

What Meta Released

Muse Spark Open is the open-weight counterpart of Muse Spark, the proprietary frontier model Meta launched in April 2026, and shares the same base model as Muse Spark 1.3. It is natively multimodal — text, image, video and audio in, text out — and trained on data from more than 100 languages, with a separate 1.8B-parameter perception encoder alongside the language model.

The licence is not Apache 2.0. Muse Spark Open ships under the Muse Model License, dated October 15, 2026, which grants free use, modification, fine-tuning, distillation, redistribution and sale but attaches two conditions: products built on it must display “Built with Muse Spark Open” prominently, and anyone offering it as a Model-as-a-Service whose group gross revenue exceeds US$20 million in any 12-month period must notify Meta and sign a separate written agreement. Meta claims no ownership of outputs, and downloads on Hugging Face are gated behind a registration form requiring a legal name, date of birth and organisation.

Weights are on Hugging Face, Amazon SageMaker AI, Google Cloud Model Garden and Microsoft Foundry Models, with hosted APIs from seven partners including Together AI and Fireworks, and support across SGLang, vLLM, TensorRT-LLM, Apple MLX and llama.cpp.

Inside the Architecture

All 62 decoder layers are MoE layers, each holding 256 experts with top-8 routing. The vLLM team’s breakdown puts almost all parameter mass in expert weights: 799B in experts, 13.5B in attention, 4.2B in MoE latent projections and 3.3B in the embedding and output layers. Attention uses 128 query heads and 16 KV heads (grouped-query) over a hidden dimension of 8192. Two details differ from a vanilla decoder: “sandwich” normalisation, with a norm layer before and after each block, and latent MoE projections that pool expert inputs into a smaller space.

Meta’s new Helion kernel DSL ships auto-tunable GPU kernels in SGLang, reporting 1.03–3.95× kernel-level speedups on NVIDIA and AMD hardware. vLLM added day-0 support with expert parallelism, PD disaggregation and a trained DSpark draft model for speculative decoding.

Benchmarks and Hardware

At xhigh reasoning effort Meta reports Terminal-Bench 2.1 of 76.4, MCP-Atlas 89.2, JobBench 61.8, GDPVal-AA v2 of 1401, GPQA Diamond 89.0, HLE-Diamond 26.1 without tools and 50.0 on ZeroBench with tools. Artificial Analysis independently scores it 39 on its Intelligence Index — 9th of 118 models — at about US$1.25 per million input tokens and US$4.25 per million output tokens, at 102 output tokens per second.

Against other open models in Meta’s own table that placement is competitive rather than dominant: Qwen3.8-2.4T-A95B scores 40, Kimi K3 44 and GLM 5.3 45, while Muse Spark Open leads the group on JobBench and MCP-Atlas. Memory is the real barrier: BF16 weights total about 1.63 TB and the model card asks for at least 2 TB of VRAM, or 700 GB for the NVFP4 release. Unsloth’s Dynamic GGUFs shrink that to a 2-bit quant targeting 250–316 GB of total memory, or 180–212 GB at 1-bit.

What This Means

The interesting part is the shape of the release, not the headline score. Meta is shipping a frontier-adjacent checkpoint under terms that are permissive for researchers and small developers but explicitly commercial-restrictive at the Model-as-a-Service scale — a middle path between Apache 2.0 and a closed API. Gating downloads behind an identity form departs further from open-weight convention.

For local-inference developers, a 2-bit quant that fits in 256 GB of unified memory is the first realistic route to running a 256-expert model on a workstation, and it keeps pressure on the assumption that only Chinese labs publish competitive open weights. Meta also reports extra safety post-training for the open release, claiming 99.2% refusal on its internal jailbreak benchmark, and states the model is not intended for users under 18.

This post was drafted with AI assistance and reviewed by RITS staff.

Sources

• Introducing Muse Spark Open, the Most Capable US Open-Weight Model to Date — Meta AI (October 15, 2026)
• Muse-Spark-Open-816B-A42B model card and Muse Model License — Hugging Face
• Muse Spark Open 816B-A42B quantised release — Unsloth on Hugging Face
• Meta Muse Spark Open — Run Locally Guide — Unsloth documentation
• Day 0 support for Muse Spark Open on vLLM — vLLM team
• Muse Spark Open (xhigh) intelligence, performance and price analysis — Artificial Analysis
• Muse Spark Is Now an Open-Weights Model — PyTorch Team at Meta
• Meta releases Muse Spark Open, a new ‘open-weight’ AI model — Agence France-Presse via The Guardian