SK Telecom Opens A.X K2: 688B Parameters and One Telling Benchmark

July 29, 2026Provided by Utku Ege Tuluk

SK Telecom released A.X K2 on July 29, 2026 — a 688-billion-parameter Mixture-of-Experts model published on Hugging Face under Apache 2.0, and the largest model yet to come out of Korea’s government-backed sovereign AI program. The headline numbers are strong: 97.1 on AIME26, 80.5 on KMMLU-Pro, and a wall of perfect 100s on needle-in-a-haystack retrieval out to 512K tokens. The more interesting number is 9.3.

Advanced

What Shipped

A.X K2 is a decoder-only MoE transformer: 688B total parameters with 33B active per token, routed across 256 experts plus one shared expert, 8 active per forward pass. It runs 61 layers (1 dense, 60 MoE), 64 attention heads, a hidden size of 7,168, and a 163,840-token vocabulary. Context length is 262,144 tokens — trained natively to 128K and extended to 256K via YaRN.

Pre-training used roughly 8.2 trillion tokens across a three-stage curriculum, weighted 72.7% English, 15.4% Korean, and 8.3% code, with Japanese, Spanish, and Chinese at about 1% each. Post-training added two supervised fine-tuning stages, multi-task RL, and a safety and red-teaming RL pass, consuming about 60.6B tokens of paired examples. The model uses a “Think-Fusion” recipe that puts reasoning-heavy thinking mode and fast non-thinking mode in a single checkpoint.

The most notable engineering choice is precision: A.X K2 was trained from scratch natively in FP8 (MXFP8, E4M3) for both forward and backward passes, and the released checkpoints ship in block-scaled FP8. That is still uncommon at this scale, where BF16 training with FP8 inference remains the default.

Sparse Gated Attention

SK Telecom credits its gains to a custom attention design it calls Sparse Gated Attention. SGA layers two mechanisms on top of Multi-head Latent Attention. A lightweight indexer scores and ranks past positions, and a selector keeps only the top-k tokens per query — k = 2048 — so attention cost stops scaling with full sequence length. Separately, a head-specific output gate is applied throughout pre-training before the output projection, which suppresses the attention-sink artifacts that otherwise show up as activation outliers.

Diagram of the A.X K2 transformer block showing GatedNorm feeding a Sparse Gated Attention sublayer, with an indexer and top-k selector supplying key-value candidates to multi-head latent attention, and a sigmoid output gate applied before the output projection — Image credit: SK Telecom / Hugging Face

Those outliers are the reason for the second component, Gated Norm, which applies input-dependent gates after RMSNorm. Taken together, the two changes are less about raw quality than about keeping activation ranges tight enough that FP8 training converges and FP4 deployment stays viable. SK Telecom reports NVFP4 experts-only W4A4 quantization holding up without measurable retrieval loss.

Serving it is not cheap. The reference vLLM deployment uses 4× B300 GPUs, with FP8 weights occupying roughly 656 GiB of aggregate GPU memory. First launch takes about 25 minutes including kernel warmup; subsequent launches drop to about 6 minutes with cached kernels.

Reading the Benchmarks

On math, A.X K2 posts 97.1 on AIME26 and 92.5 on the first round of KMO26, and SK Telecom reports 35/42 on IMO 2025 — above the gold-medal threshold. On Korean-language evaluations it leads its comparison set with 80.5 on KMMLU-Pro and 91.6 on CLIcK. The Apex result is the strangest entry in the table: 45.8, against 28.1 for the next-best model, and 1.0 for A.X K1 five months earlier. A 45× jump on a frontier math benchmark is the kind of claim that should wait for third-party replication.

Then there is the retrieval chart. Under NVFP4 quantization, A.X K2 scores a clean 100 on every needle-in-a-haystack cell, at every depth, out to 512K tokens.

Needle-in-a-haystack heatmap for A.X K2 under NVFP4 quantization at 512K context, showing a retrieval score of 100 in every cell across eight context lengths and eleven needle depths — Image credit: SK Telecom / Hugging Face

This chart is worth less than it looks. Needle-in-a-haystack is a saturated benchmark — exact-match retrieval of a planted fact is close to the easiest long-context task there is, and most frontier models have been acing it for over a year. A solid green grid is table stakes, not evidence of exceptional long-context reasoning. The informative long-context number is on the same model card and is lower: RULER averages 94.6 across 4K–256K, which is good but not a perfect score.

The genuine tell is agentic performance. A.X K2 scores 98.0 on τ²-Bench Telecom, the strongest in its comparison set — a result worth noting carefully, since τ²-Bench Telecom is a customer-support simulation rather than a test of telecom domain knowledge, and the model card marks it as evaluated by Artificial Analysis rather than in-house. But on BrowseComp, capped at 10 searches, it scores 9.3 — dead last, against 29.1 for GLM-5.1 and 26.9 for Qwen3.5. A model that resolves 98% of scripted support dialogues while failing 91% of open-ended web research tasks has not learned general agency; it has learned a benchmark’s shape.

To SK Telecom’s credit, the model card says so. Its limitations section states that “agentic performance is moderate — A.X K2 trails the strongest compared models on BrowseComp — reflecting limited agentic RL during post-training,” and advises users to “evaluate long-horizon tool-use workloads before relying on them.”

What This Means

The corporate framing around A.X K2 is sovereign AI. It is the Phase 2 deliverable of a Korean government program whose consortium includes SK Telecom, Krafton, 42dot, Rebellions, Liner, SelectStar, Seoul National University, and KAIST, and SK Telecom is positioning it for manufacturing, defense, and biotech deployments. The company says it performs at parity with Qwen3.5-397B, DeepSeek-V4-Flash, GLM-5.1, and Kimi K2.6, and claims a 32.2-point average improvement over A.X K1 across 14 benchmarks.

The table mostly supports the parity claim, with the caveat that “parity” here means trading wins across a set the vendor selected. What makes the release credible is not the wall of 100s — it is the 9.3 that SK Telecom left in the table and then explained. Apache 2.0 weights, competitor scores sourced from a third party, and a limitations section that names the model’s weakest result are more transparency than most releases at this scale offer. The benchmarks that look too good mostly turn out to be saturated rather than fabricated, which is a meaningfully different failure of evaluation — and one the field should fix by retiring the benchmark, not by doubting the lab.

Related Coverage

Kimi K3 Open Weights Ship: 2.8T Parameters, 1.4 TB to Run — the current ceiling for open-weight model scale, released two days earlier
Kimi K3 Hits Third on Artificial Analysis; Open Weights Due July 27 — on third-party evaluation as a check on vendor benchmark claims
Moonshot AI Releases Kimi K2.6 with 256K Context and 300-Agent Swarms — one of the four models SK Telecom benchmarks against
Qwen-Image-3.0: 4.5k-Token Prompts, No Benchmarks, No Weights — the opposite disclosure problem