Tencent Open-Sources Hy-SimulMT, a 3-Second Interpreter

Tencent Hunyuan released Hy-SimulMT-30B-A3B on October 26, 2026, an open-weight model that translates between Chinese and English speech while the speaker is still talking, publishing weights under Apache 2.0 on Hugging Face and ModelScope. It streams both translated text and synthesized speech, can clone the speaker’s voice in real time, and on the lab’s long-form audio evaluation lands at a StreamLAAL of about 3 seconds — the lowest latency in the comparison Tencent published, against 4.5 and 4.7 seconds for the professional interpreters it benchmarked. Simultaneous interpretation is the part of speech translation where open weights are still rare: most of the systems in Tencent’s comparison are commercial APIs, and the open ones are research releases. The weights alone would be notable. What makes the release worth reading is where the model wins, where it does not, and how much of the system is still missing.

What Simultaneous Means Here

Consecutive translation waits for a sentence, or a pause, and then translates. Simultaneous translation commits before it has the whole utterance, which forces a different trade-off: every extra second of context improves the translation and delays the listener. Hy-SimulMT is built for the second mode. It takes a continuous audio stream, emits target-language text and speech as the audio arrives, and keeps a speaker’s vocal characteristics in the output so listeners can tell who is talking.

Tencent measures the lag with StreamLAAL, a standard metric for simultaneous speech translation that averages, over the output, how far the translation trails the source. The model card is careful about one thing worth repeating: StreamLAAL is not time to first output. A system can start speaking quickly and still accumulate lag, and it can hold back early and catch up later.

The strategy the model appears to use is visible in the numbers. Its lag on the long-form benchmark is 3.3 seconds, against 4.5 seconds for interpreters holding CATTI Level 1 certification and 4.7 seconds for Level 2, measured on 18 recordings of about six minutes each, drawn from material published from September 2026 onward across ten domains. Tencent took the best score among three interpreters at each certification level. Tencent’s card quotes Wang Weiwei, Vice Dean of the School of Interpreting and Translation Studies at Guangdong University of Foreign Studies, describing the behaviour in plain terms: “To control latency, the system starts speaking first and makes corrections afterward; these corrections sound similar to an interpreter’s self-corrections.”

Inside the Model

The architecture is a Hunyuan mixture-of-experts language model with a speech front end. The checkpoint’s configuration lists 48 layers at a hidden size of 2048, grouped-query attention with 32 query heads and four key-value heads, and 128 routed experts of which eight are active per token, plus one shared expert — the arithmetic behind the 30B total and roughly 3B active in the name. A 24-layer audio encoder takes 16 kHz input, 128 mel bins at a time, and produces features the language model reads as tokens.

Two output modes ship as separate checkpoints. The translation-only build emits the target language. The Interleave build also emits the source transcript, which matters for captions, meeting records and any workflow where someone needs to check what was actually said. Each mode arrives in three formats — BF16, FP8 and GGUF — for six published checkpoints in total.

Where It Wins, and Where It Does Not

The short-form benchmarks in the card are more mixed than the headline latency result. On FLEURS the model scores 90.3, ahead of Qwen3.8-LiveTranslate at 88.4 and behind Gemini3.5-LiveTranslate at 92.4. On ACL6060 it scores 81.2 against Gemini’s 82.6. On RealSI-Sentence its 84.7 is the highest in the table. On CoVoST2 it drops to 75.5, the weakest of the strong systems, against 82.1 for Qwen3.8-LiveTranslate and 76.8 for Confucius4.

System	FLEURS	CoVoST2	ACL6060	RealSI-Sentence
Hy-SimulMT	90.3	75.5	81.2	84.7
Qwen3.8-LiveTranslate	88.4	82.1	80.5	84.4
Gemini3.5-LiveTranslate	92.4	74.4	82.6	82.2
Confucius4 R2T2 → T3PO	89.4	76.8	76.1	77.7
Seed-LiveInterpret2.0	85.8	67.7	74.8	80.0

Read the long-form chart the same way and the pattern is consistent: Hy-SimulMT sits at the left edge on latency, with a GEMBA score just under Qwen3.8-LiveTranslate’s and above the rest. The card’s own summary is that its GEMBA exceeded the compared open-source models and four of the five commercial APIs — which is accurate, and also means one commercial API beat it. Tencent also attaches its own caveats to the interpreter comparison: 18 recordings, six interpreters, and no assessment of speech quality or voice similarity.

What This Means

For anyone building meeting tools, lecture capture or live support across Chinese and English, an Apache 2.0 interpreter at this latency changes the default. The previous options were a commercial API or a pipeline of speech recognition, text translation and synthesis, where the translation step alone waits for a completed sentence. A model that emits target speech 3 to 4 seconds behind a live speaker is a different product category, and it runs from downloadable weights.

The catch is that the release is a model, not a system. Speech output and voice cloning need a TTS service the repository does not provide; the inference server is built around vLLM 0.31 with the model’s own `HYV3ForCausalLM` class; and llama.cpp support is a submitted pull request, still unmerged, that currently works through `llama-mtmd-cli` rather than the server. Chinese-English only, too — the multilingual translation line Tencent publishes separately covers far more languages.

The most useful restraint in the card comes near the end, where it notes that professional interpreters remain important where there is strict accountability, expert review or high-stakes decision-making, and that machine interpretation should prove itself first in low-risk settings. That is the right frame for a 3-second interpreter: a tool for following a talk, not for signing a contract.

This post was drafted with AI assistance and reviewed by RITS staff.

Sources

• Tencent Hunyuan — Hy-SimulMT-30B-A3B model card (October 26, 2026)
• Tencent-Hunyuan/Hy-SimulMT — SimulInfer inference service and SimulTrain tooling (GitHub)
• Hy-SimulMT-30B-A3B — model configuration (layer count, experts, audio encoder)
• Tencent Hy — Hy-SimulMT model collection, including FP8 and GGUF variants
• llama.cpp pull request #31198 — Hy-SimulMT architecture support (open, unmerged)