H3-World Turns MiniMax-H3 Into a Playable World Model

On September 1, 2026, researchers from Tencent, the National University of Singapore, and The Hong Kong Polytechnic University posted H3-World — a framework that converts MiniMax-H3, the 33B omni-modal video generator whose weights shipped in August, into an interactive world model you drive with keyboard input. It adds no action encoder and no dedicated control module. It trains 65.6M LoRA parameters, 0.199% of the backbone, on fewer than 8,000 gameplay clips.

Advanced

The Keypress Becomes a Sentence

The paper’s starting observation is that a sufficiently large video generator already responds to motion instructions written in plain language. MiniMax-H3 will move a character or pan a camera zero-shot if you ask it to. What it will not do is obey a schedule of instructions — tell it to pan left and then right, and it produces something plausible with no particular regard for when.

H3-World’s response is to keep language as the control interface and fix the timing instead. Keyboard state — 8 character keys, 8 camera keys, and a binary camera-speed flag — is aggregated over each video latent interval, with opposing keys cancelling, then rendered into a structured text clause. Formally, each action prompt is p_k = T_char(u_k) || T_cam(c_k); in practice it reads like “the character walks backward and strafes left, camera pans right slowly.” Each clause is bound to the future latent it governs.

Temporal Attention Routing

Binding a clause to a latent is not enough on its own, because MiniMax-H3’s self-attention is bidirectional — every action span can see every video latent, and control bleeds across the timeline. The paper’s mechanism, temporal attention routing, masks that flow: an action span A_k is readable only by tokens within itself and by its matched latent V_k, action spans cannot read unmatched latents or one another, and the video latents keep full bidirectional attention among themselves. Each instruction enters the visual stream through exactly one gate. Positional encoding mirrors the arrangement, placing each action span at τ(A_k) = τ(V_k) − Δ so text still precedes the video it conditions.

The clearest evidence is a controlled reversal test, where a scheduled camera pan flips from left to right at latent 15. Measured as cumulative horizontal optical flow before and after the switch, the frozen backbone registers −0.1 / 0.0 — effectively no response. Global prompting gives +0.0 / −17.3, capturing the action but not the schedule. H3-World gives +52.7 / −106.0, and reversing the schedule flips the signs to −58.7 / +121.0.

What It Cost to Train

The adaptation is small by design: LoRA rank 32, 10,000 optimisation steps at a learning rate of 1×10⁻⁴, over 7,872 clips drawn from an in-house gameplay corpus the authors call ABot-World-Explorer-500h, with 128 clips held out. Clips are 124 frames at 24 fps and 832×480. The public repository lists 4 GPUs as the training minimum and roughly 135 GB for the base weights.

The coverage figure is worth dwelling on. The action space is 9 character clauses × 16 camera clauses = 144 combinations, 135 of them structurally valid, and the training prompts touch only 83 — leaving 52 combinations the model never saw, with the top 20 accounting for 71.4% of all prompts. The compositional-generalisation claim rests on the model assembling those unseen pairs from clauses it has seen separately.

What This Means

The result that travels here is not the playable demo but the transfer path. Action-conditioned world models have generally been trained as such from the start, with a purpose-built action encoder. H3-World argues that a large video generator has already learned the relevant semantics during pretraining, and that what an adapter needs to supply is routing — when an instruction applies — rather than representation. If that holds, the marginal cost of turning a strong video model into a controllable one is a few thousand clips and a rank-32 LoRA.

It also sits at an interesting angle to the other world-model work covered here. Tencent’s HY-World 2.0 emits explicit 3D assets; Meta’s V-JEPA 2 predicts in a learned latent space. H3-World stays in pixel space and buys controllability cheaply, at the price the authors themselves name: short-horizon, fixed-length segments, with no persistent world state, real-time interaction, planning, or policy learning, and generalisation “evaluated mainly through representative examples” rather than a systematic benchmark.

One licensing wrinkle is worth flagging for anyone planning to build on it. The H3-World LoRA checkpoint is released under Apache 2.0, but MiniMax-H3 remains governed by its own community licence — the one that excludes the United States and the European Union. A permissive adapter on a territorially restricted base does not produce a permissively usable system.

Related Coverage

• MiniMax Ships H3 Weights — With the US and EU Excluded — the base model H3-World adapts, and the licence terms that carry over
• Tencent Open-Sources HY-World 2.0: A Multi-Modal 3D World Model — the explicit-3D approach to the same problem
• Introducing V-JEPA 2: Meta’s Self-Supervised Video World Model — prediction and planning in a learned latent space

This post was drafted with AI assistance and reviewed by RITS staff.

Sources

• H3-World: Turning Language Understanding into World Control — arXiv:2609.01560, submitted September 1, 2026
• H3-World project page
• Danzer1xxxxChan/H3-World — training and inference code
• DANNY621/H3-World — the rank-32 LoRA checkpoint on Hugging Face
• MiniMax H3: An Open Model Breaking the Boundaries Between Tasks and Modalities — MiniMax Research