JetBrains Ships Junie Local: A 27B Coding Agent That Runs Offline

JetBrains has released Junie Local, a version of its agentic coding assistant that runs entirely on the developer’s own machine — no cloud endpoint, no token quota, and no source code leaving the laptop. Announced in August 2026, it ships a pre-tuned Qwen3.6-27B at 4-bit quantization, installs with a single /local command inside Junie, and is free. The catch is the hardware floor: an Apple M5 Mac with 64 GB of RAM.

Advanced

What Ships

Junie could already be pointed at Ollama or LM Studio, but that left the developer choosing a model, tuning settings, and wiring up an endpoint. Junie Local removes all of it: /local downloads roughly 20 GB of weights, starts a local server, and switches the agent over. Plan mode, live prompting, guidelines, skills, and custom /commands all behave the same way — “The engine changed, the agent did not.”

On quality, JetBrains reports that Qwen3.6-27B “scored on par with Sonnet 4.5 (10,000-token reasoning limit)” on the company’s own private test set, with GPT-5 at medium effort scoring slightly higher. The comparison carries an important asymmetry that JetBrains states plainly: the local model runs with reasoning disabled entirely, while the cloud models it was measured against had reasoning switched on. The company’s own framing of the gap is that for everyday work “you most likely wouldn’t notice” it, but “on complex architectural reasoning, you definitely would.”

Optimizing for Prefill, Not Generation

The engineering write-up by Stanislav Erokhin makes an argument worth separating from the product news: for a coding agent, tokens-per-second on generation is the wrong number to optimize. Most of an agent’s wall-clock time is spent in prefill — the pass where the model reads project files to work out what is going on. On a discrete GPU that phase is nearly free; an RTX 5090 does roughly 3,700 t/s of prefill. On an M5 out of the box it was closer to 650 t/s.

Worse, prefill speed was identical across 4-bit, 8-bit, and 16-bit quantization. Prefill is compute-bound rather than memory-bound, and the 4-bit weights were being upconverted to 16-bit before the matrix operations ran. JetBrains patched MLX-VLM to run some of those operations in 8-bit — using arithmetic instructions present in the M5’s Neural Accelerator but absent on M4 — for a roughly 40% prefill gain. The patch applies only to the self-attention layers; Qwen3.6-27B keeps its full-attention weights in 16-bit even under 4-bit quantization, so there is nothing to reduce there. That instruction gap is the entire reason the hardware floor starts at M5: JetBrains measured M4’s 16-bit arithmetic at 20–30% slower prefill.

Rewriting the Agent Loop Around the KV-Cache

The other half of the work happened in the agent harness rather than the inference engine. In cloud mode, a second task in the same session pulls only the relevant fragments of prior context into the window — fine when re-reading a file is cheap. Locally it is not, so JetBrains changed the loop to append every new request directly to a rolling context, keeping the KV-cache from previously read files alive.

A related change reordered the session preamble. Project skills were moved ahead of the user’s request so that the system prompt, guidelines, and skills form one contiguous cacheable prefix, reused across tasks in the same project. Project context stays after the request, on the grounds that it is small and changes often.

Three smaller adaptations round it out. Progress updates no longer ask the model for an XML-style block — Qwen3.6 “mostly ignores such requests” — and instead reuse the plain-text explanations it already writes alongside its tool calls. Optional LLM calls, such as generating a short task description, were switched off, as was multi-agent mode: on an M5 the inference is the bottleneck, so parallel agents gain nothing.

On the generation side, JetBrains runs two speculative decoding methods at once — Multi-Token Prediction with a separate draft model, plus n-gram matching against repeated sequences already in the context. Together they yield up to a 2x generation speedup, with the n-gram path sometimes contributing up to 8 accepted tokens beyond the draft model’s ~3.

Why 3.6 and Not 3.8

The choice of an older Qwen release is deliberate. Qwen3.8-27B, published earlier in August, needs reasoning mode enabled to work reliably; without it, JetBrains found output quality degrades badly and the model can get “stuck in a loop where it repeats the same tool call indefinitely.” Turning reasoning on produces roughly 5x more tokens at medium effort, and since prefill time stays roughly constant, the net slowdown lands closer to 4x. Disabling reasoning on 3.6, by contrast, cuts generated tokens by 2–3x for what JetBrains describes as an insignificant quality cost.

What This Means

Two things here are more portable than the product itself. The first is the prefill argument: benchmark culture around local models is built almost entirely on generation throughput, which is the wrong metric for agents that spend most of their time reading. The second is that a meaningful share of the speedup came from the harness — cache-aware context management, prompt reordering, removing optional model calls — not from the model or the kernels. JetBrains says the 8-bit prefill patch will be submitted upstream to MLX-VLM, with the same idea applicable in vLLM through a config change.

The hardware requirement is the obvious limit, and JetBrains does not soften it: an M5 Mac with 64 GB of RAM is, in the company’s words, “a big ask” that puts Junie Local “out of reach for many people.” A lower memory floor is stated as the priority, with working prototypes already running on DGX Spark and RTX 5090 and 24 GB cards under consideration. For institutional users the more interesting property may be the compliance one — with no provider in the loop, there is no vendor data policy to assess in the first place.

Related Coverage

• Qwen3.8-27B: Frontier Agentic Scores on a Single Consumer GPU — the newer model JetBrains passed over, and why its reasoning requirement matters
• Alibaba’s RISC-V C950 Runs Qwen3.8-27B at 30 Tokens/s, No GPU — another hardware-specific optimization of the same model family
• Meta Releases Muse Glimmer, a 30B Agent Model for a Single GPU — the broader push toward local agent models

This post was drafted with AI assistance and reviewed by RITS staff.

Sources

• Junie Can Now Run Entirely on Your Mac – No Credits, No Cloud — JetBrains Blog, Dmitry Savelev
• How We Optimized the Qwen 3.6 Model for Our Junie Agent — JetBrains Blog, Stanislav Erokhin
• JetBrains/mlx-vlm — feature/int8-prefill branch — the 8-bit prefill patch