Gemini Adds Agentic Video Understanding, Cutting Tokens by up to 88%

On September 1, 2026, Google added agentic video understanding to the Gemini API — a processing mode that lets the model decide which parts of a video to actually look at, instead of ingesting the whole thing at a fixed frame rate. On Google’s benchmarks the change cuts token consumption by up to 88% and cost per query by up to 66% while nudging accuracy up rather than down. It is a single config flag, and it carries no additional feature fee.

Intermediate

The Problem With Fixed-Rate Video

Until now, handing Gemini a video meant static processing: the API samples the file at a fixed rate — 1 frame per second by default — converts every sampled frame into tokens, and feeds the lot into the context window. At the low media_resolution setting each frame costs 66 tokens; at high it costs 258. The arithmetic is unforgiving. A 90-minute lecture at 1 FPS is 5,400 frames before a single word of transcript is counted, and the model pays for all of them whether the answer lives at minute 3 or minute 83.

Agentic processing replaces the fixed sweep with a think-act-observe loop. Gemini is given three native video tools — get_transcript, get_frames(start, end, fps), and get_audio(start, end) — and decides for itself what to watch, at what speed, and in which modality. It can skim the transcript to locate a candidate region, then resample just that region at a high frame rate to catch a sub-second state change or cut.

The Numbers

Google published before-and-after figures for Gemini 3.7 Flash across three video benchmarks, with static processing held at high thinking level, low media resolution and 1 FPS:

• 1H-VideoQA (long video): 397.6K → 47.7K tokens per query, an 88.0% saving; accuracy 87.5% → 88.5%.
• LVBench (long video): 300.3K → 36.0K tokens, also 88.0%; accuracy 85.1% → 88.6%.
• Minerva (complex reasoning): 80.9K → 33.6K tokens, a 58.4% saving; accuracy 73.7% → 79.0% — the largest quality gain of the three.

The pattern is what you would expect: the longer the video, the more of it is irrelevant to any given question. Minerva saves the least, being a reasoning benchmark rather than a haystack search, but gains the most accuracy — targeted resampling is not merely cheaper, it sometimes sees more.

Worth flagging: the headline 88% token reduction and 66% cost reduction are not the same number, and Google does not break down the gap — a discrepancy PPC Land also noted. The plausible explanation is that agentic navigation trades cheap input tokens for fewer but pricier reasoning and output tokens, but Google has not published the split.

Google also positions the feature competitively. On a cost-versus-accuracy plot for 1H-VideoQA, it places 3.7 Flash with agentic processing at roughly $0.10 per query at about 90% accuracy, against approximately $0.60 for GPT 5.6 Terra at 80%, $0.47 for Claude Opus 5.0 at 66%, and $1.40 for GPT 5.6 Sol at 79%. These are Google’s own measurements of competitors and should be read as such.

Using It

Enabling agentic processing is a one-line change to the video part of a request:

{
    "type": "video",
    "uri": video_file.uri,
    "mime_type": video_file.mime_type,
    "processing": "agentic"
}

It is available now through the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform, for uploaded files and YouTube URLs alike, on Gemini 3.7 Flash, 3.6 Flash and 3.5 Flash-Lite; the developer documentation also lists 3.8 Flash. Billing is at ordinary token rates. Google says the mode is coming to the Gemini app for all users on the Flash and Flash-Lite models, and to YouTube’s “Ask YouTube” feature on watch pages in the coming months.

Three constraints are worth knowing before you switch it on. Navigation adds a round trip, so time to first token can rise slightly on clips under five minutes — the gains are a long-form phenomenon. Requests are capped at ten YouTube videos. And in stateless mode you must replay every processing_call and processing_result step in subsequent requests, or the model loses what it has already watched.

What This Means

The interesting part is not the percentage. It is that video has stopped being a blob you pour into a context window and become something a model navigates. Fixed-rate sampling forces a single bad trade: sample densely and pay for thousands of near-identical frames, or sample sparsely and miss the cut, the flicker, the moment the error appears on screen. Letting the model choose its own sampling rate per region dissolves that trade, which is why token count falls and accuracy rises at the same time — normally you buy one with the other.

For anyone running video through an LLM at volume — lecture capture, QA on screen recordings, archive search, compliance review — an 88% token cut is the difference between a pipeline that pencils out and one that does not. Ibrahim Syed, Founding Engineer at Ponder, told Google the company had already built its own agentic navigation layer on Gemini to find usable moments in raw footage: “Google’s Agentic Video Understanding brought that navigation into a single call, matching our recall while using roughly 3.5x fewer input tokens.” That is the honest summary of what shipped — not a new capability so much as a commoditised one, moved from application code into the API, and the third Flash-line release in two months whose headline improvement is cost per unit of work rather than a higher ceiling.

Related Coverage

• Gemini 3.7 Flash: A Big Coding Jump in a Three-Week Point Release — the model that leads the agentic video benchmarks, released August 13, 2026
• Google’s Gemini 3.6 Flash Cuts Agent Token Costs by up to 65% — the same efficiency-over-ceiling framing, applied to the whole Flash tier in July
• MiniMax Releases H3: 2K Video With Native Audio, Open Weights Promised — the generation side of the video stack

This post was drafted with AI assistance and reviewed by RITS staff.

Sources

• Introducing agentic video understanding with Gemini — Google, September 1, 2026
• Video understanding — Gemini API developer documentation
• Google Gemini’s new agent-based video analysis cuts token usage by up to 88 percent — The Decoder
• Google cuts Gemini video analysis tokens by up to 88% in agentic mode — PPC Land
• Video understanding — Gemini Enterprise Agent Platform documentation