---

Title

In a move to reinvigorate the classic encoder–decoder framework, Google has introduced T5Gemma, a family of encoder–decoder large language models (LLMs) built atop its powerful Gemma 2 architecture. Published on July 9, 2025, this offering rekindles interest in encoder–decoder systems—long favored for their superior handling of tasks like translation, summarization, and QA—bringing fresh innovations to the table (Google Developers Blog).

🎯 What Makes T5Gemma Stand Out

• Model Adaptation from Decoder‑Only Weights
  Rather than starting anew, T5Gemma adapts Gemma 2’s pretrained decoder‑only models. The weights populate both encoder and decoder layers and undergo further pre‑training using UL2 or PrefixLM strategies (Google Developers Blog).
• Support for “Unbalanced” Architectures
  T5Gemma lets you mix encoder and decoder sizes—e.g., pairing a 9B‑parameter encoder with a 2B‑parameter decoder. This configuration optimizes tasks that require deep understanding of input (via the encoder), while maintaining efficient output generation—ideal for summarization tasks, for instance (Google Developers Blog).

🚀 Performance & Efficiency Benefits

Google’s benchmarks reveal that T5Gemma models dominate the quality‑efficiency frontier compared to their decoder‑only equivalents:

• On SuperGLUE and reading‑comprehension tasks, they consistently match or outperform while reducing inference computational cost (Google Developers Blog).
• For math reasoning (GSM8K), the 9B‑9B T5Gemma exceeds accuracy of Gemma 2 9B with nearly identical latency; the 9B‑2B “unbalanced” variant boosts accuracy over 2B‑2B without slowing down (Google Developers Blog).

📈 Capabilities: Pre‑Training & Instruction Tuning

• Pre‑Training Gains
  T5Gemma’s encoder‑decoder setup achieves remarkable improvements pre‑instruction‑tuning: +9 points on GSM8K and +4 on DROP vs. Gemma 2 9B (Google Developers Blog).
• Post‑Tuning Boosts
  Instruction‑tuned T5Gemma further shines: the 2B‑2B variant outperforms the tuned Gemma 2 2B by ~12 points on MMLU and jumps from 58 %→70.7 % on GSM8K (Google Developers Blog).

📦 What’s Available

Google has open‑sourced a suite of T5Gemma models with diverse configurations (Google Developers Blog):

• Sizes: T5‑style Small, Base, Large, XL, plus Gemma 2‑based 2B, 9B, and an intermediate scale
• Variants: both pretrained and instruction‑tuned
• Configs: UL2‑ or PrefixLM‑based objectives, and unbalanced encoder/decoder sizes like 9B‑2B

🛠️ How to Get Started

• 📚 Read the detailed research paper on arXiv (Google Developers Blog)
• 🧠 Download the model checkpoints via Hugging Face or Kaggle
• 💻 Try out the provided Colab notebook or run inference through Vertex AI

✨ Why It Matters

T5Gemma revives and modernizes the encoder–decoder paradigm with cutting‑edge adaptation techniques. By combining efficiency, flexibility, and superior performance, it offers a compelling choice for developers and researchers aiming to deploy LLMs for complex language tasks. Explore the released checkpoints, fine‑tune them, and push the boundaries of encoder‑decoder LLM applications.