During its Cloud Next conference this week, Google unveiled the latest generation of its TPU AI accelerator chip.
The new chip, called Ironwood, is Google’s seventh-generation TPU and is the first optimized for inference — that is, running AI models. Scheduled to launch sometime later this year for Google Cloud customers, Ironwood will come in two configurations: a 256-chip cluster and a 9,216-chip cluster.
“Ironwood is our most powerful, capable, and energy-efficient TPU yet,” Google Cloud VP Amin Vahdat wrote in a blog post provided to TechCrunch. “And it’s purpose-built to power thinking, inferential AI models at scale.”
Ironwood arrives as competition in the AI accelerator space heats up. Nvidia may have the lead, but tech giants including Amazon and Microsoft are pushing their own in-house solutions. Amazon has its Trainium, Inferentia, and Graviton processors, available through AWS, and Microsoft hosts Azure instances for its Cobalt 100 AI chip.
Ironwood can deliver 4,614 TFLOPs of computing power at peak, according to Google’s internal benchmarking. Each chip has 192GB of dedicated RAM with bandwidth approaching 7.4 Tbps.
Ironwood has an enhanced specialized core, SparseCore, for processing the types of data common in “advanced ranking” and “recommendation” workloads (e.g. an algorithm that suggests apparel you might like). The TPU’s architecture was designed to minimize data movement and latency on-chip, resulting in power savings, Google says.
Google plans to integrate Ironwood with its AI Hypercomputer, a modular computing cluster in Google Cloud, in the near future, Vahdat added.
“Ironwood represents a unique breakthrough in the age of inference,” Vahdat said, “with increased computation power, memory capacity, […] networking advancements, and reliability.”
