Meta now runs AI in production on Google's TPUs
Google’s TPUs are now serving AI in production at Meta, marking a major shift in the AI chip market.
By Ivan Mori ·
Meta now runs AI in production on Google’s tensor processing units (TPUs), as a new PyTorch-native version of the SGLang inference engine allows TPUs to be a native option for PyTorch with minimal changes. This development follows a close collaboration between Meta, Google, and the SGLang team. A matching version of vLLM, another widely used inference engine, was also released as open source, further expanding the use of TPUs in AI workloads.
Previously, both engines used Google’s own JAX AI framework to run on TPUs. The new versions allow the hardware to be a native option for PyTorch, mostly without rewriting. This shift is part of a broader strategy to loosen Nvidia’s software lock-in, as both Google and Amazon have recently open-sourced PyTorch backends for their AI chips.
Although it is unclear where Meta’s production TPUs sit, talks last year reportedly included Google Cloud TPU rentals in 2026 with purchases in 2027, and Meta later reportedly signed a multi-year rental agreement. This development comes as a response to Meta’s earlier decision to cancel its TPU contracts from 2020 to 2023, when it opted for Nvidia’s chips instead.
On performance, a tuned vLLM setup serving Moonshot AI’s open-weight, 2.8-trillion-parameter Kimi K3 model reportedly tops 700 tokens per second for a single user on the seventh-generation Ironwood TPU. That speed figure comes from Inferact, the startup founded by vLLM’s creators that helped Google build vLLM-TorchTPU. Its test ran 16 Ironwood chips with speculative decoding, a method of speeding up large language model inference with a small draft model, against vLLM’s published setup on 16 Nvidia GB200s, which reached 452 tokens per second. Its kernels for Kimi K3 and Qwen 3.8 27B deliver roughly 1.4 to 2 times the GB200 baseline’s throughput at small batch sizes without speculative decoding.
SGLang-TorchTPU will keep adding models, starting with DeepSeek V4.1 Flash and Reflection Beam. It's also working toward Day 0 TPU support for new open models, meaning support on release day, by the first quarter of 2027, according to RadixArk. Google, Meta and RadixArk plan to share more at SGLang Summit 2026, Nov. 12-13 in San Francisco.
