Google’s new XLA Serving Engine gives teams a C++ path to deploy models trained with JAX or PyTorch across several processor types without keeping Python in the serving environment.
Google announced the open-source software on Nov. 5 and made it available under Apache 2.0. Teams can export a trained model for XSE to serve on CPUs, NVIDIA GPUs or Google Cloud TPUs. Work involving Arm’s Neoverse also extends support to Arm-based server chips.
The same export gives teams a serving path across different processor families. Google lists recommendation and computer-vision workloads among XSE’s uses, including work tied to YouTube Shorts and Google Ads. XSE handles deployment; JAX and PyTorch remain tools for model development.
OpenXLA, which dates to 2023, is designed to connect machine-learning frameworks with accelerator hardware through compilation. An exported model contains a StableHLO representation, which XSE can compile with XLA and serve through its C++ runtime. That keeps model development distinct from deployment.
An earlier Google Cloud project for PyTorch on TPUs relied on Ray to organize training and serving. Within OpenXLA, XSE provides C++ serving, while TorchTPU remains a separate Google project. Google has also developed TorchTPU to expand PyTorch support on its AI chips.
