Unifying ML infrastructure for performance across AI/ML frameworks and hardware

Presented by Eugne Burmako (Google) & Paul Baumstarck (Google) & Nitin Nitin (Nvidia)
GenAI has already shown incredible potential for world-changing product experiences, but the size and complexity of GenAI models has put intense pressure on Machine Learning (ML) software (SW) and hardware (HW) infrastructure. While ML Engineers work in a small set of training frameworks and ML hardware, the ML SW infrastructure landscape remains fragmented, which means critical performance opportunities and HW efficiencies go unaddressed, limiting end-user product experiences and creating significant cost for ML providers.
OpenXLA, an ML compiler and runtime project, provides a standardized approach to ML SW infrastructure that benefits every stakeholder in the ML ecosystem, from ML Engineers to hardware manufacturers. Led by contributions from Google, NVIDIA, Meta, AWS, Intel, Arm, and many others, OpenXLA has already demonstrated state-of-the-art performance on critical workloads and been adopted in ML production use cases across the industry.