Emerging Kubernetes tools for AI and optimizing GPU workloads

Podcast – Emerging Kubernetes tools for AI and optimizing GPU workloads

Discover how Kubernetes is evolving to support AI/ML workloads in this interview with John Platt, CTO at StormForge (now part of CloudBolt) on KubeFM’s podcast series.

This episode will cover:
 

• Notable new Kubernetes tools, such as in-place pod resizing for smoother workload rightsizing, EKS Auto Mode to streamline node management, and NOS for simplifying GPU virtualization.
• The growing adoption of Kubernetes as the preferred platform for AI/ML workloads, delivering substantial cost savings (up to 90%) over managed services like OpenAI, along with enhanced scalability and flexibility.
• Key obstacles to running AI on Kubernetes, including managing GPU drivers, ensuring CUDA version compatibility, and accommodating very large models that may exceed tens of gigabytes.