Reduce GPU waste and optimize costs with StormForge GPU Optimization. Now available close-btn
Blogs

KubeCon 2026: Five Takeaways from Salt Lake City

KubeCon 2026 in Salt Lake City was my seventh KubeCon, and it definitely felt different. AI wasn’t just a couple letters tacked onto every booth’s branding anymore. It was the first year with a dedicated AI Inference + Agentic track, and most keynotes had an agent in them somewhere. 

The conversations on the show floor ranged from teams scaling AI across their environments and running out of capacity, to teams still adopting it and working out how to govern it, and even some people who told me they don’t use Kubernetes at all and just came to learn more about AI. That one surprised me. Five themes kept coming up. 

1. AI workloads are outgrowing reactive tooling 

More teams are running AI on Kubernetes. The CNCF survey says 66% of organizations hosting generative AI models use Kubernetes for some or all of their inference. CNCF Executive Director Jonathan Bryce said in his keynote that agents now generate 70% of tracked inference traffic. 

These workloads behave differently from the microservices most clusters were tuned for. Google’s engineers described agents that spend 90 to 95% of their time waiting on a model, an API or a person. Then they act, and CPU and memory spike. Tools that react after the spike are too slow. By the time they respond, the job has been throttled or OOM killed, and a long agent step starts over. 

One platform team told us they used to skip rightsizing and buy more capacity. Today they spend about 20% of their time tuning, because it’s gotten too expensive to ignore, and they trace that to AI workloads and AI-written code. Most teams I talk to are in the same place. Requests set by hand last quarter are wrong this quarter, and the footprint grows too fast to review each one. Rightsizing has to learn from a workload’s history and forecast requests ahead of the spike. 

2. GPU footprints are small, growing and under-tooled 

Most users I talked to have a GPU footprint that is a fraction of their CPU estate. It is also the part of the bill growing fastest, and the tooling has not caught up. A GPU request is a whole device by default. One workload can hold an entire GPU while only using a small portion of its memory. 

That is why HAMi was all the rage with the GPU crowd in Salt Lake City. The CNCF project, which moved to Incubating in July, lets multiple workloads share one GPU by partitioning its memory and compute.

Kubernetes is closing the gap too. Dynamic Resource Allocation went GA in 1.34, and AWS announced EKS Auto Mode and Karpenter support for it the week of KubeCon, so a workload can now ask for a device by its attributes, such as memory, and the scheduler matches it. Reid Vandewiele went deeper on where HAMi and DRA are headed in his recap. 

Once four teams share a GPU, someone has to say who used what and who pays for it, and device-level metrics can’t answer that. That’s why we shipped GPU optimization in StormForge at the show. It shows GPU use and waste per workload, and allocates GPU cost from the actual bill, including on shared and time-sliced GPUs. 

3. The hardware crunch is turning on-prem teams inward 

Hardware is hard to get. Sam Altman is even coming for our PCs. Vendors said it again and again. Google’s Jago Macleod, who leads Kubernetes and GKE engineering, told SiliconANGLE, “There isn’t enough memory. There aren’t enough machines.” Nutanix said its board approved a $20 million on-prem hardware investment so it can generate 80% of its own AI tokens in-house. 

For on-prem Kubernetes teams, hardware delays and steep cost increases change the math on capacity planning. Buying your way out of a fully allocated cluster used to be the easy answer. The teams we talked to are looking at the clusters they already have and asking how much capacity is sitting idle inside oversized requests. They know that just because a cluster looks allocated doesn’t mean it’s being used, and more of them are kicking off rightsizing initiatives. 

Just because a cluster looks allocated doesn’t mean it’s being used.

On-prem, rightsizing has become capacity planning because compute you reclaim from an over-provisioned workload is available today, with no wait on new hardware. You can see how StormForge rightsizes workloads in the sandbox without setting anything up on your own clusters.

4. KubeVirt needs its own operating model

KubeVirt graduated at the show. That’s good news for teams trying to leave VMware. KubeVirt runs VMs next to containers on the same clusters, and a 2025 industry survey cited by CNCF found 26% of Kubernetes adopters already use it in production. 

It’s not a like-for-like move, though, and resource management is a good example. A vSphere admin typically relies on memory overcommit, but KubeVirt’s docs say it doesn’t yet support classic overcommit or ballooning, so a VM can’t hand memory back. Under node memory pressure, a hard eviction stops the VM with no grace period, which the docs compare to a power loss.  

vSphereKubeVirt
Memory overcommitCommonly relied onClassic overcommit not yet supported
BallooningSupportedNot supported, so a VM can’t hand memory back
Node memory pressureHost reclaims memory and the VM keeps runningHard eviction stops the VM with no grace period

VMs on Kubernetes follow Kubernetes rules for requests, eviction and scheduling. Teams that port their vSphere runbook feature for feature end up with reliability problems and higher costs on the other side of the migration. 

5. Teams want cost and optimization in a single platform

Consolidation was the theme all week. CVS Health showed a platform that puts dozens of clusters across three clouds and its own data centers behind one logical cluster, with OpenCost built in.  

We heard the same at the booth whether teams were running multiple flavors of Kubernetes, VMs alongside Kubernetes, or GPUs alongside the rest of their infrastructure. 

They’re tired of stitching together one tool for Kubernetes, another for VMs and a spreadsheet for network traffic just to give finance a single answer. 

Most of what we heard came back to teams wanting more out of the infrastructure they already have, whether that’s GPUs, on-prem clusters or VMs that just moved over from vSphere.

StormForge Sandbox

Get more out of the clusters you already have

See StormForge rightsizing recommendations for CPU and memory requests, with nothing to install on your own clusters.

Try the sandbox

grid pattern
Sign up for our newsletter

Exclusive insights and strategies for cloud pros. Delivered straight to your inbox.


AUTHOR
Yasmin Rajabi
Yasmin Rajabi is the Chief Operating Officer at CloudBolt Software. She is a recognized leader in the FinOps and Kubernetes communities, and her background as an engineer, product leader, and operator gives her a...
  Learn more

Related Blogs

 
thumbnail
KubeCon 2026: How AI is Changing the Kubernetes Optimization Conversation

I’ve been going to KubeCon for five years now, talking to people about rightsizing. Each year people come in knowing…

 
thumbnail
How CloudBolt’s MCP server keeps AI agents inside your guardrails

By now, most of us have watched an AI agent spin up a VM from a chat window. It’s a…

 
thumbnail
Watch an AI agent provision a server with ServiceNow approval

In this demo, a standard user asks Claude for a server and gets one back about six minutes later. CloudBolt…