AI Agents Are Having Their Kubernetes Moment
文章目录
Ten years ago, we saw infrastructure responsibilities moving out of microservices and into Kubernetes, service meshes, and proxies. Application code could focus on business logic, while the platform handled deployment, lifecycle, networking, and other operational concerns.
Now I see a similar shift happening with AI agents.
An agent calls models and tools, but increasingly that logic lives in a reusable harness. These harness based agents can execute code in a sandbox and use files to configure their skills, plugins, and MCP server connections. Session state can live on the filesystem too.
That requires stronger runtime isolation and lifecycle management, including filesystem persistence. Networking controls must also govern what data a nondeterministic agent can access and where it can send that data.
Two areas where this shift is already emerging are lifecycle management and networking.
Lifecycle and persistence move into agentic infrastructure
The runtime must isolate execution, use resources efficiently, and preserve state across sessions. Agentic infrastructure can provide that machinery to multiple harnesses.
Agent Substrate manages sandboxed workloads, checkpoints memory and filesystem state, and suspends idle workloads so workers can serve active ones. It releases compute while keeping snapshots in durable storage, then restores execution when needed.
The harness orchestrator decides when it is safe to suspend and when it is time to resume. The infrastructure handles the mechanism and storage. Deciding what belongs in the model's context remains harness work.
NVIDIA OpenShell and Kubernetes SIGs' Agent Sandbox also move sandbox management outside the harness, with different priorities. Solo's comparison of OpenShell and Agent Substrate looks at isolation, resource efficiency, and state management.
Networking moves into gateways
Agents are nondeterministic, so we need explicit controls over what they can see and what they can call. That means controlling both what reaches the harness and what leaves it.
On ingress, authenticate callers, authorize requests, and apply policies to the data and instructions passed to the harness. On egress, control which models and tools the agent can call, attach backend credentials, and restrict where it can send data.
agentgateway enforces access policies for traffic routed through it, including MCP tool calls. Its backend authentication can attach credentials outside the harness.
Changing the harness should not require rebuilding those access rules. Network controls must prevent unauthorized destinations, while the agent and its tools still need task specific checks and approval decisions.
The harness can then focus on getting the best out of the model and tools to achieve the goal: planning the work, managing context, choosing execution strategies, and improving results through verification and feedback.
If you want to start learning these technologies, check out Agent Substrate and Agent Gateway.
I write more about emerging AI architectures, agent infrastructure, and software development trends in my newsletter. Subscribe for longer-form posts: https://generativeprogrammer.com/
