Glossary Updates16 new terms added to the glossaries · 12 November 2026, 21:23 CET
AI TechDocKnowledge
  • English (US)
  • English (UK)
  • Deutsch
All context cards

Context card

Agent workloads vs. microservices and batch jobs

How does an AI agent workload differ from a microservice or a batch job in operations?

By knowledge.aitechdoc.world

Reviewed

Review log and changes

The short answer

An AI agent workload is long-lived, stateful and non-deterministic: it keeps memory across sessions, alternates between computation and waiting, calls external models and tools, and incurs cost per step. A stateless microservice answers one request and forgets it; a batch job runs a fixed sequence to completion. Agent orchestrators therefore manage identity, workspace, state, resource limits, network rules and lifecycle status instead of just replicas and restarts.

For: Platform engineers and project managers planning agent operations

Key points

  • Microservices are typically stateless and scale by replicas; agents carry state and identity.
  • Batch jobs are deterministic sequences; an agent assembles its own path at runtime.
  • Agents need workspaces, tool access, resource and cost limits, and pause/resume semantics.
  • Infrastructure-as-code alone does not describe agent behaviour: prompts, memory and tool versions do.
  • Documentation must version the executable agent configuration, not only the infrastructure.
  • ]

The context

Three workload shapes

Container platforms distinguish workloads by how they hold state. Stateless services in a microservice architecture can be replaced or replicated freely, which is why Kubernetes Deployments treat replicas as interchangeable, while StatefulSets provide stable identities and persistent storage for workloads that cannot be. Batch workloads run a defined sequence until completion and are expected to produce the same result from the same inputs.

An AI agent fits none of these shapes cleanly. It may stay active for hours or days, hold memory across sessions, idle while waiting for a human or an external system, resume with accumulated context, and choose its next step from a model response. Two runs with the same input can take different paths, and every step can call a paid model or an external tool.

What that means for orchestration

Orchestration for agents therefore manages objects a service scheduler does not need: an agent identity, a workspace with repositories, tool servers and skills, isolated execution with resource and time limits, outbound network rules, model configuration, and a lifecycle status that allows suspending an idle task and resuming it later from a checkpoint. An operating-system process boundary or a container gives isolation; it does not give authority. Agent-specific orchestrators are young, and open-source projects in this space are typically explicit that their interfaces are not yet stable.

What to document

Because agent behaviour comes from configuration rather than from code alone, the documented unit is the executable agent configuration: model and version, system instruction, skills and tool versions, allowed data sources and network targets, memory and retention rules, resource, time and cost limits, checkpoint, resume and abort conditions, and the record of executed actions with their results. Infrastructure versions alone cannot explain why an agent did what it did. Risk-management frameworks such as the NIST AI RMF ask for this kind of traceable mapping between intended function, deployed configuration and observed behaviour; recording it is a management task and does not by itself establish that an agent operated within its limits.

Questions readers ask next

Can I just run an agent as a normal container?
Technically yes, and many teams do. The gap is operational: you still need identity, state handling, tool and network restrictions, cost limits and a reliable stop function, otherwise none of these are documented or enforced.
Why are cost limits an architectural concern?
An agent that waits, retries and re-reads context consumes model calls over its whole lifetime. Without per-task budgets and stop conditions, cost grows with duration rather than with requests.
Is a pause-and-resume agent the same as a checkpointed batch job?
Only partly. Both restore saved state, but a resumed agent continues with memory and may choose a different path, so the resume point does not guarantee the same subsequent behaviour.

Sources

  1. Kubernetes StatefulSet vs. Deployment with Use Cases — Spacelift, 29 September 2026
  2. Kubernetes Deployment vs. StatefulSets — Baeldung on Ops, 29 September 2026
  3. Artificial Intelligence Risk Management Framework (AI RMF 1.0) — NIST, 26 January 2023

Review log and changes

Every context card is checked against its sources before it is published, and again whenever it changes; the date under the byline is the last review. Corrections (something was wrong) and additions (something was missing) are logged below with date and time (Berlin time). Typos, formatting and link fixes are not listed.

Reviewed

No corrections or additions since publication.