Persistent vs Ephemeral Remote Environments for Coding Agents

Coding agents now do the work that used to sit with junior engineers, and the shift to relying on them means the underlying infrastructure choices that support them have finally caught up with the hype. Anthropic's 2026 State of AI Agents Report states that more than 9 in 10 organizations use AI to help write code, and 86% have moved past pilot projects into actual production. Enterprises lead the way at 91% adoption versus 83% for small and mid-size businesses, and 42% now trust an agent to lead development work with a human checking the output rather than directing every step. This is not a pilot anymore, and the question has shifted from which model writes the best code to where each task runs and who ends up paying for it, in dollars and in risk.

That question matters more because trust hasn't kept pace with usage. Eighty percent of developers now run AI coding agents inside their workflows, yet confidence in AI accuracy actually fell, from 40% to 29% year over year. Mass adoption is colliding with growing skepticism about the output, and infrastructure is a big part of why. Persistent and ephemeral environments are not interchangeable defaults you pick based on preference. The right choice depends on what the agent is actually doing, and getting it wrong later causes unreliable agents, exposed credentials, or a cloud bill nobody can explain.

What persistent and ephemeral mean at the infrastructure level

Ephemeral environments are short-lived runtimes spun up for a single agent session or task and torn down automatically once it finishes. Nothing survives between runs; each invocation starts from a blank slate. Persistent environments do the opposite: filesystem state, memory, and even running processes carry over between invocations, because the sandbox drops into standby instead of getting destroyed, as Blaxel described in its documentation.

Speed is where the distinction becomes visible, though the article traces it to differences in cold-boot reconstruction cost versus storage cost while idle. Ephemeral setups cost nothing while idle, but every single invocation pays for a full cold-boot reconstruction from scratch. Persistent setups charge for storage while parked in standby, but on modern platforms they carry minimal cost during that idle stretch. That second point breaks an assumption a lot of engineering teams still carry over from the VM era: always-on infrastructure used to mean paying for compute every second it sat there doing nothing. Perpetual sandbox platforms have separated the cost of keeping state around from the cost of keeping a CPU spinning, and that changes the math on when persistence is affordable.

Most agent products shipping in 2026 are still session-based by default, according to computer-agents.com, even though long-running autonomous workflows are the pattern that now defines serious agentic engineering. That gap matters. Background execution environments built specifically for multi-step, multi-day agent work are what make the persistent-versus-ephemeral decision actually matter. Without that category of infrastructure, the two models look nearly identical from the outside, since nothing sticks around long enough for the difference to show.

The startup tax ephemeral environments impose on coding agents specifically

A coding agent that clones a large repo, installs its dependencies, and boots a dev server on every single invocation pays that entire setup cost again and again, before it writes one line of code. Blaxel's guide identifies this setup-cost pattern, and the numbers behind it are not abstract. In its Delty case study, cloning a repository with up to 50,000 files took over two minutes by itself. GitLab's own engineering team measured a clone of the Chromium repository, over 50 gigabytes, at roughly 95 minutes. Even something as modest as the GitLab website repo, at 8.9 gigabytes, took more than six minutes with a standard clone. None of these are worst-case horror stories pulled out to scare people. They're the tax paid on every single ephemeral invocation of a coding agent working against a real codebase.

Teams try to work around this with aggressive caching and pre-baked container images, and it helps, some. But it also drags in dependency drift and a new maintenance burden: someone now owns keeping those images current, which is its own quiet cost that rarely gets counted against the "ephemeral is cheaper" argument.

The stateless model made total sense for the workloads it was built for: resize an image, validate a webhook, transform a JSON blob. Those are one-shot, forgettable operations. A coding agent working inside a sprawling repository is not that, and treating it like it is produces the most common complaint in production today, per computer-agents.com: the agent looks brilliant in one session and then shows up the next day having forgotten everything it just learned.

None of this means ephemeral environments are the wrong choice broadly. One-shot scripts, isolated code execution, test runs, short-lived analysis jobs: these are tasks where starting from clean state is a feature, not a cost. The problem is applying that same model to work that depends on continuity.

How to match environment model to agent task type

Qovery's research points to a fairly clean two-axis way of deciding. Reach for ephemeral when the work can be verified by running code or tests in isolation, when the task is genuinely one-shot, when clean state is a security requirement rather than a nuisance, or when the agent itself is untrusted, third-party code operating on your infrastructure. Ephemeral also wins, somewhat counterintuitively, when an agent needs to touch database schemas, auth flows, or background jobs that require the full running product rather than a bare code sandbox; Qovery calls this a "full ephemeral app environment," and it's a different animal from spinning up an isolated interpreter.

Persistent environments earn their cost when an agent returns to prior work across multiple invocations, such as a PR review agent that keeps context across rounds of feedback, a coding agent embedded in one repository over weeks, or a multi-turn research session. They also win whenever the cost of rebuilding context (cloning the repo, installing packages, loading a dataset) is disproportionate to how often the task actually runs.

Most production deployments right now fall into what practitioners call the L2 to L3 autonomy band, and this happens to be exactly where the environment decision starts to bite. Agents at this level are autonomous enough to need to remember what they did yesterday, but not yet autonomous enough to manage their own infrastructure or judge when persistence is worth the cost. That judgment call still belongs to whoever is designing the system around them.

Resource footprint varies enormously by task, too, and that variance should inform the decision as much as lifecycle does. A lightweight data-analysis agent might need nothing more than a Python interpreter and a small memory allocation. A full coding agent needs a Linux environment, package managers, Git, background processes, and often exposed ports for a dev server. ML-heavy workloads push further still, needing custom images, GPU access, private networking, and real CPU and memory headroom. Snapshots, reusable images, and suspend-and-resume support (approaches Mastra has documented) let teams approximate persistence on top of ephemeral infrastructure. Worth being clear that these are workarounds bolted onto a stateless model, not a substitute for genuine persistence.

Isolation technology determines how much you can trust the boundary

Coding agents generate code at runtime that nobody reviewed in advance, and that changes the threat model completely from a normal developer workflow. Northflank's analysis frames it directly: the code an agent produces cannot be fully predicted or controlled before it runs, which means the sandbox has to assume it might be hostile.

That assumption is not paranoia. Research cited by bunnyshell.com puts the share of AI-generated code containing security flaws somewhere between 40% and 62%. Veracode's 2025 benchmark, run across more than 100 large language models on 80 coding tasks, found 45% of generated samples failed security tests tied to the OWASP Top 10, the industry's standard list of the most dangerous web application vulnerabilities. Documented incidents in the same period include agents deleting local user files, and, separately, agents deleting production databases. If an agent's own code is going to have a bug or a vulnerability in it, the goal is for that bug to crash the sandbox, not the surrounding infrastructure.

Three isolation approaches are actually built for this level of scrutiny, per Northflank. Firecracker microVMs give each session its own lightweight virtual machine with a dedicated guest kernel running through KVM, and they're designed for exactly the high-density, multi-tenant load agent platforms produce. gVisor takes a different route: a userspace kernel, called the Sentry, intercepts syscalls before they reach the host kernel, which shrinks the attack surface at the cost of some syscall compatibility gaps and added I/O latency under heavy load. One approach splits the difference, wrapping standard OCI containers inside lightweight VMs through a pluggable virtual machine monitor, giving hardware-level isolation while staying native to standard container orchestration.

Plain container isolation doesn't clear this bar. Containers share a host kernel, and a kernel-level vulnerability can expose the host node along with every other tenant running on it. For any multi-tenant agent platform, where different users' agents share underlying infrastructure, ephemeral environments backed by microVM or comparable lightweight-VM isolation aren't a nice-to-have. Application-level separation alone is not a boundary anyone should trust with untrusted, dynamically generated code.

Data cited via bunnyshell.com found that proper sandboxing cut permission prompts by 84% in high-volume agentic workflows, because the sandbox enforces limits structurally instead of asking a human to approve every risky action. Isolation depth and lifecycle model are separate decisions. A persistent environment running untrusted code needs the same hardware-level isolation an ephemeral one does. Choosing to keep state around doesn't earn you a pass on isolation, and treating the two as one decision is how teams end up with a fast, convenient sandbox that's also a soft target.

Startup latency and throughput at scale: where the environment model becomes an engineering constraint

Production agent pipelines are chasing sub-second startup, ideally under 100 milliseconds, and that target rules out traditional VMs and most standard Kubernetes-based approaches once volume gets high. The reasoning is mechanical, not aesthetic: if an environment takes two minutes to provision, the agent sitting on the other end either times out or burns tokens idling while it waits. The latency ceiling here is set by the agent's own token budget, not by whatever an infrastructure team would prefer.

The scale numbers from platforms actually running this in production make the constraint concrete. Modal reports scaling to more than 50,000 concurrent sessions, powering infrastructure for over 10,000 teams, with production users including Lovable and Quora running millions of untrusted code snippets a day. Northflank processes more than 2 million isolated workloads a month using Kata Containers and gVisor together. One platform's sandbox volume went from roughly 40,000 sessions a month in early 2024 to about 15 million a month by 2025, almost entirely on Firecracker, supporting up to 1,100 concurrent sandboxes on its higher tiers, though sessions cap at 24 hours on its Pro plan and one hour on its base tier, with users left to manage scaling themselves past a certain volume. Blaxel positions itself around the opposite pattern, a perpetual sandbox that stays on standby instead of getting torn down after each task, and backs that with SOC 2 Type II and HIPAA BAA availability. Daytona supports OCI and Docker-compatible environments with a dedicated kernel, filesystem, and network stack per sandbox, and has leaned into persistent workspaces that hold state across sessions since pivoting toward AI code execution in early 2025.

Creation latency, pool management, and how reliably an environment actually tears down are performance variables now, not background operational details someone checks once a quarter. Agent pipelines can spin up and destroy hundreds to thousands of environments an hour, per Northflank, and at that volume a slow teardown is as costly as a slow boot. Memory snapshots and suspend-and-resume, which Modal has in alpha as of May 2026, are the mechanism by which ephemeral infrastructure can start to approach persistent-style startup speed without actually being persistent. Worth watching as it matures, though it's not there yet. And the earlier point about billing applies directly here: because modern persistent platforms only charge for storage during standby, the latency advantage persistence offers no longer comes bundled with the old tax of paying for idle compute around the clock.

Governance and compliance requirements for whichever model you choose

The EU AI Act's high-risk provisions, covering standalone systems under Annex III, are now enforceable starting December 2, 2027. Regulation (EU) 2026/1744, the so-called Digital Omnibus on AI, pushed that date back from the original August 2, 2026 deadline. The obligations themselves are substantial: risk management, data governance, logging, transparency, human oversight, and cybersecurity resilience. Per salt.security, if an AI agent invokes APIs, whether internal services, third-party platforms, or MCP servers, that action layer falls inside the Act's scope.

Multi-agent architectures sit in murkier territory. The Act's text doesn't explicitly address chains of agents; Recital 99 concerns general-purpose AI models, and the claim that it extends to multi-agent systems comes from vendor interpretation rather than the statute itself. In a chain of agents, the compliance boundary extends to every agent performing a high-risk function, and whether that boundary can actually be audited after the fact depends on the environment model chosen. Penalties for violations reach €35 million or 7% of global turnover, whichever is larger. The US, by contrast, runs on voluntary standards through the NIST AI Risk Management Framework, layered with a patchwork of state rules including California's SB 53. A US standards body's cybersecurity center released a concept paper in February 2026 addressing software and AI agent identity and authorization, which is the most direct institutional attempt so far at closing the agent governance gap.

Persistent environments carry specific obligations that ephemeral ones sidestep by simply not existing long enough to accumulate them. Credentials should never persist across sessions, even when the filesystem does; minting fresh credentials per invocation and revoking them on completion is the right pattern no matter which lifecycle model is running. Log continuity across standby cycles has to be designed deliberately, since monitoring inside a sandboxed session is intentionally limited, and external log collection needs to avoid creating a side channel between tenants, a risk Northflank flags directly. And tenant boundaries need enforcement at the infrastructure level, full stop, because application-level separation isn't sufficient once agents are running arbitrary generated code.

The "Shai-Hulud campaign" in 2026 was a supply chain attack that planted malicious code inside npm and PyPI package install hooks and hijacked AI coding agent configuration files, including .claude/settings.json, to exfiltrate source code and credentials. That threat gets neutralized by scoping outbound network access inside the sandbox, not by picking persistent over ephemeral or vice versa. MCP, the Model Context Protocol, is now the dominant layer agents use for tool calls, with 97 million monthly SDK downloads as of March 2026, and any MCP server connection inside a sandbox needs outbound networking scoped carefully. Leave it open and exfiltration risk goes up; lock it down completely and legitimate tool calls stop working. Leaving outbound networking open raises exfiltration risk, and locking it down completely stops legitimate tool calls from working, and every setting carries that tension.

What a production-ready environment model looks like in practice

Northflank's March 2026 analysis boils ephemeral strategy down to three things, and each one shapes whether it holds up once real traffic hits it. How deep does isolation actually go, since process-level separation alone isn't enough for untrusted code running at scale, which makes microVM-based isolation the practical baseline rather than a premium option. How is the tension between a stateless environment and a stateful agent handled, since snapshots and suspend-resume are partial answers while genuine persistence is the direct one. And how fast, and how automated, is creation and teardown, since the throughput agent pipelines demand makes sub-second provisioning a hard requirement rather than something to aspire to eventually.

Gartner projects that 40% of enterprise applications will embed task-specific AI agents by the end of 2026, up from under 5% in 2025. That's an enormous jump in a single year, and that velocity outpaces governance maturity. Most production failures in this category won't trace back to a bad model or a clumsy prompt. They'll trace back to an environment decision nobody made deliberately: ephemeral infrastructure quietly running work that needed memory, or persistent infrastructure running untrusted code without the isolation depth that code actually required. Both mistakes are avoidable, and both come from treating persistent and ephemeral as a default instead of a choice made against the shape of the task in front of it.