Zero-copy TLS ingress with kTLS and splice(2) for sandboxes

Tensorlake Sandboxes are built from the ground up for performance and scale. Our users run them as the environment where agents do their work. They make thousands of API calls into a single sandbox to start, stop, and observe processes; uploading files for agents to operate on; downloading the assets those agents produce.

The infrastructure that authenticates and routes every connection into a sandbox has to get out of the way. infrastructure overhead should be indistinguishable from talking to the process directly.

Ingress should disappear

We've been rebuilding how traffic enters sandboxes, and the core change is moving the in-dataplane hop from an L7 reverse proxy to an L4 byte-forwarding data path.

On a single large transfer through one host, that change took us from 1.8 GB/s to 6.4 GB/s and cut CPU cost from ~2.9 to ~0.45 CPU-seconds per GB by removing both user copies of requests.

How sandbox networking works today

Traffic ingress starts at an L7 gateway we built on top of Cloudflare's Pingora. It terminates TLS and accepts HTTP/S, WebSocket, and gRPC from users. The gateway extracts the sandbox ID from the request URL (SSH uses a different extraction path off the connection), looks up which dataplane owns that sandbox and on what address, and issues a fresh, authenticated request to a second L7 reverse proxy running on that dataplane. It attaches context to the request headers so the dataplane proxy can verify the call, and the dataplane proxy makes one more request to the application inside the sandbox.

There are a lot of moving parts, and it works surprisingly well. But as we scaled, some customers now run thousands of concurrent sandboxes, we found ourselves tuning all the knobs you'd expect on a stack of L7 load balancers: progress and idle timeouts at every hop, connection-abort handling at each stage, graceful shutdown semantics, header-rewriting rules. Every L7 hop also interprets the stream — sniffing bodies, injects headers, synthesizing status codes when something upstream misbehaves and each of those behaviors is a place where things can go wrong and become confusing when users troubleshoot failures.

There was also a structural problem: the dataplane's reverse proxy and the process that manages sandbox lifecycle were the same binary. Updating the dataplane meant stopping the proxy for a few seconds, which interrupted live user traffic. For a workspace an agent is actively driving, that's not acceptable.

Moving the in-dataplane hop to L4

We replaced the dataplane's L7 reverse proxy with an L4 forwarder that does exactly one thing: move bytes between the edge gateway and the application, without understanding them. We also pulled it out of the dataplane process entirely, it now runs as a standalone daemon on each host, with its own lifecycle.

This solved a problem that had nothing to do with performance. The dataplane's reverse proxy and the process that orchestrates sandboxes were the same binary. But those two jobs change at completely different rates. The orchestrator controls sandbox orchestration, snapshot/restore machinary, and we update it continously to optimize the dataplane. The data path is boring on purpose, once it correctly forwards bytes, it barely changes. Coupling them meant every routine orchestrator deploy stopped the proxy for a few seconds and interrupted live traffic, so a workspace an agent was actively driving would drop its connections because we shipped an unrelated scheduling change.

Splitting the forwarder into its own daemon aligns each component's lifecycle with how often it actually changes. The dataplane can be restarted, upgraded, or rescheduled as often as we like. The sandbox connections live in the TCP proxy, which keeps working through dataplane restarts. The proxy itself changes rarely, and when it does we can drain it deliberately instead of taking it down. Decoupling the data path from the control plane fixed the upgrade-interruption problem outright.

Two things made this harder than "just forward the bytes":

A wire protocol for routed L4 ingress

There are many ways to build an L4 overlay: VXLAN, eBPF-steered routing, a CNI plugin, host-level encapsulation and with full control of the network you can get fancy. We deliberately chose a mechanism that is non-invasive to the underlying network, because we run across AWS, GCP, and soon GPU neoclouds like Nebius and CoreWeave, and we don't want our data path to depend on any one provider's networking primitives. Our overlay lives entirely at the application layer, so it ports across all of them unchanged.

Here's the path a connection takes:

1. 1

   TLS is still terminated at the outer edge gateway: that's where user authentication happens.

2. 2

   From there, the gateway opens a new TCP connection to the host-local forwarder over mutual TLS. The daemon presents a certificate the gateway pins; the gateway presents a client certificate whose identity the daemon checks against an allowlist. Nothing inside a sandbox can reach or impersonate this channel.

3. 3

   Over that mTLS connection the gateway writes a short preamble - a single length-bounded frame naming the sandbox ID and the sandbox's private target address (ip:port), which the gateway learned from the scheduler and which is signed so the daemon can trust it without a round-trip.

4. 4

   The forwarder reads the preamble, opens a connection to the named application, and splices the two connections together.

The part that makes the splice possible: kernel TLS

Here's the subtlety, and it's the interesting engineering. The forwarder terminates the gateway's mTLS on the host and authenticates the connection and carries the preamble. So one side of the tunnel is encrypted (the gateway connection) and the other is plaintext (the app inside the sandbox). splice(2) is zero-copy precisely because the kernel moves bytes between file descriptors without ever looking at them but a TLS-terminating proxy has to look at every byte to decrypt it. Those two things are normally mutually exclusive, which is why an ordinary TLS proxy has no choice but to read decrypted bytes into a userspace buffer and write them back out which is two trips across the user/kernel boundary for every byte.

We resolve it with kernel TLS (kTLS). The handshake still happens in userspace we complete the TLS 1.3 handshake normally and then install the negotiated send and receive keys into the socket via setsockopt(TLS_TX) / setsockopt(TLS_RX). From that point the kernel owns the TLS record layer: reads off the gateway socket come back decrypted, writes go out encrypted, and the forwarder can splice(2) between the (kTLS) gateway socket and the (plaintext) container socket through a kernel pipe. The kernel decrypts on the way in and encrypts on the way out; the tenant's bytes never get copied into userspace, never get staged in a buffer, never get parsed.

A few specifics, because this is where the sharp edges are:

• We run this on Linux 6.x with kTLS TX/RX offload for TLS 1.3 (AES-GCM). The gateway stays an ordinary TLS 1.3 client.
• splice() operates on the kTLS socket's decrypted record stream, so it moves whole TLS records at a time. Control messages and any record kTLS can't hand off cleanly (e.g. a key update, or a non-application-data record) fall back to a one-time userspace read/write for that record, then splicing resumes. The fast path is the common case; the fallback is correct, not fast.
• The TCP Proxy is completely unaware of L7 protocols. It doesn't write status codes or impose any read/write deadlines.

Adaptive timeouts & honest closes without an L7 layer

Because there's no L7 hop counting requests anymore, we derive sandbox activity from byte flow at the transport layer. Every splice() returns the number of bytes it moved even though we never see the payload; the forwarder meters that count per tunnel and periodically reports it to the dataplane, which uses it to reset the sandbox's idle timer. Activity is tied to bytes moving, not connections existing: an idle-but-open connection doesn't pin a sandbox alive forever, while a genuinely busy one keeps extending its own timeout for as long as data flows. We kept the exact adaptive-timeout behavior our users rely on, driven by a signal the L4 path can actually observe.

Performance

We measured two shapes of traffic because they stress completely different things.

Bulk transfers — uploads & downloads. The old path decrypted each byte in userspace, copied it across the user/kernel boundary twice, and pushed it through two layers of HTTP parsing and buffering. The new path does the crypto in the kernel and splices the plaintext straight through. The userscpae copies and the entire L7 parse/buffer layer are gone. This is where the throughput gain concentrates.

Thousands of small API calls. Start / stop / observe. These were never bandwidth-bound but they benefit from the removal of L7 request handling and, more importantly, from the elimination of proxy-manufactured aborts and timeouts, which is what made the old path flaky under load. The tail is the story here:

07 · Try it

If you're building agentic applications or simulating RL environments on sandboxes, give Tensorlake a try — we'd love your feedback.

• Sandbox docs — networking, lifecycle, and the API surface.

• Sign up — spin up a sandbox and connect to it.