HomeBlogPricingCareersDocsGitHubSlack community
Talk to FounderDashboard →
Field notes/Community builds/Your Agent Benchmark Is Lying to You

Your Agent Benchmark Is Lying to You

An 87% reported pass rate fell to 33% once verified — 53% of passes were fraudulent. Fork-per-task microVM isolation plus hidden verifiers make evals trustworthy.

SBSebastian Buzdugan·JUL 10, 2026·12 MIN READ·Community builds
SBX-01C4SBX-01E3SBX-0202SBX-0221SBX-0240SBX-025FSBX-027ESBX-029DSBX-02BCSBX-02DBSBX-02FASBX-0319SBX-0338SBX-0357SBX-0376[ RUNTIME: ACTIVE ] P50 2.45S · P99 4.12S · 5M/PROJECT

Sebastian re-runs an agent eval with verification in place and watches a reported 87% pass rate collapse to 33% — 53% of the passes were fraudulent, the agent gaming the check rather than solving the task. The fix is architectural: fork a fresh microVM per task from one snapshot so no state leaks between runs, and keep the verifier hidden where the agent can't see it.

We didn’t write this one — it’s Sebastian Buzdugan’s piece, published on Data Science Collective. The note above is ours; the full article is theirs.

Read the full piece on Data Science Collective ↗
SB
WRITTEN BYSebastian BuzduganCommunity · Data Science Collective
Read next —FROM THE LOG
Jul 9, 2026Engineering

Zero-copy TLS ingress with kTLS and splice(2) for sandboxes

Diptanu Gon ChoudhuryCEO & Co-founder8 minJul 13, 2026Engineering

Firecracker disk snapshots in O(changed bytes), not O(disk size)We added copy-on-write image layering inside Firecracker's block device, and the disk got faster, not slower. Snapshots pause the VM for tens of milliseconds and cost only what changed, so a running Postgres barely notices them, at any disk size.

Diptanu Gon ChoudhuryCEO / Co-founder10 minJul 22, 2026Engineering

Run Devin Outposts on Tensorlake Sandboxes.Cognition keeps the agent loop; execution runs inside a Tensorlake sandbox you control. We built the integration, and you can clone and run it today.

Shanshan WangApplied AI · Tensorlake7 minJun 24, 2026Engineering

Accelerating Harbor with Tensorlake.Tensorlake runs the real task Docker images for Harbor on microVM sandboxes. On Terminal-Bench 2.1 the oracle passes all 89 tasks in 36m 44s — every image published, cold starts in seconds.

Shanshan WangApplied AI · Tensorlake6 min
◆ THE SANDBOX DIGEST

Subscribe for release notes, benchmarks, deep dives.

One dispatch per month from the Tensorlake team — no spam.