My "secure" sandbox detected an attack and reported success anyway. Here's the marshaling bug that caused it.

Description and use

Migrating the mri sandbox to isolated-vm eliminated false positives and hidden violations, ensuring reliable isolation and correct handling of malicious code.

Problem

A sandboxed execution tool (mri) incorrectly reported successful execution of malicious code due to a marshaling bug and insufficient isolation, leading to false positives and hidden violations.

Outcome

The migration to isolated-vm and marshaling bug fix eliminated false positives and hidden violations, resulting in a robust sandbox that correctly blocks malicious code while allowing legitimate snippets.

Workflow

• 1. Built mri to run untrusted snippets in a sandbox with an execution allowlist.
• 2. Tested with three attack cases, all blocked.
• 3. Created a second benchmark with 19 legitimate cases and 14 new attacks.
• 4. Found legitimate code was incorrectly blocked due to missing bindings in the static scanner.
• 5. Discovered a marshaling bug that hid a runtime violation by converting the error to a string and returning a clean result.
• 6. Identified that Node's vm module does not provide true isolation.
• 7. Migrated to isolated-vm for a real separate V8 isolate.
• 8. Fixed the marshaling bug to ensure any violation produces a blocked verdict before serialization.
• 9. Re-ran the full benchmark; all cases passed with correct verdicts.
• 10. Documented remaining limitations (no OS-level sandboxing, no taint tracking, single-snippet evaluation).

Evidence

The legitimate-code suite failed first, and badly. 5 of 19 normal patterns got blocked...

Source

The attack suite found something worse than a missed attack... the code responsible for serializing that return value back to the host caught the resulting error, converted it to a string, and handed back a clean "executed" result.

Source

Fixed the sandbox by moving to isolated-vm, a real separate V8 isolate rather than a shared-realm context.

Source