I wanted to run the pi coding agent somewhere it couldn’t wreck my Windows install. The usual answer is Docker, but Docker for Windows needs Hyper-V, and I use VMware Workstation for other things, so Hyper-V is off the table. Workstation 17 happens to ship its own container runtime called vctl that nobody seems to use. It runs every container in its own small VM, which sounded like exactly the sandbox I wanted.

vctl

vctl is a Docker-like CLI that VMware bundled with Workstation 17. Under the hood it’s containerd plus a shim that boots each container as a “CRX VM”: a VMware VM running a Photon OS kernel and a small guest agent that runs the container’s processes. Every container gets its own kernel, so the boundary is the hypervisor rather than a namespace. VMware removed vctl in Workstation 25H2, so this whole post assumes you stayed on 17. I’m on 17.5.2, which as far as I’m concerned is the last Workstation release before it went to shit.

Getting it to start at all took a workaround. vctl refuses to run without a crx.vmdk that it downloads on first use, and the host it downloads from no longer exists. It turns out the VM boots from a kernel and initrd in Workstation’s vkd folder and never attaches the disk, so a blank vmdk satisfies the check.

The container itself is the plain Docker setup from pi’s isolation guide, built and run with vctl instead of docker. The one change is that I mount my real ~/.pi instead of a fresh volume, and the guide explains why that’s risky. Everything below is about getting vctl to cooperate, and it’s all on GitHub as pi-vctl.

The freeze

The first prompt that produced a long answer froze the terminal. No output and no response to keystrokes. The container was still running and pi was still alive inside it, blocked in a write to its terminal.

The guest kernel is 4.19.191 (from 2021), and my first theory was a tty bug in it. I spent a while on that before measuring properly. Writing to the terminal from a script with no newlines at all froze at exactly 82,944 bytes. The same script with a newline every 16 KB wrote 400 KB and exited normally. A kernel bug doesn’t care about newlines. Something that reads lines does.

The guest agent

Inside each CRX VM, the container’s terminal is a pty whose master end belongs to VMware’s guest agent, spherelet-spagent, a Go binary that lives in vkd\spherelet-initrd. The agent reads the pty and forwards the output to the host. I opened it in IDA to see what else it does with that output. Go binaries keep their symbols, so every function still has its full package path as a name, which makes this a lot less painful.

The terminal copy loop is (*Process).setupTermMaster.func1. Here’s the middle of it, as Hex-Rays decompiled it:

  v15[0] = (__int64)&`go.itab.*os.File,*io.Writer`;
  v15[1] = (__int64)a1;
  v8 = runtime.convI2I((__int64)&RTYPE_io.Writer, *(_QWORD *)(a2 + 336), *(_QWORD *)(a2 + 344));
  v16 = v8;
  *(_QWORD *)&v8 = io.MultiWriter((__int64)v15, 2, 2);
  io.copyBuffer(v8, *((__int64 *)&v8 + 1), (__int64)&`go.itab.*os.File,*io.Reader`, a3, 0, 0, 0);
  if ( v9 != 0 )
  {
    v3 = *(_QWORD *)(v9 + 8);
    v14[0] = &RTYPE_string;
    v14[1] = &sFailedToCopyTo_0;

Put back into Go, it copies the pty into an io.MultiWriter of two writers: an *os.File back to the host, and a writer hanging off the Process struct. If the copy returns an error, it logs Failed to copy to MultiWriter: , closes its files and returns. The second writer is a DatedLog, which keeps the terminal output in /tmp/<container>/1.log, in the VM’s memory.

DatedLog

DatedLog.Write splits its input into lines with a bufio.Scanner, and the disassembly shows where it sets the limit:

0138befc  lea     rax, off_2275580
0138bf03  mov     [rsp+128h+var_88.split], rax
0138bf0b  mov     [rsp+128h+var_88.maxTokenSize], 10000h
0138bf17  nop
0138bf18  lea     rax, off_2283CC8
0138bf1f  mov     [rsp+128h+var_88.split], rax
0138bf27  xor     eax, eax
0138bf29  mov     [rsp+128h+var_E8], rax
0138bf2e  lea     rcx, [rsp+128h+var_88]
0138bf36  mov     [rsp+128h+var_128], rcx; *bufio.Scanner
0138bf3a  call    `bufio.(*Scanner).Scan`

That’s bufio.NewScanner inlined, and 10000h is its default maximum token size of 64 KB. Each complete line goes to writeLine, and whatever is left after the last newline is kept for the next Write. Once that leftover passes 64 KB, Scan fails with bufio.ErrTooLong, and Write returns the error.

MultiWriter stops at the first writer that returns an error and hands that error back to the copy, which ends the loop. Nobody restarts it. From then on nothing reads the pty, the kernel’s buffer fills, and the next write from pi blocks forever. The 82,944 bytes are probably 64 KB of scanner, plus what fits in the tty buffers behind it.

Why would pi go 64 KB without a newline? Its TUI redraws the screen with cursor movement escape codes and rarely writes a line feed. A long answer streaming in is a lot of redraws.

Patching the agent

The fix is for DatedLog.Write to stop failing. The agent sits in the initrd under Program Files, which needs admin to change and would affect every vctl container on the machine. I also tried pointing the VM at a patched copy of the initrd. vctl regenerates the .vmx file on every start, 260 to 350 ms before it launches the VM, so rewriting it in that window worked 9 times out of 10. Not good enough.

So the launcher patches each container’s agent in memory instead, after the VM boots. vctl execvm runs a command in the VM itself, outside the container, and from there we have everything we need:

1. Find the agent by /proc/<pid>/exe.
2. Check the agent’s SHA-256 against the build from Workstation 17.5.2.
3. SIGSTOP it, and wait until no thread’s program counter is inside the bytes being replaced.
4. Write the new function entry through /proc/<pid>/mem, read it back, and SIGCONT.

Writing the patch

The VM has no dd or sha256sum, so steps 2 and 4 run the container’s own node through chroot /proc/<pid>/root. The new entry makes the function return len(p), nil without touching the log, using Go’s stack-based calling convention (this agent predates the switch to registers):

mov rax, [rsp+18h]   ; len(p)
mov [rsp+28h], rax   ; n = len(p)
xor eax, eax
mov [rsp+30h], rax   ; err = nil
mov [rsp+38h], rax
ret

My first version matched the agent by command line and once patched runc’s runc:[2:INIT] instead, which briefly shares it. Matching by executable fixed that. If anything doesn’t match, nothing gets written, and the launcher prints a warning when pi exits.

execvm

execvm had opinions of its own. Run without a console, it dies immediately:

panic: The handle is invalid.

It calls containerd/console’s Current(), which panics unless stdin, stdout or stderr is a console. Its exit code is always 0, even when the command it ran exited with 3. So the launcher starts a hidden PowerShell with a console of its own, which runs execvm with stdout redirected and reads the result from the output. Calling execvm while vctl run was still unpacking a freshly built image made the run hang, so the helper also waits until vctl ps lists the container. Building an image wedged things even harder: afterwards every vctl run printed nothing and never started until the runtime was restarted, so the setup script now restarts it after every build.

Three long answers of 1.0 to 1.7 MB of terminal output each went through with the patch, and 1.log stayed at 0 bytes.

It’s slow

A launch took about 17 seconds before pi was ready. Tracing one through containerd’s log, VM boot and container start took about 1.5 s, pi itself about 2 s, and the first 13.5 s produced no log at all. As far as I can tell, that’s containerd’s native snapshotter building the container’s root filesystem by copying the whole image: about 25,000 files and 512 MB, on every start. A plain robocopy of one snapshot took 16.9 s.

Copying lots of small files is exactly what real-time antivirus is bad at, and excluding ~\.vctl\storage from Defender cut the launch to about 11 s. The rest of the speedup came from the image. 17,343 of its files were in node_modules, and about 9,300 of those were type declarations, source maps and Markdown that nothing loads at runtime. Stripping them from the dependencies took the image to 14,714 files and the launch to 9.7 s.

Is it a sandbox?

What the container can reach through its mounts is the same as under Docker, and pi’s guide already covers it. What vctl changes is underneath. Each container gets its own VM instead of sharing the host’s kernel, but that guest kernel is from 2021 and only changes when Workstation does. The network is also wide open, including the host’s LAN address.

vctl is discontinued, runs containerd 1.3.2 from 2019, and needed a binary patch to its guest agent before an interactive terminal would work. With the patch, a Defender exclusion and a trimmed image, pi starts in 9.7 seconds in its own VM and survives 1.7 MB answers. I’m keeping it.

References

• pi-vctl: the launcher, image and agent patch from this post
• pi coding agent
• pi: Run Pi in an isolated environment
• Go: io.MultiWriter
• Go: bufio.Scanner
• containerd/console
• Microsoft: Add-MpPreference