How snapshot and verify work

A consistent snapshot of a running system

A snapshot of one container is easy. A snapshot of five services with state is only useful if every service's data is captured at the same instant; otherwise the fork starts with a queue that has already processed a message the database never saw.

denv's snapshot does four things in order. Databases are asked to checkpoint, so their on-disk state is complete. Every container is paused, which freezes writers. Every volume is captured together: on ZFS a recursive atomic snapshot, on a plain disk a copy taken while nothing writes. Containers are unpaused and health is re-checked, because probes that ran during the freeze failed and the stack must be known good before anyone uses it. The frozen window is measured and recorded in the phase evidence.

A version is that capture plus the pinned image digests. A fork creates volumes from the capture (clones on ZFS), renders the same services, and starts them on their own network.

Why the grader runs on a fork

Reinforcement learning finds whatever the reward overlooks. The attacks Karotte documented are concrete: symlink the answer file to the expected answer, replace it with a FIFO to hang the grader, leave a background process to rewrite the file after the check, replace python3 in the path the grader uses. Every one of them works when the grader runs inside the agent's container after the agent's turn.

denv's verify removes the shared surface. The agent's container is frozen first, so nothing of the agent's runs again. The declared paths are copied out with a tool that does not follow symlinks, into a directory the agent cannot reach, and the copy is checked: anything that is not a regular file or a directory, or that is too large or too deep, is refused and the episode scores zero without the grader running. The environment is snapshotted with the agent still frozen and forked. The grader runs on the fork, as root, in its own image, with the copy mounted read-only. It can still talk to the fork's database and services, because they are part of what is being graded. The fork is destroyed afterwards and the agent's environment is left as it was.

The sequence is recorded phase by phase, so a reward comes with the evidence of how it was produced.

What this costs

A verify is a snapshot, a fork and a container start. On a ZFS host that is a few seconds. On a plain disk the copy dominates and it is tens of seconds. For training loops with no agent process, where a sample is just files, a lighter grade-only path is planned that skips the freeze and the snapshot.