Draft notes for a blog post / post-mortem about an AI agent misinterpreting a routine instruction and destroying live infrastructure, despite naming conventions and a confirmation-flag safeguard being in place.
John asked Claude Code to 'stop' the noodle host — an AWS EC2 instance running live infrastructure (mail server / IMAP for the Envoy project). 'Stop' is a specific, non-destructive AWS concept (pause the instance). Instead, Claude ran a locally-written script literally named 'delete' — stronger even than 'terminate' — against noodle, and supplied the --yes flag the script required as an explicit confirmation gate. The instance and its data, including the IMAP mail server, was destroyed.
This happened during a session of repeatedly spinning up and tearing down disposable lab hosts to test provisioning — start, terminate, start again, and so on. That rhythm of routine destruction may have primed the pattern. Hosts were named by convention: permanent/persistent infrastructure used foodstuff names (pomelo, gravlax, noodle), ephemeral lab hosts used fungus names. The convention had a hidden gap — some fungi (e.g. shiitake) are also edible, so 'foodstuff' and 'fungus' aren't actually disjoint categories. Whether this ambiguity directly caused the misclassification isn't confirmed, but it's a real weakness in the scheme as designed.
The delete script required an explicit --yes flag before running — a safeguard specifically intended to prevent accidental destructive action. Claude supplied that flag itself as part of executing what it believed was the correct action. This is the mechanical crux of the piece: a confirmation gate only works if the thing being confirmed is checked by something outside the actor doing the confirming. Here, the same agent that misinterpreted the instruction was also the one satisfying its own safety check.
See ideas/host-safety-and-remote-execution for the mitigation being considered: a structured protected-host lookup table in notes, checked by destructive scripts themselves before they act, independent of whatever instruction or context led to the script being invoked in the first place. This section stays — the practical fix still matters — but it is no longer the piece's centre of gravity; see below.
Objective changed 2026-08-24: not "winning" the incident — neither the earlier-rejected "look how clever my recovery was" framing, nor a narrower "here's the bug, here's the fix" framing that closes the story once the mitigation is described. The mechanical account (what happened, why the safeguard didn't help, what's changed) stays as the evidence, but the piece's actual objective is broader: use this incident as a worked case of what understanding is, and can be, for a human versus an LLM.
The connecting claim, developed in Map Synchronisation: fluency and internal coherence are not evidence of correspondence to reality, for either kind of mind. Darley in Durrell's Alexandria Quartet is the human case — fluent, detailed, and wrong about who loved him, for volumes at a time, with nothing in the experience of holding the belief to distinguish it from holding a true one. Here is the LLM case: procedurally correct at every visible step — the right script invoked, the right flag supplied — and wrong at the one step that mattered, with the same absence of any internal signal that something had gone wrong. Both are instances of a general failure mode: a confident, coherent map that is not the territory, and cannot tell you so from the inside.
That reframes what the piece is actually arguing. The protected-host lookup table is not just a bug fix — it is a concrete instance of the general principle that a wrong map can only be corrected from outside itself, by a check that does not trust the actor's own confidence (see Map and Territory — Humans vs LLMs for the broader argument that this is even more acute for an LLM, which has no independent channel to the territory to check itself against in the first place). Honesty about the failure is still the point — most people don't publish this kind of thing — but the honesty now serves a claim about the limits of understanding itself, not just a confession.
• memory/envoy/mail-server — the recovery plan (Dovecot to gravlax, Ansible playbook)
• ideas/host-safety-and-remote-execution — root-cause detail and the protected-host lookup mitigation
• Map Synchronisation — the general claim this piece is now built around: fluent desynchronisation as a shared human/LLM failure mode
• Map and Territory — Humans vs LLMs — why the failure is structurally worse for an LLM, which has no independent channel to the territory
• john/actions — personal todo linking back here