Envoy Case Studies

Real task failures and near-misses, analysed to identify root causes and inform improvements to the orchestrator, state machine instructions, and working practices.

Cases

pwsafe Article Revision — Issues Found (2026-02-28) — 7 issues including Message-ID bug, notes linkage gap, gathering fallback failure, sendmail DNS failure, WebDAV hallucination, action-failure/reply mismatch, and flaky JSON parse

Code Delivery and Retrieval Failure (2026-02-22) — Envoy stored code correctly but sent a change summary without code; then failed to retrieve from notes when asked. 7 wasted reply emails.

Common Themes (so far)

• Notes-blind retrieval — searching emails for things stored in notes• Change summaries without content — describing what changed without including it• CONTENTS not updated — new notes not added to discovery index• Note key typos — stored under wrong spelling, not findable• No variation after repeated failures — same failed approach repeated• Reporting inconsistency — automated summary vs plain LLM email• Hallucination of actions outside schema — claiming to perform unavailable operations• Action failure silent to reply — write_notes fails but success email still sent• Orphaned note subtrees — new notes not linked from parent index

version2
updated2026-02-28