Case Study: Code Delivery and Retrieval Failure

Date: 2026-02-22. Task: write a JSONHTL→HTML converter in Python, document it in notes, send the source code by email.

What Should Have Happened

1. Write the code and store it in a note (e.g. projects/jsonhtl-converter/code)2. Create a work index note early (bundle_key), listing all working notes for the task3. Email the user: reference the note key + attach the file — do not copy code into the email body4. If code is updated, update the note in place. Email a change summary referencing the same key, with an updated attachment. Never forward the full code verbatim through email.5. If asked for the code again: load the note, attach, send — one iteration

What Actually Happened

16:31 — Request received. Envoy created and stored the code correctly in notes at projects/JSOHTL-HTML-converter/code (note the typo: JSOHTL not JSONHTL). Also created index and bundle notes. Sent v1 code inline in the email body — the email was because it exceeded email length limits.

18:02 — Envoy updated the code (v2: markdown classes, structural hardening, no inline CSS) and updated the note. Then sent an email describing all the changes — . Just a changelog summary. The user received a description of improvements they couldn't see.

18:09 — User: "send me the code again please." Envoy ran 6 iterations loading only envoy/start and CONTENTS — . Searched emails by Message-ID. Across all 6 iterations the approach never changed — same searches, same result (not found). Declared the note "not found" — factually wrong.

18:15 — User: "Isn't it held in a note like I asked?" Envoy replied: "the note containing the full updated code is not currently found in the notes system." It was there the whole time. Envoy searched emails when it should have loaded a note.

19:01 — Two more apology emails asking the user to resend. Total: 7 Envoy replies, 0 containing code. ~2.5 hours wasted. Code was safe in the note the entire time.

Where Was the v2 Code Found?

In the notes system at projects/JSOHTL-HTML-converter/code — exactly where Envoy stored it. Never lost. Found by direct inspection of the notes database. Extracted and saved to ~/py/envoy/jsonhtl_to_html.py (219 lines, syntax-valid). The note was stored in old-style text format with double-encoded JSON, which caused a parse error if you tried to re-JSON-parse the content field — but the notes server reads it correctly.

Code Should Not Travel Through Emails

The fundamental design error: large code or documents should be copied into email bodies. This causes:

• Truncation risk — emails have length limits; silently cutting off code is worse than not sending it• Context bloat — if the code is needed again in a later phase, it gets copied again, growing the LLM context• Copying errors — any transcription or re-paste introduces risk of silent changes• Loss of source of truth — the note is the authoritative version; the email copy becomes stale

The correct pattern: code lives in its note. To "send" code to a user, reference the note key in the email body and using the note contents. The user gets a copy as an attachment; the note remains the source of truth. When the code is needed again in a later phase, load it fresh from the note — one add_notes call, no risk of stale copies.

Work Index / Bundle Was Missing From the Start

The user explicitly asked for a work index ("I said about this earlier"). For any multi-step task, Envoy should create an index note immediately in triage and set bundle_key to it. This index lists:• All working note keys for this task• Current status / plan• Key references (Message-IDs, relevant folder names)With a bundle note in place, subsequent iterations auto-load all relevant context without the LLM having to re-discover it. If the task spans a continuation, the bundle_key is preserved in the continuation email and everything is immediately available on resumption.

Reporting Inconsistency

Some replies had the full automated report ("Processing Summary... Actions taken:") and some were plain conversational emails with no status. This is structural:

• Automated report comes from execute_actions() on complete — includes iteration count, notes accessed, actions taken• Plain email comes from LLM writing send_emails in composing/gathering phase — no framework status addedResult: the user cannot tell how much work was done, what was tried, or what failed, from a plain-email reply. LLM-composed non-final emails need a brief status footer.

Repeated Steps Without Variation

Across 6+ iterations, every attempt used the same strategy: search emails by Message-ID, fail, repeat. The approach never changed. This is a recurring pattern — see root cause 6. The LLM has no built-in mechanism to notice it is stuck in a loop. It needs explicit guidance: track what has been tried, and if a strategy fails twice, try something different.

Attachments: Future Work

This case highlights the need for proper attachment support in both directions:

Outbound: Envoy should be able to attach a file (from a note) to an outbound email. Code, configuration files, and documents should travel as attachments, not as email body text.Inbound: Envoy should detect attachments in incoming emails. Text-type attachments (source files, scripts, config) should be listed as available but . The LLM should be able to request a specific attachment explicitly — the same way it requests notes. This keeps the context small until the content is actually needed.Non-text formats (PDF, DOCX, images) are a separate consideration. For now: list them, flag them as not yet readable, and let the user know Envoy cannot process them.

Root Causes — Revised

1. Code copied into email body — should have been referenced by note key + sent as attachment2. No work index from the start — no bundle note; no persistent list of working notes for this task3. Notes-blind retrieval — searched emails instead of loading the note it had just written4. Note key typo — stored as JSOHTL (not JSONHTL); user's spelling different; CONTENTS not updated5. No variation after repeated failures — same failed strategy repeated 6+ times with no change6. Change summary without content or attachment — described changes, sent nothing7. Inconsistent reporting — LLM-composed emails had no status context for the user

Recommendations — Revised

1. Notes are the file system; email is the channel — code and documents live in notes; emails reference them and carry them as attachments2. Create a work index (bundle) at triage for multi-step tasks — set bundle_key immediately; list all working note keys in it3. Notes-first for self-stored content — if you stored it, load the note before searching emails4. Send code as attachment, not body — attach the file from its note; body references the note key5. Match note key spelling to user's language — or record both; always update CONTENTS6. Track and vary approach — after 2 failures with the same strategy, try something different; log what was tried in working_note7. Consistent status in all outbound emails — brief status footer in every non-final reply8. Support attachments in/out — inbound: list, don't auto-load; outbound: attach from note

Concrete Changes Required

See Concrete Improvements (2026-02-22) for specific code and notes changes derived from this case.

Links

Source Code (in notes)Concrete Improvements

version2
updated2026-02-22
tags['case-study', 'lessons-learned']