Envoy Design Specification

Shallow Design Specification (Working Draft)

Status: exploratory design – subject to change

Purpose: capture current architecture and decisions so context is not lost

1. Goals

Envoy is an email‑native LLM agent that:

- reads and sends email

- maintains structured long‑term memory

- reasons using GPT API calls (stateless)

- updates memory explicitly

- behaves like a human mailbox user (not a message queue)

Key priorities:

- determinism

- inspectability

- efficiency of context window

- simple tooling

- minimal “magic”

2. High‑Level Architecture

                GPT API (stateless reasoning)
                          ↑
                          │
                  Envoy Orchestrator
                (Python control logic)
                          ↑
        ┌─────────────────┴─────────────────┐
        │                                   │
   IMAP Mailbox                      GData Notes Server
 (conversation + tasks)             (persistent knowledge)

Responsibilities

GPT

- reasoning only

- no memory

- produces:

- replies

- memory updates

- actions

Orchestrator (Python)

- fetch mail

- classify

- gather context

- call GPT

- apply memory writes

- send replies

- move/delete mail

IMAP Mailbox

- conversations

- working set

- inter‑agent messaging

- short/medium‑term context

GData (GDBM)

- long‑term knowledge

- structured state

- durable truth

3. Core Philosophy

Separation of concerns

Memory model

- GPT is stateless

- all memory lives outside the model

- context is reconstructed per call

Time scales

4. Mailbox Design (IMAP)

Why IMAP (not POP3)

IMAP chosen because it provides:

- folders

- partial fetch

- server‑side search

- persistent messages

- multiple clients/agents

- “workspace” semantics

POP3 remains suitable for queue‑style systems (e.g. PopIt3 jobs), but not for Envoy’s conversational model.

5. Email Semantics

Email is not truth storage.

Email is:

- conversation

- working memory

- tasks/events

- coordination between agents

Truth and durable knowledge belong in Notes.

6. Message Types

Task Mail (queue‑like)

Examples:

- run job

- summarise thread

- scheduled reminders

- inter‑agent RPC

Lifecycle:

process → update notes → delete

Conversation Mail (human‑like)

Examples:

- design discussion

- planning

- project talk

Lifecycle:

Active → summarised → archived → optional delete

7. Folder Layout (proposed)

Inbox/
Active/
Tasks/
Agents/
Archive/
Done/

Meanings

- Inbox – new mail

- Active – ongoing threads

- Tasks – machine actions

- Agents – internal traffic

- Archive – dormant history

- Done – safe to purge

8. Thread Model

Threads are first‑class.

Thread defined by:

- Message‑ID

- In‑Reply‑To

- References

Thread storage

Each thread has:

- short summary

- longer summary

- index of messages

Stored in Notes.

Example:

/threads/envoy-design/
    summary_short
    summary_long
    message_index
    decisions
    tasks

9. Context Strategy (Efficiency Critical)

Never load entire mailbox.

Layered retrieval:

Layer 0 – current email

always loaded

Layer 1 – thread summary

tiny orientation context

Layer 2 – recent messages

optional

Layer 3 – historical messages

fetched only if explicitly needed

This keeps token use minimal.

10. Notes / Memory (GData)

Notes are the primary state store.

Characteristics:

- JSON documents

- hierarchical keys

- atomic updates

- effectively unbounded size

Categories

/projects/*
/people/*
/preferences/*
/threads/*
/tasks/*
/envoy/state/*

Rules

- all durable facts live here

- emails only trigger updates

- system must be rebuildable from notes

11. Agent Behaviour Loop

For each new message:

1. classify type 2. fetch required notes keys 3. fetch minimal email context 4. call GPT 5. apply writes to notes 6. send reply 7. move/delete message

12. GPT Contract (conceptual)

Model returns structured output:

{
  "reply": "...",
  "reads": ["keys"],
  "writes": [
    {"op": "PUT", "key": "...", "value": {...}}
  ],
  "actions": []
}

Orchestrator validates and executes.

13. Self‑Email / Multi‑Agent Design

Email also acts as a message bus.

Agents may send mail to:

- themselves

- other agents

- humans

Enables:

- deferred tasks

- coordination

- retries

- distributed behaviour

SMTP effectively becomes actor messaging.

14. Lifecycle Rules

Active thread

kept in Active/

Dormant

move to Archive/

Completed

summarise → archive → optional delete

Task mail

delete immediately after success

15. Design Principles

- explicit state > implicit memory

- small context > large history

- inspectable systems > hidden magic

- structured data > free text

- deterministic behaviour > heuristics

16. Open Questions / To Be Designed

- exact IMAP library choice

- thread completion detection

- summary format

- memory schema details

- scheduling mechanism

- concurrency rules between agents

- security/authentication for agents

17. Current Direction (Summary)

Envoy is:

An email‑native, stateless‑LLM agent with structured persistent memory,

using IMAP as a conversational workspace and GData as the cognitive store.