Envoy Folder System Design

Technical design documentation for the IMAP folder organization system and email disposition mechanism.

Overview

Envoy uses IMAP folders to organize processed emails, enabling future retrieval and preventing re-processing of handled mail. The system uses LLM-based decision making to determine appropriate folder placement.

Architecture

Cron Processing Model

1. Cron job runs orchestrator.py with search_criteria='UNSEEN' on folder='INBOX'2. Only unread emails in INBOX are fetched for processing3. After processing, LLM decides disposition via move_emails action4. Emails are moved to appropriate folders, leaving INBOX clean for next run

Automatic Folder Creation

Implementation in orchestrator.py execute_actions() (lines 502-522):

# Move emails
with IMAPClient(IMAP_HOST) as client:
    client.select_folder('INBOX')
    for email_move in response.move_emails:
        if email_move.message_id in msg_id_map:
            imap_id = msg_id_map[email_move.message_id]
            try:
                client.move_message(imap_id, email_move.folder)
            except imaplib.IMAP4.error as e:
                # Move failed - try creating the folder
                try:
                    client.create_folder(email_move.folder)
                    # Retry the move
                    client.move_message(imap_id, email_move.folder)
                except Exception as retry_error:
                    logger.exception(f"Failed to create folder")

Key behavior: If move fails (folder doesn't exist), creates folder and retries. This enables organic folder growth without pre-configuration.

Schema Support

The EmailMove schema in envoy_schema.py:

class EmailMove(EnvoyBaseModel):
    message_id: str = Field(description="Message-ID to move")
    folder: str = Field(description="Destination folder name")

LLM specifies both the email to move (by Message-ID) and destination folder name.

Search Support

The EmailSearch schema supports folder-specific searches:

class EmailSearch(EnvoyBaseModel):
    folder: str = Field(description="IMAP folder to search")
    sender: str = Field(description="Filter by sender")
    subject_contains: str = Field(description="Filter by subject")
    since: str = Field(description="Emails since date")
    before: str = Field(description="Emails before date")

Implementation in orchestrator.py search_emails_imap() (lines 213-263):

def search_emails_imap(search: 'EmailSearch') -> List[Dict[str, Any]]:
    # Determine folder to search
    folder = search.folder if search.folder else 'INBOX'
    
    # Build IMAP search criteria
    criteria_parts = []
    if search.sender:
        criteria_parts.append(f'FROM "{search.sender}"')
    # ... other criteria
    
    with IMAPClient(IMAP_HOST) as client:
        client.select_folder(folder)
        imap_ids = client.search(search_criteria)
        # Fetch and return emails

Bootstrap Discovery & Resolution

Problem Identified

Date: 2026-02-11Context: Testing folder search functionality with test email requesting "Summarize completed work from today"

Discovery: Running python3 imap_client.py imap revealed:

• Only INBOX folder existed• All 15 emails (both processed and unprocessed) were in INBOX• No Done, Archive, Active, or other folders had been created• Only distinction: SEEN (read) vs UNSEEN (unread) flags

Root cause: Early versions of envoy/start instructions didn't include folder organization guidelines. Emails were processed but remained in INBOX, only marked as SEEN.

Why Only One Email Was Fetched

User question: "If they were NOT moved to Done, why did it only count one email earlier?"

Answer: Cron uses search_criteria='UNSEEN' (unread only). Of 15 emails in INBOX:

• 14 were SEEN (previously processed, marked as read)• 1 was UNSEEN (test email manually marked unread)• Cron fetched only the 1 UNSEEN email

The SEEN flag prevented re-processing, but all emails remained in INBOX rather than being organized into folders.

Design Resolution

Rather than mandate rigid folder rules or migrate old emails, implemented flexible LLM-based approach:

1. Enhanced envoy/start (v5) with email disposition guidelines2. Created detailed guide (envoy/email-organization) linked from envoy/start3. Added bootstrap awareness — instructions note that early on, completed work may still be in INBOX4. Natural transition — as new emails arrive with updated instructions, folders get created organically5. LLM judgment — Envoy decides folder placement based on content, not rigid rules

Implementation Details

IMAP Client Folder Operations

From imap_client.py:

• list_folders() — Lists all folders in mailbox• select_folder(folder) — Selects folder for operations• create_folder(name) — Creates new folder• move_message(id, dest) — Copies to dest, marks original deleted

Connection Details

• Protocol: IMAP4_SSL (encrypted)• Port: 993 (default IMAP SSL port)• Host: 'imap' (from orchestrator.py IMAP_HOST)• Auth: .netrc credentials for machine 'imap'

Command Line Testing

The imap_client.py can be run standalone for testing:

python3 imap_client.py imap [folder]

Shows available folders, message counts, and first message preview.

Design Principles

1. Automatic vs Manual — Folders created automatically on first use, no manual setup required2. Flexible not Rigid — LLM uses judgment about folder placement, not hard-coded rules3. Organic Growth — Folder structure emerges from email patterns rather than pre-planned hierarchy4. Search-Driven — Organization optimizes for future retrieval via folder-specific searches5. Bootstrap Friendly — System works during transition from flat INBOX to organized structure

Future Enhancements

Documented in envoy/todo:

• Folder listing capability for LLM• Multi-folder search (search several folders in one iteration)• Email size checking before fetching• Progressive summarization for large email sets