See also: ask — main tool overview
Round 2 results: envoy/thread-chain-review2
notes-put utility: README/notes-put
How it was used (thread chain review)
Files passed to ask_with_files_structured.py:
- thread_chain_description.txt — system overview and feature description
- thread_chain_diff.patch — git diff from the base branch
- orchestrator_thread_chain.py — full source (2000+ lines)
- imap_client.py — IMAP wrapper
- envoy_schema.py — Pydantic models
- new_envoy_response_schema.json — JSON schema for structured LLM output
System prompt: experienced senior Python engineer, email/IMAP expert, LLM application architect.
Question: broad 'is it correct and robust?' — produced 17 numbered findings.
Model: gpt-5.3-chat-latest. gpt-4o hit 30k TPM limit with full file included.
What it found (round 1)
See: envoy/thread-chain-review
- Missing f-string: #{i} literal sent to LLM (FIXED)
- new_emails/dir() idle detection hack (FIXED)
- Cache key .strip('<>') strips multiple chars (FIXED)
- Silent except: pass in thread discovery (FIXED)
- Date sort for ancestors wrong — use References order (FIXED)
- Folder search explosion — noted
- IMAP fetch data[0] assumption — noted
- Continuation/thread cache duplication — noted
Lessons: what worked well
- Description file for system context: reviewer oriented fast, no time lost on basics
- Diff + full source: sees old vs new AND full context
- Domain-specific system prompt (IMAP, LLM apps): got domain findings
- gpt-5.3 produced numbered findings with code examples
How to improve Ask for code review
- Structured JSON output (--schema-file): get {severity, location, description, suggestion} for each finding — easier to track fixes
- Focused questions per area: IMAP robustness, prompt engineering, production failures — not one broad question
- Diff only (no full source) when diff is self-contained — saves tokens, avoids TPM limit
- Multiple passes: broad first, then targeted follow-ups
- Include test files if they exist
- --no-document-references for cleaner prose when headers aren't needed
- Save structured findings as JSON, track which were fixed vs deferred
Future Ask code review use cases
- Pre-merge review: run ask on every PR diff with standard system prompt + schema
- Security audit: OWASP-focused system prompt
- Schema drift detection: compare Pydantic model vs JSON schema vs LLM prompt description
- Prompt engineering audit: send messages[] array, ask if thread context is clear and well-ordered
- Performance review: identify O(n*m) patterns
- Refactoring suggestions: find code ready for extraction