NOTE (2026-07-08): the routing table below is stale — the map list in process_emails.py has moved on significantly (vodafone_handler, imap_deliver routes, etc. added/removed since this doc was last fully updated). For the current routing table, see popit3/email-routing. This doc is still current for the do_processing() mechanics and the size-logging feature below.
This module is the actual brain of the routing. popit3.py fetches mail and stores it; process_emails.do_processing() decides what happens to each message based on the To: address.
The map list in process_emails.py is an ordered list of (to_address, handler_function) pairs. Each incoming email is tested against all entries — an email can match MULTIPLE handlers if it matches multiple To: addresses.IMPORTANT: john.js@ has TWO entries in the map:1. newparser_jobserve.process_js_mails — parses and scores the job2. js_mail_spool.store_messages — archives a copy to jsMail/ for debuggingBoth run for every john.js@ email.Spam/unwanted addresses (e.g. john.cvl@, john.adzuna@, john.monster@, john.totaljobs@) are handled by lambda functions that return ALL UIDs for that address — effectively marking every email for deletion without reading content.
Three separate WebDAV client instances are configured:• webdav_client — base path /john/ — used for john@ mail forwarding• js_webdav_client — base path /john.js/ — used for john.js@ job alert forwarding• wf_mail_spool uses webdav_client (john path) — for john.wf@ mailWebDAV credentials come from ~/.netrc, host webdav.critchley.biz. Each client uses a different base directory on the same server.
Three MailSpool instances for local Maildir archiving:• mail_spool — ~/Mail (for john@ general mail). delete=True (default) — removes from spool after forwarding• js_mail_spool — ~/py/popit3/jsMail. delete=False — keeps archive copies• wf_mail_spool — ~/py/popit3/wfMail. delete=False — keeps archive copies
1. Open GDBM database2. For each (address, handler) in map: a. Find all emails with matching To: header b. Call handler(db, uids) → returns set of UIDs to delete c. Accumulate UIDs to delete3. Delete accumulated UIDs from GDBM4. Close databaseNote: An email going to multiple addresses (john.js@ AND john@) will be processed by ALL matching handlers. The email is only deleted from GDBM once all handlers have run and if ANY handler returns its UID for deletion.
The To: header matching uses substring matching on the decoded header. This means an email with 'To: john.js@critchley.biz, john@critchley.biz' would match BOTH john.js@ and john@ handlers. The gdata class provides the header decoding.
do_processing() now logs a second summary line alongside the existing per-recipient count: Sizes by recipient (address, count, total_bytes, avg_bytes): followed by a JSON list of (address, count, total_bytes, avg_bytes) sorted by total_bytes descending. Look for this line in out.log after each run.
Purpose: help decide which recipient/alias classes are worth building automated handlers for — high total-byte classes (e.g. large attachments, long threads) or high average-size classes stand out at a glance. This is separate from the existing count-only summary line just above it, which only shows how many messages per address, not their size.