• Para bullet items not on separate lines: In the notes viewer, bullet items within a single para block (separate strings in the array) are rendered run-on rather than on separate lines. Visible on e.g. README/runnable in the Execution Model and Features sections. Fix belongs in the viewer rendering of para arrays.
• Source control for live gdata-server: ~/py/gdata-server on gravlax is still not a git checkout (deployed by scp/rsync of individual files, not a clone). Partially resolved 2026-07-08: discovered the local pomelo clone (~/py/gdata-server) was 4 commits behind origin/m -- a multi-store --name/private-instance feature had been pushed and deployed straight to gravlax but never pulled back locally. Rebased local onto origin/m, pushed, and confirmed gravlax's live gdata_mcp_server.py is now byte-identical to git HEAD. Remaining gap: gravlax itself is still not a git clone, so this local-to-remote drift can recur -- see the deployment-verification note in gdata-server/mcp-server for the check-before-rsync procedure to catch it early, until gravlax is made an actual clone.
• Notes written in isolation: Notes have been created without updating parent pages or adding cross-links. Need a proactive convention: whenever a note is created or significantly updated, immediately check and update parent/related notes. See README/orphaning.
• Full-corpus JSONHTL lint scan, findings piped to this todo: Run the existing gdata-server/linter (lint_jsonhtl.py --all) across every note in the store, not just ad-hoc single documents. Motivating example: TFG/note-for-James carried a non-standard nested meta: {...} object duplicating top-level metadata fields (found 2026-07-27, fixed by flattening to the standard schema) — exactly the kind of "any other top-level key is flagged as unrecognised" case the linter's Document Level checks already cover, but nobody had run it against this note. Two parts: (1) do a one-off --all run now and fix what it finds; (2) extend the linter's existing "Linter output as a note" future-improvement item so that instead of (or as well as) a standalone linter/report, each run appends any newly-found irregularities as dated entries directly to this todo list, so known-error classes get tracked and actioned rather than only reported. Candidate cadence: same cron/Envoy pass as the periodic reconciliation item above, or a standalone scheduled job.
• CertHub private notes still in old markdown-in-para style: certhub/spec/api and certhub/jks-design still use markdown bold/- bullet text inside para blocks instead of proper list/table blocks, the same pattern fixed 2026-07-31 in certhub/spec (markdown table → proper table block) and location-db/gazetteer (Source values list → table). Candidates for the next full-corpus lint pass above; certhub/improvements, certhub/next-steps and certhub/portal-cert-todo not yet checked.
• JSONHTL_SCHEMA / linter drift behind JSONHTL_SPEC (svg, image, details blocks): JSONHTL_SPEC added three block types — image, details, svg — on 2026-07-17, but JSONHTL_SCHEMA (updated 2026-07-03) and gdata-server/linter's (updated 2026-07-06) "known block types" list still only cover para, heading, codeblock, list, table. Found 2026-07-27 while checking whether the linter and spec are kept in sync. Since the schema's block definitions use additionalProperties: false on a closed oneOf, any document using svg/image/details would currently fail schema validation despite being spec-legal — the reverse of the usual drift direction (spec moved ahead of schema, not schema stricter than spec by design). Fix: add block_svg/block_image/block_details definitions to JSONHTL_SCHEMA and register the three types in gdata-server/linter's Blocks section, per the existing Sync Rule between the two notes.
• Periodic reconciliation via Envoy: Cron job sends a short mail to Envoy saying something like: "Scan the notes system and look for inconsistencies — stale status, missing links, orphaned pages, outdated todos. Correct what you can directly; send a mail to ask about anything that needs a decision." No bespoke scanner needed; Envoy already has notes access and can reason about what it finds.
• Re-evaluate read-on-trigger cadence (August 2026): The TRIGGERS note currently uses a freeform cadence — agents act on a trigger whenever they recognise it, with no fixed checkpoints. Assess in August whether this is working in practice (across Claude, Codex, Envoy) or whether fixed checkpoints (e.g. read-the-map before-execute and before-commit, tied to the SCP phases) give more reliable rule-reading. Decide then whether to keep freeform or switch.
• Bootstrap connector minimalism: Keep the MCP connector bootstrap intentionally tiny. It should direct unfamiliar agents to read README first and let the Notes cascade (README → TRIGGERS → workflow → project notes) determine behaviour. This allows conventions to evolve by editing notes rather than redeploying the connector.
• Review STATE_OF_PLAY scope: STATE_OF_PLAY is titled as storage architecture but includes an Optical RX Status section. Move that section to a domain-specific note or rename/split the note if it is intentionally broader.
• MCP interface improvements: Findings from hands-on MCP tests are recorded in gdata-server/todo/mcp-interface-improvements.
• JSONHTL documentation improvements: Assessment and recommended doc/linter improvements are recorded in gdata-server/todo/jsonhtl-documentation-improvements.
• ChatGPT notes review follow-ups: Additional MCP response-shape, documentation-drift, todo-schema, and workflow-split suggestions are recorded in gdata-server/todo/chatgpt-notes-review-2026-06-29.
• Return block IDs from get by default: Structured edits via batch or patch require block IDs, yet they are only returned when include_block_ids=True is passed. Since the docs explicitly recommend ID-based over index-based ops, IDs should be returned by default (or always) to eliminate the extra round-trip before every edit.
• Section-aware / structured edits: Most index corruption comes from agents reasoning about a flat block list when they mean to operate on a section (a heading plus its bullets). An op set that addresses sections directly — move_section, append_to_section(heading) — would let edits like "add this link under Papers" be expressed as intent rather than as fragile positional block ops. Lower priority than dry_run and move_block but the same theme: raise the abstraction so the common edit is also the safe one.
→ Now specced: proposals/section-editing (nested section blocks + heading/CSS-selector editing — the section.* op family). Phase 0 (allow + render the section block, non-breaking) shipped 2026-08-04. Related: proposals/verb-taxonomy.
• Scan for broken/dangling links: No process currently checks that note-to-note links (and MCP keys referenced in links) actually resolve. Need a pass — manual or scripted — over the keyspace to find dangling links (e.g. links left pointing at deleted transient notes such as tmp/newport-rhythm-2026-07-18, deleted 2026-07-20). Related to the backlink index item above but distinct: this is about validating existing links, not indexing them.
• Append-only log subsystem: Provide a log.md equivalent (per Karpathy's LLM-WIKI.md) using KV-store structures rather than a single flat note — day-bucketed chunks (log/d/YYYY-MM-DD) with prev pointers forming a linked list, a small log/head pointer, and an optional month/root index tree. A dedicated log.append(op, title, refs, note) endpoint would encapsulate head-read → bucket-append → index-update. Full design and rationale in proposals/log-structure. Minimal first step: just day-buckets + head pointer.
• Backlink index: No way to find which notes link to a given key. A backlinks <key> query (even read-only, no auto-update) would make rename maintenance and impact analysis tractable. Currently requires manually reading all candidate parent notes.
• Link graph — visualisation AND compiled index: Two related deliverables from one link-graph over the keyspace (nodes = notes, edges = links). (a) Visual graph for navigation and spotting orphans — a panel in gdata-browser or a standalone HTML artifact (the Obsidian graph-view idea). (b) Compiled graph index — a generated GRAPH_REPORT-style note (cf. the Graphify + Obsidian + Claude Code pattern, June 2026) that an agent reads as a single lookup instead of fanning out across many gets to reconstruct relationships. It would surface hub notes (high in-degree — the ones many notes depend on), cluster related notes, and flag orphans and dangling links. The token-efficiency claim in the wild is large (the Graphify writeup cites up to 70x fewer reads by replacing grep-fan-out with one graph query); the same logic applies here — reading one compiled index beats re-deriving the link structure each session. Rebuild on change (the notes analogue of Graphify's post-commit git hook is an Envoy/cron pass or a server-side recompute when any note's links change). This is also the natural consumer of the backlink index item above — backlinks are the in-edges; the graph report is the whole edge set compiled. Ties to the graph-visualisation gap identified in the Karpathy/Obsidian comparison session and to LLM-WIKI.md (compile-don't-re-derive).
• Private notes instance: Add a second notes_web.py / gdata-server instance on a separate port (e.g. 8022) serving a private key namespace. Should share the same OAuth auth layer as the existing instance (port 8021). Key design questions: separate database file vs. key prefix separation within the same DB; how the notes browser selects which instance to connect to.
• Version-change summary field: The README pattern if you already know version N, skip this relies on the reader knowing what changed. A short changed string field per document (e.g. "changed": "added megadoc trap section") would let an LLM decide whether to re-read without doing so, making the version-skip optimisation actually work.
• Pretty visualisation / graph view: Add a visual graph of note links (inspired by Obsidian's graph view). Nodes = notes, edges = links. Useful for navigation and spotting orphans. Could be a panel in gdata-browser or a standalone HTML artifact.
• Runnable sheet input fields fillable via control API: Currently sheet.inputs.set only works on input fields declared via input_prompts in the codeblock spec. Input fields that appear at runtime via input() calls cannot be pre-filled or submitted through the control socket. Both cases should be settable via the API so that runnable sheets can be driven programmatically without manual interaction.
• Proposal: implement table.fill_na: Replace null (or empty) cells in a table with a supplied fill value, optionally restricted to a named column. Signature: table.fill_na(block, value, column=None). Was listed in the supported-ops error string but never implemented and has never been called (zero hits in server log as of 2026-06-26). Low priority; add when there is a real use case.
• Add dry_run support (raised priority): A dry_run: true option for patch and batch returning the resulting document without persisting it. Promoted from nice-to-have after a concrete incident on 2026-06-28: a multi-op CONTENTS batch mixing deletes and positional inserts corrupted the index (links in wrong sections, an unrelated entry deleted, a duplicated item) and needed a full-document put to repair. dry_run would have surfaced the bad result before it was written — it is the strongest single guard against blind structural edits, since the agent cannot see the rendered document. Pair with returning the post-edit block list so the caller can diff.
• Reduce fragile JSON/string reparsing: Only unwrap JSON-encoded strings for block, fields, or ops when the incoming value is actually a string. Prefer native JSON objects from MCP callers. This should help avoid apostrophe/double-decoding problems.
• Make batch reliable from ChatGPT: batch is the documented safest path for multi-step edits, but it failed from ChatGPT at dispatch/resource resolution rather than with a Notes-level error. Fixing this should be high priority because it enables atomic edits with if_rev and block IDs.
• Prefer IDs by default for edit workflows: Since safe patching requires stable block IDs, either return block IDs from all get calls by default or add a dedicated get_for_edit(key) convenience endpoint that always returns {document, rev, block_ids}.
• Concept index (composition layer): A second navigational layer alongside the link graph — an inverse concept → notes map built by matching-pursuit decomposition of note embeddings at ingest, so retrieval is plain lookup rather than query-time similarity. Same compile-don't-re-derive logic as the link-graph item above, on the composition axis rather than the link axis. Pre-design rationale, mechanism (TBDs), honest costs, and the paired hierarchical-CONTENTS restructuring in ideas/concept-factorisation.
• Verb API implementation plan: Ordered build sequence (reorder → dry_run → MOVE/COPY → ETag → metadata ops), with the core-once/three-interfaces constraint, the gravlax clone/deploy reality, per-phase tests, and a definition of done. Starting note for the Claude Code session: gdata-server/todo/verb-api-implementation.
• Multiple same-anchor insert_after ops reverse-order (low priority): within one batch, N insert_after ops all targeting the same block_id insert in reverse of array order (each new block goes immediately after the anchor, pushing the previous one down). Documented as a quirk in README/mcp-note-editing and hit again 2026-07-22. Deliberately low priority: it's fully self-correcting — a single reorder fixes it and the reversal is obvious the moment you outline. The wasteful failure mode is the recovery, not the quirk: escalating to a whole-document put to fix one mis-placed block (also 2026-07-22). If ever tackled at source, the clean fix is to let insert_after accept an ordered list of blocks and insert them in array order. Meanwhile the mitigation is a workflow rule, now in README/agent (“Use targeted edits when safe” → recovery discipline): insert bottom-up or via reorder, and never escalate to put for a one-block slip.
• Verb taxonomy (overarching): Keep get/put as basic whole-document KV; move all structured ops (patch/batch/move/diff/dry_run) onto PATCH and POST /{key}, and map notes operations onto HTTP extension methods (MOVE for key rename, COPY, PROPFIND/PROPPATCH for metadata, If-Match/ETag for concurrency in place of if_rev). Many items below (dry_run, move, richer returns, metadata reads, prefer-IDs) are facets of this single design — see proposals/verb-taxonomy.
• Return richer success data from edits: Successful patch and batch calls should return at least the new rev, changed/inserted block IDs, and ideally the changed block or a compact summary. That makes verification easier for agents.
• Add simple convenience operations: Consider agent-friendly helpers such as append_para, append_heading, or auto_update_meta. Example: append a paragraph while automatically incrementing version and setting updated to today. This avoids forcing callers to construct full JSONHTL for common append-only notes. The log subsystem is the canonical use case motivating an atomic append_entry op.
• Plain get(key) response issue in ChatGPT: get(key) without include_block_ids succeeds but returns an empty resource in the ChatGPT connector, while get(key, include_block_ids=true) returns the full document. Investigate the plain-get response rendering/resource path. This may be a connector rendering issue rather than a storage issue.
• Clarify patch as single-op convenience and batch as preferred multi-op path: Document patch for simple one-shot edits and batch for coordinated edits, especially when metadata needs updating in the same logical change.
• 2026-06-28: reorder op implemented across HTTP (POST+PATCH), MCP tool, and CLI. Strict validation (422 on missing/unknown/duplicate IDs), if_rev optimistic concurrency. 18 HTTP + 11 MCP tests. Deployed to gravlax.
• 2026-06-28: Logged a structural-edit incident and acted on it. A multi-op CONTENTS batch mixing deletes with positional inserts corrupted the index; repaired via full put. Root-cause analysis in troubleshooting (sequential-application trap). Actions taken: raised dry_run priority, added move_block and section-aware edit requests above, and recorded the working rule to prefer full put for reorganisations.
• ~~MCP SSE -32602 on reconnect~~: FIXED 2026-06-28. Root cause: SSE client caches session state and skips initialize after server restart. Fix: changed ~/.claude.json "type": "sse" → "type": "http" (Streamable HTTP, stateless). See note gdata-server/session-2026-06-28.
• 2026-05-28: Generalized MEMORY for all LLM assistants, added MEMORY/claude and MEMORY/codex, refreshed the historical Envoy status wording, split README/tools into a tooling index plus README/command-line-tools and README/mcp-tools, normalized CLI codeblocks, updated current gdata-server deployment summary, and cleaned up the duplicate Future heading.
• 2026-07-07: HTML renderer metadata list formatting fixed: updated notes_web.py in /home/john/git/gdata-server so list metadata such as tags renders as comma-separated text instead of Python repr syntax. Deployed manually with rsync -av /home/john/git/gdata-server/notes_web.py john@gravlax.critchley.biz:~/py/gdata-server/ and restarted gdata-mcp-server.service. Verified README/mcp-note-editing renders tags documentation, tools, mcp, notes.
• ~~notes load should not silently store ops-shaped input as a document~~: FIXED 2026-07-08. PUT (and load/write, which are PUT under the hood) now rejects a JSON array shaped like an ops/patch payload (a list of objects with an 'op' key) instead of silently storing it verbatim. Other non-dict JSON (lists, strings, numbers) is still accepted -- only the specific ops-shaped corruption pattern is blocked. Fixed in both gdata_server.py (handle_PUT_request) and gdata_mcp_server.py (db_put, backs the MCP put/patch/batch tools). Deployed to gravlax and verified live.
• ~~Batch/patch apostrophe encoding bug~~: FIXED 2026-07-08. _parse_json_robust (used to tolerate \' where a plain apostrophe was meant) had a redundant duplicate condition and silently re-raised the pre-repair error on double failure, hiding what the repair attempt actually produced. Simplified the check and added a combined error message showing both the original and post-repair parse errors. Same fix applied to both gdata_server.py and gdata_mcp_server.py; test suite (196 tests) passes; deployed to gravlax.
• 2026-07-10: outline / block-head read op implemented and deployed: added plain-text block previews plus stable IDs via REST POST /{key} {"op":"outline"} and MCP outline tool. Added tests in test_patch_api.py and test_mcp_tools.py; verified live on gravlax. During deployment, drift check caught live-only 2026-07-08 gdata_mcp_server.py fixes missing from the local clone and merged them before rsync; source-control reconciliation remains a process issue.
• 2026-07-27: "HTML renderer leaks raw meta object" — root cause found, not a renderer bug: investigated by comparing TFG/note-for-James (broken footer) against writing/failure-does-not-increase-authority-linkedin (renders fine). The linkedin note uses the standard flat top-level metadata schema (title/version/created/updated/tags alongside content, per README/metadata). The James note instead had a non-standard nested meta: {...} object duplicating title/created/tags alongside separate top-level version/updated fields — the renderer doesn't recognise the nested meta key so dumps it raw, and the duplicated version/updated produced the second footer line. Fixed by flattening TFG/note-for-James to the standard schema via put (rev r16). Was a data problem specific to that one note, not a systemic renderer flaw; no server-side fix needed.
• ~~Cell name label low contrast in runnable sheet~~: FIXED 2026-07-27. Set an explicit dark foreground colour (wx.Colour(30, 40, 70)) on self.name_label in sheet_ui.py so the cell name renders clearly against the light blue-grey (#ECF0F7) header strip.
• 2026-07-28: Per-key HTML view implemented (Claude Code): notes_web.py content-negotiates on Accept at /notes/{key} (proxied publicly as https://www.critchley.biz/notes/{key}), serving rendered HTML by default plus JSON/YAML/Markdown alternates. Full design in gdata-server/http-formats. Confirmed live for fitness/bodypump-attendance (SVG chart and table render correctly). Closes the gap raised 2026-07-27.
• 2026-08-04: Source control for live gdata-server — RESOLVED. ~/py/gdata-server on gravlax is now a real git clone: git init in place → fetch → reset origin/m, then checked out all tracked files to HEAD (2 stale runtime files refreshed, 129 missing populated). Operational/host-local files (secrets, .bak-, backups/, gdata_module/, patches/, resetpw) are in .git/info/exclude; the repo's .gitignore covers .gdbm/.log/__pycache__/tmp/. Working tree clean, tracks origin/m, git pull deploys, and gdata_mcp_server.py/notes_web.py/gdata_oauth.py hash-match HEAD. GitHub SSH wired in ~/.ssh/config (repo-scoped deploy key generated, pending registration; working account key in use meanwhile). Verified live: both services active, import gdata OK, public /notes/ 200, private gate 302/200. Closes the deploy-drift class and unblocks the Envoy execution-sandbox prerequisite.
• 2026-08-04: JSONHTL_SCHEMA / linter drift (svg, image, details) — RESOLVED. Added block_svg/block_image/block_details to JSONHTL_SCHEMA (v4) and registered them in gdata-server/linter (v5); spec-legal docs using those blocks now validate (verified live via jsonschema against the served schema). The same pass added the nested section block (Phase 0 of proposals/section-editing) to spec/schema/linter and both renderers.
✅ Shipped 2026-08-05 — codeblock canonicalisation (visible, not silent). Root cause of the certhub/dap note rendering empty code on the web: codeblocks stored with language/text were accepted on write, tolerated by desktop/markdown renderers, and only failed on notes_web. Fix: jsonhtl_canon.py shared reader (all renderers), db_put normalises + self-heals, REST GET/PUT emit 203 Non-Authoritative Information + Warning: 299 + X-GData-Warnings for non-canonical docs (canonical stays 200), MCP surfaces warnings in-band. Deployed to gravlax (both services) with no outage; whole corpus migrated to canonical (public 0/501, private 0/92). Design + as-built: proposals/codeblock-canonicalisation. Follow-ups there: shared block constructors, explicit schema version, render-check (empty-from-non-empty = error), and the stale ~/dbcerts/output/fix_cert_pwds_note.json template (Codex-owned).