This note documents the MCP access work done on 2026-05-28 to make the notes server usable from Codex.
The combined server is gdata_mcp_server.py under ~/py/gdata-server on gravlax. It serves REST on 127.0.0.1:8020 and MCP on 127.0.0.1:8023 against .agent_notes.gdbm. Public access is proxied via https://www.critchley.biz.
• gdata_oauth.py now supports OAuth Dynamic Client Registration via POST /oauth/register.
• OAuth authorization-server metadata now advertises registration_endpoint.
• Dynamically registered clients are stored in the OAuth token JSON store and are accepted by /oauth/authorize and /oauth/token.
• Token endpoint auth supports both client_secret_post and none for dynamically registered loopback clients.
• The MCP bearer-auth challenge now includes resource_metadata pointing to the protected-resource metadata URL, which helps clients discover OAuth metadata.
Local Codex config at /home/john/.codex/config.toml has a notes MCP server entry for https://www.critchley.biz/mcp/ with an Authorization bearer header. After restarting Codex, the notes MCP tools loaded successfully.
During setup the OAuth password was temporarily changed to jc1234 to mint a token. The previous OAUTH_PASSWORD hash was restored afterward. The backup hash is retained at ~/py/gdata-server/.oauth_password_hash.backup.
See gdata-server/testing for the full test suite reference (test_mcp_tools.py, test_patch_api.py, test_reorder.py, test_json_robust.py, test_gdata_mcp_rest.py, test_mcp_transports.py).
The live directory ~/py/gdata-server is not a git working tree and has no detected upstream remote. The upstream repository was identified as git@github.com:john-critchley/gdata-server.git and cloned to ~/tmp/gdata-server-github on branch m.
Update 2026-08-04: resolved — ~/py/gdata-server IS now a git clone of origin/m (converted in place). The statement above describes the prior state.
The OAuth patch was applied in that clone, committed as c277099 Add OAuth dynamic registration for MCP clients, and pushed to origin/m on GitHub. The saved patch remains at ~/py/gdata-server/patches/gdata_oauth_codex_mcp_oauth_20260528.patch.
Push used the dedicated SSH key at ~/.ssh/github_gdata_server_ed25519 after its public key was added to GitHub.
MCP interface testing and recommended improvements are documented in gdata-server/todo/mcp-interface-improvements.
Superseded 2026-08-04: ~/py/gdata-server on gravlax is now a real git clone tracking origin/m. Deploy is: git -C ~/py/gdata-server pull --ff-only then restart the affected service(s). The rsync-from-pomelo steps and the "verify before rsyncing" drift procedure below are historical — local↔live drift is no longer possible since the live tree IS the clone. Host-local/operational files are in .git/info/exclude. See gdata-server/todo (2026-08-04, "Source control … RESOLVED").
gravlax is accessible as ssh gravlax (defined in /etc/hosts on pomelo as 13.63.176.7). The live directory ~/py/gdata-server on gravlax is NOT a git working tree. Deploy by rsyncing changed files directly from pomelo — do not use the GitHub clone on gravlax as a deploy mechanism (the clone at ~/tmp/gdata-server-github is for source control only). After pushing to GitHub on pomelo, deploy with:
# Run from pomelo — rsync changed files then restart
rsync -av gdata_mcp_server.py notes_client.py gravlax:~/py/gdata-server/
ssh gravlax "sudo systemctl restart gdata-mcp-server.service"Copy additional files as needed (e.g. gdata_server.py, gdata_mcp_server.py). After restarting, verify with:
ssh gravlax "sudo systemctl status gdata-mcp-server.service | head -6"Note: if the restart fails to bind port 8020, a stale process may be holding it. Kill it with:
ssh gravlax "ps aux | grep gdata_mcp_server | grep -v grep" # find PID
ssh gravlax "sudo kill <PID>"The running process was restarted manually with the existing environment. If this service is later moved to systemd or another supervisor, ensure OAuth environment variables and the current code path are preserved.
The MCP server is accessible from pomelo via stunnel. Local stunnel (/etc/stunnel/stunnel.conf on pomelo) connects 127.0.0.1:8023 → gravlax:18023 → gravlax:127.0.0.1:8023. Claude Code on pomelo is configured in ~/.claude.json with transport type http (Streamable HTTP, stateless).
Do NOT use "type": "sse" in the Claude Code MCP config. The SSE client caches session state and after a server restart it skips the initialize handshake on reconnect, causing MCP error -32602 (Invalid request parameters) on every tool call. The server log shows: WARNING - Failed to validate request: Received request before initialization was complete.
Use "type": "http" (Streamable HTTP) instead. This is stateless — every POST to /mcp/ is self-contained, no session needed. The correct entry in ~/.claude.json:
"gdata": {
"type": "http",
"url": "http://127.0.0.1:8023/mcp/",
"headers": { "Authorization": "Bearer <token>" }
}Fixed two bugs (full technical detail in troubleshooting): notes load/PUT silently storing an ops-shaped payload as a document (permanently breaking patch on that key), and _parse_json_robust mishandling apostrophes across every ops/block/fields/order call site. Found and fixed two more instances of the apostrophe bug beyond the original report: the MCP batch/reorder tool handlers were discarding the repaired error message, and gdata_mcp_server.py's own REST batch route (separate code path from the MCP batch tool) still used plain json.loads with no repair at all. Added test_json_robust.py and test_gdata_mcp_rest.py, extended test_mcp_tools.py and test_patch_api.py. 295+ tests passing.
The rsync-the-whole-file instruction above is dangerous if the local clone is behind origin. Discovered 2026-07-08: this local clone (~/py/gdata-server on pomelo) was 4 commits behind origin/m (missing the multi-store --name/private-instance feature, commits 38f3871..dc650fe). Gravlax's live file already had that feature deployed. A blind rsync of the stale local file would have silently reverted it in production.
Correct procedure before any deploy: git fetch origin && git log HEAD..origin/m --oneline to check for missed upstream commits -- pull/rebase first if anything shows up. Then, whether or not gravlax's live file exactly matches your local HEAD, diff before overwriting: ssh gravlax "cat path/to/file.py" > /tmp-equivalent-under-$HOME, then diff <(git show HEAD:file.py) that-file. If they differ beyond your intended change, do NOT rsync/scp the whole file -- extract just your diff as a patch (git diff <old> <new> -- file.py) and apply it to gravlax's actual current copy (patch -p1 gravlax-copy < your.patch), verify the resulting diff shows only your intended change, then deploy that merged result. This is exactly how the 2026-07-08 fixes were deployed, and it caught the divergence before anything was lost.
Implemented and deployed a read-only outline operation for safe block addressing. REST supports POST /{key} {"op":"outline","preview_chars":N}; MCP exposes an outline tool after client tool metadata refresh. Response shape is {rev, blocks:[{id,index,type,preview}, ...]}. Verified live on gravlax against README after restarting gdata-mcp-server.service.
Deployment note: the pre-rsync drift check caught that live ~/py/gdata-server/gdata_mcp_server.py on gravlax still had the 2026-07-08 JSON parsing / ops-shaped PUT safety fixes that the local clone lacked. Those live-only changes were merged into the local file before deployment, preventing a regression. Source-control reconciliation is still needed so the clone and origin carry the same fixes as production.
Symptom: Codex Apps Notes tools such as notes.get and notes.batch returned HTTP 504 after about 120 seconds, while the local pomelo notes CLI against 127.0.0.1:8021 still worked. Reading this note first showed the production path: Codex connector → https://www.critchley.biz/mcp/ → gravlax Apache → gdata_mcp_server.py on 127.0.0.1:8023; REST is on 127.0.0.1:8020.
curl http://127.0.0.1:8020/hosts/radagast/source-upgrade returned 200 in milliseconds, and unauthenticated curl http://127.0.0.1:8023/mcp/ returned the expected 401 bearer challenge.curl https://www.critchley.biz/.well-known/oauth-protected-resource/mcp timed out.:443 Recv-Q 506/511 and :80 Recv-Q 458/511; /var/log/apache2/error.log repeated AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.; many Apache children were stuck and had to be SIGKILLed during restart.sudo systemctl restart apache2 on gravlax. After restart, listener backlogs cleared, OAuth metadata returned 200, notes HTML returned 200, unauthenticated MCP returned 401, and Codex notes.get returned successfully.Troubleshooting order for future 504s: first check public Apache before debugging GDBM or MCP tool code: ssh gravlax "sudo ss -ltnp | egrep ':80|:443|:8020|:8023'; sudo tail -80 /var/log/apache2/error.log". If scoreboard is full or listener backlogs are high while 127.0.0.1:8020 and 127.0.0.1:8023 are healthy, restart Apache and then investigate why workers are sticking.
Open problem, not solved by the restart: this Apache scoreboard exhaustion has been seen before and then appeared to clear, so treat it as a recurring production problem rather than a one-off. The restart is only a service-restoration step. Deeper follow-up should identify what keeps Apache workers stuck long enough to fill the event MPM scoreboard: long-lived MCP/SSE/proxy connections, bad keepalive/proxy timeout settings, external scanners holding sockets, WSGI handlers, or an undersized/incorrect mpm_event configuration. Collect evidence before the next restart if possible: apachectl fullstatus if mod_status is enabled, per-state connection counts from ss -tanp grouped by peer and state, Apache MaxRequestWorkers/ServerLimit/ThreadsPerChild/KeepAliveTimeout/ProxyTimeout settings, and vhost access/error logs around the first AH03490 entry.
Temporary watchdog installed 2026-09-19: because cron must run from pomelo rather than gravlax, /home/john/bin/check-gravlax-apache-scoreboard now runs hourly from John's pomelo crontab at minute 17. It SSHes to gravlax, reads only new bytes from /var/log/apache2/error.log, and if it sees new AH03490 / scoreboard is full lines it runs sudo systemctl restart apache2 on gravlax. State is kept in /home/john/.cache/gravlax-apache-scoreboard.state so historical errors do not trigger restarts. Actions are logged locally with logger tag gravlax-apache-scoreboard. The first run on install baselined the current log end and exited 0 without restarting Apache.