Rainbow — Display Loss Incident, 2026-10-16

Parent: Rainbow development

Reported symptoms

John reported that the monitor lost signal and the computer required pulling its power. He later reported a changed CPU fan sound, with no further evidence about SSH/audio responsiveness. Do not describe a CPU, thermal, GPU or power fault as established.

Surviving evidence

Previous boot's journal stops at 11:59:13 BST. Current boot began at 12:08:39 BST. Rainbow build files were modified around 11:59:47–11:59:53. The renderer and test executable paths exist but are zero-length; build.ninja, .ninja_log and several other generated files are also zero-length. This is consistent with incomplete/unflushed writes after an abrupt power-off, but does not identify the original fault. No output directory, optics cache or CTest result directory existed when inspected.

Last persisted sysstat sample at 11:50:01 BST: about 97.6% aggregate CPU idle and about 16 GiB available memory. These ten-minute samples do not cover the final build interval, so they cannot exclude a later CPU load spike or memory pressure. No surviving near-end kernel OOM, thermal shutdown, panic or GPU hang/reset entry was found. pstore was empty and no new crash report was present. Historical temperature/fan data were unavailable.

Xorg.0.log.old contains repeated AMDGPU flip queue/Page flip failed: Invalid argument warnings earlier in the session; its last entries are device events around 11:44. The earlier display warnings are a lead, not evidence that a GPU error caused the final blackout. Post-reboot AMDGPU initialization/reset messages are startup activity, not proof of a pre-reboot failure. Journald reports an unclean/corrupt journal on restart.

Transcript limitations

The last initially visible tool result showed a CMake configuration failure. Filesystem evidence then demonstrated later build writes, so the initial statement that no build had started was corrected. The local Codex transcript contains a corrupted record at line 544; surviving records did not recover the later build command. Do not claim to know the exact compile concurrency, completion or whether tests ran. There is no surviving evidence of a completed render.

Actions and next evidence

Development workloads were stopped while read-only investigation ran. Source and tests survive; generated build outputs need rebuilding. No stress test, kernel/driver change or hardware setting change was performed. John requested remote telemetry; the implemented logger and gravlax receiver are described in rainbow/dev/monitoring. Continuous local collection needs user-service activation. Resume modest monitored work only after checking fresh remote acknowledgements; inspect available temperatures, power, GPU/kernel messages and workload timing if another failure occurs.

Telemetry activation and recovery

Follow-up after remote logging was enabled

John explicitly authorized localhost SSH for service activation. With real continuous remote acknowledgements verified, modest builds/renders resumed and completed, producing PNG/EXR images. No kernel events were captured during the resumed work; maximum sampled CPU temperature reached 86 C during compilation and a later post-render sample was 61.5 C. These observations do not identify or rule out a cause for the earlier blackout. See rainbow/dev/review for successful validation and profiles.

version 2  ·  created 2026-10-16  ·  updated 2026-10-16