jpeg-obscura Session Retrospective: Lessons Learned

Reflections from v0.5 bootstrap on effective goal accomplishment — general principles, domain-specific patterns, and process improvements.

General Principles (Broadly Applicable)

1. Verification-Driven Development

When restored images looked wrong, I didn't accept them at face value. Instead: quantified the problem (pixel-level delta) → traced root cause (missing salt) → verified fix worked (0.15-0.28 bytes).

Why: Visual inspection deceives. Metrics are truth.

Application: For image/video/media projects, always measure pixel-level quality. For data transforms, always compare input ↔ output quantitatively.

2. Deterministic Round-Trip Testing as Gold Standard

Before trusting file-based restoration, tested in-memory: load → scramble → restore all in RAM. Result: 0.23 bytes = mathematically perfect.

This proved the algorithm was correct before blaming JPEG codec or file format.

Why: Separates algorithm bugs from I/O artifacts.

Application: For reversible transforms (encryption, compression, permutation), always have in-memory round-trip test as reference. File-based testing inherits I/O codec artifacts.

3. Know Your Error Surface Early

Understanding JPEG lossy encoding's effect: 0.15-0.28 bytes Δ is good (codec artifact); 49 bytes Δ would be bad (permutation failed).

This let me distinguish signal from noise in metrics.

Why: False negatives (rejecting good results) waste time; false positives (accepting bad results) hide bugs.

Application: Characterize your I/O medium's noise floor before testing. For JPEG: ~0.5-2 byte uncertainty/pixel. For crypto: zero uncertainty. This changes what metrics you trust.

4. Context Capture as Debugging Tool

The bug (unrestored images) was solved by capturing context: capture salt at scramble → use immediately for restore.

Result: perfect restoration. The algorithm was correct; the problem was information loss between steps.

Why: Intermediate state loss is a common pipeline bug.

Application: When a pipeline fails, ask 'What state is being lost between steps?' Often the fix is capturing and threading that state through.

Domain-Specific Principles (Image Transformation)

5. Block-Grid Snapping Reduces Surprises

Forcing regions to 8×8 block boundaries eliminates partial-block edge cases and makes permutation math clean.

Users expect pixel-aligned regions anyway.

Application: For block-based transforms (JPEG, video codecs, DFT), always snap geometric requests to block grid. Document the snapping.

6. Deterministic Streams Beat True Randomness for Reproducibility

HMAC-SHA256 counter-mode gives reproducible permutations (same key → same shuffle), cross-language compatibility, and no random seed management.

True RNG requires seed storage. Deterministic generation requires only key.

Application: For reversible operations (scrambling, diffusion, erasure coding), use deterministic PRNG keyed from user passphrase. Never trust true RNG for reproducibility.

7. Permutation-Only is Visually Misleading But Sound Foundation

Permutation alone isn't cryptographically secure (99.3% pixel change looks 'secure' but isn't). However: it's a perfect foundation for v1 (add diffusion), demonstrates the core concept, and restoration metrics prove it works.

Permutation is the hardest part technically; diffusion layers on top.

Application: For security phasing: implement hard part (reversible core) first with weak security. Add layers later. Users benefit now; future versions add strength without rewrite.

8. Defer Complex I/O Design (Wisely)

v0.5 deliberately doesn't embed metadata in APP15 because: core algorithm is decoupled from metadata strategy, multiple strategies can be tried in v1 without rewriting transform code.

Samples work fine with external salt capture.

Why: Premature I/O design locks you into one approach.

Application: For versioned features, defer complex I/O (metadata, headers, serialization) until core algorithm is proven. This lets you iterate on algorithm without breaking file format.

Process Principles (Effective Collaboration)

9. Notes System > Git Markdown (For Evolving Knowledge)

Moved all documentation to notes system early: easy to link/cross-reference, easy to update without committing, git stays clean (code only).

Why: Code reviews bloat when .md docs change. Notes iteration is separate from code review.

Application: Use git for stable deliverables (code, scripts, samples). Use notes for documentation that evolves. Link from CLAUDE.md to notes.

10. Capture Commit Context in Memory

Recorded git baseline in memory with commit hash, tag, metrics achieved, next phase dependencies.

Why: Future sessions can recover exact state. Git history alone doesn't capture why a commit matters.

Application: On every major milestone, record git hash + context in memory. Makes context-switching painless.

11. Incremental Validation with Real Data

Used actual images from WebDAV (3840×2160, HEIC-converted), not toy data. Caught real EXIF orientation issues, tested actual JPEG quality, observed real file sizes.

Why: 10×10 test images hide bugs that 4K images expose.

Application: Use production-scale data early. Don't wait for 'real' data later.

Technical Patterns That Worked

12. Key Derivation with Separate Keys

Pattern: scrypt(pass, salt) → (perm_key + check_key). Split output into separate keys for different purposes, prevents cross-contamination, allows future versioning.

Application: For any KDF, split output strategically. Don't reuse one key for multiple roles.

13. Fisher-Yates with Rejection Sampling

Counter-mode stream generates random values, rejection-sample to unbiased randbelow(). Simpler than rejection-free algorithms, standard math, deterministic stream.

Application: For deterministic shuffling, combine counter-mode PRNG with Knuth-Yates rejection sampling.

14. In-Memory Permutation Before File I/O

Correct: permute_blocks(array, region, p, 8) in-memory, then save_image(array, 'file.jpg', quality=95) with one lossy encode.

Wrong: load → permute → save → load → permute → save (double-encodes).

Why: JPEG encode happens once, pixel permutation happens cleanly in memory first.

Application: For image transforms + JPEG: do all pixel manipulation in-memory, JPEG encode only once at end.

Why This Session Succeeded

1. Clear definition of success (samples with verified restoration)

2. Metrics-first debugging (quantified rather than assumed)

3. Staged validation (in-memory → file-based → samples)

4. Root cause analysis (salt loss, not algorithm failure)

5. Decoupled concerns (core algorithm independent of metadata strategy)

6. Context capture (commit hashes in memory for recovery)

Next Session Prep

Baseline at git tag v0.5-baseline (commit 9acab30) — all code checked in, working tree clean.

Samples on WebDAV demonstrate working tool.

Documentation at ideas/jpeg-obscura/* in notes system.

To Carry Forward

Use metrics, not intuition. Test algorithms in-memory before files. Defer complex I/O decisions until core is proven. Document git context in memory, not just git history.

version 1  ·  created 2026-08-31  ·  updated 2026-08-31  ·  tags jpeg-obscura, retrospective, process, lessons