Project Testing Patterns

version 3  ·  body ## Rule Test code lives in the repository, separately from application code. Never leave tests only in `/tmp/` or other ephemeral locations. ## Two-tier test structure For every project, establish two tiers of tests: ### Tier 1 — Standalone tests - **No external build system required** — a single `g++` / `cc` command compiles and runs them - **No heavy dependencies** — no gtest, no full application build, no database, no network - **Fast** — run in seconds; must be the first thing checked after any change - **Location:** `src/test/<component>_standalone_test.cpp` or a `tests/standalone/` directory - **Build instruction:** embedded as a comment block at the top of the file - **Coverage:** pure unit tests of the component being developed, edge cases, error paths, regressions for every bug found Write standalone tests *first*, before the integration tests. They catch bugs cheaply. ### Tier 2 — Integration tests - **Use the project's test framework** (gtest, pytest, etc.) - **Exercise the full stack** — real database, real file I/O, real API surface - **Location:** `src/test/`, `tests/`, or wherever the project's test runner expects them - **CMakeLists / build system:** wired in so they run as part of `cmake --build; ctest` - **Coverage:** end-to-end scenarios, regression for bugs found in integration ## What to test in a new feature 1. **Happy path** — the normal case works end-to-end 2. **Error paths** — every failure mode returns the right error, doesn't crash 3. **Edge cases** — empty inputs, boundary values, off-by-one conditions 4. **Round-trip fidelity** — data written equals data read back (byte-identical if applicable) 5. **Idempotency** — repeated operations don't corrupt state ## On finding a bug 1. Write a test that reproduces it *before* fixing it (confirm the test fails) 2. Fix the bug 3. Confirm the test passes 4. Document the bug and fix in `<project>/dev/decisions` ## Test documentation Document every test suite in `<project>/dev/testing`: - File location - How to build and run (exact command) - What it covers (table of sections/cases) - Any prerequisites (test data, plugins, env vars) ## Isolated runtime environments When a test command adds to a language or tool search path, preserve the caller's existing path unless deliberate isolation is the purpose of the test. Replacing it can silently hide user-local or project dependencies and create a false failure. For example: ```bash PERL5LIB="test/lib:src/api:$PERL5LIB" prove ... ``` If isolation is intentional, explicitly list every required dependency and record that fact in the test documentation. ## Reviewing a branch after a base-branch refresh After merging or rebasing the base branch, review the actual branch delta before treating every hunk as feature work: 1. Check the changed-file set and whitespace with `git diff --name-status origin/master...HEAD` and `git diff --check origin/master...HEAD`. 2. Inspect shared-file hunks in context. 3. For an unexpected exact string, use `git log -S '<string>' -- <file>` to identify the commit that introduced or removed it. This catches unrelated changes accidentally carried through a refresh while preserving intended nearby edits.  ·  children