e2e-test-strategy

E2E Test Strategy

A test pyramid that catches the bugs that matter, runs fast, and doesn't flake.

Procedure

The Pyramid

       ╱  E2E  ╲              few — 5-20 critical flows
      ╱─────────╲
     ╱Integration╲           some — key API + DB combinations
    ╱─────────────╲
   ╱     Unit     ╲         many — every function, every branch
  ╱─────────────────╲

Don't invert it. Lots of E2E = slow, flaky, low-signal.

Decision Tree

For any test case, ask in order:

Can I test it with a pure function? → Unit
Can I test it with one module + mocked deps? → Integration
Does it span network / browser / external service? → E2E

Behavior	Layer
`validateEmail(s)` rejects bad input	Unit
`UserRepo.create()` inserts and returns	Integration
`POST /signup` returns 201 + sends email	E2E
Drag-drop reorders list	E2E
Pricing tier changes entitlement	Integration
Stripe webhook triggers subscription	E2E

Which E2E Flows Matter

E2E is expensive. Pick the 5-20 flows where a regression would cost the most.

Always E2E	Never E2E
Sign-up → first value moment	Internal helpers
Core monetization (checkout, subscription)	Pure UI styling
Critical auth (login, password reset)	Bulk data imports
Cross-service contracts (webhooks)	Background jobs
Critical admin actions

Rule: "What is the user-visible failure cost if this regresses?" Catastrophic → E2E. Cosmetic → unit or skip.

Test Data: Seeding

Bad: production DB or saved dumps.

Good: deterministic per-test seeding.

async function seedUser(opts: { plan?: 'free' | 'pro' } = {}) {
  const id = `test-${randomUUID()}`;
  await db.users.insert({ id, email: `${id}@test.com`, plan: opts.plan ?? 'free' });
  return id;
}

Rules:

Each test seeds only what it needs
Cleanup in afterEach, not afterAll
Unique IDs per run (parallel-safe)
Pin time-relative data to fixtures

Anti-patterns

Putting every flow in E2E because it is end-to-end.
Sharing one seeded user across parallel tests.
Relying on production data or saved dumps.
A test that passes only when run alone.

References

details

Verification Checklist

[ ] Each behavior sits at the cheapest layer that can catch it.
[ ] E2E set is limited to costly regressions.
[ ] Tests seed deterministic, unique data.
[ ] Auth state is created without manual steps.
[ ] The suite passes in parallel and in isolation.

Test Integrity Rules

Snapshot byte-exact prompt assembly for a real skill, the fallback, and a schema-augmented request. A one-space template change must fail.
Probe provider capability once and mark skips explicitly; a missing provider is a skip, not a silent pass.
Install a raising stub in mocked-provider tests so an accidental real call fails loudly.
Reset module caches and env overrides before each test; a spec that passes only alone is a test defect.
Verify the world, not the self-report: re-read the file, row, or response. Assert untouched files are byte-identical.
A guard only guards if the regression fails it. Introduce the regression once, watch red, revert.

Install and discovery

Contents