Runbook 06 — Release gates (time-to-signal) + decision rubric

Scope

Define release gates as named tests with explicit decision metadata.
Split gates into SMOKE and REGRESSION suites.
Print a gate contract block on failure (CI log).
Retain failure evidence (artifacts) when enabled in config/CI.

Gate contract Time-to-signal

Requirement: each gate declares id, capability, persona, failureMeans, releaseDecision (BLOCK/WARN), mitigation, owner. Missing fields = hard fail at gate definition time.

System contract

INPUT	Deterministic test outcomes (pass/fail) Tests annotated with gate metadata (id, capability, decision, owner)
OUTPUT	Immediate release signal (BLOCK or WARN) Human-readable gate contract printed in CI logs
DETERMINISM	Each test encodes its own decision rules No external interpretation required Same failure always produces same decision
FAILURE SIGNAL	Missing contract → hard fail Gate failure → BLOCK or WARN decision emitted No silent or ambiguous failures

Exit criteria

✔ SMOKE suite runs independently and completes with fast signal.
✔ REGRESSION suite runs independently and covers deeper workflows.
✔ Each gate has decision metadata (BLOCK or WARN).
✔ On failure, CI log includes a single readable contract block (no extra runs required to classify).
✔ Failure evidence exists when enabled: HTML report + per-test artifacts.

Decision rubric

Decision	Definition	Action
BLOCK	Capability required for release is not verified.	Stop release. Apply fix/rollback. Re-run gate once after change.
WARN	Capability is degraded and mitigation exists.	Release may proceed with recorded mitigation + follow-up work item.

Rule

If mitigation is undefined or not actionable, set decision to BLOCK.

Gate structure

Implementation uses helpers/releaseGate.js to enforce required fields and print failure contracts.

test(
"RW.SMOKE.001 — Login establishes authenticated session",
releaseGate(
{
  id: "RW.SMOKE.001",
  capability: "Authentication: establish session",
  persona: "Registered user",
  failureMeans: "Users cannot authenticate; authenticated journeys unavailable",
  releaseDecision: "BLOCK",
  mitigation: "Rollback auth change; validate /api/users/login and storageState",
  owner: "Web + API",
},
["home"],
async ({ home }) => {
  await home.open();
  await home.expectLoggedInNav();
}
)
);

Validation

Required field missing = gate definition hard fails (no silent partial gates).

Suite policy

SMOKE	Fast signal on core flows. Default decision = BLOCK unless mitigation is explicit.
REGRESSION	Deeper flows and consistency checks. May include WARN gates when mitigation exists.
Ownership	Each gate has an `owner`. CI log output must route ownership from the contract block.

Run commands

Environment contract

export WEB_BASE_URL="http://sut.testlab:3000"
export API_BASE_URL="http://sut.testlab:3001/api"
export RW_USER_EMAIL="rwuser_pw@yourdomain.test"
export RW_USER_PASSWORD="yourpassword"

Run SMOKE

npx playwright test specs/sprint6/Sprint6.release-gates.smoke.spec.js \
  --project=auth-chromium --workers=1 --reporter=line

Run REGRESSION

npx playwright test specs/sprint6/Sprint6.release-gates.regression.spec.js \
  --project=auth-chromium --workers=1 --reporter=line

Run all Sprint 6

npx playwright test specs/sprint6 \
  --project=auth-chromium --workers=1 --reporter=line

Project names

Valid projects: auth-chromium, unauth-chromium. Do not run --project=chromium unless it exists in config.

Expected outputs

✔ Gate failure prints a single contract block with all fields populated.
✔ Gate annotations include capability/persona/decision/owner (reporting).
✔ Failure evidence exists when enabled: trace/screenshot/video + HTML report.
✔ Go/no-go decision can be made from CI log + artifacts.

Failure modes

Gate fails without contract block: wrapper not used. Action: route test through releaseGate(...).
Contract fields missing: wrapper validation fails. Action: populate required fields.
Auth gate fails: storageState missing/invalid or creds invalid. Action: validate fixtures/global-setup.js and RW_USER_*.
API-seeded gate fails: API base URL invalid or missing /api. Action: confirm API_BASE_URL ends with /api and is reachable.
Unauth gate not unauth: test executed with auth state. Action: run under unauth-chromium or override storageState explicitly.
Artifacts missing: artifact collection not enabled or paths not uploaded. Action: verify Playwright config + CI artifacts:paths.

Fix contract inputs (env/auth/data/routing) when gate fails.
Re-run once after change.

Don’t

Do not add waits to “stabilize” a gate.
Do not rerun without a change.

System flow

UPSTREAM	Sprint 5 — Deterministic state (idempotent, isolated tests)
DOWNSTREAM	Sprint 14 — Failure artifacts (structured evidence) Sprint 15 — Failure classification (triage-report.json) Sprint 17 — Automated release decision (release-gate.json)

Why this matters

This runbook converts test failures into business decisions. Without this layer, failures are just noise. With it, they become actionable release signals.