Runbook 17 — Release gate agent

Objective

• Encode release gate reasoning into explicit, auditable policy.
• Drive GO / WARN / BLOCK from suite intent, business capability, and triage evidence.
• Preserve the distinction between fast stop-signal and broader confidence signal.
• Make release-night decisions reviewable from artifacts alone.

Definition of done

• ✔ Gate output is driven by explicit policy, not narrative summary text.
• ✔ Every BLOCK or WARN includes capability-level rationale.
• ✔ Smoke and regression outcomes are clearly distinguished.
• ✔ Reviewers can explain the decision from artifacts alone.

Decision policy

{
  "testId": "auth-chromium::specs/smoke/login.spec.js::user can sign in",
  "suite": "smoke",
  "capability": "Authentication",
  "impact": "Users cannot access protected features.",
  "gateBehavior": "BLOCK",
  "ownerHint": "Auth / platform"
}

Mechanics

• Validate evidence completeness.
• Evaluate smoke outcomes first.
• Evaluate regression outcomes second.
• Attach business capability impact and owner hint.
• Emit final GO / WARN / BLOCK decision with rationale.

if (missingRequiredEvidence) return 'HOLD';
if (hasFailedSmokeBlockingCapability) return 'BLOCK';
if (hasRegressionFailures) return 'WARN';
return 'GO';

Output schema

{
  "failures": [
    {
      "capability": "Article Publishing",
      "suite": "smoke",
      "impact": "Core content creation path is impaired.",
      "classification": "LOCATOR",
      "ownerHint": "UI / test automation review"
    }
  ],
  "policy": {
    "smokeFailuresBlock": true,
    "regressionFailuresWarn": true
  }

}
      

      

        
Commands
        
node scripts/release-gate-agent.js
cat test-results/release-gate/release-gate.json | jq .

        

          GitLab shape
          
release_gate:
  stage: report
  script:
    - node scripts/release-gate-agent.js
    - cat test-results/release-gate/release-gate.json
  artifacts:
    when: always
    paths:
      - test-results/release-gate/
        
      

      

        
Expected outputs
        

          
✔ One explicit gate decision per run.
          
✔ Capability-level rationale attached to each warning or block.
          
✔ A reviewer can explain the decision without opening raw logs first.
          
✔ The policy remains stable across reviewers and runs.
        
      

      

        
Failure modes
        

          
Pass percentage replaces risk logic: action: anchor the gate to capability and impact.
          
Regression failures block everything: action: preserve smoke vs regression intent.
          
Agent hides uncertainty: action: call out missing evidence or ambiguous classification.
          
No ownership hint: action: include ownerHint in the risk map.
        
      

      

        
Operational value
        

          
Explicit release-night decision support.
          
Clear separation between stop-signal and confidence-signal suites.
          
Auditable release rationale tied to business risk.
          
Stronger stakeholder communication than raw dashboard percentages.