What this lab is

QA Test Lab is a production-realistic environment built to stay current with modern automation, CI/CD, and test infrastructure—without the cost or operational overhead of full enterprise systems. It’s designed to surface and/or prevent the kinds of failures that appear in real deployment environments.

Why it matters in real teams

• Faster time-to-signal on build and release risk
• Lower flake noise and more trustworthy pipeline outcomes
• Clearer release decisions through explicit gates
• Reproducible failures diagnosable from artifacts alone
• Faster onboarding through runbooks and conventions rather than tribal knowledge

Core idea

The lab separates responsibilities across three machines. The MBP hosts the system under test, Mac Mini 1 executes CI/CD through a local GitLab Runner, and Mac Mini 2 acts as the dedicated Playwright test runner. Both CI/CD and local proof runs must cross the runner-to-SUT boundary.

Reference diagram (text)

    [Mac Mini 1 / CI-CD]
      local GitLab Runner
            |
            |  pipeline execution / validation
            v
    [MBP / SUT]
      Ubuntu VM
      Docker Compose:
        - web  -> :3000
        - api  -> :3001
        - db   -> internal
    
    [Mac Mini 2 / Test Runner]
      Playwright local execution
            |
            |  proof / debug / smoke validation
            v
    [MBP / SUT]
            

System capability: Establish a Deterministic Environment Baseline

• Environment can be recreated from scratch with identical results
• Access (SSH/keys) is consistent across sessions
• System can be reset to a known-good snapshot
• Baseline eliminates configuration drift

System capability: Enforce Real Runner-to-System Boundaries

• Tests execute against a non-localhost system
• Runner reaches system through a stable hostname
• Localhost shortcuts are explicitly rejected
• Execution path matches real CI conditions

System capability: Enforce Explicit Environment Targeting

• BASE_URL must be defined or execution fails
• Environment selection is explicit and validated
• Tests cannot run with implicit defaults
• Incorrect targets are blocked before execution

System capability: Make Failures Diagnosable from Artifacts

• Failures include trace, screenshot, and logs
• CI artifacts are sufficient for root cause analysis
• No rerun is required to understand failures
• Failure context is preserved deterministically

System capability: Classify and Eliminate Flake by Root Cause

• Failures are categorized by flake type
• Timing issues are replaced with deterministic waits
• Flake patterns are documented and repeatable

System capability: Control Test Data and State Deterministically

• Tests do not depend on shared state
• Each test can run independently and in parallel
• Data setup and cleanup are deterministic
• Repeated runs produce identical outcomes

System capability: Enforce Risk-Based Release Gates

• Release decision is explicitly BLOCK or WARN
• Each gate maps to a defined risk
• Failures include capability and impact context
• Time-to-signal is minimized for release confidence

System capability: Compose Tests from Domain-Level Primitives

• Tests use reusable UI and API fixtures
• Auth and setup logic are centralized
• Test steps reflect business actions
• Execution is consistent across test suites

System capability: Maintain UI Automation at Scale

• Page objects support large test suites effciently
• Locator strategy is consistent and maintainable
• New pages follow a defined structure
• UI changes require minimal refactoring

System capability: Derive Test Coverage from Live Systems

• Same URL produces identical test-plan output across runs
• Navigation and capabilities are discovered automatically
• At least one real user path is identified
• Locator map is generated alongside the test plan

System capability: Create a Structured Failure Artifact Contract

• Failures are stored as structured JSON artifacts
• Schema is consistent across all test runs
• Artifacts include environment and evidence links
• Failure data is machine-readable and reusable

System capability: Classify Failures Deterministically

• Failures are categorized (assertion, env, data, etc.)
• Root cause hints are generated automatically
• Classification is consistent across runs
• Triage output is structured and actionable

System capability: Enforce Intelligent Locator Strategy

• Locators are derived from accessibility roles
• Locator strategies are validated automatically
• Locator map is generated and reusable
• Locator changes are detectable and traceable

System capability: Synthesize Action Methods from Typed UI Elements

• Validated locators are transformed into typed element intents
• Primitive action methods are generated deterministically
• Composite actions are created only when confidence is high
• Artifacts become a safe bridge into page-object and test-skeleton generation

System capability: Automate Risk-Based Release Decisions

• Release decision is generated automatically
• Decision is based on test signals and classification
• Output includes BLOCK or WARN with rationale
• Decision is reproducible from artifacts

What it proves:

Deliverable:

• Runbook 09 — Cypress API harness (setup + first 3 tests)

What it proves:

Deliverable:

• Runbook 10 — Cypress auth + deterministic test data

What it proves:

Deliverable:

• Runbook 11 — Cypress CI integration + evidence bundle

What it proves:

Deliverable:

• Runbook 12 — JMeter baseline (CLI, reports, parameterization)

What it proves:

Deliverable:

• Runbook 13 — JMeter concurrency bottleneck modeling (unique vs contention)

CI Architect

Mission: Make pipelines boring and trustworthy.

Outputs

• MR pipeline always runs
• Artifacts always available
• Promotion path dev → qa → stage

Rules

• No manual steps except deliberate approvals
• Failures diagnosable from artifacts alone

Flake Detective

Mission: Reduce noise without hiding risk.

Outputs

• Flake taxonomy
• At least one principled flake elimination per cycle

Rules

• No waitForTimeout as a fix
• Don’t increase timeouts until the failure mode is explained

Release Gatekeeper

Mission: Prevent silent regressions from shipping.

Outputs

• Minimal go/no-go suite
• Rationale mapping tests → risks

Rules

• Every gate maps to a risk
• Coverage ≠ safety to ship

New Hire

Mission: Make the system understandable in 30 minutes.

Outputs

• One-page setup
• Top 10 troubleshooting failures

Rules

• If it’s not documented, it doesn’t exist
• No tribal knowledge dependencies

What each runbook includes

Each runbook has the goal, exact commands, expected output, common failure modes, and why the system is built that way.

Runbook index

• Runbook 00 — Golden VM baseline
• Runbook 01 — Runner → SUT connectivity and naming
• Runbook 02 — Compose deploy (dev) + health check
• Runbook 02a — baseURL discipline and proof
• Runbook 03 — Diagnosability via Intentional Failure
• Runbook 04 — CI artifacts + debug without rerun
• Sprint 4 Runbook — NAT + Port Forwarding Mode
• Runbook 05 — Test data + state reset
• Runbook 06 — Time-to-signal + release gates
• Runbook 07 — Custom fixtures (UI + API) + composition rules
• Runbook 08 — Page Objects at scale
• Runbook 09 — Cypress API harness
• Runbook 09.5 — URL → Test Plan Agent
• Runbook 14 — Artifact Contract
• Runbook 15 — Deterministic Triage
• Runbook 16 — Locator Intelligence Agent
• Runbook 17 — Release Gate Agent

Runbooks are command + expected-output documents. Examples are sanitized.