Objective + success criteria

Objective: Validate that tests executed by a CI runner reach the System Under Test (SUT) over a real network boundary using stable naming (e.g., sut.testlab). In default mode, localhost HTTP must not be an acceptable target because it can mask routing, name resolution, firewall, and port exposure failures.

Rule (default): HTTP checks and Playwright UI navigation must target ${WEB_BASE_URL} resolved by name (not localhost).

Mode contract: This runbook has two valid execution modes.
BRIDGED/LAN (default): reach the SUT by stable name (e.g., sut.testlab). Localhost HTTP is forbidden.
NAT RECORDING (temporary): reach the SUT via local VirtualBox port-forwards (127.0.0.1:3000, 127.0.0.1:3001). Localhost HTTP is allowed, but must be explicitly enabled via CI_NAT_MODE=1.

Compatibility note: Runbook 00 uses localhost port-forwarding for SSH baseline access (127.0.0.1:2222 → VM:22). That remains acceptable for bootstrap access. This runbook governs SUT HTTP (web/API) proof with a mode-aware boundary rule.

Success criteria:

• Name resolution proof (default mode): ${SUT_HOST} resolves inside the CI job to the intended SUT address (DNS or injected hosts mapping).
• Boundary proof (mode-aware): when CI_NAT_MODE!=1, localhost must not return SUT HTTP; when CI_NAT_MODE=1, localhost must return SUT HTTP via port-forwards.
• TCP transport proof: TCP handshake succeeds to ${SUT_HOST}:${WEB_PORT} and ${SUT_HOST}:${API_PORT} (or to localhost ports in NAT mode).
• Remote identity proof: web and API responses match Conduit (HTML title for web; JSON shape for API).
• Automation proof: at least one Playwright UI test passes while targeting ${WEB_BASE_URL}.

# Declare which topology this CI runner is using (keep raw IPs private)
# Preferred (production-like): CI runner is on a different host and must use sut.testlab over LAN/VPN.
# Temporary (recording-safe): NAT mode uses localhost web/api via VirtualBox port-forwards and is enabled via CI_NAT_MODE=1.

# Conduit services (canonical):
# BRIDGED/LAN: WEB http://sut.testlab:3000 | API http://sut.testlab:3001/api
# NAT (recording): WEB http://127.0.0.1:3000 | API http://127.0.0.1:3001/api

# Forbidden for BRIDGED/LAN proof:
# - port-forwards/proxies/tunnels exposing SUT web/api to localhost

# Allowed only for NAT RECORDING mode:
# - VirtualBox NAT port forwarding that exposes 3000/3001 on 127.0.0.1 (CI_NAT_MODE=1)

New hire guide — how environment selection works

If you are new to the project: most “mystery failures” in automation are not code failures—they are environment selection failures. This framework supports multiple environments via environment files. Your job is to pick the right environment file for the run you are executing.

What you are selecting

How to choose an environment (recommended patterns)

Recommended: use dotenvx to load an explicit env file per run. This mirrors CI behavior and avoids “my terminal had old variables.”

# Dev
dotenvx run -f .env.dev -- npx playwright test

# QA
dotenvx run -f .env.qa -- npx playwright test

Recording-safe NAT example (temporary)

Use this only during Sprint 00–04 recordings when the SUT is in NAT mode and exposed via localhost port-forwards.

CI_NAT_MODE=1 dotenvx run -f .env.sprint4 -- bash -lc '
  echo "CI_NAT_MODE=$CI_NAT_MODE"
  echo "WEB_BASE_URL=$WEB_BASE_URL"
  echo "API_BASE_URL=$API_BASE_URL"
  npx playwright test --project=auth-chromium -g "@smoke"
'

How to confirm you loaded the right environment

dotenvx run -f .env.qa -- bash -lc 'echo "TEST_ENV=$TEST_ENV"; echo "WEB_BASE_URL=$WEB_BASE_URL"; echo "API_BASE_URL=$API_BASE_URL"; echo "CI_NAT_MODE=${CI_NAT_MODE:-0}"'

Optional: load an environment into your current shell (use carefully)

This approach is convenient, but it can cause confusion if you forget what is currently loaded. Prefer dotenvx run -f ... for repeatability.

# zsh / bash
set -a
source .env.qa
set +a

echo "TEST_ENV=$TEST_ENV"
echo "WEB_BASE_URL=$WEB_BASE_URL"
echo "API_BASE_URL=$API_BASE_URL"
echo "CI_NAT_MODE=${CI_NAT_MODE:-0}"

# Mistake: "tests hit localhost and pass"
# Cause: NAT mode or local port-forward makes localhost look like SUT.
# Fix: assert the intended mode. If BRIDGED: localhost must fail. If NAT: localhost must succeed.

# Mistake: "CI_NAT_MODE is 0 even though I set it"
# Cause: protected variables do not apply to non-protected branches.
# Fix: unprotect CI_NAT_MODE or run on a protected ref.

# Mistake: "API login fails"
# Cause: API_BASE_URL wrong (missing /api) or credentials incorrect.
# Fix: confirm API_BASE_URL, curl API_BASE_URL/tags, validate RW creds.

Dependencies

Required prerequisites

• SUT is running and services are bound to a reachable interface (not loopback-only in BRIDGED/LAN mode).
• CI runner network can reach the SUT (routed LAN/VPN, correct runner tags, firewall rules in place).
• Stable naming is configured for the SUT hostname (DNS or hosts injection) in BRIDGED/LAN mode.
• Boundary enforcement is mode-aware:
  BRIDGED/LAN: no port-forwards/proxies/tunnels may expose SUT web/api via localhost in CI.
  NAT RECORDING: localhost port-forwards are expected and must be explicitly enabled via CI_NAT_MODE=1.

# Option 1: Real DNS (preferred)
# - sut.testlab is resolvable via lab DNS/VPN DNS for CI runners.

# Option 2: Inject hosts mapping (common)
# - GitLab Runner host: /etc/hosts entry for sut.testlab
# - Docker executor: add-host/extra_hosts (runner-level config)
# - Kubernetes: hostAliases or cluster DNS record (private)

# Note: Operator-mode hidden HTML is not secrecy. Keep private evidence in a private repo.

Environment contract (the “real” variables this framework uses)

This project uses environment files (e.g., .env.dev, .env.qa, .env.sprint4) that define TEST_ENV, WEB_BASE_URL, and API_BASE_URL. CI_NAT_MODE is an explicit mode switch used during temporary NAT recording runs. Some legacy scripts may still read BASE_URL; treat it as optional compatibility only.

# Default (BRIDGED/LAN)
export TEST_ENV="qa"
export WEB_BASE_URL="http://sut.testlab:3000"
export API_BASE_URL="http://sut.testlab:3001/api"
export CI_NAT_MODE="0"

# Temporary (NAT recording mode)
export CI_NAT_MODE="1"
export WEB_BASE_URL="http://127.0.0.1:3000"
export API_BASE_URL="http://127.0.0.1:3001/api"

# Optional (compatibility only)
export BASE_URL="${WEB_BASE_URL}"

Confirm values (CI)

echo "Runner host: $(hostname)"
echo "TEST_ENV=${TEST_ENV}"
echo "CI_NAT_MODE=${CI_NAT_MODE:-0}"
echo "WEB_BASE_URL=${WEB_BASE_URL}"
echo "API_BASE_URL=${API_BASE_URL}"

# Derive SUT_HOST from WEB_BASE_URL when you need name resolution/TCP checks:
SUT_HOST="$(echo "$WEB_BASE_URL" | sed -E 's#^https?://([^:/]+).*#\1#')"
echo "SUT_HOST=$SUT_HOST"

# Derive ports (default Conduit):
WEB_PORT="3000"
API_PORT="3001"

Proof blocks (CI, fail-fast)

These blocks match the environment contract (WEB_BASE_URL, API_BASE_URL, CI_NAT_MODE). Where we need host/ports, we derive them deterministically.

0) Derive SUT_HOST + ports (CI)

# Derive host from WEB_BASE_URL
SUT_HOST="$(echo "$WEB_BASE_URL" | sed -E 's#^https?://([^:/]+).*#\1#')"
WEB_PORT="3000"
API_PORT="3001"

echo "CI_NAT_MODE=${CI_NAT_MODE:-0}"
echo "SUT_HOST=${SUT_HOST}"
echo "WEB_PORT=${WEB_PORT}"
echo "API_PORT=${API_PORT}"

1) Name resolution proof (CI)

Proves the CI runner can resolve the SUT hostname in BRIDGED/LAN mode. In NAT mode (CI_NAT_MODE=1) the host may be 127.0.0.1, so name resolution is informational.

if [ "${CI_NAT_MODE:-0}" = "1" ]; then
  echo "CI_NAT_MODE=1 → name resolution is informational (SUT_HOST may be localhost)"
  getent hosts "${SUT_HOST}" || true
else
  getent hosts "${SUT_HOST}" \
    || nslookup "${SUT_HOST}" \
    || dig +short "${SUT_HOST}"
fi

2) Boundary proof (mode-aware) — localhost HTTP

Deterministic:
• BRIDGED/LAN (CI_NAT_MODE!=1): localhost must NOT return SUT HTTP.
• NAT RECORDING (CI_NAT_MODE=1): localhost MUST return SUT HTTP (port-forwards present).

if [ "${CI_NAT_MODE:-0}" = "1" ]; then
  echo "CI_NAT_MODE=1 → localhost HTTP is expected (NAT port-forwards)"
  curl -fsS --max-time 5 "http://127.0.0.1:${WEB_PORT}/" >/dev/null \
    && echo "PASS: localhost returned HTTP (NAT mode)" \
    || { echo "FAIL: localhost did not respond in NAT mode"; exit 1; }
else
  echo "CI_NAT_MODE!=1 → localhost HTTP must NOT be reachable"
  curl -fsS --max-time 2 "http://localhost:${WEB_PORT}/" >/dev/null \
    && { echo "FAIL: localhost returned HTTP (boundary bypass)"; exit 1; } \
    || echo "PASS: localhost did not respond (boundary enforced)"
fi

# If BRIDGED/LAN mode unexpectedly sees localhost HTTP, identify the listener.
# Linux:
ss -lntp | egrep ":(3000|3001)\b" || true

# macOS runner host:
lsof -nP -iTCP:3000 -sTCP:LISTEN || true
lsof -nP -iTCP:3001 -sTCP:LISTEN || true

3) TCP transport proof (CI)

Success condition: exit code 0 for both handshakes, with host determined by WEB_BASE_URL/API_BASE_URL.

WEB_HOST="$(echo "$WEB_BASE_URL" | sed -E 's#^https?://([^:/]+).*#\1#')"
API_HOST="$(echo "$API_BASE_URL" | sed -E 's#^https?://([^:/]+).*#\1#')"

nc -vz -w 2 "${WEB_HOST}" "${WEB_PORT}"; echo "web_exit_code=$?"
nc -vz -w 2 "${API_HOST}" "${API_PORT}"; echo "api_exit_code=$?"

4) Remote identity proof (CI)

Web UI identity (Conduit)

curl -sS -I "${WEB_BASE_URL}/" | head -n 1
curl -sS "${WEB_BASE_URL}/" | grep -i "<title" | head -n 1

API identity (Conduit)

curl -sS -I "${API_BASE_URL}/tags" | head -n 1
curl -sS "${API_BASE_URL}/tags" | head -c 200

5) Automation proof (CI)

Replace the spec path/tag with your actual Sprint 1 connectivity spec. Contract: load ${WEB_BASE_URL} and assert a Conduit UI signature (e.g., navbar contains “conduit”).

dotenvx run -f .env.qa -- \
npx playwright test specs/sprint1/Sprint1.runner-to-sut.connectivity.spec.js \
  -g "RW.NET.001" \
  --project=chromium \
  --reporter=line

if [ "${CI_NAT_MODE:-0}" != "1" ]; then
  echo "${WEB_BASE_URL}" | grep -Eqi 'localhost|127\.0\.0\.1|0\.0\.0\.0' \
    && { echo "FAIL: WEB_BASE_URL points to localhost in BRIDGED/LAN mode"; exit 1; } \
    || echo "PASS: WEB_BASE_URL is not localhost (BRIDGED/LAN mode)"
fi

GitLab CI job snippet (drop-in)

This job loads a specific env file and then runs the proof blocks and one Playwright proof test. Adjust tags/image to match the runner. Keep secrets masked.

BRIDGED/LAN example (default)

ci_runner_to_sut_proof_bridged:
  stage: test
  image: mcr.microsoft.com/playwright:v1.47.2-jammy
  variables:
    CI_NAT_MODE: "0"
  script:
    - dotenvx run -f .env.qa -- bash -lc '
        echo "Runner host: $(hostname)";
        echo "CI_NAT_MODE=${CI_NAT_MODE:-0}";
        echo "WEB_BASE_URL=$WEB_BASE_URL";
        echo "API_BASE_URL=$API_BASE_URL";

        SUT_HOST="$(echo "$WEB_BASE_URL" | sed -E "s#^https?://([^:/]+).*#\\1#")";
        WEB_PORT="3000"; API_PORT="3001";

        # Name resolution proof
        getent hosts "$SUT_HOST" || nslookup "$SUT_HOST" || dig +short "$SUT_HOST";

        # Localhost boundary proof (must NOT respond)
        curl -fsS --max-time 2 "http://localhost:$WEB_PORT/" >/dev/null \
          && { echo "FAIL: localhost returned HTTP (boundary bypass)"; exit 1; } \
          || echo "PASS: localhost did not respond (boundary enforced)";

        # TCP + identity proofs
        nc -vz -w 2 "$SUT_HOST" "$WEB_PORT";
        nc -vz -w 2 "$SUT_HOST" "$API_PORT";

        curl -sS "$WEB_BASE_URL/" | grep -i "

ci_runner_to_sut_proof_nat_recording:
  stage: test
  image: mcr.microsoft.com/playwright:v1.47.2-jammy
  variables:
    CI_NAT_MODE: "1"
  script:
    - dotenvx run -f .env.sprint4 -- bash -lc '
        echo "Runner host: $(hostname)";
        echo "CI_NAT_MODE=${CI_NAT_MODE:-0}";

        # Force localhost contract for NAT mode (recording-safe)
        export WEB_BASE_URL="http://127.0.0.1:3000";
        export API_BASE_URL="http://127.0.0.1:3001/api";

        echo "WEB_BASE_URL=$WEB_BASE_URL";
        echo "API_BASE_URL=$API_BASE_URL";

        WEB_PORT="3000"; API_PORT="3001";

        # Localhost boundary proof (must respond)
        curl -fsS --max-time 5 "http://127.0.0.1:$WEB_PORT/" >/dev/null \
          && echo "PASS: localhost returned HTTP (NAT mode)" \
          || { echo "FAIL: localhost did not respond in NAT mode"; exit 1; }

        # TCP + identity proofs (localhost)
        nc -vz -w 2 "127.0.0.1" "$WEB_PORT";
        nc -vz -w 2 "127.0.0.1" "$API_PORT";

        curl -sS "$WEB_BASE_URL/" | grep -i "

# If CI cannot reach the SUT in BRIDGED/LAN mode:
# - Tag job to a runner on the correct network segment.
# - Ensure VPN is active on the runner host if required.
# - Confirm firewall rules allow runner → SUT on 3000/3001.
# - Ensure SUT services are exposed on 0.0.0.0 and mapped correctly in docker-compose.