Vetrix Docs

Pipeline Job Configuration & Setup (by job class)

Companion to pipeline-reference.md (variable/field reference) and authoring-vetrix-pipelines.md (how to write a pipeline file). This document covers what each class of CI job needs from the environment to run correctly — the runner knobs, network, services, secrets, and server-side prerequisites.

Vetrix jobs are only as healthy as their provisioning. A job that is correct in YAML still fails if the runner isn't given the socket, network, frontend tree, registry host, or credentials it depends on. The three classes below are the ones with non-trivial setup.

Shared runner provisioning knobs

Most of the per-class setup reduces to a handful of worker/runner env knobs (set on the worker container; see runners.md and docker-compose.yml):

Knob Purpose
RUNNER_DOCKER_SOCKET Forwards the host Docker socket into job containers so docker-dependent jobs (build, integration, e2e) reach a daemon (Docker-out-of-Docker). Empty = no socket in jobs.
RUNNER_JOB_NETWORK Attaches launched job containers to a named Docker network so they can reach the dev-stack nginx/web/api (e.g. *.gitvetrix.test). Empty = default network (often not able to resolve the stack).
RUNNER_REPOS_ROOT / RUNNER_WORKSPACE_DIR Bare-repo mount + per-job workspace root. Workspace is populated by git archive (no .git) by default.
VETRIX_WORKSPACE_GIT (per-job var) Opt-in: populate the workspace via a real --local clone (.git present) instead of git archive, for jobs whose commands: run git.
VETRIX_EXTERNAL_URL → CI_REGISTRY The worker derives the instance-relative registry host from the instance URL and injects CI_REGISTRY. Without it, the registry host falls back to the prod default.
JWT_SECRET Signing key for the per-job VETRIX_JOB_TOKEN; must be identical on the worker and the server, else minted tokens fail verification.

Bucket 2 — Browser / frontend end-to-end + accessibility

Jobs: a11y-gate, web-a11y-e2e, repositories-landing-responsive, security-scannerfix-cross-browser, slow-chromium-e2e.

These drive real browsers (Playwright / Selenium / Chromium) against the live running frontend, so they have the heaviest setup.

What the job declares

a11y-gate:
  stage: test
  image: mcr.microsoft.com/playwright:v1.49.0-jammy   # ships the browsers
  secret: true                                         # redacts the injected creds
  only: [ master, develop, staging ]
  variables:
    A11Y_BASE_URL: "https://www.gitvetrix.test"     # the LIVE web app under test
    E2E_BASE_URL:  "https://www.gitvetrix.test"     # bootstrap reachability probe target
    A11Y_USER: "$A11Y_USER"                            # ← injected from the runner host
    A11Y_PASS: "$A11Y_PASS"                            # ← injected from the runner host
    A11Y_FAIL_ON: "serious,critical"
    DOCKER_HOST: "unix:///var/run/docker.sock"
  commands:
    - apt-get update && apt-get install -y --no-install-recommends docker.io curl ca-certificates
    - bash scripts/cicd/e2e-bootstrap.sh               # verify socket + wait for the live stack
    - |
      bash -c 'set +e; mkdir -p a11y-report; node scripts/a11y/gate.mjs; exit $?'
  artifacts:
    paths: [ a11y-report/ ]

To operate correctly, the environment MUST provide

  1. A reachable live frontend stack. A11Y_BASE_URL/E2E_BASE_URL must resolve and respond from inside the job container. This is the most common failure point.
  2. Job-network attachment (RUNNER_JOB_NETWORK). The job container only reaches www/api.gitvetrix.test if it is attached to the dev-stack network. Without it the bootstrap probe times out and the job exits non-zero by design (e2e-bootstrap.sh fails fast with a reachability diagnostic).
  3. The Docker socket (RUNNER_DOCKER_SOCKET). The bootstrap runs docker version to confirm the mounted socket; the image carries only the docker CLI, never an in-container daemon.
  4. The frontend web/ tree + its npm deps. The a11y/e2e specs and tools (axe, pa11y, Playwright specs under web/src/__tests__/e2e/…) live in the frontend tree. On the backend repo the operator must make web/ available in the workspace (see each job's DEPENDENCY NOTE header).
  5. Injected credentials (never in YAML). A11Y_USER/A11Y_PASS (and the Chromatic / cross-browser token for security-scannerfix-cross-browser) are injected into the runner-host process env; the job references them as "$VAR" and sets secret: true so the expanded values are redacted in logs.
  6. bash execution. The bootstrap and gate are wrapped in explicit bash so the worker's sh -c (dash) join cannot break &&/set +e logic.

Why they fail on a bare runner: if the live stack isn't reachable (no RUNNER_JOB_NETWORK, stack down) or the web/ tree / creds aren't provided, these jobs fail at bootstrap — an environment-provisioning gap, not a code or YAML error.

Bucket 3 — Database-backed integration

Jobs: go-integration-test, go-integration-test-collation-glibc.

These need a real Postgres, started as a sibling container via the mounted Docker socket (Vetrix has no services: — see authoring-vetrix-pipelines.md §8).

What the job declares (shape shared by both)

go-integration-test:
  stage: test
  image: golang:1.25
  secret: true                       # redacts the dev-default DSN in logs
  variables:
    POSTGRES_USER: "vetrix"
    POSTGRES_PASSWORD: "vetrix"
    POSTGRES_DB: "vetrix"
  commands:
    - apt-get update && apt-get install -y --no-install-recommends docker.io ca-certificates
    - docker rm -f vetrix-it-pg 2>/dev/null || true            # clear leftovers
    - docker run -d --name vetrix-it-pg -e POSTGRES_USER=$POSTGRES_USER ... postgres:16-alpine
    - |                                                          # wait for pg_isready
      for i in $(seq 1 30); do docker exec vetrix-it-pg pg_isready -U $POSTGRES_USER -d $POSTGRES_DB && break; sleep 2; done
    - PG_IP=$(docker inspect -f "{{.NetworkSettings.IPAddress}}" vetrix-it-pg)
    - export TEST_DSN="postgres://$POSTGRES_USER:$POSTGRES_PASSWORD@$PG_IP:5432/$POSTGRES_DB?sslmode=disable"
    - export DATABASE_URL="$TEST_DSN"
    - export VETRIX_TEST_DESTRUCTIVE_MIGRATIONS=1               # SAFE: DSN points at the throwaway container
    - go run ./cmd/vetrix-migrate -dsn "$TEST_DSN" -dir db/migrations
    - |                                                          # test + teardown in ONE shell (no after_script:)
      set +e
      go vet ./...; vrc=$?
      go test -tags integration -count=1 ./...; trc=$?
      docker rm -f vetrix-it-pg >/dev/null 2>&1 || true
      if [ $vrc -ne 0 ]; then exit $vrc; fi
      exit $trc

To operate correctly, the environment MUST provide

  1. The Docker socket (RUNNER_DOCKER_SOCKET). Used to docker run the sibling Postgres. No host port-publish — connectivity is by the container's bridge IP (docker inspect … .IPAddress), so the runner's job network must allow container-to-container traffic.
  2. Schema applied before tests. cmd/vetrix-migrate runs db/migrations to head against the ephemeral DB (same internal/db.Migrate the server uses).
  3. Throwaway-DB discipline. VETRIX_TEST_DESTRUCTIVE_MIGRATIONS=1 is only safe because TEST_DSN points at the docker rm-ed sibling container — it must never be exported against a shared dev DB (mydev_postgres); the internal/db guard skips the destructive down/up suite otherwise.
  4. In-shell teardown. The docker rm -f runs in the same set +e block as the test so cleanup happens even on failure (Vetrix has no after_script:).
  5. A green suite. go-integration-test runs the whole go test -tags integration ./...; any failing test fails the job. Greening this job requires fixing any failing tests (or scoping the suite), not changing the job config.

go-integration-test-collation-glibc — the one extra requirement

This job runs only the collation guard (anchored -run), and its Postgres must be a glibc image with a locale-aware collation so the divergence cross-check actually executes:

  image: golang:1.25
  variables:
    POSTGRES_INITDB_ARGS: "--locale=en_US.utf8"   # locale-aware cluster (NOT byte-wise)
    LANG: "en_US.utf8"
  commands:
    - docker run -d --name vetrix-it-pg-glibc -e POSTGRES_INITDB_ARGS="--locale=en_US.utf8" ... postgres:16   # glibc (Debian), NOT -alpine
    - ... go test -tags integration -run '^TestVBE189_CanonicalTagSet_GoMatchesSQLBucketKey$' ./internal/cicd/scheduler/ ...
  • Image must be glibc postgres:16 (Debian-based), not postgres:16-alpine: musl collates byte-wise even when labelled en_US.utf8, so on alpine the guard t.Skipfs loudly (a PASS). Only glibc + --locale=en_US.utf8 makes the guard EXECUTE (locally: "DIVERGENCE GUARD EXECUTED: 6/9 cases diverged").
  • To operate correctly the runner must be able to pull the glibc postgres:16 image and complete initdb --locale=en_US.utf8 over the mounted socket.

Bucket 4 — Registry publish (build-image)

Job: build-image. Builds the server image and pushes it to the instance's OCI registry using a per-job token. This is the most setup-sensitive job because it spans the worker (token mint + host derivation), the server (registry authz), and the docker client (push handshake).

What the job declares

build-image:
  stage: publish
  image: docker:25-cli
  only: [ master, develop ]
  variables:
    DOCKER_HOST: "unix:///var/run/docker.sock"
    NOTIFY_WEBHOOK_URL: "https://hooks.example.test/REPLACE_ME"   # per-repo; placeholder is non-resolving on purpose
  commands:
    - docker version                                              # fail fast if the socket isn't mounted
    - apk add --no-cache curl
    - |
      set +e
      docker build --target server \
        --build-arg VERSION="${CI_COMMIT_SHA}" --build-arg COMMIT="${CI_COMMIT_SHA}" \
        -t "vetrix:${CI_COMMIT_SHA}" -f Dockerfile . \
        && echo "${VETRIX_JOB_TOKEN}" | docker login "${CI_REGISTRY}" --username x-vetrix-runner --password-stdin \
        && docker tag "vetrix:${CI_COMMIT_SHA}" "${CI_REGISTRY}/vetrix/vetrix:${CI_COMMIT_SHA}" \
        && docker push "${CI_REGISTRY}/vetrix/vetrix:${CI_COMMIT_SHA}"
      rc=$?
      docker logout "${CI_REGISTRY}" 2>/dev/null || true          # token must not outlive the job
      # ... notify NOTIFY_WEBHOOK_URL with the outcome ...
      exit $rc

To operate correctly, the environment MUST provide

  1. The Docker socket (RUNNER_DOCKER_SOCKET). docker version is run first, outside the rc-capture, so a missing socket fails fast as a runner-config error (not a build outcome).
  2. A correct, instance-relative CI_REGISTRY. The worker derives it from the instance external URL, so VETRIX_EXTERNAL_URL must be set on the worker (e.g. https://api.gitvetrix.test on dev) — otherwise it falls back to the prod default and the push targets the wrong host. You can also set CI_REGISTRY/RUNNER_CI_REGISTRY explicitly.
  3. The per-job token path, not a static credential. Auth is the ephemeral VETRIX_JOB_TOKEN (scoped registry:write, bound to repo/pipeline/job, auto-expiring), injected by the worker and consumed via docker login --password-stdin. Never wire a static/account-level registry credential or a cached config.json — that is explicitly out of the model.
  4. JWT_SECRET identical on worker and server, so the minted token verifies server-side.
  5. A server that authorizes the push. The registry must accept the per-job token for the full push: the read leg (HEAD blob / manifest probes) and emit upload Location headers carrying the full owner/repo path.
  6. A docker client that completes the auth handshake. docker login must succeed and the docker client's push auth/retry flow against the Basic-auth registry must complete; an unauthorized at the push leg after a successful login is a docker-client / CI-config matter, not a server defect.
  7. only: master/develop gating so the publish job runs on the right branches, and an optional per-repo NOTIFY_WEBHOOK_URL (the default placeholder intentionally doesn't resolve, so an unconfigured pipeline logs a continue-on-failure notice rather than failing the job).

Quick provisioning matrix

Job class Socket Job network Service / image Frontend web/ Secrets Server-side prereq
Bucket 2 (browser/e2e/a11y) ✅ ✅ (reach live stack) Playwright/Selenium image ✅ A11Y_USER/PASS, Chromatic live web+api+nginx up
Bucket 3 (DB integration) ✅ container-to-container sibling `postgres:16(-alpine glibc)` — DSN (redacted)
Bucket 4 (registry publish) ✅ reach registry host docker:25-cli — VETRIX_JOB_TOKEN (per-job) CI_REGISTRY host + registry authz