Vetrix Docs

Runbook — enable SBOM attestation on a repository's pipeline

Take a repository from "no SBOM" to a registered pipeline artifact carrying an attached CycloneDX attestation, using only vetrix-ci.yml and operator-side environment provisioning. Every command below is copy-pasteable; nothing here requires the web UI except the optional visual check in section 4.

Audience: pipeline authors and CI/CD operators.

Current shipped state — read this before you start.

  • Attestations are signed differently per lane. vetrix/vetrix (scripts/ci/sbom-publish.sh, used by the publish-server-image* jobs) attaches unsigned (--sign=false); its resulting row carries signature_required: false and no signer_subject. vetrix/vetrix-frontend (vetrix-ci.yml's sbom-web job) attaches signed (--sign=true) — flipped from --sign=false once the signing daemon reached production and the server's attestation config was wired up to use it — and its resulting row carries signature_required: true with a populated signer_subject. There is no single "current shipped state" for this flag; check the job you're mirroring before copying a recipe below.
  • The SBOM describes what ships — the built container image — never the source tree. There is no dir:. fallback in either recipe, by design: a source-tree scan silently attests something other than the artifact.
  • Both control-plane calls — registering the pipeline artifact and attaching the attestation to it — authenticate with the same per-job VETRIX_JOB_TOKEN. The long-lived CI_SBOM_TOKEN service bearer the attach step used to take is retired from both shipped lanes; do not put it in a new pipeline. Section 1.2 has the detail; section 5 lists the symptoms of getting the credential wrong.

This runbook is about vetrix-ci.yml pipelines. The vetrix/sbom@<SHA> composite step described in SBOM — Software Bill of Materials belongs to the separate automations subsystem (.vetrix/automations/*.yml, jobs: / steps: / uses: syntax). Those keys are rejected by the pipeline parser: steps: and jobs: inside a job definition are both hard parse errors under the v1 and the v2 schema (internal/cicd/parser/v1.go incompatibleJobKeys, internal/cicd/parser/v2.go droppedJobKeysV2). Do not copy composite-step YAML into a pipeline file. Everything a pipeline needs is in sections 2 and 3 below.


1. Prerequisites matrix

Name Supplied by How the job references it Absent →
CI_API_URL Runner worker, automatically, for every job Read from the environment. Never declare it in variables: Job must skip or fail loudly; never fall back to a production host
VETRIX_JOB_TOKEN Runner, minted per job Read from the environment. Never declare it Both the artifact registration and the attestation attach fail authentication
VETRIX_JOB_ID Runner, per job Read from the environment. Never declare it Registration URL has no job segment
CI_REGISTRY, CI_COMMIT_SHA, CI_COMMIT_REF_NAME Runner, per job Read from the environment Image reference / ref guard cannot be built
syft + vetrix-sbom The job image, or pinned tool images over the Docker socket See "Scanner availability" below No SBOM can be produced or attached

1.1 CI_API_URL — the control-plane base URL

Highest precedence: a CI_API_URL configured at an admin variable-store scope — repo, org, or instance — overrides everything else (cmd/worker/branchvars.go ResolveAPIURLOverride). That approved-scope set deliberately excludes the branch scope (and, transitively, committed YAML): only an admin-configured store value can redirect the control plane this way.

Absent that override, the worker resolves a fallback value once at start-up and injects it into every job container. The resolution order (cmd/worker/main.go resolveServerURL) is:

  1. VETRIX_SERVER_URL
  2. VETRIX_EXTERNAL_URL
  3. EXTERNAL_URL
  4. SERVER_EXTERNAL_URL
  5. https://<host> derived from CI_REGISTRY, else from RUNNER_CI_REGISTRY

Full precedence, narrowest-effective first: the admin-scope override, then the worker-resolved value above, then omitted. If neither resolves, the executor emits no empty -e CI_API_URL=. A job therefore sees either a usable value or nothing at all.

CI_API_URL is a reserved canonical name (cmd/worker/docker_executor.go reservedCanonicalEnvKeys). Reserved names are emitted after the job's own variables: block, so a committed variables: CI_API_URL: entry or a branch-scoped variable-store overlay cannot clobber the resolved value. Declaring it in YAML has no effect and only obscures where the value comes from.

The fail-loud contract. A job that needs the control plane must never invent a default. Either abort:

if [ -z "${CI_API_URL}" ]; then
  echo "CI_API_URL is not set: the worker did not inject the control-plane URL." >&2
  echo "Refusing to default to a production control plane. Set the worker's instance" >&2
  echo "external URL (VETRIX_EXTERNAL_URL) and restart the worker." >&2
  exit 1
fi

…or skip the attestation work while still doing the rest of the job (section 2 uses this form, so an unprovisioned worker still publishes its image).

Operator note: the worker reads this environment at process start. Merging a change to a branch does not re-read it — the persistent worker daemon has to be restarted or redeployed before a newly-set VETRIX_EXTERNAL_URL reaches any job.

1.2 VETRIX_JOB_TOKEN — the one credential for both control-plane calls

Both shipped lanes authenticate the artifact-register POST and the vetrix-sbom attach step with the same per-job VETRIX_JOB_TOKEN. Nothing else has to be provisioned.

The attach endpoint accepts two credential classes on one route (internal/api/artifact_attestations_handler.go):

Credential Required permission Reach
Session / access token (web UI, a human with a PAT) ci:write on the repo Every artifact in every repo that identity holds ci:write on — but for kind: sbom only; see the sixth gate below
Per-job VETRIX_JOB_TOKEN (minted at dispatch) registry:write Only artifacts belonging to the token's own pipeline, in its own repo; expires shortly after the job

The job-token path is the strictly tighter one. It runs a five-gate ladder — the token must verify, carry registry:write, be per-job scoped (a repo-only registry token is refused), match the repo resolved from the URL, and its pipeline must own the artifact being attested — and it needs no standing credential on any worker host. Because the register and attach steps run back-to-back in the same job, the token's claims match the artifact by construction.

Tighter on those five gates, but not narrower everywhere. A sixth gate runs inside the handler's kind switch, after all five have passed, and it cuts the other way: for kind: image-signature the credential must be a per-job token — jobClaims non-nil with a non-Nil JobID — and a session / ci:write bearer is refused 403 image-signature attestations require a per-job token (internal/api/artifact_attestations_handler.go, the attestation.KindImageSignature case). Only the job-token path can write that kind, which is why the session row's reach in the table above is kind: sbom only. The SBOM path both classes use is unchanged, so nothing in this runbook is affected — but do not read that table as "session credentials reach everything a job token reaches, and more".

For the scopes each credential class carries, see CI/CD API reference — Auth & scopes.

CI_SBOM_TOKEN is retired from the pipeline recipes. It was a long-lived ci:write service bearer held in the runner-host process environment, and it reached a job container by exactly one mechanism — a self-referential variables: declaration, CI_SBOM_TOKEN: "${CI_SBOM_TOKEN}", resolved against Runner.hostEnvMap (internal/cicd/runner.go). Nothing else pushes it in: the executor injects only the canonical reserved keys. Deleting that declaration therefore is the host-injection removal, and neither shipped lane carries it any more. The backend build enforces this — internal/cicd/parser/vbe1815_ci_sbom_token_retired_test.go fails if any job reintroduces the name, in variables: or in a commands: body. Do not add it to a new pipeline.

The session ci:write path itself is unchanged and still supported; it is how the web UI and a human with a PAT attach an attestation out of band. What is retired is standing a long-lived copy of such a credential on runner hosts so a pipeline can use it. The CLI is credential-agnostic — --token takes any bearer (cmd/vetrix-sbom/main.go) — so which credential a lane uses is decided entirely by what the YAML passes.

Secret-carrying variables: brace form and log redaction

Keep secret: true on the job even though the recipes below declare no secret of their own. secret: true is what puts a job's expanded variable values into the runner's log-redaction set (RedactSecrets, internal/cicd/runner.go), and the runner mints VETRIX_JOB_TOKEN into the job's variable map before interpolation. The mint is a runner step, not a parser one: RunJob clones the job's Variables map and assigns vars["VETRIX_JOB_TOKEN"] = tok (internal/cicd/runner.go). Do not look for it in the interpolator — InterpolateJobReservedFields (internal/cicd/parser/v1.go) takes neither a token nor a minter and seeds nothing; it only expands ${...} references in a map it is handed. Dropping the flag "because there is no secret variable left in the YAML" leaks the per-job token into the job log: the redaction set starts from a sweep of job.Variables under if job.Secret || … after the mint, and that sweep is the only step that could add the token to the set — with the flag dropped, job.Secret is false and the token's name carries no SECRET_ prefix, so the token never enters the set regardless of what else does. The runner separately, and unconditionally, appends the job's resolved branch/repo variable-store values onto the same set — those are redacted regardless of the secret: flag — so the log sink itself is not reliably left unarmed: a job with the flag dropped and no variables: of its own can still get a redacting sink whenever the branch/repo store resolves something for it. That does not help the token, which the sweep excluded either way. Only when the branch/repo store resolves nothing for a job does dropping the flag also leave the sink unarmed, with the real token sitting unredacted in the map — the leak holds regardless.

Any variable you do declare must use the brace form. The interpolator's pattern is \$\{([A-Za-z_][A-Za-z0-9_]*)\} — brace form only (internal/cicd/parser/v1.go varRe). A bare $NAME is shipped verbatim as a literal, never resolves, and arms the log redactor with that placeholder instead of the real value. The same rule governs the self-referential shape NAME: "${NAME}" used for any operator-injected host variable: it is not a typo and not a cycle, because the interpolator resolves a self-reference from the runner-side source (host environment plus push-context variables), deliberately excluding the job's own variables: map.

Log redaction. secret: true on the job genuinely arms whole-job log redaction — under both the v1 and the v2 schema. When armed, the expanded values of every one of that job's variables are collected into the redaction set, and matching text is replaced with *** in captured logs. This has always been true: secret is a first-class JobConfig field going back to before the v2 schema existed (internal/cicd/parser/v1.go), and v2 carries it through unchanged — internal/cicd/parser/v2.go's toPipelineConfig computes Secret: len(j.Secrets) > 0 || j.Secret, so a bare secret: true on a v2 job arms redaction exactly as it does under v1. secrets: is not a replacement for secret: true, and it is not narrower — a non-empty secrets: list arms the same whole-job redaction, not just the named variables. The difference is a side effect secret: true does not have — see below.

Three facts worth knowing:

  • Any variable whose name begins with SECRET_ is redacted per-variable even without the flag.
  • Under the v2 schema the job field secrets: (a list of variable names) is a second, independent way to switch on redaction: a non-empty secrets: list also arms whole-job redaction (via the same len(j.Secrets) > 0 term above), on top of anything secret: true already armed. Note that each name listed in secrets: also has its value overwritten with the empty string when the v2 job is adapted to the shared job config, so do not list a variable you also need a value for. That overwrite is a parse-time seeding, and a run-time mint overwrites it in turn — which is why it does not blank VETRIX_JOB_TOKEN (§3). It blanks every name nothing mints. secret: true has no such side effect.
  • Redaction only ever matches a variable's expanded value — never the literal reference text. The interpolator's pattern (varRe, the callout just above) is brace-only: ${NAME} expands and its resolved value is what gets added to the redaction set, but a bare $NAME (no braces) never expands at all. A bare reference is therefore shipped as a harmless literal string, and if that variable is also covered by secret: true or secrets:, the redactor ends up armed with the harmless literal instead of the real secret value — the real value, having never been substituted in, reaches the log unredacted wherever it appears verbatim. Always use the brace form ${NAME} for any variable you intend secret: true / secrets: to actually protect.

1.3 VETRIX_JOB_TOKEN / VETRIX_JOB_ID — never declared

Both are reserved canonical names injected by the executor. VETRIX_JOB_TOKEN is a per-job JWT carrying registry:write plus repo_id / pipeline_id / job_id claims; VETRIX_JOB_ID is the job UUID that forms the artifact-registration path segment.

Declaring either in variables: would hardcode a literal or shadow the injected value. Read them straight from the environment. RunJob seeds VETRIX_JOB_TOKEN into the job's variable map itself — the vars["VETRIX_JOB_TOKEN"] = tok assignment in internal/cicd/runner.go, not anything in the parser — which is why secret: true covers it without — and only without — a declaration of your own (§1.2).

1.4 Scanner availability

Two provisioning routes work today. Pick by what the job image can do.

(a) Pinned tool images over the Docker socket. Works from any job image that has a POSIX shell and can reach the daemon. syft runs as anchore/syft:v1.18.0; the vetrix-sbom CLI is built from cmd/vetrix-sbom through a streamed Docker build context. This is what section 2 uses.

Requires the operator to have opted the runner into socket pass-through by setting RUNNER_DOCKER_SOCKET on the worker. When it is unset no socket is mounted — that is the default, because mounting the host socket grants jobs root-equivalent access to the runner host. With it set, the socket appears at /var/run/docker.sock inside the job, so DOCKER_HOST: "unix:///var/run/docker.sock" reaches the host daemon.

(b) An image that already ships both. vetrix/runner-security carries syft v1.18.0 at /usr/local/bin/syft and the vetrix-sbom CLI at /usr/local/bin/vetrix-sbom (deployments/runner-security/Dockerfile). It ships no Docker daemon and no crane / oras / skopeo, so scans must use syft's own registry client — see section 3. It also runs as USER nobody with WORKDIR /workspace, and declares ENTRYPOINT ["/usr/local/bin/vetrix-security-scan"] — its own composite-action orchestrator binary, not a shell that execs its arguments.

Before choosing route (b):

  • Confirm the image is actually present in the registry the runner pulls from. Nothing in this runbook's recipes (sections 2 and 3) builds or publishes vetrix/runner-security — provisioning it into that registry is a separate, out-of-band operational step. Route (a) has no such dependency.
  • No entrypoint check is actually needed, despite the image's own ENTRYPOINT not execing its arguments. A baked entrypoint that ignores its arguments used to swallow a job's commands: silently; the runner now emits an explicit --entrypoint /bin/sh whenever the shell script built from a job's commands: is non-empty (cmd/worker/docker_executor.go buildDockerRunArgs, via buildShellScript), overriding any image ENTRYPOINT unconditionally. buildShellScript right-trims each commands: entry and drops any entry that trims to empty, so a job whose commands: block is present but contains only blank/whitespace entries builds an empty script and takes the same no-override path as a job with no commands: at all (the composite-action dispatch shape) — both run the image's own baked entrypoint as-is.

2. v1 schema recipe — fold the SBOM into the image-publish job

Fold, do not split. Scanning in the same job that built the image means the image under attestation is the local, already-authenticated build: no unauthenticated registry probe, no cross-job ordering hazard, and no window in which the digest you register differs from the digest you scanned.

The v1 job-key allowlist is stage, image, commands, variables, only, except, secret, environment, allow_failure, and artifacts.paths. before_script, after_script, script, services, runs-on, steps, and jobs are hard parse errors. There is no needs: under v1, so cross-job hand-off is not available — another reason to fold. Full schema: Authoring a Vetrix CI/CD Pipeline File.

stages:
  - build
  - test
  - publish

publish-server-image:
  stage: publish
  image: docker:25-cli
  secret: true
  only:
    - develop
  variables:
    DOCKER_HOST: "unix:///var/run/docker.sock"
    # VETRIX_JOB_TOKEN / VETRIX_JOB_ID are injected by the worker and are
    # deliberately NOT declared here. `secret: true` above still matters: the
    # runner seeds VETRIX_JOB_TOKEN into this job's variables, and that is what
    # the log redactor is built from (§1.2).
  commands:
    # Fail fast on a missing socket mount before the slow build.
    - docker version
    - |
      set -e

      # run_step <description> <command> [args...]. `exit` inside a POSIX shell
      # function exits the shell, so a failed step ends the job carrying that
      # step's own status. The job shell here is Alpine ash: no bashisms.
      run_step() {
        _desc="$1"
        shift
        set +e
        "$@"
        _rc=$?
        set -e
        if [ "$_rc" -ne 0 ]; then
          echo "publish-server-image: $_desc FAILED (exit $_rc)" >&2
          exit "$_rc"
        fi
      }

      # v1 has no after_script:; an EXIT trap is the always-run teardown.
      trap 'docker logout "${CI_REGISTRY}" >/dev/null 2>&1 || true' EXIT

      run_step "docker build" \
        docker build --target server -t "myimage:${CI_COMMIT_SHA}" -f Dockerfile .
      run_step "docker login" \
        sh -c 'echo "${VETRIX_JOB_TOKEN}" | docker login "${CI_REGISTRY}" --username x-vetrix-runner --password-stdin'
      run_step "docker tag" \
        docker tag "myimage:${CI_COMMIT_SHA}" "${CI_REGISTRY}/<owner>/<repo>:${CI_COMMIT_SHA}"
      run_step "docker push" \
        docker push "${CI_REGISTRY}/<owner>/<repo>:${CI_COMMIT_SHA}"

      # The SBOM control-plane round-trips need CI_API_URL. Gate on it so an
      # unprovisioned worker still publishes the image, and so the attestation
      # activates by itself once the worker injects the value.
      if [ -n "${CI_API_URL}" ]; then
        (
        set -e

        # docker:25-cli is Alpine ash and ships NO curl.
        if ! command -v curl >/dev/null 2>&1; then
          run_step "install curl" apk add --no-cache curl ca-certificates
        fi

        # Resolve the REAL pushed digest and on-disk size; fail closed on an
        # empty or malformed value. RepoDigests[0] is the registry-assigned
        # manifest digest, populated by the push above.
        DIGEST_REF=$(docker image inspect --format '{{index .RepoDigests 0}}' "${CI_REGISTRY}/<owner>/<repo>:${CI_COMMIT_SHA}")
        SHA256=$(printf '%s' "$DIGEST_REF" | sed -n 's/.*@\(sha256:[0-9a-f]\{64\}\).*/\1/p')
        SIZE=$(docker image inspect --format '{{.Size}}' "myimage:${CI_COMMIT_SHA}")
        STORAGE_KEY="${CI_REGISTRY}/<owner>/<repo>:${CI_COMMIT_SHA}"
        if ! printf 'X%s' "$SHA256" | grep -q '^Xsha256:[0-9a-f]\{64\}$'; then
          printf 'could not resolve pushed image digest (got: %s)\n' "$SHA256" >&2
          exit 1
        fi
        case "$SIZE" in
          ''|*[!0-9]*)
            printf 'image size not numeric: %s\n' "$SIZE" >&2
            exit 1
            ;;
        esac

        # Scan the SHIPPED LOCAL image over the mounted socket. No `dir:.`
        # fallback. syft v1.18.0 writes CycloneDX to stdout for
        # `-o cyclonedx-json` (no `=path`), redirected into the workspace here.
        run_step "syft scan (cyclonedx-json)" \
          docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
            anchore/syft:v1.18.0 "docker:myimage:${CI_COMMIT_SHA}" -o cyclonedx-json > sbom.json
        if [ ! -s sbom.json ]; then
          echo "syft produced no SBOM — refusing to attach" >&2
          exit 1
        fi

        # Register the artifact. Credential here is VETRIX_JOB_TOKEN (per-job,
        # registry:write) — the same token the attach step below uses.
        # Capture the HTTP status as well as the body: `-w` appends
        # "\n<status>", and curl's own exit is captured separately so a
        # transport failure is not mistaken for an unparseable body.
        set +e
        REG_RAW=$(curl -s -w '\n%{http_code}' -X POST "${CI_API_URL}/api/v1/jobs/${VETRIX_JOB_ID}/artifacts" \
          -H "Authorization: Bearer ${VETRIX_JOB_TOKEN}" \
          -H "Content-Type: application/json" \
          -d "{\"name\":\"server-image\",\"storage_key\":\"$STORAGE_KEY\",\"size_bytes\":$SIZE,\"sha256\":\"$SHA256\"}")
        rc=$?
        set -e
        if [ "$rc" -ne 0 ]; then
          echo "artifact registration could not reach the API (curl exit $rc)" >&2
          exit 1
        fi
        HTTP_CODE=$(printf '%s' "$REG_RAW" | tail -n 1)
        RESP=$(printf '%s' "$REG_RAW" | sed '$d')
        case "$HTTP_CODE" in
          2??) printf 'Artifact registration accepted: HTTP %s\n' "$HTTP_CODE" ;;
          '')  echo "artifact registration returned no HTTP status line" >&2; exit 1 ;;
          *)   printf 'artifact registration REJECTED: HTTP %s\n%s\n' "$HTTP_CODE" "$RESP" >&2; exit 1 ;;
        esac

        # Depth-aware, jq-free extraction of the genuine TOP-LEVEL "id".
        # docker:25-cli ships no jq and no python3. Fold newlines, strip the
        # outer braces, delete every innermost {...}/[...] until stable (that
        # removes exactly the nested objects/arrays), then take the
        # member-anchored "id" — which never matches job_id or pipeline_id.
        _art_flat=$(printf '%s' "$RESP" | tr -d '\n\r')
        _art_inner=$(printf '%s' "$_art_flat" | sed -e 's/^[[:space:]]*{//' -e 's/}[[:space:]]*$//')
        while :; do
          _art_next=$(printf '%s' "$_art_inner" | sed -e 's/{[^{}]*}//g' -e 's/\[[^][]*\]//g')
          [ "$_art_next" = "$_art_inner" ] && break
          _art_inner=$_art_next
        done
        ART_ID=$(printf '%s' "$_art_inner" | tr ',' '\n' \
          | sed -n 's/^[[:space:]]*"id"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' \
          | head -n 1)
        if [ -z "$ART_ID" ] || [ "$ART_ID" = "null" ]; then
          printf 'HTTP %s but no top-level "id" in the body: %s\n' "$HTTP_CODE" "$RESP" >&2
          exit 1
        fi
        # Fail closed unless the id is UUID-shaped. A wrong id handed to attach
        # would bind the attestation to another artifact with no error signal.
        if ! expr "X$ART_ID" : 'X[0-9a-fA-F]\{8\}-[0-9a-fA-F]\{4\}-[0-9a-fA-F]\{4\}-[0-9a-fA-F]\{4\}-[0-9a-fA-F]\{12\}$' >/dev/null 2>&1; then
          printf 'extracted artifact id is not UUID-shaped: "%s"\n' "$ART_ID" >&2
          exit 1
        fi
        printf 'Registered artifact ID: %s\n' "$ART_ID"

        # Build the attach CLI from the STREAMED build context. A
        # `-v "$PWD":/src` bind mount would resolve against the HOST filesystem,
        # where the in-container /workspace does not exist.
        printf 'FROM golang:1.25-alpine AS b\nRUN apk add --no-cache git ca-certificates\nWORKDIR /build\nCOPY go.mod go.sum* ./\nRUN go mod download\nCOPY . .\nRUN CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /vetrix-sbom ./cmd/vetrix-sbom\nFROM scratch\nCOPY --from=b /vetrix-sbom /vetrix-sbom\n' > /tmp/vetrix-sbom.Dockerfile
        run_step "docker build vetrix-sbom" \
          docker build -t "vetrix-sbom-build:${CI_COMMIT_SHA}" -f /tmp/vetrix-sbom.Dockerfile .
        _sbom_cid=$(docker create "vetrix-sbom-build:${CI_COMMIT_SHA}" /vetrix-sbom) || exit 1
        run_step "extract vetrix-sbom" docker cp "${_sbom_cid}:/vetrix-sbom" ./vetrix-sbom
        docker rm "${_sbom_cid}" >/dev/null 2>&1 || true

        # Attach. --token is the SAME per-job VETRIX_JOB_TOKEN the register
        # call above used — the attach endpoint accepts it, and its claims
        # already match this artifact. --server is the worker-injected
        # CI_API_URL, never a hardcoded host.
        run_step "vetrix-sbom attach" ./vetrix-sbom attach \
          --server "${CI_API_URL}" \
          --token "${VETRIX_JOB_TOKEN}" \
          --repo "<owner>/<repo>" \
          --artifact "$ART_ID" \
          --format "cyclonedx-json" \
          --sbom "sbom.json" \
          --sign=false
        )
      else
        printf 'WARNING: CI_API_URL was not injected by the worker — publishing without an SBOM attestation.\n' >&2
      fi

Substitute <owner>, <repo>, the image name, and the --target for your repository.

2.1 The fail-closed guards, and why each exists

Guard Prevents
sha256 shape check on the resolved digest Registering a placeholder such as an all-zeros digest, which makes the artifact row unverifiable
Numeric check on SIZE Registering a placeholder size (1 is the classic one)
[ ! -s sbom.json ] Attaching an empty payload — the CLI and the API both reject it, but later and less clearly
HTTP-status capture on the register call Treating a 4xx body as a successful registration
Depth-aware top-level "id" extraction Picking up a nested job_id / pipeline_id instead of the artifact id
UUID-shape check on ART_ID Binding the attestation to a different artifact with no error signal

2.2 Best-effort mode (optional)

The block above is fail-loud: any SBOM sub-step failure ends the job. To keep a blocking publish job green while the attestation path is still being provisioned, wrap the subshell in a status downgrade:

      if [ -n "${CI_API_URL}" ]; then
        set +e
        (
          set -e
          # ... all SBOM sub-steps ...
        )
        _sbom_rc=$?
        set -e
        if [ "$_sbom_rc" -ne 0 ]; then
          printf 'WARNING: SBOM sub-steps failed (exit %s) — continuing; attestation SKIPPED this run.\n' "$_sbom_rc" >&2
        fi
      fi

set -e and exit inside the subshell terminate only the subshell. The subshell also resets the inherited EXIT trap to default, so a docker logout trap fires only when the main job shell exits.

Understand the cost before adopting it: a green job no longer proves an attestation exists. Verify with section 4, not with the job's exit status. Remove the downgrade once the path has been observed green.


3. v2 schema recipe — a dedicated job behind needs:

Use this when the pipeline declares version: 2 and an earlier job already pushed the image. v2 adds needs:, which gates dispatch on the named jobs completing, so the scan job can run separately from the build without an ordering hazard.

v2 differences that matter here:

  • Jobs live under a top-level jobs: mapping. Declaring them at the document root — the v1 layout — is rejected with pipeline defines no jobs (missing jobs: block).
  • needs: targets must be in the same or an earlier stage. Same-stage edges are allowed.
  • v2 adds a secrets: field (a list of variable names) — a mechanism v1 does not have. secret: true remains valid on v2 jobs too (internal/cicd/parser/v2.go's JobV2.Secret field, a v1-compat alias); see §1.2 for how the two combine.
  • The image ships no Docker daemon, so the scan reads the image from the registry using syft's own registry client, authenticated by SYFT_REGISTRY_AUTH_*.
version: 2

stages:
  - build
  - test
  - publish

jobs:
  build-image:
    stage: publish
    image: docker:25-cli
    only:
      - master
    variables:
      DOCKER_HOST: "unix:///var/run/docker.sock"
    commands:
      - docker version
      # ... build, login with VETRIX_JOB_TOKEN, tag, push ...

  sbom-web:
    stage: publish
    image: vetrix/runner-security
    secret: true
    needs:
      - build-image
    only:
      - master
    # No `variables:` block at all: every value this job needs is a reserved
    # name the worker injects. `secret: true` above is still required — see the
    # note under the block.
    commands:
      # Ref guard — see §3.1. Keep it FIRST: attaching an SBOM from another ref
      # would register artifacts and attestations against the wrong image.
      # `case` (not `if`) tolerates an unset CI_COMMIT_REF_NAME.
      - |
        case "${CI_COMMIT_REF_NAME:-}" in
          refs/heads/master) ;;
          *) echo "sbom-web: skipping — ref ${CI_COMMIT_REF_NAME:-<unset>} is not refs/heads/master"; exit 0 ;;
        esac
      - |
        set -e

        run_step() {
          _desc="$1"
          shift
          set +e
          "$@"
          _rc=$?
          set -e
          if [ "$_rc" -ne 0 ]; then
            echo "sbom-web: $_desc FAILED (exit $_rc)" >&2
            exit "$_rc"
          fi
        }

        # No production fallback — fail fast on an unset control-plane URL.
        if [ -z "${CI_API_URL}" ]; then
          echo "CI_API_URL is not set: the worker did not inject the control-plane URL." >&2
          echo "Refusing to default to a production control plane. Set the worker's" >&2
          echo "instance external URL (VETRIX_EXTERNAL_URL)." >&2
          exit 1
        fi

        IMG="${CI_REGISTRY}/<owner>/<repo>:${CI_COMMIT_SHA}"

        # Daemon-free registry auth for syft. This image has no docker daemon
        # (so no `docker login`) and no crane/oras/skopeo. syft's own registry
        # client authenticates the pull from these variables. The credential is
        # the same per-job token the build job logged in with.
        export SYFT_REGISTRY_AUTH_AUTHORITY="${CI_REGISTRY}"
        export SYFT_REGISTRY_AUTH_USERNAME="x-vetrix-runner"
        export SYFT_REGISTRY_AUTH_PASSWORD="${VETRIX_JOB_TOKEN}"

        if ! command -v syft >/dev/null 2>&1; then
          echo "sbom-web: syft not found on the image" >&2
          exit 1
        fi

        # ONE authenticated pass emits BOTH the CycloneDX payload we attach AND
        # the syft-json image metadata we register from, so the digest
        # registered is exactly the digest the SBOM was built from — there is no
        # resolve-then-scan window. `registry:` is the only source scheme used;
        # there is no unauthenticated manifest probe and no `dir:.` fallback, so
        # an unpullable image fails the job CLOSED rather than attesting the
        # git checkout.
        run_step "syft scan (registry: cyclonedx-json + syft-json)" \
          syft scan "registry:${IMG}" \
            -o cyclonedx-json=sbom.json \
            -o syft-json=image-meta.json
        if [ ! -s sbom.json ]; then
          echo "sbom-web: syft produced no SBOM — refusing to attach" >&2
          exit 1
        fi
        if [ ! -s image-meta.json ]; then
          echo "sbom-web: syft produced no image metadata — cannot resolve digest/size" >&2
          exit 1
        fi

        # jq IS present on this image. manifestDigest is the registry manifest
        # digest; imageSize is the image's byte size. Fail CLOSED on a
        # non-sha256 digest or a non-positive size.
        SHA256=$(jq -r '.source.metadata.manifestDigest // empty' image-meta.json)
        SIZE=$(jq -r '.source.metadata.imageSize // empty' image-meta.json)
        STORAGE_KEY="$IMG"
        if ! printf 'X%s' "$SHA256" | grep -q '^Xsha256:[0-9a-f]\{64\}$'; then
          printf 'sbom-web: could not resolve manifest digest (got: %s)\n' "$SHA256" >&2
          exit 1
        fi
        case "$SIZE" in
          ''|*[!0-9]*) printf 'sbom-web: image size not numeric: %s\n' "$SIZE" >&2; exit 1 ;;
        esac
        if [ "$SIZE" -le 0 ]; then
          printf 'sbom-web: image size must be positive (got: %s)\n' "$SIZE" >&2
          exit 1
        fi

        # Register the artifact. Credential here is VETRIX_JOB_TOKEN (per-job,
        # registry:write) — the same token the attach step below uses, and the
        # same token the syft registry pull above authenticated with.
        # Capture the HTTP status as well as the body: `-w` appends
        # "\n<status>", and curl's own exit is captured separately so a
        # transport failure is not mistaken for an unparseable body.
        set +e
        REG_RAW=$(curl -s -w '\n%{http_code}' -X POST "${CI_API_URL}/api/v1/jobs/${VETRIX_JOB_ID}/artifacts" \
          -H "Authorization: Bearer ${VETRIX_JOB_TOKEN}" \
          -H "Content-Type: application/json" \
          -d "{\"name\":\"web-image\",\"storage_key\":\"$STORAGE_KEY\",\"size_bytes\":$SIZE,\"sha256\":\"$SHA256\"}")
        rc=$?
        set -e
        if [ "$rc" -ne 0 ]; then
          echo "sbom-web: artifact registration could not reach the API (curl exit $rc)" >&2
          exit 1
        fi
        HTTP_CODE=$(printf '%s' "$REG_RAW" | tail -n 1)
        RESP=$(printf '%s' "$REG_RAW" | sed '$d')
        case "$HTTP_CODE" in
          2??) printf 'Artifact registration accepted: HTTP %s\n' "$HTTP_CODE" ;;
          '')  echo "sbom-web: artifact registration returned no HTTP status line" >&2; exit 1 ;;
          *)   printf 'sbom-web: artifact registration REJECTED: HTTP %s\n%s\n' "$HTTP_CODE" "$RESP" >&2; exit 1 ;;
        esac

        # Depth-aware, jq-free extraction of the genuine TOP-LEVEL "id" — the
        # same idiom §2 uses, byte-for-byte. jq IS on this image (used for
        # SHA256/SIZE above), but the register response gets this extraction
        # anyway rather than a second parsing approach for the same shape of
        # problem. Fold newlines, strip the outer braces, delete every
        # innermost {...}/[...] until stable (that removes exactly the nested
        # objects/arrays), then take the member-anchored "id" — which never
        # matches job_id or pipeline_id.
        _art_flat=$(printf '%s' "$RESP" | tr -d '\n\r')
        _art_inner=$(printf '%s' "$_art_flat" | sed -e 's/^[[:space:]]*{//' -e 's/}[[:space:]]*$//')
        while :; do
          _art_next=$(printf '%s' "$_art_inner" | sed -e 's/{[^{}]*}//g' -e 's/\[[^][]*\]//g')
          [ "$_art_next" = "$_art_inner" ] && break
          _art_inner=$_art_next
        done
        ART_ID=$(printf '%s' "$_art_inner" | tr ',' '\n' \
          | sed -n 's/^[[:space:]]*"id"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' \
          | head -n 1)
        if [ -z "$ART_ID" ] || [ "$ART_ID" = "null" ]; then
          printf 'sbom-web: HTTP %s but no top-level "id" in the body: %s\n' "$HTTP_CODE" "$RESP" >&2
          exit 1
        fi
        # Fail closed unless the id is UUID-shaped. A wrong id handed to attach
        # would bind the attestation to another artifact with no error signal.
        if ! expr "X$ART_ID" : 'X[0-9a-fA-F]\{8\}-[0-9a-fA-F]\{4\}-[0-9a-fA-F]\{4\}-[0-9a-fA-F]\{4\}-[0-9a-fA-F]\{12\}$' >/dev/null 2>&1; then
          printf 'sbom-web: extracted artifact id is not UUID-shaped: "%s"\n' "$ART_ID" >&2
          exit 1
        fi
        printf 'Registered artifact ID: %s\n' "$ART_ID"

        run_step "vetrix-sbom attach" vetrix-sbom attach \
          --server "${CI_API_URL}" \
          --token "${VETRIX_JOB_TOKEN}" \
          --repo "<owner>/<repo>" \
          --artifact "$ART_ID" \
          --format "cyclonedx-json" \
          --sbom "sbom.json" \
          --sign=true

The register call and the artifact-id extraction are byte-for-byte the same as section 2 — only the artifact name (web-image rather than server-image) and the digest/size source differ.

This worked example declares no variables: at all, yet still carries secret: true — and that is deliberate, not leftover. Under v2, secret: true arms whole-job log redaction over the expanded values of the job's variables, the same v1-compat alias v1 has always had (JobConfig.Secret, §1.2), wired through by internal/cicd/parser/v2.go's JobV2.Secret field. RunJob seeds VETRIX_JOB_TOKEN into that variable map itself — the vars["VETRIX_JOB_TOKEN"] = tok assignment in internal/cicd/runner.go, a runner step, not a parser one (§1.2) — so the flag is what keeps the per-job token, passed to --token and to the register call's Authorization header, out of the captured log. Removing it because the YAML names no secret is exactly the mistake to avoid.

secrets: is not the thing to reach for here either — though not for the reason it is tempting to give. Naming a variable in a v2 secrets: list does overwrite its value with the empty string, but at parse time: toPipelineConfig seeds jc.Variables[s] = "" for each listed name (internal/cicd/parser/v2.go). RunJob's mint then assigns vars["VETRIX_JOB_TOKEN"] = tok unconditionally at run time, over whatever the parsed config held. So listing this particular name would not blank it — the seeding that strips an ordinary variable is simply overwritten for the one name that has a minter behind it. That is the difference between the retired CI_SBOM_TOKEN, which nothing mints and which the seeding therefore really does blank, and VETRIX_JOB_TOKEN, which the runner re-supplies. Do not carry the old token's behaviour across to this one.

Keep secret: true anyway, for three reasons that do hold:

  • It buys nothing to switch. secrets: arms the same whole-job redaction, not a narrower per-name one: toPipelineConfig computes Secret: len(j.Secrets) > 0 || j.Secret (internal/cicd/parser/v2.go), so a non-empty list and a bare flag land on the identical JobConfig.Secret gate.
  • It is the only form that works under both schemas. v1 has no secrets: field at all, so section 2's recipe must use secret: true; one flag across both recipes is one rule to get right.
  • The blanking side effect is harmless only for this name — and, for a different reason, for the executor-owned reserved canonical names. For an ordinary job variable it is live: list one you actually need a value for and the job gets an empty string. It is equally inert on CI_API_URL, CI_REGISTRY, and the rest of the set cmd/worker/docker_executor.go builds as reservedEnvKeysDroppedFromJobVariables — not because anything re-mints them the way the runner re-mints VETRIX_JOB_TOKEN, but because the executor drops every one of them out of job.Variables before assembling the container environment and emits its own resolved value afterward. secret: true has no such side effect — it switches on redaction without touching any variable's value.

3.1 The runner-level ref guard

The shipped v2 parser does carry only: and except: into the shared job config (internal/cicd/parser/v2.go toPipelineConfig), where the same branch filter evaluates them for v1 and v2 alike. The first-command ref guard is therefore belt-and-braces, not a substitute for only: — keep both. It costs one case statement and it protects a job running against an instance whose parser build predates that carry-through.

Two details make the guard correct:

  • CI_COMMIT_REF_NAME carries the pipeline's ref, so on a branch pipeline it reads refs/heads/<branch>. only: patterns match the short branch name, because the scheduler strips refs/heads/ before filtering. Compare against the form the variable actually holds — the two are not interchangeable.
  • only:/except: matching (internal/cicd/parser/v1.go matchBranch) is a plain prefix/suffix test, not a slash-aware glob: a trailing * checks strings.HasPrefix and a leading * checks strings.HasSuffix, with no exclusion for /. So * spans path separators — feature/* matches feature/a/b, not just one path segment. (matchBranch's own doc comment says "matches any run of non-slash characters"; that comment is stale — the code above is what actually runs.) Anything without a leading or trailing * is an exact match.

allow_failure: true (a first-class boolean job field under both schemas, default false) makes a failure soft so it cannot block a deploy off the same branch. Like the best-effort pattern in §2.2, it means a green pipeline does not prove the attestation landed. The recipe above does not carry it, matching the shipped sbom-web job: that job carried the flag from its introduction until a master pipeline was observed with it at state: success, then removed it in the same change that flipped the signing flag (sbomWebJobContract.test.ts now asserts the flag's absence). If you're bringing up a new lane and want the same soft-start safety net while the path is unverified, add allow_failure: true back and remove it once you've observed the job green, the same way sbom-web did.


4. Verification

Three checks, cheapest first. All API paths below are relative to ${CI_API_URL} (or your instance's API base); every call needs a bearer credential with ci:read.

1. Find the artifact id.

curl -s -H "Authorization: Bearer $TOKEN" \
  "$API/api/v1/repos/<owner>/<repo>/pipelines/<pipeline-id>/artifacts"

Returns a bare JSON array. Look for the artifact your job registered by name (server-image, web-image, …) and take its id. Its sha256 must be the real image digest and size_bytes the real byte size — a 0000… digest or a size_bytes of 1 is the placeholder trap in section 5.

2. List the attestations on that artifact.

curl -s -H "Authorization: Bearer $TOKEN" \
  "$API/api/v1/repos/<owner>/<repo>/artifacts/<artifact-id>/attestations"

Note the base path: attestations hang off /artifacts/{id} directly, not under /pipelines/{id}/. Newest first. A successful attach shows a row with kind: "sbom", format: "cyclonedx-json", predicate_type: "https://cyclonedx.org/bom", and a payload_sha256. signature_required and signer_subject differ by lane (see the header above): on the still-unsigned vetrix/vetrix lane expect signature_required: false and no signer_subject; on the signed vetrix/vetrix-frontend lane expect signature_required: true with a populated signer_subject — an absent or empty value there, on a run where the attach step itself succeeded, means the signing daemon did not sign, which is worth its own bug report rather than being read as expected behavior.

3. Download the payload bytes.

curl -s -H "Authorization: Bearer $TOKEN" -OJ \
  "$API/api/v1/repos/<owner>/<repo>/artifacts/<artifact-id>/attestations/<attestation-id>/payload"

Streams the exact stored envelope. It is always served as an attachment with X-Content-Type-Options: nosniff, so a browser never renders it inline.

Endpoint details, status codes, and response shapes: CI/CD API reference.

Optional visual check. Open the repository's pipeline run detail page. Below the coverage report is a panel headed SBOM attestations, described as "Software bill-of-materials and provenance attestations attached to this pipeline's artifacts". It fetches the pipeline's artifacts collection and issues one attestation query per artifact, so it lists every row on every artifact — a pipeline is never assumed to have exactly one SBOM. Each row shows a Signed / Unsigned badge (from signature_required), the format, a truncated SHA-256 with a copy control, a relative creation time, and a Download button. The panel has four mutually exclusive states: loading, an error alert with Retry, the empty state ("No SBOM attestations for this pipeline."), and the populated list.


5. Failure modes

Symptom Cause Fix
Job log: WARNING: CI_API_URL was not injected by the worker The worker resolved no instance external URL, so the variable was omitted entirely Set VETRIX_EXTERNAL_URL (or another accepted name, §1.1) in the worker's process environment and restart the worker. Merging a branch does not re-read it
Job green, no attestation anywhere The best-effort pattern (§2.2) or allow_failure: true (§3) downgraded a real failure to a warning Read the job log for the WARNING: line and the preceding ... FAILED (exit N); verify with section 4, never with the job's exit status
Attach step: attestation server returned 401 The bearer did not verify — an empty ${VETRIX_JOB_TOKEN} (typically a bare $NAME reference that never expanded, §1.2), or a job token that has already expired Pass --token "${VETRIX_JOB_TOKEN}" in brace form, and keep the attach in the same job as the register call so the token is still live
Attach step: attestation server returned 403 A verifying credential the attach gates refuse: a job token without registry:write, a repo-only registry token that is not per-job scoped, a token whose repo does not match the URL, or one whose pipeline does not own the artifact. For a session credential, no CI write access on the repo Attach from the same job that registered the artifact, so the token's pipeline owns it (§1.2). For a human/UI attach, use a credential with ci:write on that repo
Register call: HTTP 401 / 403 token missing registry:write scope A session/service ci:write bearer was sent to the register endpoint. That endpoint takes the per-job token only Send ${VETRIX_JOB_TOKEN} to /api/v1/jobs/{id}/artifacts — and to the attach endpoint too; one credential serves both (§1.2)
Register call: 403 job token scope does not match job id The {id} path segment is not VETRIX_JOB_ID Build the URL from ${VETRIX_JOB_ID}, never from a pipeline id or a job number
Register call: 400 invalid request body naming an unknown field The register body decoder rejects unknown fields Send only name, storage_key, size_bytes, sha256 (plus the optional retention_days / retention / paths)
Register call: 400 storage_key must be a relative path without '..' segments An absolute or traversing storage_key Use the image reference form <registry>/<owner>/<repo>:<tag>
sh: curl: not found, exit 127 docker:25-cli is Alpine and ships no curl apk add --no-cache curl ca-certificates, guarded by command -v curl so it stays idempotent — and keep it inside the CI_API_URL gate so an unprovisioned worker incurs no fetch
Artifact row has an all-zeros sha256 or size_bytes: 1 Placeholder values posted instead of resolved ones Resolve the real digest and size (§2 / §3) and fail closed on a malformed value. An artifact registered with a placeholder digest cannot be verified against the image and the attestation on it is worthless
Attestation attached to the wrong artifact, no error A nested job_id / pipeline_id was extracted instead of the top-level "id" Use the depth-aware extraction plus the UUID-shape guard (§2.1)
Attach: 422 with {"error":"unsupported sbom format","supported":[...]} --format outside the closed set Use cyclonedx-json (the default), spdx-json, or spdx-tag-value
Attach: 400 sbom payload is empty sbom.json produced no bytes The [ ! -s sbom.json ] guard should have caught this first; check the syft step's exit status and the scan target
Attach: 503 attestation service not configured The deployment has no attestation service wired Operator action; nothing in the pipeline file can work around it
Parse error: job field `steps` is a GitHub Actions convention Composite-step YAML pasted into vetrix-ci.yml Use the recipes in sections 2 and 3. steps:, jobs:, script:, runs-on:, services:, before_script:, and after_script: inside a job are all hard errors
Parse error: pipeline defines no jobs (missing jobs: block) version: 2 with jobs at the document root Nest every job under the top-level jobs: mapping
A literal $NAME string appears in the job log where a value was expected A bare $NAME reference — the interpolator only expands the brace form, so the bare reference ships as a literal and never resolves Use ${NAME} everywhere, in variables: and in commands: alike (§1.2)
The job declares CI_SBOM_TOKEN and the attach still 401s That credential is retired: the runner host no longer injects it, so the self-referential declaration expands to nothing Delete the declaration and authenticate the attach with ${VETRIX_JOB_TOKEN} (§1.2)
The per-job token appears unredacted in the job log secret: true was dropped from the job on the grounds that its variables: block names no secret. The runner seeds VETRIX_JOB_TOKEN into that map, so the flag is still what arms redaction Keep secret: true on any job that carries the token (§1.2)
A declared secret's resolved value appears unredacted in the job log A bare $NAME reference (no braces) anywhere in the job's variables: never resolves (varRe is brace-only, §1.2) — the redaction set is built from each variable's expanded value, so a bare-form entry arms the redactor with that harmless literal instead of the real secret. This is not a v2-only gap: secret: true (v1) and secret: true / secrets: (v2, via internal/cicd/parser/v2.go's JobV2.Secret field) both genuinely trigger whole-job redaction through the same shared JobConfig.Secret gate Use the brace form ${NAME} for every secret-carrying variable, under either schema (§1.2)
The scan attests the source tree instead of the image A dir:. fallback was added to "make the scan pass" Never add one. An unpullable or unbuilt image must fail the job