Runbook — enable SBOM attestation on a repository's pipeline
Take a repository from "no SBOM" to a registered pipeline artifact carrying an attached
CycloneDX attestation, using only vetrix-ci.yml and operator-side environment
provisioning. Every command below is copy-pasteable; nothing here requires the web UI
except the optional visual check in section 4.
Audience: pipeline authors and CI/CD operators.
Current shipped state — read this before you start.
- Attestations are signed differently per lane.
vetrix/vetrix(scripts/ci/sbom-publish.sh, used by thepublish-server-image*jobs) attaches unsigned (--sign=false); its resulting row carriessignature_required: falseand nosigner_subject.vetrix/vetrix-frontend(vetrix-ci.yml'ssbom-webjob) attaches signed (--sign=true) — flipped from--sign=falseonce the signing daemon reached production and the server's attestation config was wired up to use it — and its resulting row carriessignature_required: truewith a populatedsigner_subject. There is no single "current shipped state" for this flag; check the job you're mirroring before copying a recipe below. - The SBOM describes what ships — the built container image — never the source tree.
There is no
dir:.fallback in either recipe, by design: a source-tree scan silently attests something other than the artifact. - Both control-plane calls — registering the pipeline artifact and attaching the
attestation to it — authenticate with the same per-job
VETRIX_JOB_TOKEN. The long-livedCI_SBOM_TOKENservice bearer the attach step used to take is retired from both shipped lanes; do not put it in a new pipeline. Section 1.2 has the detail; section 5 lists the symptoms of getting the credential wrong.
This runbook is about
vetrix-ci.ymlpipelines. Thevetrix/sbom@<SHA>composite step described in SBOM — Software Bill of Materials belongs to the separate automations subsystem (.vetrix/automations/*.yml,jobs:/steps:/uses:syntax). Those keys are rejected by the pipeline parser:steps:andjobs:inside a job definition are both hard parse errors under the v1 and the v2 schema (internal/cicd/parser/v1.goincompatibleJobKeys,internal/cicd/parser/v2.godroppedJobKeysV2). Do not copy composite-step YAML into a pipeline file. Everything a pipeline needs is in sections 2 and 3 below.
1. Prerequisites matrix
| Name | Supplied by | How the job references it | Absent → |
|---|---|---|---|
CI_API_URL |
Runner worker, automatically, for every job | Read from the environment. Never declare it in variables: |
Job must skip or fail loudly; never fall back to a production host |
VETRIX_JOB_TOKEN |
Runner, minted per job | Read from the environment. Never declare it | Both the artifact registration and the attestation attach fail authentication |
VETRIX_JOB_ID |
Runner, per job | Read from the environment. Never declare it | Registration URL has no job segment |
CI_REGISTRY, CI_COMMIT_SHA, CI_COMMIT_REF_NAME |
Runner, per job | Read from the environment | Image reference / ref guard cannot be built |
syft + vetrix-sbom |
The job image, or pinned tool images over the Docker socket | See "Scanner availability" below | No SBOM can be produced or attached |
1.1 CI_API_URL — the control-plane base URL
Highest precedence: a CI_API_URL configured at an admin variable-store scope —
repo, org, or instance — overrides everything else (cmd/worker/branchvars.go
ResolveAPIURLOverride). That approved-scope set deliberately excludes the branch
scope (and, transitively, committed YAML): only an admin-configured store value can
redirect the control plane this way.
Absent that override, the worker resolves a fallback value once at start-up and injects it
into every job container. The resolution order (cmd/worker/main.go resolveServerURL)
is:
VETRIX_SERVER_URLVETRIX_EXTERNAL_URLEXTERNAL_URLSERVER_EXTERNAL_URLhttps://<host>derived fromCI_REGISTRY, else fromRUNNER_CI_REGISTRY
Full precedence, narrowest-effective first: the admin-scope override, then the
worker-resolved value above, then omitted. If neither resolves, the executor emits no
empty -e CI_API_URL=. A job therefore sees either a usable value or nothing at all.
CI_API_URL is a reserved canonical name (cmd/worker/docker_executor.go
reservedCanonicalEnvKeys). Reserved names are emitted after the job's own
variables: block, so a committed variables: CI_API_URL: entry or a branch-scoped
variable-store overlay cannot clobber the resolved value. Declaring it in YAML has no
effect and only obscures where the value comes from.
The fail-loud contract. A job that needs the control plane must never invent a default. Either abort:
if [ -z "${CI_API_URL}" ]; then
echo "CI_API_URL is not set: the worker did not inject the control-plane URL." >&2
echo "Refusing to default to a production control plane. Set the worker's instance" >&2
echo "external URL (VETRIX_EXTERNAL_URL) and restart the worker." >&2
exit 1
fi
…or skip the attestation work while still doing the rest of the job (section 2 uses this form, so an unprovisioned worker still publishes its image).
Operator note: the worker reads this environment at process start. Merging a change to a
branch does not re-read it — the persistent worker daemon has to be restarted or
redeployed before a newly-set VETRIX_EXTERNAL_URL reaches any job.
1.2 VETRIX_JOB_TOKEN — the one credential for both control-plane calls
Both shipped lanes authenticate the artifact-register POST and the vetrix-sbom attach step with the same per-job VETRIX_JOB_TOKEN. Nothing else has to be provisioned.
The attach endpoint accepts two credential classes on one route
(internal/api/artifact_attestations_handler.go):
| Credential | Required permission | Reach |
|---|---|---|
| Session / access token (web UI, a human with a PAT) | ci:write on the repo |
Every artifact in every repo that identity holds ci:write on — but for kind: sbom only; see the sixth gate below |
Per-job VETRIX_JOB_TOKEN (minted at dispatch) |
registry:write |
Only artifacts belonging to the token's own pipeline, in its own repo; expires shortly after the job |
The job-token path is the strictly tighter one. It runs a five-gate ladder — the token must
verify, carry registry:write, be per-job scoped (a repo-only registry token is refused),
match the repo resolved from the URL, and its pipeline must own the artifact being attested
— and it needs no standing credential on any worker host. Because the register and attach
steps run back-to-back in the same job, the token's claims match the artifact by
construction.
Tighter on those five gates, but not narrower everywhere. A sixth gate runs inside the
handler's kind switch, after all five have passed, and it cuts the other way: for
kind: image-signature the credential must be a per-job token — jobClaims non-nil with a
non-Nil JobID — and a session / ci:write bearer is refused 403
image-signature attestations require a per-job token
(internal/api/artifact_attestations_handler.go, the attestation.KindImageSignature
case). Only the job-token path can write that kind, which is why the session row's reach in
the table above is kind: sbom only. The SBOM path both classes use is unchanged, so
nothing in this runbook is affected — but do not read that table as "session credentials
reach everything a job token reaches, and more".
For the scopes each credential class carries, see CI/CD API reference — Auth & scopes.
CI_SBOM_TOKEN is retired from the pipeline recipes. It was a long-lived ci:write
service bearer held in the runner-host process environment, and it reached a job container
by exactly one mechanism — a self-referential variables: declaration,
CI_SBOM_TOKEN: "${CI_SBOM_TOKEN}", resolved against Runner.hostEnvMap
(internal/cicd/runner.go). Nothing else pushes it in: the executor injects only the
canonical reserved keys. Deleting that declaration therefore is the host-injection
removal, and neither shipped lane carries it any more. The backend build enforces this —
internal/cicd/parser/vbe1815_ci_sbom_token_retired_test.go fails if any job reintroduces
the name, in variables: or in a commands: body. Do not add it to a new pipeline.
The session ci:write path itself is unchanged and still supported; it is how the web
UI and a human with a PAT attach an attestation out of band. What is retired is standing a
long-lived copy of such a credential on runner hosts so a pipeline can use it. The CLI is
credential-agnostic — --token takes any bearer (cmd/vetrix-sbom/main.go) — so which
credential a lane uses is decided entirely by what the YAML passes.
Secret-carrying variables: brace form and log redaction
Keep secret: true on the job even though the recipes below declare no secret of their
own. secret: true is what puts a job's expanded variable values into the runner's
log-redaction set (RedactSecrets, internal/cicd/runner.go), and the runner mints
VETRIX_JOB_TOKEN into the job's variable map before interpolation. The mint is a runner
step, not a parser one: RunJob clones the job's Variables map and assigns
vars["VETRIX_JOB_TOKEN"] = tok (internal/cicd/runner.go). Do not look for it in the
interpolator — InterpolateJobReservedFields (internal/cicd/parser/v1.go) takes neither a
token nor a minter and seeds nothing; it only expands ${...} references in a map it is
handed. Dropping the flag "because there is no secret variable left in the YAML" leaks the
per-job token into the job log: the redaction set starts from a sweep of job.Variables
under if job.Secret || … after the mint, and that sweep is the only step that could add
the token to the set — with the flag dropped, job.Secret is false and the token's name
carries no SECRET_ prefix, so the token never enters the set regardless of what else does.
The runner separately, and unconditionally, appends the job's resolved branch/repo
variable-store values onto the same set — those are redacted regardless of the secret:
flag — so the log sink itself is not reliably left unarmed: a job with the flag dropped and
no variables: of its own can still get a redacting sink whenever the branch/repo store
resolves something for it. That does not help the token, which the sweep excluded either
way. Only when the branch/repo store resolves nothing for a job does dropping the flag also
leave the sink unarmed, with the real token sitting unredacted in the map — the leak holds
regardless.
Any variable you do declare must use the brace form. The interpolator's pattern is
\$\{([A-Za-z_][A-Za-z0-9_]*)\} — brace form only (internal/cicd/parser/v1.go varRe).
A bare $NAME is shipped verbatim as a literal, never resolves, and arms the log redactor
with that placeholder instead of the real value. The same rule governs the self-referential
shape NAME: "${NAME}" used for any operator-injected host variable: it is not a typo and
not a cycle, because the interpolator resolves a self-reference from the runner-side source
(host environment plus push-context variables), deliberately excluding the job's own
variables: map.
Log redaction. secret: true on the job genuinely arms whole-job log redaction —
under both the v1 and the v2 schema. When armed, the expanded values of every one of
that job's variables are collected into the redaction set, and matching text is replaced
with *** in captured logs. This has always been true: secret is a first-class
JobConfig field going back to before the v2 schema existed (internal/cicd/parser/v1.go),
and v2 carries it through unchanged — internal/cicd/parser/v2.go's toPipelineConfig
computes Secret: len(j.Secrets) > 0 || j.Secret, so a bare secret: true on a v2 job
arms redaction exactly as it does under v1. secrets: is not a replacement for
secret: true, and it is not narrower — a non-empty secrets: list arms the same
whole-job redaction, not just the named variables. The difference is a side effect
secret: true does not have — see below.
Three facts worth knowing:
- Any variable whose name begins with
SECRET_is redacted per-variable even without the flag. - Under the v2 schema the job field
secrets:(a list of variable names) is a second, independent way to switch on redaction: a non-emptysecrets:list also arms whole-job redaction (via the samelen(j.Secrets) > 0term above), on top of anythingsecret: truealready armed. Note that each name listed insecrets:also has its value overwritten with the empty string when the v2 job is adapted to the shared job config, so do not list a variable you also need a value for. That overwrite is a parse-time seeding, and a run-time mint overwrites it in turn — which is why it does not blankVETRIX_JOB_TOKEN(§3). It blanks every name nothing mints.secret: truehas no such side effect. - Redaction only ever matches a variable's expanded value — never the literal reference
text. The interpolator's pattern (
varRe, the callout just above) is brace-only:${NAME}expands and its resolved value is what gets added to the redaction set, but a bare$NAME(no braces) never expands at all. A bare reference is therefore shipped as a harmless literal string, and if that variable is also covered bysecret: trueorsecrets:, the redactor ends up armed with the harmless literal instead of the real secret value — the real value, having never been substituted in, reaches the log unredacted wherever it appears verbatim. Always use the brace form${NAME}for any variable you intendsecret: true/secrets:to actually protect.
1.3 VETRIX_JOB_TOKEN / VETRIX_JOB_ID — never declared
Both are reserved canonical names injected by the executor. VETRIX_JOB_TOKEN is a
per-job JWT carrying registry:write plus repo_id / pipeline_id / job_id claims;
VETRIX_JOB_ID is the job UUID that forms the artifact-registration path segment.
Declaring either in variables: would hardcode a literal or shadow the injected value.
Read them straight from the environment. RunJob seeds VETRIX_JOB_TOKEN into the job's
variable map itself — the vars["VETRIX_JOB_TOKEN"] = tok assignment in
internal/cicd/runner.go, not anything in the parser — which is why secret: true covers
it without — and only without — a declaration of your own (§1.2).
1.4 Scanner availability
Two provisioning routes work today. Pick by what the job image can do.
(a) Pinned tool images over the Docker socket. Works from any job image that has a
POSIX shell and can reach the daemon. syft runs as anchore/syft:v1.18.0; the
vetrix-sbom CLI is built from cmd/vetrix-sbom through a streamed Docker build context.
This is what section 2 uses.
Requires the operator to have opted the runner into socket pass-through by setting
RUNNER_DOCKER_SOCKET on the worker. When it is unset no socket is mounted — that is
the default, because mounting the host socket grants jobs root-equivalent access to the
runner host. With it set, the socket appears at /var/run/docker.sock inside the job, so
DOCKER_HOST: "unix:///var/run/docker.sock" reaches the host daemon.
(b) An image that already ships both. vetrix/runner-security carries syft v1.18.0
at /usr/local/bin/syft and the vetrix-sbom CLI at /usr/local/bin/vetrix-sbom
(deployments/runner-security/Dockerfile). It ships no Docker daemon and no
crane / oras / skopeo, so scans must use syft's own registry client — see
section 3. It also runs as USER nobody with WORKDIR /workspace, and declares
ENTRYPOINT ["/usr/local/bin/vetrix-security-scan"] — its own composite-action
orchestrator binary, not a shell that execs its arguments.
Before choosing route (b):
- Confirm the image is actually present in the registry the runner pulls from. Nothing in
this runbook's recipes (sections 2 and 3) builds or publishes
vetrix/runner-security— provisioning it into that registry is a separate, out-of-band operational step. Route (a) has no such dependency. - No entrypoint check is actually needed, despite the image's own
ENTRYPOINTnot execing its arguments. A baked entrypoint that ignores its arguments used to swallow a job'scommands:silently; the runner now emits an explicit--entrypoint /bin/shwhenever the shell script built from a job'scommands:is non-empty (cmd/worker/docker_executor.gobuildDockerRunArgs, viabuildShellScript), overriding any imageENTRYPOINTunconditionally.buildShellScriptright-trims eachcommands:entry and drops any entry that trims to empty, so a job whosecommands:block is present but contains only blank/whitespace entries builds an empty script and takes the same no-override path as a job with nocommands:at all (the composite-action dispatch shape) — both run the image's own baked entrypoint as-is.
2. v1 schema recipe — fold the SBOM into the image-publish job
Fold, do not split. Scanning in the same job that built the image means the image under attestation is the local, already-authenticated build: no unauthenticated registry probe, no cross-job ordering hazard, and no window in which the digest you register differs from the digest you scanned.
The v1 job-key allowlist is stage, image, commands, variables, only, except,
secret, environment, allow_failure, and artifacts.paths. before_script,
after_script, script, services, runs-on, steps, and jobs are hard parse
errors. There is no needs: under v1, so cross-job hand-off is not available — another
reason to fold. Full schema: Authoring a Vetrix CI/CD Pipeline File.
stages:
- build
- test
- publish
publish-server-image:
stage: publish
image: docker:25-cli
secret: true
only:
- develop
variables:
DOCKER_HOST: "unix:///var/run/docker.sock"
# VETRIX_JOB_TOKEN / VETRIX_JOB_ID are injected by the worker and are
# deliberately NOT declared here. `secret: true` above still matters: the
# runner seeds VETRIX_JOB_TOKEN into this job's variables, and that is what
# the log redactor is built from (§1.2).
commands:
# Fail fast on a missing socket mount before the slow build.
- docker version
- |
set -e
# run_step <description> <command> [args...]. `exit` inside a POSIX shell
# function exits the shell, so a failed step ends the job carrying that
# step's own status. The job shell here is Alpine ash: no bashisms.
run_step() {
_desc="$1"
shift
set +e
"$@"
_rc=$?
set -e
if [ "$_rc" -ne 0 ]; then
echo "publish-server-image: $_desc FAILED (exit $_rc)" >&2
exit "$_rc"
fi
}
# v1 has no after_script:; an EXIT trap is the always-run teardown.
trap 'docker logout "${CI_REGISTRY}" >/dev/null 2>&1 || true' EXIT
run_step "docker build" \
docker build --target server -t "myimage:${CI_COMMIT_SHA}" -f Dockerfile .
run_step "docker login" \
sh -c 'echo "${VETRIX_JOB_TOKEN}" | docker login "${CI_REGISTRY}" --username x-vetrix-runner --password-stdin'
run_step "docker tag" \
docker tag "myimage:${CI_COMMIT_SHA}" "${CI_REGISTRY}/<owner>/<repo>:${CI_COMMIT_SHA}"
run_step "docker push" \
docker push "${CI_REGISTRY}/<owner>/<repo>:${CI_COMMIT_SHA}"
# The SBOM control-plane round-trips need CI_API_URL. Gate on it so an
# unprovisioned worker still publishes the image, and so the attestation
# activates by itself once the worker injects the value.
if [ -n "${CI_API_URL}" ]; then
(
set -e
# docker:25-cli is Alpine ash and ships NO curl.
if ! command -v curl >/dev/null 2>&1; then
run_step "install curl" apk add --no-cache curl ca-certificates
fi
# Resolve the REAL pushed digest and on-disk size; fail closed on an
# empty or malformed value. RepoDigests[0] is the registry-assigned
# manifest digest, populated by the push above.
DIGEST_REF=$(docker image inspect --format '{{index .RepoDigests 0}}' "${CI_REGISTRY}/<owner>/<repo>:${CI_COMMIT_SHA}")
SHA256=$(printf '%s' "$DIGEST_REF" | sed -n 's/.*@\(sha256:[0-9a-f]\{64\}\).*/\1/p')
SIZE=$(docker image inspect --format '{{.Size}}' "myimage:${CI_COMMIT_SHA}")
STORAGE_KEY="${CI_REGISTRY}/<owner>/<repo>:${CI_COMMIT_SHA}"
if ! printf 'X%s' "$SHA256" | grep -q '^Xsha256:[0-9a-f]\{64\}$'; then
printf 'could not resolve pushed image digest (got: %s)\n' "$SHA256" >&2
exit 1
fi
case "$SIZE" in
''|*[!0-9]*)
printf 'image size not numeric: %s\n' "$SIZE" >&2
exit 1
;;
esac
# Scan the SHIPPED LOCAL image over the mounted socket. No `dir:.`
# fallback. syft v1.18.0 writes CycloneDX to stdout for
# `-o cyclonedx-json` (no `=path`), redirected into the workspace here.
run_step "syft scan (cyclonedx-json)" \
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
anchore/syft:v1.18.0 "docker:myimage:${CI_COMMIT_SHA}" -o cyclonedx-json > sbom.json
if [ ! -s sbom.json ]; then
echo "syft produced no SBOM — refusing to attach" >&2
exit 1
fi
# Register the artifact. Credential here is VETRIX_JOB_TOKEN (per-job,
# registry:write) — the same token the attach step below uses.
# Capture the HTTP status as well as the body: `-w` appends
# "\n<status>", and curl's own exit is captured separately so a
# transport failure is not mistaken for an unparseable body.
set +e
REG_RAW=$(curl -s -w '\n%{http_code}' -X POST "${CI_API_URL}/api/v1/jobs/${VETRIX_JOB_ID}/artifacts" \
-H "Authorization: Bearer ${VETRIX_JOB_TOKEN}" \
-H "Content-Type: application/json" \
-d "{\"name\":\"server-image\",\"storage_key\":\"$STORAGE_KEY\",\"size_bytes\":$SIZE,\"sha256\":\"$SHA256\"}")
rc=$?
set -e
if [ "$rc" -ne 0 ]; then
echo "artifact registration could not reach the API (curl exit $rc)" >&2
exit 1
fi
HTTP_CODE=$(printf '%s' "$REG_RAW" | tail -n 1)
RESP=$(printf '%s' "$REG_RAW" | sed '$d')
case "$HTTP_CODE" in
2??) printf 'Artifact registration accepted: HTTP %s\n' "$HTTP_CODE" ;;
'') echo "artifact registration returned no HTTP status line" >&2; exit 1 ;;
*) printf 'artifact registration REJECTED: HTTP %s\n%s\n' "$HTTP_CODE" "$RESP" >&2; exit 1 ;;
esac
# Depth-aware, jq-free extraction of the genuine TOP-LEVEL "id".
# docker:25-cli ships no jq and no python3. Fold newlines, strip the
# outer braces, delete every innermost {...}/[...] until stable (that
# removes exactly the nested objects/arrays), then take the
# member-anchored "id" — which never matches job_id or pipeline_id.
_art_flat=$(printf '%s' "$RESP" | tr -d '\n\r')
_art_inner=$(printf '%s' "$_art_flat" | sed -e 's/^[[:space:]]*{//' -e 's/}[[:space:]]*$//')
while :; do
_art_next=$(printf '%s' "$_art_inner" | sed -e 's/{[^{}]*}//g' -e 's/\[[^][]*\]//g')
[ "$_art_next" = "$_art_inner" ] && break
_art_inner=$_art_next
done
ART_ID=$(printf '%s' "$_art_inner" | tr ',' '\n' \
| sed -n 's/^[[:space:]]*"id"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' \
| head -n 1)
if [ -z "$ART_ID" ] || [ "$ART_ID" = "null" ]; then
printf 'HTTP %s but no top-level "id" in the body: %s\n' "$HTTP_CODE" "$RESP" >&2
exit 1
fi
# Fail closed unless the id is UUID-shaped. A wrong id handed to attach
# would bind the attestation to another artifact with no error signal.
if ! expr "X$ART_ID" : 'X[0-9a-fA-F]\{8\}-[0-9a-fA-F]\{4\}-[0-9a-fA-F]\{4\}-[0-9a-fA-F]\{4\}-[0-9a-fA-F]\{12\}$' >/dev/null 2>&1; then
printf 'extracted artifact id is not UUID-shaped: "%s"\n' "$ART_ID" >&2
exit 1
fi
printf 'Registered artifact ID: %s\n' "$ART_ID"
# Build the attach CLI from the STREAMED build context. A
# `-v "$PWD":/src` bind mount would resolve against the HOST filesystem,
# where the in-container /workspace does not exist.
printf 'FROM golang:1.25-alpine AS b\nRUN apk add --no-cache git ca-certificates\nWORKDIR /build\nCOPY go.mod go.sum* ./\nRUN go mod download\nCOPY . .\nRUN CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /vetrix-sbom ./cmd/vetrix-sbom\nFROM scratch\nCOPY --from=b /vetrix-sbom /vetrix-sbom\n' > /tmp/vetrix-sbom.Dockerfile
run_step "docker build vetrix-sbom" \
docker build -t "vetrix-sbom-build:${CI_COMMIT_SHA}" -f /tmp/vetrix-sbom.Dockerfile .
_sbom_cid=$(docker create "vetrix-sbom-build:${CI_COMMIT_SHA}" /vetrix-sbom) || exit 1
run_step "extract vetrix-sbom" docker cp "${_sbom_cid}:/vetrix-sbom" ./vetrix-sbom
docker rm "${_sbom_cid}" >/dev/null 2>&1 || true
# Attach. --token is the SAME per-job VETRIX_JOB_TOKEN the register
# call above used — the attach endpoint accepts it, and its claims
# already match this artifact. --server is the worker-injected
# CI_API_URL, never a hardcoded host.
run_step "vetrix-sbom attach" ./vetrix-sbom attach \
--server "${CI_API_URL}" \
--token "${VETRIX_JOB_TOKEN}" \
--repo "<owner>/<repo>" \
--artifact "$ART_ID" \
--format "cyclonedx-json" \
--sbom "sbom.json" \
--sign=false
)
else
printf 'WARNING: CI_API_URL was not injected by the worker — publishing without an SBOM attestation.\n' >&2
fi
Substitute <owner>, <repo>, the image name, and the --target for your repository.
2.1 The fail-closed guards, and why each exists
| Guard | Prevents |
|---|---|
sha256 shape check on the resolved digest |
Registering a placeholder such as an all-zeros digest, which makes the artifact row unverifiable |
Numeric check on SIZE |
Registering a placeholder size (1 is the classic one) |
[ ! -s sbom.json ] |
Attaching an empty payload — the CLI and the API both reject it, but later and less clearly |
| HTTP-status capture on the register call | Treating a 4xx body as a successful registration |
Depth-aware top-level "id" extraction |
Picking up a nested job_id / pipeline_id instead of the artifact id |
UUID-shape check on ART_ID |
Binding the attestation to a different artifact with no error signal |
2.2 Best-effort mode (optional)
The block above is fail-loud: any SBOM sub-step failure ends the job. To keep a blocking publish job green while the attestation path is still being provisioned, wrap the subshell in a status downgrade:
if [ -n "${CI_API_URL}" ]; then
set +e
(
set -e
# ... all SBOM sub-steps ...
)
_sbom_rc=$?
set -e
if [ "$_sbom_rc" -ne 0 ]; then
printf 'WARNING: SBOM sub-steps failed (exit %s) — continuing; attestation SKIPPED this run.\n' "$_sbom_rc" >&2
fi
fi
set -e and exit inside the subshell terminate only the subshell. The subshell also
resets the inherited EXIT trap to default, so a docker logout trap fires only when the
main job shell exits.
Understand the cost before adopting it: a green job no longer proves an attestation exists. Verify with section 4, not with the job's exit status. Remove the downgrade once the path has been observed green.
3. v2 schema recipe — a dedicated job behind needs:
Use this when the pipeline declares version: 2 and an earlier job already pushed the
image. v2 adds needs:, which gates dispatch on the named jobs completing, so the scan
job can run separately from the build without an ordering hazard.
v2 differences that matter here:
- Jobs live under a top-level
jobs:mapping. Declaring them at the document root — the v1 layout — is rejected withpipeline defines no jobs (missing jobs: block). needs:targets must be in the same or an earlier stage. Same-stage edges are allowed.- v2 adds a
secrets:field (a list of variable names) — a mechanism v1 does not have.secret: trueremains valid on v2 jobs too (internal/cicd/parser/v2.go'sJobV2.Secretfield, a v1-compat alias); see §1.2 for how the two combine. - The image ships no Docker daemon, so the scan reads the image from the registry
using syft's own registry client, authenticated by
SYFT_REGISTRY_AUTH_*.
version: 2
stages:
- build
- test
- publish
jobs:
build-image:
stage: publish
image: docker:25-cli
only:
- master
variables:
DOCKER_HOST: "unix:///var/run/docker.sock"
commands:
- docker version
# ... build, login with VETRIX_JOB_TOKEN, tag, push ...
sbom-web:
stage: publish
image: vetrix/runner-security
secret: true
needs:
- build-image
only:
- master
# No `variables:` block at all: every value this job needs is a reserved
# name the worker injects. `secret: true` above is still required — see the
# note under the block.
commands:
# Ref guard — see §3.1. Keep it FIRST: attaching an SBOM from another ref
# would register artifacts and attestations against the wrong image.
# `case` (not `if`) tolerates an unset CI_COMMIT_REF_NAME.
- |
case "${CI_COMMIT_REF_NAME:-}" in
refs/heads/master) ;;
*) echo "sbom-web: skipping — ref ${CI_COMMIT_REF_NAME:-<unset>} is not refs/heads/master"; exit 0 ;;
esac
- |
set -e
run_step() {
_desc="$1"
shift
set +e
"$@"
_rc=$?
set -e
if [ "$_rc" -ne 0 ]; then
echo "sbom-web: $_desc FAILED (exit $_rc)" >&2
exit "$_rc"
fi
}
# No production fallback — fail fast on an unset control-plane URL.
if [ -z "${CI_API_URL}" ]; then
echo "CI_API_URL is not set: the worker did not inject the control-plane URL." >&2
echo "Refusing to default to a production control plane. Set the worker's" >&2
echo "instance external URL (VETRIX_EXTERNAL_URL)." >&2
exit 1
fi
IMG="${CI_REGISTRY}/<owner>/<repo>:${CI_COMMIT_SHA}"
# Daemon-free registry auth for syft. This image has no docker daemon
# (so no `docker login`) and no crane/oras/skopeo. syft's own registry
# client authenticates the pull from these variables. The credential is
# the same per-job token the build job logged in with.
export SYFT_REGISTRY_AUTH_AUTHORITY="${CI_REGISTRY}"
export SYFT_REGISTRY_AUTH_USERNAME="x-vetrix-runner"
export SYFT_REGISTRY_AUTH_PASSWORD="${VETRIX_JOB_TOKEN}"
if ! command -v syft >/dev/null 2>&1; then
echo "sbom-web: syft not found on the image" >&2
exit 1
fi
# ONE authenticated pass emits BOTH the CycloneDX payload we attach AND
# the syft-json image metadata we register from, so the digest
# registered is exactly the digest the SBOM was built from — there is no
# resolve-then-scan window. `registry:` is the only source scheme used;
# there is no unauthenticated manifest probe and no `dir:.` fallback, so
# an unpullable image fails the job CLOSED rather than attesting the
# git checkout.
run_step "syft scan (registry: cyclonedx-json + syft-json)" \
syft scan "registry:${IMG}" \
-o cyclonedx-json=sbom.json \
-o syft-json=image-meta.json
if [ ! -s sbom.json ]; then
echo "sbom-web: syft produced no SBOM — refusing to attach" >&2
exit 1
fi
if [ ! -s image-meta.json ]; then
echo "sbom-web: syft produced no image metadata — cannot resolve digest/size" >&2
exit 1
fi
# jq IS present on this image. manifestDigest is the registry manifest
# digest; imageSize is the image's byte size. Fail CLOSED on a
# non-sha256 digest or a non-positive size.
SHA256=$(jq -r '.source.metadata.manifestDigest // empty' image-meta.json)
SIZE=$(jq -r '.source.metadata.imageSize // empty' image-meta.json)
STORAGE_KEY="$IMG"
if ! printf 'X%s' "$SHA256" | grep -q '^Xsha256:[0-9a-f]\{64\}$'; then
printf 'sbom-web: could not resolve manifest digest (got: %s)\n' "$SHA256" >&2
exit 1
fi
case "$SIZE" in
''|*[!0-9]*) printf 'sbom-web: image size not numeric: %s\n' "$SIZE" >&2; exit 1 ;;
esac
if [ "$SIZE" -le 0 ]; then
printf 'sbom-web: image size must be positive (got: %s)\n' "$SIZE" >&2
exit 1
fi
# Register the artifact. Credential here is VETRIX_JOB_TOKEN (per-job,
# registry:write) — the same token the attach step below uses, and the
# same token the syft registry pull above authenticated with.
# Capture the HTTP status as well as the body: `-w` appends
# "\n<status>", and curl's own exit is captured separately so a
# transport failure is not mistaken for an unparseable body.
set +e
REG_RAW=$(curl -s -w '\n%{http_code}' -X POST "${CI_API_URL}/api/v1/jobs/${VETRIX_JOB_ID}/artifacts" \
-H "Authorization: Bearer ${VETRIX_JOB_TOKEN}" \
-H "Content-Type: application/json" \
-d "{\"name\":\"web-image\",\"storage_key\":\"$STORAGE_KEY\",\"size_bytes\":$SIZE,\"sha256\":\"$SHA256\"}")
rc=$?
set -e
if [ "$rc" -ne 0 ]; then
echo "sbom-web: artifact registration could not reach the API (curl exit $rc)" >&2
exit 1
fi
HTTP_CODE=$(printf '%s' "$REG_RAW" | tail -n 1)
RESP=$(printf '%s' "$REG_RAW" | sed '$d')
case "$HTTP_CODE" in
2??) printf 'Artifact registration accepted: HTTP %s\n' "$HTTP_CODE" ;;
'') echo "sbom-web: artifact registration returned no HTTP status line" >&2; exit 1 ;;
*) printf 'sbom-web: artifact registration REJECTED: HTTP %s\n%s\n' "$HTTP_CODE" "$RESP" >&2; exit 1 ;;
esac
# Depth-aware, jq-free extraction of the genuine TOP-LEVEL "id" — the
# same idiom §2 uses, byte-for-byte. jq IS on this image (used for
# SHA256/SIZE above), but the register response gets this extraction
# anyway rather than a second parsing approach for the same shape of
# problem. Fold newlines, strip the outer braces, delete every
# innermost {...}/[...] until stable (that removes exactly the nested
# objects/arrays), then take the member-anchored "id" — which never
# matches job_id or pipeline_id.
_art_flat=$(printf '%s' "$RESP" | tr -d '\n\r')
_art_inner=$(printf '%s' "$_art_flat" | sed -e 's/^[[:space:]]*{//' -e 's/}[[:space:]]*$//')
while :; do
_art_next=$(printf '%s' "$_art_inner" | sed -e 's/{[^{}]*}//g' -e 's/\[[^][]*\]//g')
[ "$_art_next" = "$_art_inner" ] && break
_art_inner=$_art_next
done
ART_ID=$(printf '%s' "$_art_inner" | tr ',' '\n' \
| sed -n 's/^[[:space:]]*"id"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' \
| head -n 1)
if [ -z "$ART_ID" ] || [ "$ART_ID" = "null" ]; then
printf 'sbom-web: HTTP %s but no top-level "id" in the body: %s\n' "$HTTP_CODE" "$RESP" >&2
exit 1
fi
# Fail closed unless the id is UUID-shaped. A wrong id handed to attach
# would bind the attestation to another artifact with no error signal.
if ! expr "X$ART_ID" : 'X[0-9a-fA-F]\{8\}-[0-9a-fA-F]\{4\}-[0-9a-fA-F]\{4\}-[0-9a-fA-F]\{4\}-[0-9a-fA-F]\{12\}$' >/dev/null 2>&1; then
printf 'sbom-web: extracted artifact id is not UUID-shaped: "%s"\n' "$ART_ID" >&2
exit 1
fi
printf 'Registered artifact ID: %s\n' "$ART_ID"
run_step "vetrix-sbom attach" vetrix-sbom attach \
--server "${CI_API_URL}" \
--token "${VETRIX_JOB_TOKEN}" \
--repo "<owner>/<repo>" \
--artifact "$ART_ID" \
--format "cyclonedx-json" \
--sbom "sbom.json" \
--sign=true
The register call and the artifact-id extraction are byte-for-byte the same as section 2 —
only the artifact name (web-image rather than server-image) and the digest/size
source differ.
This worked example declares no variables: at all, yet still carries secret: true —
and that is deliberate, not leftover. Under v2, secret: true arms whole-job log redaction
over the expanded values of the job's variables, the same v1-compat alias v1 has always had
(JobConfig.Secret, §1.2), wired through by internal/cicd/parser/v2.go's JobV2.Secret
field. RunJob seeds VETRIX_JOB_TOKEN into that variable map itself — the
vars["VETRIX_JOB_TOKEN"] = tok assignment in internal/cicd/runner.go, a runner step, not
a parser one (§1.2) — so the flag is what keeps the per-job token, passed to --token and
to the register call's Authorization header, out of the captured log. Removing it because
the YAML names no secret is exactly the mistake to avoid.
secrets: is not the thing to reach for here either — though not for the reason it is
tempting to give. Naming a variable in a v2 secrets: list does overwrite its value with
the empty string, but at parse time: toPipelineConfig seeds jc.Variables[s] = "" for
each listed name (internal/cicd/parser/v2.go). RunJob's mint then assigns
vars["VETRIX_JOB_TOKEN"] = tok unconditionally at run time, over whatever the parsed
config held. So listing this particular name would not blank it — the seeding that strips
an ordinary variable is simply overwritten for the one name that has a minter behind it.
That is the difference between the retired CI_SBOM_TOKEN, which nothing mints and which
the seeding therefore really does blank, and VETRIX_JOB_TOKEN, which the runner
re-supplies. Do not carry the old token's behaviour across to this one.
Keep secret: true anyway, for three reasons that do hold:
- It buys nothing to switch.
secrets:arms the same whole-job redaction, not a narrower per-name one:toPipelineConfigcomputesSecret: len(j.Secrets) > 0 || j.Secret(internal/cicd/parser/v2.go), so a non-empty list and a bare flag land on the identicalJobConfig.Secretgate. - It is the only form that works under both schemas. v1 has no
secrets:field at all, so section 2's recipe must usesecret: true; one flag across both recipes is one rule to get right. - The blanking side effect is harmless only for this name — and, for a different reason,
for the executor-owned reserved canonical names. For an ordinary job variable it is
live: list one you actually need a value for and the job gets an empty string. It is
equally inert on
CI_API_URL,CI_REGISTRY, and the rest of the setcmd/worker/docker_executor.gobuilds asreservedEnvKeysDroppedFromJobVariables— not because anything re-mints them the way the runner re-mintsVETRIX_JOB_TOKEN, but because the executor drops every one of them out ofjob.Variablesbefore assembling the container environment and emits its own resolved value afterward.secret: truehas no such side effect — it switches on redaction without touching any variable's value.
3.1 The runner-level ref guard
The shipped v2 parser does carry only: and except: into the shared job config
(internal/cicd/parser/v2.go toPipelineConfig), where the same branch filter evaluates
them for v1 and v2 alike. The first-command ref guard is therefore belt-and-braces, not a
substitute for only: — keep both. It costs one case statement and it protects a job
running against an instance whose parser build predates that carry-through.
Two details make the guard correct:
CI_COMMIT_REF_NAMEcarries the pipeline's ref, so on a branch pipeline it readsrefs/heads/<branch>.only:patterns match the short branch name, because the scheduler stripsrefs/heads/before filtering. Compare against the form the variable actually holds — the two are not interchangeable.only:/except:matching (internal/cicd/parser/v1.gomatchBranch) is a plain prefix/suffix test, not a slash-aware glob: a trailing*checksstrings.HasPrefixand a leading*checksstrings.HasSuffix, with no exclusion for/. So*spans path separators —feature/*matchesfeature/a/b, not just one path segment. (matchBranch's own doc comment says "matches any run of non-slash characters"; that comment is stale — the code above is what actually runs.) Anything without a leading or trailing*is an exact match.
allow_failure: true (a first-class boolean job field under both schemas, default
false) makes a failure soft so it cannot block a deploy off the same branch. Like the
best-effort pattern in §2.2, it means a green pipeline does not prove the attestation
landed. The recipe above does not carry it, matching the shipped sbom-web job: that
job carried the flag from its introduction until a master pipeline was observed with it at
state: success, then removed it in the same change that flipped the signing flag
(sbomWebJobContract.test.ts now asserts the flag's absence). If you're bringing up a
new lane and want the same soft-start safety net while the path is unverified, add
allow_failure: true back and remove it once you've observed the job green, the same way
sbom-web did.
4. Verification
Three checks, cheapest first. All API paths below are relative to ${CI_API_URL} (or your
instance's API base); every call needs a bearer credential with ci:read.
1. Find the artifact id.
curl -s -H "Authorization: Bearer $TOKEN" \
"$API/api/v1/repos/<owner>/<repo>/pipelines/<pipeline-id>/artifacts"
Returns a bare JSON array. Look for the artifact your job registered by name
(server-image, web-image, …) and take its id. Its sha256 must be the real image
digest and size_bytes the real byte size — a 0000… digest or a size_bytes of 1 is
the placeholder trap in section 5.
2. List the attestations on that artifact.
curl -s -H "Authorization: Bearer $TOKEN" \
"$API/api/v1/repos/<owner>/<repo>/artifacts/<artifact-id>/attestations"
Note the base path: attestations hang off /artifacts/{id} directly, not under
/pipelines/{id}/. Newest first. A successful attach shows a row with kind: "sbom",
format: "cyclonedx-json", predicate_type: "https://cyclonedx.org/bom", and a
payload_sha256. signature_required and signer_subject differ by lane (see the header
above): on the still-unsigned vetrix/vetrix lane expect signature_required: false and
no signer_subject; on the signed vetrix/vetrix-frontend lane expect
signature_required: true with a populated signer_subject — an absent or empty value
there, on a run where the attach step itself succeeded, means the signing daemon did not
sign, which is worth its own bug report rather than being read as expected behavior.
3. Download the payload bytes.
curl -s -H "Authorization: Bearer $TOKEN" -OJ \
"$API/api/v1/repos/<owner>/<repo>/artifacts/<artifact-id>/attestations/<attestation-id>/payload"
Streams the exact stored envelope. It is always served as an attachment with
X-Content-Type-Options: nosniff, so a browser never renders it inline.
Endpoint details, status codes, and response shapes: CI/CD API reference.
Optional visual check. Open the repository's pipeline run detail page. Below the
coverage report is a panel headed SBOM attestations, described as "Software
bill-of-materials and provenance attestations attached to this pipeline's artifacts". It
fetches the pipeline's artifacts collection and issues one attestation query per artifact,
so it lists every row on every artifact — a pipeline is never assumed to have exactly one
SBOM. Each row shows a Signed / Unsigned badge (from signature_required), the
format, a truncated SHA-256 with a copy control, a relative creation time, and a
Download button. The panel has four mutually exclusive states: loading, an error alert
with Retry, the empty state ("No SBOM attestations for this pipeline."), and the
populated list.
5. Failure modes
| Symptom | Cause | Fix |
|---|---|---|
Job log: WARNING: CI_API_URL was not injected by the worker |
The worker resolved no instance external URL, so the variable was omitted entirely | Set VETRIX_EXTERNAL_URL (or another accepted name, §1.1) in the worker's process environment and restart the worker. Merging a branch does not re-read it |
| Job green, no attestation anywhere | The best-effort pattern (§2.2) or allow_failure: true (§3) downgraded a real failure to a warning |
Read the job log for the WARNING: line and the preceding ... FAILED (exit N); verify with section 4, never with the job's exit status |
Attach step: attestation server returned 401 |
The bearer did not verify — an empty ${VETRIX_JOB_TOKEN} (typically a bare $NAME reference that never expanded, §1.2), or a job token that has already expired |
Pass --token "${VETRIX_JOB_TOKEN}" in brace form, and keep the attach in the same job as the register call so the token is still live |
Attach step: attestation server returned 403 |
A verifying credential the attach gates refuse: a job token without registry:write, a repo-only registry token that is not per-job scoped, a token whose repo does not match the URL, or one whose pipeline does not own the artifact. For a session credential, no CI write access on the repo |
Attach from the same job that registered the artifact, so the token's pipeline owns it (§1.2). For a human/UI attach, use a credential with ci:write on that repo |
Register call: HTTP 401 / 403 token missing registry:write scope |
A session/service ci:write bearer was sent to the register endpoint. That endpoint takes the per-job token only |
Send ${VETRIX_JOB_TOKEN} to /api/v1/jobs/{id}/artifacts — and to the attach endpoint too; one credential serves both (§1.2) |
Register call: 403 job token scope does not match job id |
The {id} path segment is not VETRIX_JOB_ID |
Build the URL from ${VETRIX_JOB_ID}, never from a pipeline id or a job number |
Register call: 400 invalid request body naming an unknown field |
The register body decoder rejects unknown fields | Send only name, storage_key, size_bytes, sha256 (plus the optional retention_days / retention / paths) |
Register call: 400 storage_key must be a relative path without '..' segments |
An absolute or traversing storage_key |
Use the image reference form <registry>/<owner>/<repo>:<tag> |
sh: curl: not found, exit 127 |
docker:25-cli is Alpine and ships no curl |
apk add --no-cache curl ca-certificates, guarded by command -v curl so it stays idempotent — and keep it inside the CI_API_URL gate so an unprovisioned worker incurs no fetch |
Artifact row has an all-zeros sha256 or size_bytes: 1 |
Placeholder values posted instead of resolved ones | Resolve the real digest and size (§2 / §3) and fail closed on a malformed value. An artifact registered with a placeholder digest cannot be verified against the image and the attestation on it is worthless |
| Attestation attached to the wrong artifact, no error | A nested job_id / pipeline_id was extracted instead of the top-level "id" |
Use the depth-aware extraction plus the UUID-shape guard (§2.1) |
Attach: 422 with {"error":"unsupported sbom format","supported":[...]} |
--format outside the closed set |
Use cyclonedx-json (the default), spdx-json, or spdx-tag-value |
Attach: 400 sbom payload is empty |
sbom.json produced no bytes |
The [ ! -s sbom.json ] guard should have caught this first; check the syft step's exit status and the scan target |
Attach: 503 attestation service not configured |
The deployment has no attestation service wired | Operator action; nothing in the pipeline file can work around it |
Parse error: job field `steps` is a GitHub Actions convention |
Composite-step YAML pasted into vetrix-ci.yml |
Use the recipes in sections 2 and 3. steps:, jobs:, script:, runs-on:, services:, before_script:, and after_script: inside a job are all hard errors |
Parse error: pipeline defines no jobs (missing jobs: block) |
version: 2 with jobs at the document root |
Nest every job under the top-level jobs: mapping |
A literal $NAME string appears in the job log where a value was expected |
A bare $NAME reference — the interpolator only expands the brace form, so the bare reference ships as a literal and never resolves |
Use ${NAME} everywhere, in variables: and in commands: alike (§1.2) |
The job declares CI_SBOM_TOKEN and the attach still 401s |
That credential is retired: the runner host no longer injects it, so the self-referential declaration expands to nothing | Delete the declaration and authenticate the attach with ${VETRIX_JOB_TOKEN} (§1.2) |
| The per-job token appears unredacted in the job log | secret: true was dropped from the job on the grounds that its variables: block names no secret. The runner seeds VETRIX_JOB_TOKEN into that map, so the flag is still what arms redaction |
Keep secret: true on any job that carries the token (§1.2) |
| A declared secret's resolved value appears unredacted in the job log | A bare $NAME reference (no braces) anywhere in the job's variables: never resolves (varRe is brace-only, §1.2) — the redaction set is built from each variable's expanded value, so a bare-form entry arms the redactor with that harmless literal instead of the real secret. This is not a v2-only gap: secret: true (v1) and secret: true / secrets: (v2, via internal/cicd/parser/v2.go's JobV2.Secret field) both genuinely trigger whole-job redaction through the same shared JobConfig.Secret gate |
Use the brace form ${NAME} for every secret-carrying variable, under either schema (§1.2) |
| The scan attests the source tree instead of the image | A dir:. fallback was added to "make the scan pass" |
Never add one. An unpullable or unbuilt image must fail the job |