Push auto-trigger operator runbook

Operator-facing reference and rollout runbook for the push auto-trigger — the mechanism that fires a CI/CD pipeline in-process when a branch is pushed to the bundled git transports (SSH / smart-HTTP). This document is operator-facing; it restates behavior and points at the source so an operator can verify each claim.

Section	Verb
§1 How the push trigger works	understand the mechanism + semantics
§2 What the trigger is not — no hook binary required	distinguish triggering from enforcement
§3 Loop avoidance: mirror / runner pushes	understand why replicated pushes don't retrigger
§4 Rollout: enable the gate on `develop` and `staging`	replayable rollout runbook
§5 Troubleshooting	diagnose a push that did / didn't trigger

Implementation-state note. engine.ScheduleForBranch (the gated scheduler this trigger calls) lives in internal/cicd/engine.go, and the in-process transport wiring invokes ScheduleForBranch(..., "push") on the success branch of receive-pack. Where an exact symbol or call site is named below, verify against the merged implementation before relying on the literal path. The behavior and semantics described here are stable; the exact wiring location may shift.

1. How the push trigger works

1.1 Mechanism

A successful push to a branch over the bundled git transports (SSH on :2222, smart-HTTP on the HTTP port) fires a pipeline schedule in-process — there is no spool directory, no external dispatcher process, and no extra hop. The transport layer, on the success branch of a receive-pack run, calls the engine directly:

engine.ScheduleForBranch(ctx, cfg, repoID, commitSHA, ref, branch, "push")

This is the same in-process post-receive pattern the code-search reindex already uses: both the HTTP handler (internal/git/protocol_http.go) and the SSH handler (internal/git/protocol_ssh.go) run their post-receive work only on a successful write (receive-pack exit / CGI status < 400). The push trigger hangs off that same success gate. See the postReceiveIndexer dispatch in those two files for the established pattern; the schedule call sits alongside it.

1.2 `trigger="push"` semantics

The pipeline row records its origin in the trigger column. An auto-triggered pipeline carries trigger="push". Contrast the manual REST trigger (POST /api/v1/repos/{owner}/{repo}/pipelines, internal/api/pipelines.go Trigger), which records trigger="manual". Both paths funnel through engine.Schedule → engine.ScheduleForBranch, so they get identical per-job branch gating (§1.3); the only difference is the recorded trigger value and how the ref/SHA are supplied (transport-supplied vs. request body).

What to observe: a pipeline created by a push has trigger: "push" in its API representation; a pipeline created by the REST endpoint has trigger: "manual".

1.3 Config discovery is per-pushed-ref

The trigger schedules from the vetrix-ci.yml (or .ci/pipeline.yml / .vetrix/pipeline.yml) found in the tree of the pushed ref, not from any single global location. cicd.DiscoverConfig reads the pipeline file out of the pushed ref's tree, walking the documented search path (.ci/pipeline.yml → .vetrix/pipeline.yml → vetrix-ci.yml; see internal/cicd/discovery.go and the discovery note in pipelines.go).

The operational consequence — which drives the entire rollout runbook in §4 — is that the config the trigger sees on a push to develop is whatever lives on develop, and the config it sees on a push to staging is whatever lives on staging. These are independent trees. A vetrix-ci.yml edit on one branch has no effect on the other until the file is propagated to that branch.

1.4 Per-job branch gating

Before any pipeline row is created, ScheduleForBranch filters cfg.Jobs through parser.JobRunsOnBranch(job, branch) using the short branch name (e.g. develop, not refs/heads/develop). The filter rules (internal/cicd/parser/v1.go):

If a job's only: is non-empty and the branch matches none of its patterns → the job is dropped.
If the branch matches any pattern in the job's except: → the job is dropped.
Otherwise the job runs.

Pattern matching is simple glob: an exact match, *, a prefix*, or a *suffix (* matches any run of non-slash characters).

If every job is gated out, ScheduleForBranch returns ErrNoRunnableJobs and creates no pipeline row — the push succeeds, but nothing is scheduled. This is the intended way to keep a branch "quiet": gate every job out for it.

What to observe: after an all-gated-out push, there is no new pipeline for that commit (the pipelines list for the repo shows no new row for the pushed SHA), and the push itself still reports success to the client.

2. What the trigger is not — no hook binary required

Auto-triggering does not require any external hook binary. This is the single most common point of operator confusion, so it is called out explicitly.

The push trigger runs in-process inside the server off the transport success path (§1.1). It does not depend on:

VETRIX_HOOK_BINARY
the cmd/vetrix-hook binary
the pre-receive / post-receive shell scripts in internal/git/hook_scripts.go

2.1 What the hook binary is for: pre-receive enforcement only

The VETRIX_HOOK_BINARY / cmd/vetrix-hook mechanism exists solely for pre-receive ENFORCEMENT — rejecting a push before it lands. That covers checks such as:

protected-branch rules,
force-push rejection,
push-time secret scanning.

It is an admission control path: it decides whether the push is allowed at all. It has nothing to do with scheduling pipelines.

The two are wired at different points in the push lifecycle:

	Pre-receive enforcement	Push auto-trigger
Purpose	accept / reject the push	schedule a pipeline after the push lands
Runs	before refs are updated	after a successful write
Mechanism	`pre-receive` script execs `$VETRIX_HOOK_BINARY`	in-process call to `engine.ScheduleForBranch`
Needs a binary?	yes (else the script logs `VETRIX_HOOK_BINARY not set; skipping pre-receive checks` and allows the push)	no

Operator takeaway. You do not need to set VETRIX_HOOK_BINARY to get auto-triggering. A deployment with VETRIX_HOOK_BINARY unset still auto-triggers pipelines on push; it just skips pre-receive enforcement (and logs the warning above on every push). Conversely, configuring a hook binary does not, by itself, schedule any pipeline — that is the in-process trigger's job.

3. Loop avoidance: mirror / runner pushes

A push that the platform itself generates — a mirror / replication push or a runner / worker push — must not retrigger a pipeline, or pushes would feed back on themselves. This is satisfied structurally, not by a runtime guard:

Mirror / replication pushes (internal/mirror/push_mirror.go) push to a remote URL with go-git directly. They never go through the bundled SSH / smart-HTTP receive-pack transports, so they never reach the transport success path the trigger hangs off. This is the same structural exclusion that keeps mirror pushes out of the git.push analytics event — see the comment block at the top of internal/git/analytics_push_event.go ("Mirror / replication pushes … bypass the HTTP / SSH transports entirely … so [the exclusion] is satisfied structurally, without an explicit guard").
The trigger only fires from the inbound transport receive-pack success path. Any platform-internal write that does not transit those transports cannot retrigger.

What to observe: trigger a mirror sync (or inspect the mirror sync log via the mirror admin surface) and confirm no new trigger: "push" pipeline appears on the mirror-source repo as a result of the replication push. The only trigger: "push" pipelines should be the ones from genuine inbound developer pushes.

4. Rollout: enable the gate on `develop` and `staging`

This is the replayable runbook for enabling the push auto-trigger on the develop and staging branches by adding the appropriate only: filter to the gate job in vetrix-ci.yml. Every step names what to observe.

Why two branches, two edits. Because DiscoverConfig reads vetrix-ci.yml from the pushed ref's tree (§1.3), the gate-job only: configuration must physically exist on both develop and staging. Editing it on develop alone leaves staging ungated. Step §4.4 propagates the file to staging as an explicit operator action — do not skip it.

The example below uses a single gate job named build-test that should run on develop and staging and nowhere else. Adapt the job name to your repo.

4.0 Pre-flight

Confirm the trigger path is live on this deployment before relying on it:

# A test push to a feature branch should create a trigger="push" pipeline
# (assuming the job is not gated out for that branch).
curl -fsSL "https://api.gitvetrix.com/api/v1/repos/{owner}/{repo}/pipelines?per_page=5" \
    -H "Authorization: Bearer $VETRIX_OPERATOR_TOKEN"

Observe: the most recent rows show trigger: "push" for recent inbound pushes. If every recent pipeline is trigger: "manual" and none are "push", the push-trigger wiring is not active on this deployment — stop and confirm the build state before continuing.

4.1 Edit the gate job on `develop`

On a working clone, on the develop branch, set the gate job's only: to the two target branches:

# vetrix-ci.yml (on develop)
jobs:
  build-test:
    stage: test
    image: golang:1.23
    only:
      - develop
      - staging
    script:
      - go test ./...

Commit and push the change to develop.

Observe: the push to develop itself creates a trigger: "push" pipeline (the job's only: now includes develop, so it is runnable):

curl -fsSL "https://api.gitvetrix.com/api/v1/repos/{owner}/{repo}/pipelines?per_page=3" \
    -H "Authorization: Bearer $VETRIX_OPERATOR_TOKEN"

The newest row should have trigger: "push", ref ending in develop, and commit_sha equal to the commit you just pushed.

4.2 Confirm a gated-out branch creates no pipeline

Push a no-op commit to a branch that is not in the only: list (e.g. a throwaway feature/gate-check branch):

Observe: the push succeeds but no new pipeline row appears for that commit (re-run the pipelines list; the gated branch's SHA is absent). This is the ErrNoRunnableJobs path (§1.4) — every job is gated out for that branch, so no pipeline is created. This is the positive confirmation that the only: filter is taking effect on develop's config.

4.3 Propagate the config to `staging` — REQUIRED

staging is still ungated until vetrix-ci.yml carries the same only: block on staging's tree. Bring the file across, e.g.:

git checkout staging
git checkout develop -- vetrix-ci.yml   # take develop's vetrix-ci.yml onto staging
git commit -m "Propagate gate-job only: filter to staging"
git push origin staging

(Use whatever propagation mechanism your branch policy allows — a merge or cherry-pick of the develop change is equally fine. The requirement is only that the resulting staging tree contains the same only: block.)

Observe: the push to staging creates a trigger: "push" pipeline (the job's only: includes staging, so it is runnable). Confirm via the pipelines list that the newest row has trigger: "push" and ref ending in staging.

4.4 Verify both branches are gated

Final confirmation. Push a trivial no-op commit to each branch in turn and confirm a pipeline is created for each:

# After a push to develop, then a push to staging:
curl -fsSL "https://api.gitvetrix.com/api/v1/repos/{owner}/{repo}/pipelines?per_page=10" \
    -H "Authorization: Bearer $VETRIX_OPERATOR_TOKEN"

Observe: there is one trigger: "push" pipeline for the develop commit and one for the staging commit, and pushes to any branch outside the only: list create no pipeline. If staging pushes still create no pipeline, §4.3 did not land — re-check that staging's vetrix-ci.yml actually contains the only: block (the config is per-ref; a stale staging tree is the most common failure).

4.5 Rollback of the rollout

To disable the gate on a branch, remove that branch from the job's only: list (or add it to except:) on that branch's tree and push. The change is per-ref, exactly like enabling it: removing staging from only: on develop's tree does nothing to staging — you must edit staging's tree.

Observe: after the rollback push, a subsequent no-op push to the de-gated branch creates no pipeline (ErrNoRunnableJobs).

5. Troubleshooting

A push did not create a pipeline.

Is the deployment running the push-trigger wiring? Re-run §4.0 — if no recent pipeline is trigger: "push", the trigger is not live.
Is the branch gated out? If every job's only: excludes the pushed branch (or except: includes it), ScheduleForBranch returns ErrNoRunnableJobs and creates nothing — this is expected (§1.4). Check the vetrix-ci.yml on the pushed ref's tree, not on another branch (§1.3).
Did the push actually succeed? The trigger only fires on a successful receive-pack (write, exit < 400). A push rejected by pre-receive enforcement (§2) never reaches the trigger.
Is a pipeline config present on the pushed ref? DiscoverConfig must find .ci/pipeline.yml, .vetrix/pipeline.yml, or vetrix-ci.yml in that ref's tree; otherwise there is nothing to schedule.

A mirror or runner push created an unexpected pipeline. This should not happen — those pushes bypass the transports (§3). If you see a trigger: "push" pipeline you can attribute to a replication or runner push, treat it as a defect and capture the source repo, ref, and SHA before filing.

VETRIX_HOOK_BINARY not set warning on every push. This is the pre-receive enforcement path (§2), not the trigger. It means enforcement is being skipped; auto-triggering is unaffected. Configure the hook binary only if you want enforcement.

6. Live devstack dry-run checklist

This section is the replayable operator checklist for the live HTTP+SSH push matrix against a deployed rc devstack. It exists because the in-process automated coverage cannot stand up a real worker, real Docker, or a real receive-pack transport in the unit-test sandbox. Run this against a built & deployed rc devstack with a running worker.

Prerequisites (from §4.0 and the §2 notes):

rc code built and deployed; one worker process running with RUNNER_ID set and /var/run/docker.sock mounted with the correct group_add GID (see the Docker deployment guide §7.1 in the system-docs repo).
A repo reachable over both transports — exercises ssh:// repo-arg owner resolution (the leading-slash fix in parseRepoArg, internal/git/protocol_ssh.go). The fix is not org-specific; it applies equally to user-owned repo SSH clones (orgs are just rows in the users table). Use the canonical vetrix/vetrix repo here as the convenient fixture, not a scope limit.
A vetrix-ci.yml on develop and staging shaped like §4.1: a gate job only:[develop,staging] and a release job only:[master].

Each row: perform the action, then assert the exact observable.

#	Action	Exact observable to assert
6.1	`git push https://<host>/vetrix/vetrix.git develop` (smart-HTTP)	A new pipeline appears with `trigger: "push"`, `ref: refs/heads/develop`, `commit_sha` == the pushed tip. The `gate` job is present and runs; the `release` job is absent.
6.2	`git push ssh://git@<host>:2222/vetrix/vetrix.git staging` (SSH)	Same as 6.1 with `ref: refs/heads/staging`. The `ssh://` push succeeds — it does not 404. This exercises `ssh://` repo-arg owner resolution: the leading-slash fix in `parseRepoArg`, `internal/git/protocol_ssh.go`. The fix is not org-specific: `ssh://` URLs send the repo path absolute, so `parseRepoArg` strips the leading slash before `SplitN` to avoid an empty owner segment. It applies equally to user-owned repo SSH clones (orgs are just rows in the users table). The `vetrix/vetrix` repo here is the convenient fixture, not a scope limit.
6.3	After 6.1/6.2 jobs finish, open the job detail / log view for a finished `gate` job	Finished-job logs are retrievable from the durable store, not only the live SSE stream. Logs persist after the job reaches a terminal state.
6.4	`git push <host>/vetrix/vetrix.git v9.9.9` (a tag)	The push succeeds. No new `trigger: "push"` pipeline is created (pipelines are branch-scoped).
6.5	`git push <host>/vetrix/vetrix.git :develop` (branch delete)	The delete succeeds. No new pipeline is created.
6.6	Push to a branch whose `vetrix-ci.yml` gates every job out (e.g. a feature branch with only `release:[master]`), or to a repo with no config	The push succeeds. No pipeline is created (`ErrNoRunnableJobs` / config-not-found benign skip).
6.7	Trigger a mirror sync (or runner push) on the repo (§3)	No new `trigger: "push"` pipeline appears as a result of the replication/runner push. Only genuine inbound developer pushes produce `trigger: "push"` rows.
6.8	For 6.1, open the audit log for the repo	A `pipeline.trigger` audit row exists with `source: "push"`, the correct `branch`/`ref`, and the actor set to the pushing user.

If any row's observable does not hold, capture the repo, ref, SHA, and the relevant worker / server log lines and file a bug before continuing.