Vetrix Docs

Code-search incremental indexing (post-receive)

Operator reference for the post-receive code-search reindex hook.

What this is

After a git push lands on a tracked branch, Vetrix automatically walks the repository's tree on the pushed branch and updates code_search_index (and, via the outbox scheduler, the OpenSearch vetrix-code and vetrix-symbols indexes). This is the steady-state counterpart to the one-shot POST /api/v1/admin/search/code/backfill endpoint used for empty-deployment seeding.

Both the HTTP smart-protocol path (POST /<owner>/<repo>.git/git- receive-pack) and the SSH transport (git push over port 2222) carry the hook — there is no operator action required to enable it.

Latency contract

Step Bound Notes
Push response returned to client within normal git-protocol budget The reindex runs on a background goroutine; the push response does not block on the tree walk.
code_search_index rows visible < 5 s typical, dominated by tree-walk cost Idempotent; re-runs of the same (repo_id, file_path, ref) tuple replace existing rows via ON CONFLICT DO UPDATE.
search_index_outbox row enqueued Same transaction as the code_search_index upsert Outbox enqueue is in the same tx as the index write.
Searchable via GET /api/v1/search/code?q=... < 30 s typical end-to-end Scheduler dispatch cadence + OpenSearch refresh interval. Tune by widening the scheduler tick if your cluster has lots of background pressure.

The 30 s envelope is the end-to-end SLO an operator can quote to a developer asking "I just pushed — why isn't my new file searchable yet?" Anything longer than ~60 s suggests:

  1. The outbox scheduler is wedged (check scheduler.go runtime metrics or its log lines).
  2. OpenSearch is slow / unhealthy (check /api/v1/admin/search/health if available; otherwise watch the adapter's failure counter).
  3. The repository tree is unusually large (10k+ files). Repeated pushes against a megarepo will queue several reindexes in a row; that is normal but visible as latency on the second push if it lands while the first is still walking.

Skip cases

The hook deliberately does not invoke IndexRepository for:

  • Delete pushes (git push origin :feature) — there is no ref to walk. code_search_index rows for the deleted branch are left in place; they fall out of search results when the next reindex of the same (repo_id, ref) runs against a non-existent branch, or when the OpenSearch document TTL expires. There is no dedicated GC of stale branch refs.
  • Tag pushes (refs/tags/*) — the code-search index is branch-scoped. Tag content overlaps the branch the tag was created from, so indexing tags would double-write every blob without adding any new searchable content.
  • Non-branch refs (refs/notes/*, refs/keep-around/*, etc.) — the indexer is keyed to (repo_id, ref) where ref is a branch name. Indexing non-branch refs would pollute the keyspace.

A delete-push of a branch followed by a normal create on the same branch name will re-populate the index on the create push.

Failure mode

The dispatcher swallows IndexRepository errors and logs a WARN slog event with protocol, repo_id, branch, and err fields:

WARN post-receive: code-search reindex failed protocol=http
  repo_id=... branch=main err=...

The push itself succeeds — the receive-pack response is already on the wire by the time the reindex goroutine runs. This is intentional: a failing reindex must not surface to the developer as a failed push. The next push to the same branch will re-trigger the walk and typically succeeds (the failure is usually a transient lock on a pack file or an OpenSearch hiccup).

If a specific repo's reindex fails persistently, run the admin backfill endpoint to force a fresh walk:

curl -X POST https://api.gitvetrix.com/api/v1/admin/search/code/backfill \
  -H "Authorization: Bearer ${ADMIN_TOKEN}"

(This walks every repo in the deployment, not just the failing one; there is no per-repo recovery endpoint today.)

Diagnosis: a push lands but search doesn't update

  1. Confirm the slog WARN line for the affected repo is absent — if it's present, the reindex hit an error; jump to the failure mode section.
  2. Confirm the push was a branch push, not a tag or delete push (see skip cases above).
  3. Check code_search_index for rows with the expected (repo_id, ref) tuple. If the rows are present, the Postgres side worked and the failure is downstream:
    • Check search_index_outbox for un-claimed rows tagged code / code_search_index.
    • Check the outbox scheduler is running (no log error / stall).
    • Check the CodeAdapter's failure counter.
  4. If code_search_index rows are absent, the walk skipped them. Common causes:
    • The branch is empty (no blobs).
    • Every file in the branch is binary (Manager.GetBlob returned isBinary=true for every entry).
    • Every file exceeds maxIndexFileSize (512 KB).
  5. If none of the above applies, fall back to running the admin backfill endpoint — that re-walks the default branch and re-establishes a known-good baseline.

Architecture

  • The RepoIndexer interface is declared in internal/git/post_ receive_indexer.go. *search.Indexer satisfies it verbatim.
  • cmd/server/main.go constructs a single *search.Indexer near the *api.Services literal and passes it to both transports — sshServer.SetRepoIndexer(...) directly and the HTTP path via Services.SearchIndexer → gitHTTP.SetRepoIndexer(...) in internal/api/router.go.
  • The dispatcher (postReceiveIndexer in internal/git/post_ receive_indexer.go) runs IndexRepository on a background goroutine with a detached context (context.WithoutCancel) so the reindex survives the push response.
  • IndexRepository (internal/search/code.go) walks the tree and calls Store.UpsertFile per blob. UpsertFile writes both the code_search_index row and the search_index_outbox row in a single transaction, preserving the outbox-in-tx invariant the scheduler relies on.