Image Rebuild Pipelines

Every container image we own (pdns-recursor, pdns-authoritative, netbox, vault-autoinit, paperless-backup, homelabctl2) is rebuilt by the same three-phase pipeline pattern. Shared job templates live under .gitlab/pipelines/_shared/; per-trigger consumers under .gitlab/pipelines/schedule/ and .gitlab/pipelines/default/ extend those templates and add the right rules: / needs: wiring.

The three phases

1. build (kaniko)

_shared/<image>-image-rebuild.yml defines a hidden .build-<image>-image template. Trigger-specific consumers (schedule/<image>-image-rebuild.yml, default/<image>-image-rebuild.yml) extends: it and add rules:.

The build:

• reads the version (either from a VERSION file in the build context or from /etc/<image>-version inside the just-built image, depending on the image)
• runs kaniko to push to a per-pipeline staging tag ${IMAGE_NAME}:staging-${CI_PIPELINE_ID}[-<sub>]
• emits the version as a dotenv (<IMAGE>_VERSION=...) so the tag job can read it

2. tag (crane)

scripts/ci/rebuild/tag-image.sh is the workhorse — every .tag-* template delegates to it after exporting IMAGE_NAME, STAGING, TAG_VAR, and either VERSION or VERSION_FROM_ROOTFS. The script:

1. Resolves the upstream version (env or from a rootfs path via crane export).
2. Lists existing tags on the image with crane ls.
3. Picks the final tag:
   • <version> if absent from the registry, or
   • <version>-N bumping the highest existing counter (for repeat rebuilds at the same upstream version, e.g. weekly OS-patch refreshes).
4. Retags staging → <final> and latest, deletes the staging tag.
5. Writes TAG_<NAME>=<final> to tag.env (consumed as a dotenv artifact by the next job).

3. bump-manifest

scripts/ci/rebuild/bump-manifest.sh opens a bot MR pinning the deployment manifest(s) to the new tag. It takes positional pairs of <file> <sed-script>, so a single bump job can patch multiple files (e.g. both pdns deployments in one MR).

The bot:

• creates a branch bot/rebuild-<image>-images-<ts>
• runs sed against each <file> to swap the image: registry…:<old> → image: registry…:<new> line
• commits with chore(<component>): rebuild image (<tag>)
• pushes with GitLab push options to open an MR targeting main, with merge_request.remove_source_branch set
• uses $PAT_GITLAB_TOKEN (project CI variable, masked) as the push credential

The MR sits awaiting review/approval — Flux only picks up the new tag once the MR is merged.

Versioning conventions

Image	Version source	When -N bumps	When the base version bumps
pdns-recursor, pdns-authoritative	/etc/pdns-version inside the staged image	Same upstream pdns version, repeat OS-patch rebuilds	Upstream pdns release picked up by base image
netbox	FROM netboxcommunity/netbox:vX.Y.Z in images/netbox/Dockerfile	pip plugin updates, config-only changes at same upstream	You bump the FROM tag
vault-autoinit	code/vault-autoinit/VERSION (SemVer)	Scheduled OS-patch / bao rebuilds at same SemVer	You hand-bump VERSION when autoinit.sh changes in a way consumers must react to
paperless-backup	images/container/paperless-k8s-backup/VERSION (SemVer)	Scheduled debian / kubectl rebuilds at same SemVer	You hand-bump VERSION when paperless_backup_script.bash changes breakingly
homelabctl2	code/homelabctl2/VERSION (SemVer)	Scheduled OS-patch / Go toolchain rebuilds at same SemVer; default-branch rebuilds on any change in code/homelabctl2/**	You hand-bump VERSION when the CLI surface changes in a way the in-cluster Job must react to

When editing autoinit.sh, paperless_backup_script.bash, or making a CLI-breaking change to homelabctl2, bump the matching VERSION file in the same MR so the next rebuild ships under the new SemVer (no -N), making the change obvious in the bump-MR diff.

Adding a new image rebuild

1. Add a _shared/<image>-image-rebuild.yml with three templates: .build-<image>-image, .tag-<image>-image, .bump-<image>-manifest. Model it on _shared/homelabctl2-image-rebuild.yml (VERSION-file based) or _shared/pdns-image-rebuild.yml (rootfs-based version).
2. Add a schedule/<image>-image-rebuild.yml consumer gated by $RUN_REBUILD_<IMAGE> == "true".
3. Add a default/<image>-image-rebuild.yml consumer gated by changes: on the relevant source paths.
4. Wire schedule/main.yml (and optionally default/main.yml) to include: the new files.
5. Add a row to the declarative schedule list in code/homelabctl2/cmd/homelabctl2/gitlab/schedules/schedules.go, then run homelabctl2 gitlab schedules sync --apply to create the GitLab schedule.

The PAT_GITLAB_TOKEN CI variable is already in place for the bump-MR push — no per-image setup needed there.