> ## Documentation Index > Fetch the complete documentation index at: https://docs.servers.dev/llms.txt > Use this file to discover all available pages before exploring further. # Image registry > How build images are stored and pulled, and how the registry authenticates. Build pods push one image per Ktor module (`/atoll//:`) and the nodes pull it by digest when an app pod starts. This page describes the registry used for self-hosting and how it is secured. ## Why zot | | zot | Harbor | | - | - | - | | Shape | One Go binary, one volume | \~8 services (core, jobservice, portal, registry, Trivy...) plus its own Postgres and Redis | | Footprint | \~100 MB RAM | 2-4 GB RAM | | Auth | htpasswd, LDAP, OIDC, API keys, **bearer tokens from an external token server**; per-repository policies | Users, projects, robot accounts with an API | | Extras | GC, retention, metrics, OCI artifacts, sync/mirroring | UI, replication, vulnerability scanning, quotas | The platform needs a private store for its own images, and the platform itself knows which build may push which repository. zot keeps that store small and lets the platform become the token issuer (phase 2 below), so per-project access needs no second user database. Harbor would work too (a robot account per project, created through its API), but it adds a large stateful system and couples the server to Harbor's API. ## Topology zot runs in the `registry` namespace behind Traefik on `registry.example.com` with a Let's Encrypt certificate. It keeps blobs in the SeaweedFS bucket `atoll-registry` (S3 user `registry`, limited to that bucket), so images share SeaweedFS' replication and the zot pod itself holds no state. Its NetworkPolicy only admits Traefik. Build pods push and nodes pull through that one name (`ATOLL_REGISTRY_PUSH` = `ATOLL_REGISTRY_PULL` = `registry.example.com`, `ATOLL_REGISTRY_INSECURE=false`), so there is no plain-HTTP path and no node-level mirror to configure. The `atoll-builds` NetworkPolicy allows build pods to reach the Traefik pods for that (`networkPolicy.builds.allowIngressController`). ## Phase 1: static users (this install) `scripts/create-secrets.sh` creates three zot users (`zot-htpasswd`, passwords in `registry/zot-users`): | User | Rights | Used by | Secret | | - | - | - | - | | `atoll-push` | read, create, update `atoll/**` | build pods | `atoll-builds/atoll-registry-push` (dockerconfigjson) | | `atoll-pull` | read `atoll/**`, `platform/**` | kubelet, for app pods and the builder image | `atoll-apps/atoll-registry-pull`, `atoll-builds/atoll-registry-pull` | | `admin` | everything | publishing the platform's own images to `platform/` | — | Anonymous access is off and nobody can delete except `admin`. ### How the server uses them The chart exports the Secret names as `ATOLL_REGISTRY_PUSH_SECRET`, `ATOLL_REGISTRY_PULL_SECRET` and `ATOLL_BUILDER_PULL_SECRET` (all optional; `ATOLL_REGISTRY_INSECURE` defaults to `false`). * **Build pods** run the user's Gradle build in the init container `build`, which has no registry credentials, no `DOCKER_CONFIG` and no service account token. It leaves one image layer per Ktor module in an emptyDir. The container `publish` then pushes them: only it mounts the push Secret (key `.dockerconfigjson` as `config.json` at `/var/run/atoll/docker`, `DOCKER_CONFIG` pointing there). It runs as its own uid with a read-only root filesystem, without the project's build env vars, refuses links and special files in the build's outputs, checks every name it reads from them and only hands the layer files to crane as data. The build's processes are gone before it starts. The builder pull Secret goes into the pod's `imagePullSecrets`. A build whose push Secret is missing, not of type `kubernetes.io/dockerconfigjson` or without the `.dockerconfigjson` key fails before its Job is created. The credentials never appear in the Job spec or the build log. * **App pods** get the pull Secret as `imagePullSecrets`. Its `auths` key must be the host in `ATOLL_REGISTRY_PULL` (the bundled registry writes both the push and the pull host). * **Digests**: `crane append` prints the digest of the image it pushed, `crane mutate` continues from that digest (not the tag) and prints the final digest, which `publish` reports as `repo:@sha256:...`. The server accepts only digest-pinned images under `atoll//` with the build's deployment id as tag, and the app Deployment references that digest. Promotions roll out the stored reference again, so they run exactly the image that was built. Deployments from before digest pinning must be promoted with `--rebuild`. * **Version check**: the server refuses a builder image older than itself ("The builder image is older than this server; set ATOLL\_BUILDER\_IMAGE ..."). Upgrade the server and `builds.image` together. **Existing apps:** Deployments created before the pull Secret was set get it on their next rollout only. Until then their new pods pull without credentials and fail with `ImagePullBackOff` once anonymous pulls are off. Redeploy each Ktor app once, or give the nodes credentials (below) until you have. **Remaining risk:** every build shares `atoll-push`. The user's build can no longer read it, but anything that runs in the `publish` container (the builder image itself) could push to another project's repository. Deploying by digest makes such a push harmless for running apps and promotions: an overwritten tag is never pulled. Phase 2 removes the shared credential. Nodes can also pull with node-level credentials instead of the pull Secret: `deploy/selfhost/k3s/registries.yaml.example`. ## Phase 2: per-build tokens issued by the platform zot supports Docker registry token authentication: on a 401 the client fetches a short-lived JWT from a "realm" URL and zot checks its signature and the `access` claims for the requested repository. ``` crane push ──► zot: 401, WWW-Authenticate: Bearer realm="https://atoll.example.com/registry/token",service="atoll-registry" crane ──► GET /registry/token?scope=repository:atoll/hello/server:push,pull (Basic build:) Server: build is running, project is "hello", scope is under atoll/hello/ ──► JWT (5 min, push+pull on that repo) crane push ──► zot (Bearer JWT) ──► accepted kubelet ──► same realm with Basic pull: ──► JWT with pull on atoll/* ``` zot config: ```json theme={null} "auth": { "bearer": { "realm": "https://atoll.example.com/registry/token", "service": "atoll-registry", "cert": "/etc/zot/token.crt" } } ``` Server changes for phase 2: | # | Change | | - | - | | 1 | Token endpoint (public route, e.g. `GET /registry/token`) implementing the Docker token spec: Basic auth with either a running build's token (scope limited to `repository:atoll//*` with `push,pull`) or the pull password (`pull` on `atoll/*`); signs ES256/RS256 JWTs (`iss`, `sub`, `aud`=service, `exp` ≤ 5 min, `access`) | | 2 | Settings: `ATOLL_REGISTRY_TOKEN_KEY` (PEM private key, Secret), `ATOLL_REGISTRY_TOKEN_SERVICE` (`atoll-registry`), `ATOLL_REGISTRY_PULL_PASSWORD` (Secret) | | 3 | Build Job: drop the shared push secret; `build.sh` passes `build:` to crane (`crane auth login` into a private `DOCKER_CONFIG` before running Gradle, removed afterwards) | | 4 | The pull secret holds `pull:`; nothing changes for nodes | The per-build token is already random and only valid while that build runs, so a malicious build can only push to its own project. On the deploy side `create-secrets.sh` and the zot values change; the chart already has everything else. ## Bundled registry (evaluation) `registry.bundled.enabled` runs a single zot pod inside the chart, over plain HTTP. It always requires a login: without `registry.bundled.htpasswdSecret` the chart generates `atoll-push` and `atoll-pull` once (Secret `atoll-registry-auth`, kept on uninstall) and writes `atoll-registry-push` (builds namespace) and `atoll-registry-pull` (builds and apps namespaces), which it also exports as `ATOLL_REGISTRY_PUSH_SECRET` and `_PULL_SECRET`. Both users only reach `atoll/**`. The Service is ClusterIP; only build pods may connect. For node pulls on kind/k3d see [install.md](https://docs.servers.dev/self-hosting/install.md#without-the-helmfile). This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.