Air-Gapped Deployment

This guide explains how to deploy applications in an air-gapped (offline) Kubernetes environment using Capsailer.

Overview

Deploying in an air-gapped environment involves four steps:

Build a bundle on a connected machine (Building Bundles).
Transfer the bundle and the capsailer binary to the air-gapped side.
Make sure there is a registry the cluster can pull from.
Push the images, then install the charts.

Choose Your Registry

	Registry you already have	Registry Capsailer deploys
Typical for	Harbor, Artifactory, Nexus, a shared `registry:2`	A cluster with nothing yet
Set up	Nothing	`capsailer registry`
Push images	`push --external-registry <host>`	`push --namespace <ns>`
Charts	Install from the extracted `.tgz`	Published to ChartMuseum, installed with `helm repo add`
Needs	`capsailer` only	`capsailer`, `kubectl`, a default storage class

If you already have a registry, use it. It is the simplest setup and has the fewest moving parts.

Transferring the Bundle

After building your bundle in a connected environment, you need to transfer it to the air-gapped environment:

# Copy the bundle and the capsailer binary
scp capsailer capsailer-bundle.tar.gz user@airgapped:~/

Setting Up a Registry

Option A: A registry you already have

Nothing to set up. Make sure the machine you run capsailer push on can reach the registry, and you have credentials if it needs them.

Option B: Let Capsailer deploy one

This registry has no TLS and no authentication, which is fine to get started but not for production. See Limitations.

./capsailer registry --namespace my-registry

This deploys a container registry and a ChartMuseum (for Helm charts) in the namespace. See the registry command for details, including how nodes pull from it.

Pushing Artifacts

To a registry you already have

# Check the names first
./capsailer push --bundle capsailer-bundle.tar.gz --external-registry registry.example.com:5000 --dry-run

# Push
./capsailer push --bundle capsailer-bundle.tar.gz --external-registry registry.example.com:5000 \
  --username myuser --password-stdin < password.txt

Charts are not uploaded to external registries. They stay in the bundle:

mkdir work && cd work
../capsailer unpack --file ../capsailer-bundle.tar.gz
helm install my-release charts/hello-world-0.1.0.tgz

To the registry Capsailer deployed

./capsailer push --bundle capsailer-bundle.tar.gz --namespace my-registry

Images go to the registry. Charts go to ChartMuseum.

Pushing a Vendor `docker save` Archive

Vendors often ship images as a docker save tarball whose image names have no registry (for example nutanix/nai-api:v2.8.1). Pass the tarball to push, and Capsailer rewrites every name to your registry:

# Check the mapping first
./capsailer push --bundle nai-images.tar --external-registry 10.55.1.16:5000 --dry-run

# Push all images in the archive
./capsailer push --bundle nai-images.tar --external-registry 10.55.1.16:5000 --username admin --password-stdin < password.txt

See the push command reference for the rewrite rules.

Deploying Applications

From the bundle (works with any registry):

helm install my-release charts/hello-world-0.1.0.tgz

From the built-in ChartMuseum:

kubectl port-forward -n my-registry svc/chartmuseum 8080:8080 &
helm repo add local-charts http://localhost:8080
helm repo update
helm install my-release local-charts/hello-world

If you built with --rewrite-image-references, the charts already use images from your registry.

Rewriting Image References

When deploying applications in air-gapped environments, you need to ensure that container image references in Helm charts point to your private registry. Capsailer provides a built-in feature to automatically rewrite these references during the build process:

# Build a bundle with image reference rewriting
capsailer build --manifest manifest.yaml --output bundle.tar.gz --rewrite-image-references --registry-url registry.local:5000

This will: - Download all images and charts specified in the manifest - Rewrite all image references in Helm charts to use your private registry - Package everything into a portable bundle

When you deploy these charts in your air-gapped environment, they use images from your private registry without manual changes, as long as those images are in the bundle and the image names are in the chart's top-level values.yaml. Images hard-coded in templates, in subchart defaults or in your own values files and custom resources are not rewritten. Run helm template and check every image: line. See Limitations. See Finding the images a chart needs.

The value for --registry-url must be an address your cluster nodes can pull from. If you deploy the registry with capsailer registry, see Pulling from the registry.

How image names are mapped

build and push use the same naming rule, so the names in the rewritten charts are the names push creates. The registry host is replaced, the rest of the path and the tag are kept, and Docker Hub official images have no library/ part:

Image in the manifest or chart	Name in your registry (`registry.local:5000`)
`bitnami/nginx:1.25`	`registry.local:5000/bitnami/nginx:1.25`
`nginx:1.25` or `library/nginx:1.25`	`registry.local:5000/nginx:1.25`
`quay.io/acme/proxy:2.0`	`registry.local:5000/acme/proxy:2.0`

The registry can include a path, for example harbor.test.com:5000/nai. Use the same value for --registry-url when building and for --external-registry (or --target-prefix) when pushing.

Two different images that would end up under the same name (for example quay.io/foo/bar:1 and docker.io/foo/bar:1) make push stop before uploading anything. Keep one of them out of the bundle, or push with --keep-source-registry. That option puts the source registry in the name, so charts rewritten by build will not match it.

Values that are not image references, such as Helm template expressions or chart repository URLs, are left unchanged.

Handling Kubernetes Operators

Kubernetes operators often require special handling in air-gapped environments because:

Operators typically reference container images in their Custom Resources (CRs)
These image references need to be rewritten to point to your private registry

Operator Image References in CRs

When deploying operators, you'll often need to specify container images in the Custom Resource (CR) definitions. For example, a database operator might require you to specify the database image in its CR:

apiVersion: database.example.com/v1
kind: Database
metadata:
  name: my-database
spec:
  # Image reference that needs to be rewritten
  image: docker.io/postgres:14.5
  replicas: 3

Strategies for Handling Operators

Include Operator Images in Your Manifest: Make sure to include all images required by the operator in your Capsailer manifest.
Manually Update CRs: After deploying the operator, update the image references in your CRs to point to your private registry:

spec:
  # Updated to use private registry
  image: registry.local:5000/postgres:14.5

Use Helm Values for Operators: If deploying operators via Helm, use values files to override image references:

# values.yaml
operator:
  image: registry.local:5000/operator:v1.0.0

# Images used by the operator's CRs
defaultImages:
  postgres: registry.local:5000/postgres:14.5
  redis: registry.local:5000/redis:7.0

Leverage Capsailer's Image Rewriting: If your operator is deployed via a Helm chart that includes CR templates, Capsailer's image reference rewriting feature will automatically update those references.

Example: PostgreSQL Operator

# In your manifest.yaml
images:
  - postgres:14.5
  - postgres-operator:v1.10.0
  # Any additional images the operator might need

charts:
  - name: postgres-operator
    repo: https://example.com/charts
    version: 1.10.0

After deploying with Capsailer:

# Deploy the operator
helm install postgres-operator local-charts/postgres-operator

# Create a database CR with the rewritten image reference
cat <<EOF | kubectl apply -f -
apiVersion: database.example.com/v1
kind: Database
metadata:
  name: my-database
spec:
  image: registry.local:5000/postgres:14.5
  replicas: 3
EOF

Complete Workflow

In the connected environment:

vi manifest.yaml
capsailer init --manifest manifest.yaml
capsailer build --manifest manifest.yaml --output capsailer-bundle.tar.gz \
  --rewrite-image-references --registry-url registry.example.com:5000

Transfer:

scp capsailer capsailer-bundle.tar.gz user@airgapped:~/

In the air-gapped environment:

./capsailer push --bundle capsailer-bundle.tar.gz --external-registry registry.example.com:5000 --dry-run
./capsailer push --bundle capsailer-bundle.tar.gz --external-registry registry.example.com:5000

mkdir work && cd work
../capsailer unpack --file ../capsailer-bundle.tar.gz
helm install my-release charts/hello-world-0.1.0.tgz

If something fails, see Troubleshooting.