How to Migrate a Docker Compose Application to Kubernetes

A Docker Compose application can start successfully in Kubernetes while its existing data remains inaccessible. Converting the Compose file doesn't migrate your database or persistent files, and incorrect volume mounts or service configuration can prevent the application from using restored data. You need to migrate these components and verify that the application still works before retiring the original deployment.

In this article, you'll migrate a Docker Compose application to Kubernetes, configure application services and persistent storage, restore the database and files, and verify that existing workspaces and pages are available after migration.

Prepare the Docker Compose Application

Before migrating to Kubernetes, inspect your existing Docker Compose application to identify its services and configuration.

Check the status of your running services:

docker compose ps

Review the Compose configuration with docker compose config to identify the container images, ports, environment variables, networks, and volumes used by each service:

docker compose config

For this migration, the self-hosted AFFiNE application uses PostgreSQL for its database and stores uploaded files and configuration in persistent directories. Back up the database and these directories before proceeding, as described in AFFiNE's backup and restore guide.

Create a directory for the backups:

mkdir -p /opt/affine-backup

Create a PostgreSQL database backup with pg_dump:

docker compose exec -T postgres \
  pg_dump -U affine -d affine \
  > /opt/affine-backup/affine.sql

Back up AFFiNE's persistent files and configuration:

tar -czf /opt/affine-backup/affine-storage-config.tar.gz \
  -C /opt/affine data/storage config

These commands assume that your Compose project is located at /opt/affine, the PostgreSQL service is named postgres, and the database and user are both named affine. Run the commands from the Compose project directory and adjust these values if your configuration differs.

Verify that both backup files exist:

ls -lh /opt/affine-backup/

Keep the original deployment and these backups until you have restored the data and verified that the migrated application works correctly.

Configure Kubernetes Storage and Networking

AFFiNE stores uploaded files and configuration in persistent directories. In Kubernetes, use a PersistentVolumeClaim (PVC) to provision storage for these files.

Check the available StorageClasses:

kubectl get storageclass

This example uses Civo's civo-volume StorageClass. If you're using another provider, replace it with a StorageClass available in your cluster.

Create a file named affine-storage-pvc.yaml:

nano affine-storage-pvc.yaml

Add the following configuration:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: affine-storage
  namespace: affine
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: civo-volume
  resources:
    requests:
      storage: 5Gi

Apply the manifest:

kubectl apply -f affine-storage-pvc.yaml

Verify that the PVC was created:

kubectl get pvc -n affine

The claim may initially show Pending if the StorageClass uses the WaitForFirstConsumer volume binding mode. Kubernetes provisions the volume when a Pod starts using the claim.

AFFiNE also needs network access to PostgreSQL and Redis. The Kubernetes Services for these dependencies provide stable internal endpoints:

Configure the AFFiNE Deployment to connect to these endpoints. Mount the affine-storage PVC at /root/.affine/storage and /root/.affine/config to match the paths used by the original Docker Compose application.

To expose AFFiNE outside the cluster, use a ClusterIP Service for the application and route external HTTP or HTTPS traffic through an Ingress Controller. You'll configure these resources when deploying the application. For a step-by-step setup, see how to expose an application with Kubernetes Ingress.

The storage claim defines where Kubernetes stores the files, but it doesn't copy your existing data into the volume. You'll restore the database and persistent files in the migration section.

Deploy the Application to Kubernetes

Deploy PostgreSQL and Redis before starting AFFiNE. Restore the existing database and persistent files before running the database migration Job.

Create a Kubernetes Secret for the PostgreSQL credentials. To decide which values belong in a Secret rather than a ConfigMap, see Kubernetes ConfigMap vs Secret explained.

kubectl create secret generic affine-postgres-secret \
  --namespace affine \
  --from-literal=POSTGRES_USER=affine \
  --from-literal=POSTGRES_PASSWORD='REPLACE_WITH_A_STRONG_PASSWORD' \
  --from-literal=POSTGRES_DB=affine

Replace the password placeholder with a strong password. Use the same credentials in AFFiNE's database connection URL.

Create a file named affine-postgres.yaml:

nano affine-postgres.yaml

Add the following configuration:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: affine-postgres-data
  namespace: affine
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: civo-volume
  resources:
    requests:
      storage: 10Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: affine-postgres
  namespace: affine
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: affine-postgres
  template:
    metadata:
      labels:
        app: affine-postgres
    spec:
      containers:
        - name: postgres
          image: pgvector/pgvector:pg16
          ports:
            - name: postgres
              containerPort: 5432
          env:
            - name: POSTGRES_USER
              valueFrom:
                secretKeyRef:
                  name: affine-postgres-secret
                  key: POSTGRES_USER
            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: affine-postgres-secret
                  key: POSTGRES_PASSWORD
            - name: POSTGRES_DB
              valueFrom:
                secretKeyRef:
                  name: affine-postgres-secret
                  key: POSTGRES_DB
            - name: PGDATA
              value: /var/lib/postgresql/data/pgdata
          volumeMounts:
            - name: postgres-data
              mountPath: /var/lib/postgresql/data
          readinessProbe:
            exec:
              command:
                - sh
                - -c
                - pg_isready -U "$POSTGRES_USER" -d "$POSTGRES_DB"
            initialDelaySeconds: 5
            periodSeconds: 5
          resources:
            requests:
              cpu: 250m
              memory: 512Mi
            limits:
              memory: 1Gi
      volumes:
        - name: postgres-data
          persistentVolumeClaim:
            claimName: affine-postgres-data
---
apiVersion: v1
kind: Service
metadata:
  name: postgres
  namespace: affine
spec:
  type: ClusterIP
  selector:
    app: affine-postgres
  ports:
    - name: postgres
      port: 5432
      targetPort: postgres

The PGDATA setting places PostgreSQL's data files in a subdirectory of the mounted volume. This avoids initialization errors when the volume contains a lost+found directory.

Apply the manifest and wait for PostgreSQL to become ready:

kubectl apply -f affine-postgres.yaml
 
kubectl rollout status deployment/affine-postgres \
  -n affine \
  --timeout=180s

Verify the resources:

kubectl get pods,services,pvc -n affine

Next, create a file named affine-redis.yaml:

nano affine-redis.yaml

Add the following configuration:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: affine-redis
  namespace: affine
spec:
  replicas: 1
  selector:
    matchLabels:
      app: affine-redis
  template:
    metadata:
      labels:
        app: affine-redis
    spec:
      containers:
        - name: redis
          image: redis:7
          ports:
            - name: redis
              containerPort: 6379
          readinessProbe:
            exec:
              command:
                - redis-cli
                - ping
            initialDelaySeconds: 5
            periodSeconds: 5
          resources:
            requests:
              cpu: 100m
              memory: 128Mi
            limits:
              memory: 256Mi
---
apiVersion: v1
kind: Service
metadata:
  name: redis
  namespace: affine
spec:
  type: ClusterIP
  selector:
    app: affine-redis
  ports:
    - name: redis
      port: 6379
      targetPort: redis

Apply the manifest and wait for Redis to become ready:

kubectl apply -f affine-redis.yaml
 
kubectl rollout status deployment/affine-redis \
  -n affine \
  --timeout=120s

Confirm that both Services are available:

kubectl get services -n affine

AFFiNE will connect to PostgreSQL at postgres:5432 and Redis at redis:6379.

At this point, restore the database dump and persistent files created in the preparation section. Follow the restore procedure in the next section before continuing. Do not run the migration Job against an empty database if you need to preserve existing data.

After restoring the database, create a Secret for AFFiNE's database connection:

kubectl create secret generic affine-app-secret \
  --namespace affine \
  --from-literal=DATABASE_URL='postgresql://affine:REPLACE_WITH_THE_SAME_PASSWORD@postgres:5432/affine'

Replace the password placeholder with the PostgreSQL password configured earlier. Percent-encode URL-reserved characters in the password before using it in the connection URL.

Create a file named affine-migration.yaml:

nano affine-migration.yaml

Add the migration Job manifest:

apiVersion: batch/v1
kind: Job
metadata:
  name: affine-migration
  namespace: affine
spec:
  backoffLimit: 1
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: migration
          image: ghcr.io/toeverything/affine:stable
          command:
            - node
            - ./scripts/self-host-predeploy.js
          env:
            - name: REDIS_SERVER_HOST
              value: redis
            - name: DATABASE_URL
              valueFrom:
                secretKeyRef:
                  name: affine-app-secret
                  key: DATABASE_URL
            - name: DEPLOYMENT_TYPE
              value: selfhosted

Apply the Job and wait for it to complete:

kubectl apply -f affine-migration.yaml
 
kubectl wait --for=condition=complete \
  job/affine-migration \
  -n affine \
  --timeout=300s
 
kubectl logs job/affine-migration -n affine

If the Job fails, inspect its logs and resolve the error before deploying AFFiNE.

Create a file named affine-app.yaml:

nano affine-app.yaml

Add the Deployment and Service configuration:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: affine
  namespace: affine
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: affine
  template:
    metadata:
      labels:
        app: affine
    spec:
      containers:
        - name: affine
          image: ghcr.io/toeverything/affine:stable
          ports:
            - name: http
              containerPort: 3010
          env:
            - name: REDIS_SERVER_HOST
              value: redis
            - name: DATABASE_URL
              valueFrom:
                secretKeyRef:
                  name: affine-app-secret
                  key: DATABASE_URL
            - name: DEPLOYMENT_TYPE
              value: selfhosted
          volumeMounts:
            - name: affine-storage
              mountPath: /root/.affine/storage
              subPath: data/storage
            - name: affine-storage
              mountPath: /root/.affine/config
              subPath: config
          resources:
            requests:
              cpu: 500m
              memory: 1Gi
            limits:
              memory: 2Gi
      volumes:
        - name: affine-storage
          persistentVolumeClaim:
            claimName: affine-storage
---
apiVersion: v1
kind: Service
metadata:
  name: affine
  namespace: affine
spec:
  type: ClusterIP
  selector:
    app: affine
  ports:
    - name: http
      port: 3010
      targetPort: http

Apply the manifest and wait for the Deployment to become ready:

kubectl apply -f affine-app.yaml
 
kubectl rollout status deployment/affine \
  -n affine \
  --timeout=300s

Verify the Pods and Services:

kubectl get pods,services -n affine

The AFFiNE, PostgreSQL, and Redis Pods should be running and ready. The affine Service exposes port 3010 within the cluster.

To test AFFiNE locally, forward the Service port:

kubectl port-forward -n affine service/affine 3010:3010

Keep the command running and open http://localhost:3010 in your browser.

Confirm that AFFiNE loads and that your existing workspaces, pages, and uploaded files are available. Keep the original Compose deployment and backups until you've verified the migrated application and its data.

Migrate the Database and Persistent Files

Restore the PostgreSQL database and persistent files from your backups. Run the following commands from a machine configured to access your Kubernetes cluster with kubectl.

Check that PostgreSQL is running and the storage claims are bound:

kubectl get pods,pvc -n affine

Copy the backup files from your source server to the machine running kubectl if they aren't already available there. The following commands assume the files are located at ~/affine.sql and ~/affine-storage-config.tar.gz.

Restore the database dump:

kubectl exec -i -n affine deployment/affine-postgres -- \
  psql -U affine -d affine \
  < ~/affine.sql

The target database must exist and be empty before you restore the dump. If it already contains data, inspect it before proceeding to avoid conflicting database objects.

Verify that the existing workspace records are present:

kubectl exec -n affine deployment/affine-postgres -- \
  psql -U affine -d affine -c \
  'SELECT id, name, created_at FROM workspaces ORDER BY created_at;'

The query should return the workspaces stored in your backup. If no rows are returned when you expect existing workspaces, investigate the restore before continuing.

Next, restore AFFiNE's persistent files and configuration. Create a temporary Pod that mounts the affine-storage PersistentVolumeClaim (PVC).

Create a file named affine-storage-restore.yaml:

nano affine-storage-restore.yaml

Add the following configuration:

apiVersion: v1
kind: Pod
metadata:
  name: affine-storage-restore
  namespace: affine
spec:
  restartPolicy: Never
  containers:
    - name: restore
      image: busybox:1.36
      command:
        - sh
        - -c
        - sleep 36000
      volumeMounts:
        - name: affine-storage
          mountPath: /restore
  volumes:
    - name: affine-storage
      persistentVolumeClaim:
        claimName: affine-storage

Create the Pod and wait for it to become ready:

kubectl apply -f affine-storage-restore.yaml
 
kubectl wait --for=condition=Ready \
  pod/affine-storage-restore \
  -n affine \
  --timeout=180s

Extract the backup archive into the mounted volume:

kubectl exec -i -n affine affine-storage-restore -- \
  tar -xzf - -C /restore \
  < ~/affine-storage-config.tar.gz

The archive contains the data/storage and config directories. Extracting it into /restore recreates the directory paths used by the AFFiNE Deployment.

Verify that the files were restored:

kubectl exec -n affine affine-storage-restore -- sh -c \
  'ls -lah /restore/config &&
   ls -lah /restore/data/storage &&
   test -s /restore/config/private.key &&
   echo "Private key exists"'

Confirm that the expected configuration files and storage directories are present. The private.key check applies to the example backup; if your deployment uses a different configuration, verify the files it requires.

After restoring the database and files, remove the temporary Pod:

kubectl delete pod affine-storage-restore -n affine

Run the pre-deployment migration Job from the previous section and wait for it to complete successfully before starting the main AFFiNE application.

Once AFFiNE starts, sign in and verify that your existing workspaces, pages, and uploaded files are available. Keep the original Docker Compose deployment and backups until you've confirmed that the migrated application works as expected.

Verify the Migrated Application

Verify that AFFiNE is running and that your existing data is accessible.

Check the status of the application and its dependencies:

kubectl get pods,services -n affine

Confirm that the AFFiNE, PostgreSQL, and Redis Pods are ready. If a Pod isn't ready, inspect its logs to identify the issue:

kubectl logs -n affine deployment/affine
kubectl logs -n affine deployment/affine-postgres
kubectl logs -n affine deployment/affine-redis

You can also inspect the application Deployment and its events:

kubectl describe deployment affine -n affine

To access AFFiNE, forward the application's Service port to your local machine:

kubectl port-forward -n affine service/affine 3010:3010

Open http://localhost:3010 in your browser and sign in to AFFiNE.

Verify that your existing workspaces and pages are available. Open a few pages and confirm that their content loads correctly. If your deployment contains uploaded files, check that you can open those files as well.

If existing content is missing, check that the database backup was restored successfully and that the affine-storage PersistentVolumeClaim (PVC) is mounted at the correct paths in the AFFiNE Deployment.

Once these checks pass, keep the original Docker Compose deployment and backups until you've confirmed that the migrated application works as expected.

Clean Up the Original Deployment

Remove the original Docker Compose deployment only after you've verified that AFFiNE and its existing data work correctly in Kubernetes.

Before cleaning up the source server, verify your database and file backups. Store them outside the original server and Kubernetes cluster.

Stop the original Compose services:

cd /opt/affine
docker compose down

The docker compose down command stops and removes the Compose containers and networks. It doesn't remove named volumes unless you include the --volumes option. Files stored in bind-mounted directories remain on the source server.

Check that the original services have stopped:

docker compose ps

Keep the database dump and persistent-file archive until you're confident that the Kubernetes deployment is stable and your recovery requirements are covered. Once you no longer need the original environment, remove its resources according to your data-retention requirements.

When This Breaks Down

If AFFiNE doesn't start or existing data is missing, check the application logs, database connection, and storage configuration.

Check the application logs for errors:

kubectl logs -n affine deployment/affine

If the Pod isn't ready, inspect its status and events:

kubectl get pods -n affine
kubectl describe deployment affine -n affine

If the Pod status shows CrashLoopBackOff or ImagePullBackOff, see how to fix CrashLoopBackOff in Kubernetes or how to fix ImagePullBackOff in Kubernetes.

If AFFiNE can't connect to PostgreSQL, check that the database Pod is ready and the postgres Service exists:

kubectl get pods,services -n affine
kubectl logs -n affine deployment/affine-postgres

Verify that the DATABASE_URL environment variable contains the correct database name, username, password, and Service hostname. If you use a Kubernetes Secret for database credentials, confirm that the Deployment references the correct Secret.

If existing workspaces or uploaded files are missing, verify that the database backup was restored successfully and that the affine-storage PersistentVolumeClaim (PVC) is mounted at the expected paths.

Check the storage claim and list the contents of the mounted directory:

kubectl get pvc -n affine
 
kubectl exec -n affine deployment/affine -- \
  ls -lah /root/.affine/storage

Confirm that the expected files are present. If the PVC is still Pending, inspect its status and events to identify storage provisioning or scheduling issues:

kubectl describe pvc affine-storage -n affine

Fix the underlying issue before changing other resources. After applying a correction, check the Pod status and logs again, then verify that AFFiNE can access your existing workspaces, pages, and uploaded files.

Keep the original Docker Compose deployment and backups until the Kubernetes deployment works correctly and you have a reliable recovery path.