How to Deploy a Stateful Application on Kubernetes

A stateful application can appear to work correctly until its Pod is recreated. If the application stores data on the Pod's ephemeral filesystem, that data can be lost when Kubernetes replaces the Pod. This can leave you with a running application that has unexpectedly lost its data. The challenge is giving the application storage that persists independently of the Pod while Kubernetes continues to manage the workload.

In this article, you'll deploy a stateful application on Kubernetes using a StatefulSet and persistent storage, configure a PersistentVolumeClaim (PVC) through volumeClaimTemplates, and verify that the application's data remains available after its Pod is recreated.

Why storing data in the Pod fails

A stateful application can lose its data even when the Kubernetes workload is running normally. If the application writes data to a Pod's ephemeral filesystem, that data is tied to the Pod rather than stored on persistent storage.

For example, this Deployment uses an emptyDir volume mounted at /data:

apiVersion: apps/v1
kind: Deployment
metadata:
name: stateful-app
spec:
replicas: 1
selector:
    matchLabels:
    app: stateful-app
template:
    metadata:
    labels:
        app: stateful-app
    spec:
    containers:
        - name: app
        image: nginx:1.27
        volumeMounts:
            - name: data
            mountPath: /data
    volumes:
        - name: data
        emptyDir: {}

The emptyDir volume exists for the lifetime of the Pod. If Kubernetes deletes the Pod and creates a replacement, the replacement gets a new emptyDir volume.

The application can return to a Running state, but the data written to /data is gone. This is the failure mode we need to avoid when deploying a stateful application.

Persistent storage separates the application's data from the Pod lifecycle. Kubernetes provides PersistentVolumes and PersistentVolumeClaims for this purpose, while a StatefulSet can create a dedicated PVC for each Pod through volumeClaimTemplates.

How StatefulSet preserves application data

A StatefulSet gives each Pod a stable identity and can associate that Pod with dedicated persistent storage. Unlike a Deployment, StatefulSet Pods are not treated as interchangeable replicas. Kubernetes keeps track of each Pod's identity and its associated storage. Kubernetes StatefulSets documentation

Open larger view

For example, a StatefulSet with three replicas creates Pods with predictable names such as app-0, app-1, and app-2. With volumeClaimTemplates, each Pod can also receive its own PersistentVolumeClaim instead of sharing a single volume.

If app-0 is deleted, Kubernetes recreates app-0 with the same identity. The replacement Pod can mount the existing PersistentVolumeClaim, allowing the application to access the data stored by the previous Pod.

This is the key difference for stateful workloads. The StatefulSet manages the Pod's identity and lifecycle, while the PersistentVolume and PersistentVolumeClaim provide storage that can outlive the individual Pod. Kubernetes Persistent Volumes documentation

Create the application configuration

Start with a headless Service so the StatefulSet Pods can receive stable network identities. The Service does not expose the application outside the cluster; it provides the DNS records StatefulSet uses to address individual Pods.

Create a file named stateful-app.yaml with the following configuration:

    apiVersion: v1
    kind: Service
    metadata:
      name: stateful-app
    spec:
      clusterIP: None
      selector:
        app: stateful-app
    ports:
      - port: 80
        targetPort: 80

The clusterIP: None setting makes this a headless Service. Kubernetes can then create DNS records for the individual StatefulSet Pods instead of assigning the Service a single cluster IP.

Next, add the StatefulSet configuration to the same file. This example uses nginx:1.27 and creates a dedicated 1 GiB PersistentVolumeClaim for each Pod:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: stateful-app
spec:
  serviceName: stateful-app
  replicas: 1
  selector:
    matchLabels:
      app: stateful-app
  template:
    metadata:
      labels:
        app: stateful-app
    spec:
      containers:
        - name: app
          image: nginx:1.27
          ports:
            - containerPort: 80
          volumeMounts:
            - name: data
              mountPath: /data
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi

The volumeClaimTemplates section creates a separate PersistentVolumeClaim for each StatefulSet Pod. Kubernetes names each claim using the template name and Pod identity, such as data-stateful-app-0. The claim remains available when the Pod is recreated, allowing the replacement Pod to use the same persistent storage.

How to deploy a StatefulSet

A StatefulSet needs a stable network identity and persistent storage for each Pod. The following manifest creates a headless Service and a StatefulSet with two Pods, each with its own PersistentVolumeClaim.

Create a file named statefulset.yaml:

apiVersion: v1
kind: Service
metadata:
  name: stateful-app
spec:
  clusterIP: None
  selector:
    app: stateful-app
  ports:
    - port: 80
      targetPort: 80
 
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: stateful-app
spec:
  serviceName: stateful-app
  replicas: 2
  selector:
    matchLabels:
      app: stateful-app
  template:
    metadata:
      labels:
        app: stateful-app
    spec:
      containers:
        - name: app
          image: nginx:1.27
          ports:
            - containerPort: 80
              name: http
          volumeMounts:
            - name: app-data
              mountPath: /usr/share/nginx/html
 
  volumeClaimTemplates:
    - metadata:
        name: app-data
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi

The serviceName connects the StatefulSet to the headless Service, while volumeClaimTemplates tells Kubernetes to create a separate PVC for each Pod.

Apply the manifest:

kubectl apply -f statefulset.yaml

Check that the StatefulSet was created:

kubectl get statefulset

You should see the StatefulSet with two desired replicas:

NAME            READY   AGE
stateful-app    2/2     ...

Check the Pods:

kubectl get pods

Kubernetes should create predictable Pod names based on the StatefulSet name and ordinal:

NAME              READY   STATUS    RESTARTS   AGE
stateful-app-0    1/1     Running   0          ...
stateful-app-1    1/1     Running   0          ...

Finally, check the PersistentVolumeClaims:

kubectl get pvc

You should see a separate PVC for each Pod:

NAME                     STATUS   VOLUME   CAPACITY   ACCESS MODES
app-data-stateful-app-0  Bound    ...      1Gi        RWO
app-data-stateful-app-1  Bound    ...      1Gi        RWO

If both PVCs show Bound, Kubernetes has successfully provisioned persistent storage for the StatefulSet Pods.

Note: The PVCs depend on dynamic volume provisioning in your cluster. If no suitable StorageClass is available, the PVCs can remain in Pending instead of becoming Bound.

Apply best practices for stateful applications

A working StatefulSet is not enough for a production workload. You also need to configure storage, resource usage, health checks, and data recovery appropriately.

Apply these practices when running stateful applications on Kubernetes:

1. Choose storage based on the application's requirements. Use a StorageClass that provides the performance and availability your application needs. A StatefulSet does not make the underlying storage system resilient by itself.

2. Set CPU and memory requests and limits. Define resource requirements for each container so Kubernetes can schedule the workload reliably and prevent it from consuming resources needed by other applications.

3. Configure health checks carefully. Use readiness and liveness probes that match the application's startup and recovery behavior. Aggressive probes can restart an application before it has enough time to become ready.

4. Keep backups separate from persistent storage. A PVC preserves data when a Pod is recreated, but it is not a backup. Use a separate backup strategy and regularly test that the data can be restored.

5. Test recovery before production use. Delete a Pod and verify that it reconnects to its existing PVC. Also test application recovery and backup restoration so you know how the workload behaves when something fails.

These practices complement the StatefulSet rather than replacing it. The StatefulSet provides stable Pod identities and persistent storage, while storage configuration, backups, health checks, and recovery procedures determine how reliably the application operates in production.

How to verify the StatefulSet

Once the StatefulSet is running, verify that Kubernetes maintains the Pod's identity and persistent storage. Test this by writing data to one Pod, deleting it, and confirming that the recreated Pod can still access the data.

First, confirm that both Pods are running:

kubectl get pods

You should see both Pods in the Running state:

NAME              READY   STATUS    RESTARTS   AGE
stateful-app-0    1/1     Running   0          ...
stateful-app-1    1/1     Running   0          ...

Next, verify that Kubernetes created a separate PersistentVolumeClaim (PVC) for each Pod:

kubectl get pvc

Both PVCs should show a Bound status:

NAME                     STATUS   VOLUME   CAPACITY   ACCESS MODES
app-data-stateful-app-0  Bound    ...      1Gi        RWO
app-data-stateful-app-1  Bound    ...      1Gi        RWO

Now write test data to the persistent volume mounted by stateful-app-0:

kubectl exec stateful-app-0 -- sh -c 'echo "StatefulSet data" > /usr/share/nginx/html/data.txt'

Verify that the file was created:

kubectl exec stateful-app-0 -- cat /usr/share/nginx/html/data.txt

The command should return:

StatefulSet data

Now delete stateful-app-0:

kubectl delete pod stateful-app-0

The StatefulSet controller should recreate the Pod with the same ordinal name. Check its status:

kubectl get pods

You should see stateful-app-0 running again:

NAME              READY   STATUS    RESTARTS   AGE
stateful-app-0    1/1     Running   0          ...
stateful-app-1    1/1     Running   0          ...

Finally, verify that the test data is still available:

kubectl exec stateful-app-0 -- cat /usr/share/nginx/html/data.txt

If the recreated Pod does not reach the Running state, check its events and container logs to identify the failure. For a deeper walkthrough of diagnosing this condition, see How to Fix the "CrashLoopBackOff" Error in Kubernetes.

The command should return:

StatefulSet data

The data remains available because the recreated Pod reconnects to the existing PVC instead of receiving a new empty volume. This confirms that the StatefulSet maintains the Pod's stable identity and persistent storage across Pod recreation.

When this approach breaks down

A StatefulSet is useful when an application needs stable Pod identities or persistent storage, but it is not the right workload controller for every application.

• The application is stateless. If Pods can be replaced without preserving their identity or data, a Deployment is simpler and avoids unnecessary StatefulSet configuration.
• The storage layer is not highly available. A StatefulSet manages the relationship between Pods and their PVCs, but it does not make the underlying storage system resilient. Storage availability still depends on the cluster's StorageClass and storage infrastructure.
• The application needs data replication or backups. A StatefulSet provides stable identities and persistent volumes, but it does not automatically replicate application data or create backups. Databases and distributed systems still need their own replication, backup, and recovery mechanisms.

For simple stateful workloads, a StatefulSet with persistent storage may be enough. For production databases and distributed systems, you still need separate strategies for replication, backups, and recovery.