Deploying a MySQL Master‑Slave Cluster on Kubernetes Using StatefulSet and Local Persistent Volumes

In Kubernetes, ReplicaSet can create multiple identical Pods, but they are stateless; StatefulSet is used to manage stateful Pods that require stable network identities and persistent storage, ensuring that a replacement Pod retains the same name, IP, and data.

The goal of this tutorial is to set up a MySQL master‑slave cluster using a StatefulSet, employing local storage for simplicity in a test environment while noting that production should use dynamic storage solutions such as GCE, NFS, or Ceph.

Required components include a Kubernetes master, a node where all test Pods run, and DNS services enabled.

Experiment objectives are to create a MySQL master‑slave cluster, allow horizontal scaling of slave nodes, restrict write operations to the master, permit reads from both master and slaves, and ensure slaves synchronize data from the master.

Local storage is achieved by defining a StorageClass with provisioner: kubernetes.io/no-provisioner and volumeBindingMode: WaitForFirstConsumer , which enables delayed binding of PersistentVolumes to Pods.

StorageClass definition:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

Three PersistentVolumes are pre‑created on the node (static provisioning) to hold MySQL data directories:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: example-mysql-pv
spec:
  capacity:
    storage: 15Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  storageClassName: local-storage
  local:
    path: /data/svr/projects/mysql
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - 172.31.170.51

Similarly, example-mysql-pv-2 and example-mysql-pv-3 are defined for additional replicas.

Apply the PVs:

kubectl apply -f 01-persistentVolume-{1..3}.yaml

Create the StorageClass:

kubectl apply -f 02-storageclass.yaml

Create a dedicated namespace for MySQL:

apiVersion: v1
kind: Namespace
metadata:
  name: mysql
  labels:
    app: mysql

Apply the namespace:

kubectl apply -f 03-mysql-namespace.yaml

Define a ConfigMap containing separate MySQL configuration files for master and slave:

apiVersion: v1
kind: ConfigMap
metadata:
  name: mysql
  namespace: mysql
  labels:
    app: mysql
data:
  master.cnf: |
    # Master configuration
    [mysqld]
    log-bin=mysqllog
    skip-name-resolve
  slave.cnf: |
    # Slave configuration
    [mysqld]
    super-read-only
    skip-name-resolve
    log-bin=mysql-bin
    replicate-ignore-db=mysql

Create the ConfigMap:

kubectl apply -f 04-mysql-configmap.yaml

Create a Secret to store the MySQL root password (base64‑encoded):

apiVersion: v1
kind: Secret
metadata:
  name: mysql-secret
  namespace: mysql
  labels:
    app: mysql
type: Opaque
data:
  password: MTIzNDU2  # echo -n "123456" | base64

Apply the Secret:

kubectl apply -f 05-mysql-secret.yaml

Define Services for write (master) and read (master or slaves) traffic:

apiVersion: v1
kind: Service
metadata:
  name: mysql
  namespace: mysql
  labels:
    app: mysql
spec:
  ports:
  - name: mysql
    port: 3306
  clusterIP: None
  selector:
    app: mysql
---
apiVersion: v1
kind: Service
metadata:
  name: mysql-read
  namespace: mysql
  labels:
    app: mysql
spec:
  ports:
  - name: mysql
    port: 3306
  selector:
    app: mysql

Apply the Services:

kubectl apply -f 06-mysql-services.yaml

The core StatefulSet manifest creates two replicas (master and slave), uses an init container to generate a unique server-id and copy the appropriate configuration, a second init container to clone data from the previous pod using xtrabackup, and a sidecar container that serves backup streams for cloning:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mysql
  namespace: mysql
  labels:
    app: mysql
spec:
  selector:
    matchLabels:
      app: mysql
  serviceName: mysql
  replicas: 2
  template:
    metadata:
      labels:
        app: mysql
    spec:
      initContainers:
      - name: init-mysql
        image: mysql:5.7
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-secret
              key: password
        command:
        - bash
        - "-c"
        - |
          set -ex
          [[ $(hostname) =~ -([0-9]+)$ ]] || exit 1
          ordinal=${BASH_REMATCH[1]}
          echo [mysqld] > /mnt/conf.d/server-id.cnf
          echo server-id=$((100 + ordinal)) >> /mnt/conf.d/server-id.cnf
          if [[ ${ordinal} -eq 0 ]]; then
            cp /mnt/config-map/master.cnf /mnt/conf.d
          else
            cp /mnt/config-map/slave.cnf /mnt/conf.d
          fi
        volumeMounts:
        - name: conf
          mountPath: /mnt/conf.d
        - name: config-map
          mountPath: /mnt/config-map
      - name: clone-mysql
        image: gcr.io/google-samples/xtrabackup:1.0
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-secret
              key: password
        command:
        - bash
        - "-c"
        - |
          set -ex
          [[ -d /var/lib/mysql/mysql ]] && exit 0
          [[ $(hostname) =~ -([0-9]+)$ ]] || exit 1
          ordinal=${BASH_REMATCH[1]}
          [[ $ordinal == 0 ]] && exit 0
          ncat --recv-only mysql-$((ordinal-1)).mysql 3307 | xbstream -x -C /var/lib/mysql
          xtrabackup --prepare --target-dir=/var/lib/mysql
        volumeMounts:
        - name: data
          mountPath: /var/lib/mysql
          subPath: mysql
        - name: conf
          mountPath: /etc/mysql/conf.d
      containers:
      - name: mysql
        image: mysql:5.7
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-secret
              key: password
        ports:
        - name: mysql
          containerPort: 3306
        volumeMounts:
        - name: data
          mountPath: /var/lib/mysql
          subPath: mysql
        - name: conf
          mountPath: /etc/mysql/conf.d
        livenessProbe:
          exec:
            command: ["mysqladmin", "ping", "-uroot", "-p${MYSQL_ROOT_PASSWORD}"]
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          exec:
            command: ["mysqladmin", "ping", "-uroot", "-p${MYSQL_ROOT_PASSWORD}"]
          initialDelaySeconds: 5
          periodSeconds: 2
      - name: xtrabackup
        image: gcr.io/google-samples/xtrabackup:1.0
        ports:
        - name: xtrabackup
          containerPort: 3307
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-secret
              key: password
        command:
        - bash
        - "-c"
        - |
          set -ex
          cd /var/lib/mysql
          if [[ -f xtrabackup_slave_info ]]; then
            mv xtrabackup_slave_info change_master_to.sql.in
            rm -f xtrabackup_binlog_info
          elif [[ -f xtrabackup_binlog_info ]]; then
            [[ $(cat xtrabackup_binlog_info) =~ ^(.*?)[[:space:]]+(.*?)$ ]] || exit 1
            rm xtrabackup_binlog_info
            echo "CHANGE MASTER TO MASTER_LOG_FILE='${BASH_REMATCH[1]}',\
                  MASTER_LOG_POS=${BASH_REMATCH[2]}" > change_master_to.sql.in
          fi
          if [[ -f change_master_to.sql.in ]]; then
            echo "Waiting for mysqld to be ready"
            until mysql -h 127.0.0.1 -uroot -p${MYSQL_ROOT_PASSWORD} -e "SELECT 1"; do sleep 1; done
            echo "Initializing replication from clone position"
            mv change_master_to.sql.in change_master_to.sql.orig
            mysql -h 127.0.0.1 -uroot -p${MYSQL_ROOT_PASSWORD} <
Deploy the StatefulSet:
kubectl apply -f 07-mysql-statefulset.yaml
After deployment, two Pods (mysql-0 and mysql-1) become ready; the master is mysql-0 and the slave is mysql-1.
Service verification commands show that write operations should target
mysql-0.mysql
while read operations can use the
mysql-read
service, which load‑balances across both pods.
Replication status can be checked with:
kubectl -n mysql exec mysql-1 -c mysql -- bash -c "mysql -uroot -p123456 -e 'show slave status \G'"
Creating a database, table, and inserting data on the master:
kubectl -n mysql exec mysql-0 -c mysql -- bash -c "mysql -uroot -p123456 -e 'create database test'"
kubectl -n mysql exec mysql-0 -c mysql -- bash -c "mysql -uroot -p123456 -e 'use test; create table counter(c int);'"
kubectl -n mysql exec mysql-0 -c mysql -- bash -c "mysql -uroot -p123456 -e 'use test; insert into counter values(123)'"
Querying the slave confirms the data has been replicated:
kubectl -n mysql exec mysql-1 -c mysql -- bash -c "mysql -uroot -p123456 -e 'use test; select * from counter'"
Scaling the StatefulSet to three replicas adds a new slave (mysql-2); verifying the new pod shows it also contains the replicated data.
kubectl -n mysql scale statefulset mysql --replicas=3
kubectl -n mysql exec mysql-2 -c mysql -- bash -c "mysql -uroot -p123456 -e 'use test; select * from counter'"