Change StorageClass of an already running cluster

[dtapia (Daniel)]

I've created a cluster with a specific storageClass, but now I need to change the disks to something cheaper, but I get this error when trying to update the ElasticSearch CRD.

Error: UPGRADE FAILED: cannot patch "elasticsearch-7" with kind Elasticsearch: 
admission webhook "elastic-es-validation-v1.k8s.elastic.co" denied the request: 
Elasticsearch.elasticsearch.k8s.elastic.co "elasticsearch-7" is invalid: 
spec.nodeSet[0].volumeClaimTemplates: Invalid value: 
[]v1.PersistentVolumeClaim{v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", 
APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"elasticsearch-data", GenerateName:"", 
Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, 
CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}},
 DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), 
Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:
[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:
[]v1.ManagedFieldsEntry(nil)}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:
[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), 
Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), 
Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1374389534
72, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"", Format:"BinarySI"}}}, 
VolumeName:"", StorageClassName:(*string)(0xc00812aff0), VolumeMode:
(*v1.PersistentVolumeMode)(nil), DataSource:(*v1.TypedLocalObjectReference)(nil)}, 
Status:v1.PersistentVolumeClaimStatus{Phase:"", AccessModes:
[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:
[]v1.PersistentVolumeClaimCondition(nil)}}}: Volume claim templates cannot be modified

Is this possible or supported? Or do I just have to create a new cluster and sync the data?

[sebgl (Sebastien Guilloux)]

You cannot change the existing storage class in an existing nodeSet.
What you can do however is rename the existing nodeSet along with the storage class modification.

For example:

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: quickstart
spec:
  version: 7.11.2
  nodeSets:
  - name: my-nodes
    count: 3
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        resources:
          requests:
            storage: 10Gi
        storageClassName: old-sc

to:

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: quickstart
spec:
  version: 7.11.2
  nodeSets:
  - name: new-nodes
    count: 3
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        resources:
          requests:
            storage: 10Gi
        storageClassName: my-new-storage-class

See how both nodeSet name and storage class name have changed in that example.

Behind the scenes, the operator will create Pods for the new nodeSet, and migrate data away from Pods of the old nodeSet before removing them. The data migration can take some time depending on how much data you have.