Issue with deploying Elasticsearch helm chart with dedicated nodes and deleting and re-deploying the release

Elastic Stack Elasticsearch
1

I have been using the elasticsearch helm chart from elastic for a long time now, but recently wanted to switch my architecture to use dedicated master and data nodes.
I looked at the example provided here: https://github.com/elastic/helm-charts/tree/7.9/elasticsearch/examples/multi
and added 2 releases of the chart in my code. This installed properly with both master and data nodes starting correctly with their respective pvc's.

Issue happened when i deleted both the helm releases (master and data) and tried to re-deploy the helm release (i did that because i wanted to update the PVC size in the statefulset and with the current chart you cannot update the volume size value without deleting and deploying the statefulset). Related to: https://github.com/elastic/helm-charts/issues/492
Specific error is this:

image
image2176×320 50 KB

when i re deployed, of course the PVC's were still there, and when my helm command to deploy master nodes ran, the pods got scheduled but the master nodes never formed the cluster.

My question is, is this expected because there are no data nodes ? But since to use this chart as described in the example, 1 release will have to happen before the other, cannot happen together.
How do i overcome this issue ?

Of course if i remove the PVC's (delete them) i see deploy works fine.

Any help will be appreciated and i can provide more info if needed.

1 Like
2

In a nutshell, i am trying to understand how should one proceed with resizing of the PVC's for both the releases.

  1. If i want to resize data nodes, can I delete just that release and deploy it back ? will the cluster be stable with only master nodes in it. ?
  2. What if i need to resize the pvc for master nodes, can i take them down (just master release) without messing up the elasticsearch cluster (which will be with only data nodes and no master nodes) ?
3

Looks like i figured out the solution. Hopefully it's helpful for others.
I ll try to add this information to the helm chart repo as part of the multi example.

PVC's for master or data release has to be updated 1 at a time.
Steps:

  1. Update the value for the pvc for data release. helm upgrade will fail because of the issue i mentioned in my original question above. Delete just the data nodes specific helm release, and re deploy it to get the updated statefulset (with new value for the volumeclaimtemplate).
  2. Similarly do the same for master release, if that needs to be re sized.
1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.