[Questions ES/Helm charts] Difference between two types of services

[usamaahmadkhan (Usama Ahmad)]

Hi, I was exploring Helm charts for Elasticsearch to deploy on my k8s cluster. I have 2 questions:

1. I successfully deployed 1 master 1 client and 2 data nodes. 6 services were created 2 for each distinct type (normal and headless). Can anyone explain to me what is the function of these two types?
   Referring to the following service templates:
   https://github.com/elastic/helm-charts/blob/master/elasticsearch/templates/service.yaml

2. How can I pass sharding related configurations via these charts?

[michael.russell]

1. This is a standard pattern for statefulsets. It is explained in detail in https://github.com/elastic/helm-charts/tree/master/elasticsearch#clustering-and-node-discovery
2. Can you be more specific about what configuration it is you are trying to set? A link to the relevant docs pages would help.

[usamaahmadkhan (Usama Ahmad)]

Hi Michael,
I want to specify no of shards per index as given here in docs:
https://www.elastic.co/guide/en/elasticsearch/reference/7.4/indices-split-index.html

Similarly I want to configure replicas as well. How can I specify these settings via helm charts?

Things I tried:
Tried passing index.no_of_shards and index.no_of_replicas via esConfig in values.yaml but following error occured on the statefulset pods log and the pod fails readiness probe

{"type": "server", "timestamp": "2019-10-21T12:15:28,205Z", "level": "WARN", "component": "o.e.c.s.SettingsModule", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-0", "message": "\n*************************************************************************************\nFound index level settings on node level configuration.\n\nSince elasticsearch 5.x index level settings can NOT be set on the nodes \nconfiguration like the elasticsearch.yaml, in system properties or command line \narguments.In order to upgrade all indices the settings must be updated via the \n/${index}/_settings API. Unless all settings are dynamic all indices must be closed \nin order to apply the upgradeIndices created in the future should use index templates \nto set default values. \n\nPlease ensure all required values are updated on all indices by executing: \n\ncurl -XPUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -d '{\n  \"index.number_of_replicas\" : \"2\",\n  \"index.number_of_shards\" : \"6\"\n}'\n*************************************************************************************\n" }

[michael.russell]

There is currently no supported way to do this via the helm charts. What we do internally (on my team at Elastic) is manage cluster settings via the same automation that is deploying our helm charts.

There is an open issue with a similar request https://github.com/elastic/helm-charts/issues/249