Elastic Search Index Refresh Interval elastic.yaml file in Kubernetes

I am new to Elasticsearch. I have a 3 data node and 3 master node elastic cluster deployed in Kubernetes. It was working well until recently there is a large intake of data. Now, I am in a stage where I need to apply index refresh interval configuration to 120s to allow optimal usage of the cluster. I am able to do it on individual index level using this method. but I am not able to do at the cluster level. I have a process that creates a new index everyday and do not have more than 15 indexes totally at any point of time in the cluster. So, currently, I am doing it manually by using the method afore mentioned/using Kibana UI. I tried to do this couple of ways both failed.

  1. Used the settings PUT method to force the index settings at the global level and it gives an error of no requests in the range

PUT /_cluster/settings -d { "index" : { "refresh_interval" : "120s" } }

  1. I used the elasticsearch yaml to set this value for the data node and the data node fails to come up.

elasticsearch yaml

    apiVersion: elasticsearch.k8s.elastic.co/v1
    kind: Elasticsearch
    metadata:
      name: elastic-cluster
      namespace: elastic-system
    spec:
      version: 7.6.2
      nodeSets:
        - name: master
          count: 3
          config:
            node.master: true
            node.data: false
            node.ingest: false
            node.ml: false
            node.store.allow_mmap: false
          podTemplate:
    			...
        - name: data1
          count: 3
          config:
            node.master: false
            node.data: true
            node.ingest: true
            node.ml: false
            node.store.allow_mmap: false
    		index.refresh_interval: 120s # I added it here
          podTemplate:
    			...

There is a third way, through kibana UI -> settings-> Elasticsearch -> Index management -> index template. But there is no index template for me to start with. Nevertheless, the Elasticsearch creates an index daily with the date. So, I do not want to mess with the existing template.

Can anyone suggest me a better way to do this. Thanks for your time.

This is something you need to control through an index template. This will apply to newly created indices and you can use a list of prefixes with wildcards to control which indices it will apply to.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.