How to change the cluster_update_settings time?


(andy_zhou) #1

the default is 30s ,but in big cluster this is small? how to change the time
thanks.

{
  "error" : {
    "root_cause" : [
      {
        "type" : "process_cluster_event_timeout_exception",
        "reason" : "failed to process cluster event (cluster_update_settings) within 30s"
      }
    ],
    "type" : "process_cluster_event_timeout_exception",
    "reason" : "failed to process cluster event (cluster_update_settings) within 30s"
  },
  "status" : 503

(Christian Dahlqvist) #2

That should be fine also for larger clusters. It may be worthwhile investigating while it is taking so long instead of trying to increase it.

Which version of Elasticsearch are you using? How many nodes do you have in your cluster? How many indices and shards do you have in the cluster?


(andy_zhou) #3

5.4.3
in cluseter 10W+ shards 54 nodes 36 data nodes 500+index
when init the cluster or large data bulk to es cluseter ,
i can't change the settings of cluster
the pending tasks 1000+
queue 10W+


(Christian Dahlqvist) #4

I do not understand this metric. What is it?

You may also want to read this blog post.


(andy_zhou) #5

sorry 10W+ =100k+
1w=ten thousand


(Christian Dahlqvist) #6

That is over 2800 shards per data node, which is far too much. I suspect you will need to reduce this. Have a look at the blog post I linked to for some guidelines on shard count and size.


(andy_zhou) #7

can give me a link? witch the best shards in one data node? can give me example.


(Christian Dahlqvist) #8

It is in my previous post where I asked about the unit.

Increasing queue sizes and timeouts like you have done does not solve the problems you are having in the cluster, just putting a band-aid on it. I would recommend trying to address the underlying issues while your cluster still is in a workable state. The worse state you cluster gets in, the more difficult it will be to solve any problem properly.


(andy_zhou) #9

can give me a blog link about you sea?


(Christian Dahlqvist) #10

(andy_zhou) #11

this time can change ?
i need delete data about index. but can't up it . output timeout 30s.


(Christian Dahlqvist) #12

I think this might be what you are looking for.


(system) #13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.