Hello
My customer would like to monitor shards number on Elasticsearch node. He would like to recive email alerting that the limit is about to exceed max value. How to arrange that?
As far as i know shards number is index.routing.allocation.total_shards_per_node value. I can get
shard number selecting last result from GET _cat/shards?s=shard.
Is there anything I'm missing or don't understand or wrong understand?
I'm rather beginner in administrating ELK, so I just want to make sure that my 'thinking mode' is OK
shard number is hard set when index is created. So it's not really useful to monitor it since whoever created it knows.
You can get it by listing all the indices and it'll tell you per index. Or list all the nodes and it'll tell you how many shards are residing on each node.
index.routing.allocation.total_shards_per_node is a cluster configuration setting. It's not what you want.
Knowing what your customer really cares about is probably more important.
Hi Waitangi,
You can setup those alerts if you enable stack monitoring, you can also create dashboards like below . All stack monitoring data will be stored in .monitoring-* data-streams and you can apply threshold rules/watchers on it to trigger email.
Hi Venkata Raja
I don't see any rules related with "Shards number" There is a rule related with "Shared size".
I don't have any subscriptions / licences. Maybe there is a rule related with "Shard number" that requires additional licence.
Hi Waitangi,
You can leverage Elasticsearch query rule type to achieve that.Here is the sample where i have created rule on cluster total shards.
Hi
I'm still trying to monitor shard number in my elastic. Now I have perl script that curl's query "GET /_cluster/stats?human&pretty" to get total shards that is on my cluster. It's about 1700.
As I have read the default value for total_shards_per_node is 1000, and that value is not set for my cluster, so the query 'GET /_cluster/settings?human&pretty' return evary value it has except total_shards_per_node.
My Elastic cluster has 1 node so number of shards exceed the default value. How coult it be?
And how can I get the value for maximal shards that can be created.
All I need is write an email when shard number is about to exceed the specified limit. But what is the limit?
How the number of shards per cluster refers to number of shards per node and number of shards per index?
Hi,
You will get node shards here,
GET _nodes/stats?filter_path=nodes.*.indices.shard_stats.total_count
Replace * with node name you have
All I need is write an email when shard number is about to exceed the specified limit. But what is the limit?
shard limit is 20* heap_size
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.