We have an ES installation where we have 4000 indexes and 40,000 shards. Its clear that this is not a good idea, but we cant get Elastic to start to do anything about it. Any ideas how we can get Elastic to start up? We don't need some of the old indexes but we cant do much until we can get Kibana up so we can use the API.
Hi @chrism1, and welcome. Yes, 40k shards is a lot. Can you tell us a bit more? What version are you running? How many nodes are there in the cluster? Are you using dedicated master nodes? In what way will it not start up? Can you share any logs?
Hi David,
Thanks for your response. We are running version 6.3 of Elasticsearch. There are 4 nodes in the cluster. They are all running as Master, client and slave.
When we restart the nodes come up with status red. It then gets to the point about 15% of the shards are status yellow and becomes unresponsive. It appears to keep going but making hardly any progress and taking minutes to respond to any API request.
regards
Chris
Ok, that sounds promising. Since the nodes are starting up and forming a cluster, you will be able to delete any unnecessary indices via the delete indices HTTP API as soon as they've started. No need to wait for the cluster to become healthy:
curl -XDELETE http://node-address:9200/index-to-delete
You can also get a list of the indices via the usual APIs, e.g.:
curl http://node-address:9200/_cat/indices
Thanks David,
It takes a long time for the delete to return and we have a lot to delete. What about closing the indexes we don't need - would this operation be faster?
I couldn't say if that'd be quicker, sorry, but it seems worth trying.
Thanks
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.