If you delete the index using DELETE API, does it clear the fielddata / cache? Or is it suggested that we stop the index before deleting it?
I want to make sure that we don't keep increasing memory footprint (JVM/ OS Memory) forever. Thanks so much.
If you delete the index you delete anything else related to it, including any caches.
Thanks for the confirmation.
If you close the index, does it get rid of all resources that it would consume other than the disk space?
What do you mean by stop the index?
If you close an index it removes any caches, it still takes some resources because the cluster maintains the index info in cluster state (in memory).
I am guessing that is purely metadata like mentioned in the documentation and doesn't occupy too much space. We have observed that having too many indices/shards slows down the cluster and there isn't any easy way to recovery from that state. For example, we have 47k shards on a cluster of 8 nodes (5 data + 3 dedicated master nodes).
We also have dedicated master nodes (3 to be specific), so suppose the that master nodes take care keeping index information.
Each shard needs a certain amount of resources to exist.
Yeah, that's a little overboard.
What version are you on.
_shrink API to reduce the count for old indices.
A quick question
@animageofmine, are you using the default number of shards or have you customised them?
customized. It is based on the number of expected documents for index. Why do you ask?
Just wondering if you were using defaults or not
ok :-). Default is way to high for most of the cases for our system.
If you get a chance to look at this question, I would really appreciate that.
We use transport client (elastic4s) to communicate with ES cluster via consul. One of the problems we have seen so far is that the client keeps sending queries to the dead nodes.
For example, if the data node node1 is dockerized and if you stop the process inside the container (for testing purpose), but the container is still on, consul does not redirect the queries to that node. However, we still see the queries routed to the dead node node1. I know the client is cluster aware, but I am not s…
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.