Hello,
Is there some optimal number of documents to be present in an index? for example the indexes are created daily and go as foo-2018-02-20 and so on. Some of the indexes are really small (<100 documents with average size per document around 10kb per document) while some are really large in comparison to this (around 300K documents with similar document size) The total number of documents is currently 29 million across 8K indexes. I am not running into any performance reasons but just curious as to what would be an optimal number so I can try to tweak the smaller indexes to have some larger number so it can scale better.
Thanks
< 2 ^ 32 -1 :p, cause that's a lucene limit.
Look at shard sizing instead. We suggest <50GB, some suggest <30GB, the real number depends on a few things that you may want to test if performance is important. High level though, more shards = faster writes, less shards = faster reads.
However given you are using time based indices, you can do things like _shrink and split pretty easily, so you can experiment. Also check out https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster, which is a great resource.
1 Like
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.