Hello Guys ,
Sorry for my noob question, but I can't find where can I setup the index size?. I just see in docs about shard cache or buffer size.
Thank you
Hi! There are no index size limit. Index max size limited by available hardware (in my case with RAM) and some shard-level limits (2.1B documents per shard and recommended shard size up to 20-40GB (soft limit to ensure reasonably fast index relocation speed over the net)).
Hello Rusty,
Thank you for you answer, with curator I can handle the index. I'll try with it.
thank you !!!!
That means that my index size is limited by the shards and would be amount of shards * max shard size (40 GB or what is it?). When the index size gets close to that I would have to add a shard to the index, correct? Is that possible or do I have to create a new index with let's say double the shards and move the old index to the new one ?
The only limit is 2.1Billion documents per shard (it's Lucene core limitation), 20-40GBytes is soft limit (recommended size in terms of shard relocation time and shard recovery time).
You should plan expected capacity in advance or use for sample time/name based indexes (you_index-DATETIME_OR_COUNTER_OR_UID_OR_MISC and use wildcard to search like "/you_index-*/_search").
True, 2.1 Billion documents was the restriction. Thanks .