From docs, i read that query performance remains same if queried on 10 indices of 1 shard Vs 1 index of 10 shards.
- What about bulk indexing performance for the same situation: Bulk indexing on 10 indices of 1 shard Vs 1 index of 10 shards?
I have continuous incoming of logs into the system throughout the day. Queries have to be supported on whole data(for a year) but it will be less frequent. i am planning to use roll over api (https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-rollover-index.html) to roll over index when it has become large.
With roll over api, It is a tradeoff to figure out
- How much data to be kept in a single index vs rolling over to new index when index becomes large?
3..Also, how many shards should we keep in an index even if we configure roll over? Keeping whole data in a single shard and roll over vs keeping data in 5 shards(eg) and roll over
I would like to get recommendations as baseline to start my tests.