We use Elasticsearch(7.x) to store application/system logs and our indices are time based where new index is created everyday. We have a requirment where different indices are created for every log types and also for every diff application per day. This results in too many indices created everyday and with having retention period of 15 days, this is increasing the default
cluster.max_shards_per_node value of 1000. Each index is not too big in size( few in gbs/mbs/kbs also)
Can you please suggest us can we safely exceed this limit by increasing any env parameters like JVM. Any guidance on memory usage based on shards will be very helpful.
Thanks in advance,