Cluster currently has [1000]/[1000] maximum normal shards open

Actually, recommended would be to setup your shard sizing (index rollover) on size and not on time. You would do this to combat oversharding.

In your case, lets assume you have 4 hot nodes, each with 100gb of space.
That means you can store 4 * 100gb = 400gb of data on those nodes. As this is hot you will use 1primary and 1replica shard, reducing the usable space by half -> 200gb. Your shards should be between 30 and 50gb (a lot of reasons but for now just believe me :wink: ) .

Here comes the math:

  • space = 4 * 100gb = 400gb/2 = 200gb -> 200gb
  • indices room = 200gb / 40gb = 5 --> 5 indices

In this example you can hold 5 indices on your hot tier (the ingest tier) and once an index reaches the primary shard size of 40gb it does a roll over and moves to your next tier (warm).

2 Likes