In our current running architecture we are not having hot /warm nodes. So all open indices are almost equally distributed across all data nodes. All our queries specify multiple indices to query, we dont use wildcards.
In our new cluster we are planning to use hot warm architecture with dedicated hot nodes where we will keep around 10 of recent indices and rest of indices would be on warm nodes.
Now the question -
how will hot/warm architecture or availability of limited number of indices on hot nodes would enhance
- Query performance
- indexing performance
- Overall cluster health.
Data Usecase - Significant high reads/writes on approx 7 recent indices and significantly less reads on older indices. even lesser writes on older than 7 indices.
Read traffic - 65000 request per minute during peak.
write traffic - 1500 request per minute
total indices - 90
Old cluster = 21 data nodes , 3 client+master nodes
elasticsearch version 1.5
each index size 30 gb
no of shards - 4
replication factor 1
recent indices have increased replication factor 3 to enhance read searches.
new cluster being built = 3 master+3client + data [x hot+ y warm] nodes
elasticsearch version 6.2
node types - hot/warm
Index Size - 30 Gb
no. of shards- 4