Hello,
I currently have a 6 node cluster in Azure (3 master, 3 data) with the 3 data nodes behind a loadbalancer. Kibana and Logstash point to the loadbalancer for the queries of and sending to Elasticsearch.
I would like to implement the Hot-Warm-Cold architecture a part of a 6 year logging solution.
I have created another 7 nodes. 3 will be for Warm and 4 will be for Cold storage.
Following the guides, I have added node attributes to the elasticsearch.yml's like so:
node.attr.data: 'hot'
node.attr.data: 'warm'
node.attr.data: 'cold'
And Kibana has picked up on them to be used in creating a ILM policy.
My question here is, should the Warm and Cold nodes be behind the loadbalancer?
If my logstash configs have the { ilm_enabled => true } setting then it should not send data to the Warm or Cold nodes, correct?
Any guidance would be great! I do want to keep the loadbalancer and may even switch to a gateway.
Thanks!