ILM with a loadbalancer

Hello,

I currently have a 6 node cluster in Azure (3 master, 3 data) with the 3 data nodes behind a loadbalancer. Kibana and Logstash point to the loadbalancer for the queries of and sending to Elasticsearch.

I would like to implement the Hot-Warm-Cold architecture a part of a 6 year logging solution.

I have created another 7 nodes. 3 will be for Warm and 4 will be for Cold storage.

Following the guides, I have added node attributes to the elasticsearch.yml's like so:
node.attr.data: 'hot'
node.attr.data: 'warm'
node.attr.data: 'cold'

And Kibana has picked up on them to be used in creating a ILM policy.

My question here is, should the Warm and Cold nodes be behind the loadbalancer?
If my logstash configs have the { ilm_enabled => true } setting then it should not send data to the Warm or Cold nodes, correct?

Any guidance would be great! I do want to keep the loadbalancer and may even switch to a gateway.

Thanks!

Am looking for any guidance that could be offered. Anyone doing something similar? Or, have a document I can refer to?

To answer my own question: it does't seem to matter.

Logstash and filebeat both point at the Azure loadbalancer.
Only the hot nodes are attached to the load balancer.
The master nodes are not attached to the load balancer.

When I was troubleshooting my ILM policy I noticed that documents were sent to all nodes, hot, warm and cold. I guess the master handles all that?

Currently, I have ILM working correctly with a hot-warm-cold-delete policy and only the hot nodes behind the loadbalancer.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.