Correct way to send data to ES cluster

[tomer (tomer zaks)]

Hi,

How to send logs to the ES if I have 1 master node and two data nodes? Should I send all logs to master or all to data or it doesnt matter?

(I was unable to see such information in the master/data node manual)

I also saw the logstash -> output-plugins -> ElasticSearch. It write there:

It is important to exclude dedicated master nodes from the hosts list to prevent LS from sending bulk requests to the master nodes. So this parameter should only reference either data or client nodes in Elasticsearch.

Does this mean not to send the logs to the data node?

[Christian_Dahlqvist]

Don't run with a single master node as that will be a single point of failure. Make you two data nodes master eligible too so that you have three master eligible nodes in he cluster. Make sure you update minimum_master_nodes to 2 and then send all requests to the master/data nodes.

[tomer (tomer zaks)]

Thanks!
If I make the data nodes as master eligible, do I need also to create a new master only node? (I rather not, since I am working on a project that is lagging behind).

From my understanding I could send from lets say the logstash to the master and data nodes with no difference.

[Christian_Dahlqvist]

You should always aim to have 3 master eligible nodes, so do not remove the dedicated master node. This allows Elasticsearch to elect a master even if one node is unavailable, which it is not if there are only 2 nodes in the cluster.

[tomer (tomer zaks)]

Perfect thanks!

Just to make sure so I send my logs to the data nodes (that are also master eligible) but not to send logs to the master only node?

[Christian_Dahlqvist]

Exactly.