Sending data from 2 logstash nodes to an elasticsearch cluster

Elastic Stack Elasticsearch
1

Hi Folks,

I have 2 logstash nodes (version -8.6.2) that i want to send data to 2 elasticsearch nodes (version -8.6.2) ( a third node will be added soon to the cluster) . Do i just mention the elasticsearch nodes' in the ES output section like below , would that be enough?

 hosts => [ "https://10.27.101.63:9200","https://10.27.101.247:9200"]

Would the data be sent to both ? or since its a cluster i'd just need to send to the master and would that take care of sharding and placing data on both nodes ?

master is 10.27.101.63

currently i have both elasticsearch nodes set in [master ,data] roles

2

The configuration you showed will send the data to one of the nodes in the cluster, which is the correct behaviour. When you send a bulk request to one of the nodes in the cluster it will write the data across the cluster, so you should not try to send data twice. The master node role is not involved in indexing (unless mappings or other things affecting the cluster state changes) so data should be sent to any of the data nodes.

1 Like
3

And is it load balanced automatically ? so as of now the data seems to be going the master /data node - 10.27.101.63 or would it always goto this one node all the time ?

4

No. It can go to either node.

1 Like
5

Thank you for the swift response :slight_smile:

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.