Does filebeat transmit data to the master node of the elasticsearch cluster or to the data node

If I have a es cluster which has three data nodes (B,C,D) and a master node (A). Now I want to use filebeat to collect logs and directly transfer date to the es cluster.Which node should I transfer the data to?
data nodes?

    hosts: ["dataNode.IP:9200"]

master node?

    hosts: ["masterNode.IP:9200"]

Hi @CatLoveFishma Welcome to the community!

Great Question you can configure filebeat to send data to your data / ingest / coordinating nodes since that is where the most traffic will go.... Master Nodes only keep track of the state of the cluster they do not write data

hosts: ["dataNodeA.IP:9200", "dataNodeB.IP:9200", "dataNodeC.IP:9200", ]

I will comment your cluster would be more resilient if you just left your 3 data nodes as data + master nodes... rather than just having a single dedicated master node (if it fails your cluster will fail). 3 Data + Master Nodes is a very common and resilient initial cluster design.

1 Like

Thank you
But I still have a question. If I have 12 filebeats (F1, ...,F12),and I have three data nodes (B,C,D).
The best allocation strategy is to transfer the data of 12 filebeans to 3 data nodes on average?

Yes.. don't overthink it let filebeat and Elasticsearch do the work for you.

All 12 beats point to the three data notes.

When clusters grow, people often put a load balancer in front of the data nodes and then just point all the filebeats or other ingest processes to the load balancer.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.