My Setup: Have 50+ application servers which generates huge logs and filebeat ships them to multiple LS nodes and then gets shipped into 1 ES node[ 6 node on ES cluster => 5 data nodes and 1 client node]
Version of Components:
elasticsearch-5.5.2 , logstash-2.4.1,kibana-5.5.2-linux-x86_64, filebeat-1.2.0-1.x86_64/ filebeat-5.4.0-1.x86_64
All data flows via filebeat => respective logstash => 1 ES client node (then to ELK cluster)
We totally have 3 logstash load balancing all 50 application servers. Instead of sending to one ES client node can i sent to multiple ES data nodes? is that possible? How can i measure the ingest rate of ES and logstash easily.
Instead of sending to one ES client node can i sent to multiple ES data nodes? is that possible?
Yes it is possible. See
There's probably many ways to measure and monitor ingest rates but for ES I usually look at Kibana's monitoring page which comes with the free version of x-pack
Looks like it offers Logstash monitoring as well.
Thanks A_B for the reference links.
Before pointing to multiple ES nodes i would prefer to monitor the ES ingest rate so that i can compare the performance metrics. As said above will use free version of X-pack and test in replica setup and then install in production.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.