Hi,
Our setup as below:
- FIlebeat configured on multiple servers.
- Elastic search in 3 node cluster
- Single node logstash server with persistent queue
- Single node kibana interface.
We would like to make 2 node logstash in high-availability with persistent queue facility. For load balancing, we are considering keepalived solution with floating ip. That floating ip will configure on all filebeat client. Our concern is regarding to persistent queue during the fail-over scenario . (For now, we are not thinking to configure any third party messages queuing application i.e. RabbitMQ )
We are not using any shared device between 2 logstash nodes. So, our persistent queue will be generate individually on both node whoever has floating ip.
Suppose, Our A server will goes down unexpectedly and floating IP shift from A server to B server. Logstash will start with new queue file on B server. How logstash will behave after fail-back floating IP to A server. How persistent queuing will work in this scenario?
- Does logatsh consider and execute older queue file of A server ? (Before goes down)
- Does logstash consider and execute pending queue on B server ? (After fail-back to A server)
Let me know in case any more info require.
Kindly help.
Regards
Nishant