Logstash Database pipeline resiliency

Elastic Stack Logstash
1

Hello Team,

I have a use case where Ineed to deploy database pipeline on 2 nodes, but i want to implement in a way that if one node(machine) goes down then only 2nd will pick up from that point and until the second goes down the first will wait.
I dont have the option of docker and kubernetes.
could you please help me understand if this is possible.

Thank you for your help and support.

Thank you,
Aditya

2

logstash does not support clustering in the sense that nodes are coordinated. They all run independently.

3

Yeah right, any thoughts on handling the situaltion.
I was thinking of keeping shared lastcrawledtime file (if possible) and keep scheduling at different time so that i wil not crawl duplicates and 2 pipeline will not query at same time.
any thoughts on this approach.

4

logstash is not designed to do what you want it to do. That does not mean it is impossible, but it will not be simple.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.