Regarding Data transfer from Filebeat to Logstash


(Karunesh Upadhyay) #1

Hi

Filebeat is sending files to logstash and transfer got stopped because of connection break then when will happen?
Data will be processed form starting or it will continue from same point?

Ex-

We have 5 files, 2 files has transferred , 3rd file is transferring from File beat to Logstash, before 3rd file completion connection lost and filebeat stop.

So when we restart the filebeat then it will start from starting of 3rd file or from the same point where it was there before connection lost?

Thanks and Regards,
Karunesh Upadhyay


(Steffen Siering) #2

filebeat uses send-at-least-once semantics. Every line send must be ACKed by logstash. All lines not ACked will be send again. On restart or if network unavailable (filebeat will try to auto-reconnect), filebeat will try to continue from last ACKed position.


(system) #3