When Logstash sends ACK to input source(Example- Filebeat)

[prashant.bajpai (pBajpai)]

I have read about the at-least-once-delivery commitment of filebeat and what I understood is that until the ack of sent logline is not received by filebeat, that line will be sent again (in case of filebeat re-start).
Now supppose, In my solution, I am using Filebeat, Logstash, and one other component that logstash is using for filtering. And after filtering the logstash sends the line to elasticsearch.

Now here are below checkpoints where we can loss data :

1. Filebeat got shutdown without receiving ack from logstash - In this case we know that line will be sent again by filebeat.
2. Suppose Filebeat sent a line, and logstash applies filtering on it with the external component and then when It tries to send to elasticsearch and the same time logstash/elasticsearch got crashed, So will we loss this data.

My question is:

Basically logstash processes data in below sequence:

INPUT --> FILTER --> OUTPUT

So I want to know at which step the logstash will send ACK to filebeat. I want to basically understand how the ACKS are being sent and when.
I mean below flow with ACKS sent-
FILEBEAT -> LOGSTASH -> filter -> LOGSTASH -> ELASTICSEARCH
I want to understand when :

1. Logstash sends ACK to filebeat
2. Elasticsearch send ACK to logstash

I tried to search it on google and ELK official websites but didn't get the information in details.

Can somebody help me in understanding these details ?
Thanks in advance.

[magnusbaeck (Magnus Bäck)]

I believe the input plugin will send the ack as soon as it has pushed the event to Logstash's internal queue, so if Logstash dies shortly after that you may lose messages. However, as of recently that queue can be stored on disk with the persistent queue functionality which will remove that problem.

https://www.elastic.co/guide/en/logstash/current/persistent-queues.html

[prashant.bajpai (pBajpai)]

@magnusbaeck, thank you for your answer.