Does Elasticsearch lose logs in the event of an unexpected outage?

Elastic Stack Elasticsearch
1

I use Filebeat, Logstash, and Elasticsearch to collect logs.
If a server with Elasticsearch and Logstash set up goes down unexpectedly, what happens to the logs sent by Filebeat? Will I lose them?

If the server on which Filebeat is set up goes down, I believe that if I restart it, Filebeat will send logs starting from the last log it sent, but what happens if Filebeat keeps running?

2

Hey Ogawa,

If Filebeat is not able to deliver the logs to Logstash, it will continue sending the logs where it where Logstash last accepted the logs if Logstash comes back up. See How Filebeat works | Filebeat Reference [8.4] | Elastic

I hope this answers your question,

Tim

3

Thanks for answering.
That is the best answer.

I knew that Filebeat kept the last location of the logs it delivered, but I was wondering if Logstash or Elasticsearch would continue from the last recognized location as well in case it did not successfully receive the logs from Filebeat.

This gave me peace of mind.

The state is used to remember the last offset a harvester was reading from and to ensure all log lines are sent. If the output, such as Elasticsearch or Logstash, is not reachable, Filebeat keeps track of the last lines sent and will continue reading the files as soon as the output becomes available again.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.