Does logstash continue getting log from Kafka if it cannot send log to ELS?

Elastic Stack Logstash
1

I want to build a topology like this, but I don't know the behavior of logstash when elasticsearch dies ( Logstash cannot send the log to Elasticsearch). There are 2 situations that I'm thinking

  1. Logstash will stop getting logs from Kafka.
  2. Logstash keep getting logs from Kafka and save to its queue, if elasticsearch is online, logstash will send logs from the queue to ELS.

Please help me explain more about this.

image
image700×338 41.6 KB

2

Hi,

LogStash will keep receiving events even if ElasticSearch is unavailable. There are 2 points though:

  1. By default LogStash will keep the events in memory. If LogStash is restarted all pending events are lost. To solve this use a persistent queue
  2. The storage space for pending messages is fixed. If the queue is full no new messages are received. The maximum size can be defined for persistent queues

Best regards
Wolfram

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.