What happens if Logstash stops indexing from Redis?

Elastic Stack Logstash
1

Hello,
Consider this scenario:
Logstash shipper ships to Redis, and a Logstash indexer consumes messages from it.
Suddenly, the indexer stops due to server shutdown.

  1. What is going to happen to Redis? Will it be fed messages endlessly until it's out of memory?

  2. Let's assume I set a maxmemory limit and no eviction policy in the Redis configuration file.
    2.1) Is that the right solution? My assumption is that the Logstash shipper is going to start getting errors for messages outgoing to Redis, and existing queue in Redis will be kept. Is that correct?
    2.2) What is going to happen to the Logstash shipper? How long is it going to hold messages for? What is its buffer size?

Thanks!

2

What is going to happen to Redis? Will it be fed messages endlessly until it's out of memory?

I believe so, yes.

What is going to happen to the Logstash shipper? How long is it going to hold messages for? What is its buffer size?

Logstash has no internal buffer to speak of. If you're shipping local files then they will be the buffer (i.e. Logstash will simply stop reading from them if it can't send the events to Redis) and you'll be fine until the files rotate. However, if you're receiving messages over the network you might have a problem (especially if you're using UDP).

1 Like