Can filebeat buffer data for a second?


(Mathias) #1

Hi,

I believe my performance problems with filebeat and logstash are due too many events in short time.
I have notice in other systems that performance is improved when I buffer my events and upload the buffer once per second.
I wonder if this is possible with current filebeat configuration options?

Br Mathias


(Steffen Siering) #2

The spooler in filebeat buffers events for up to 2048 events by default. If buffer is full or after flush-timeout events (1 second by default), the buffered events/lines are published.


(Mark Walkom) #3

You can also send to redis in 1.X if you want to use that sort of broker.
Kafka is coming in 5.X.


(system) #4