Logstash Kafka input, set interval/frequency

Elastic Stack Logstash
1

My setup is like

FileBeat --> Kafka --> Logstash --> Syslog Server

Q: How to limit/interval frequency in input or output of Logstash.

I'm using Logstash here to format the event logs to be sent to Syslog server. The problem here is Filebeat is sending high events/sec to Kafka and when I send those to Syslog server, it is unable to handle the load. Is there a way to limit the Events/sec in either input plugin or output plugin of Logstash?

2

Look at the throttle filter. It will simply add a sleep delay, alternatively you could set the workers to 1 thread and a small batch size e.g. 10.

Its not often we get requests to slow Logstash down :grinning:

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.