Trying to use the throttle plugin in logstash to drop messages if I am receiving them at too high a rate. My goal is to:
drop excess messages if more than 100 received in a 10 second window
To accomplish this, I use the config below, but it doesn't do what I expected. Instead, when I start logstash, it allows the first 100 messages through and then throttles the rest indefinitely -- all the messages received after the first 100 are dropped. How do I accomplish the goal above?
logstash 2.4.0
throttle 4.0.1
filter {
throttle {
after_count => 100
period => 10
key => "%{type}"
add_tag => "throttled"
}
if "throttled" in [tags] {
drop { }
}
}