# Throttle filter: Notify on throttle **URL:** \ **Category:** Logstash\ **Created:** [January 22, 2021, 1:12am UTC](https://discuss.elastic.co/t/throttle-filter-notify-on-throttle/261861 "2021-01-22T01:12:26Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![holobolo0815](https://avatars.discourse-cdn.com/v4/letter/h/ecc23a/32.png) [@holobolo0815](https://discuss.elastic.co/u/holobolo0815)\ **Post date:** [January 22, 2021, 1:12am UTC](https://discuss.elastic.co/t/throttle-filter-notify-on-throttle/261861/1 "2021-01-22T01:12:27Z")
When using the throttle filter, how would I go about notifying **one** event once the throttle has triggered? Meaning if I allow 5 events per minute, I'd like to know somehow if there has been a 6th event and send a mail or output some kind of "special" event to ES. I don't care about subsequent events during that time period. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [January 22, 2021, 5:29pm UTC](https://discuss.elastic.co/t/throttle-filter-notify-on-throttle/261861/2 "2021-01-22T17:29:49Z")
A throttle filter checks how many events have occurred in a period and if the number is outside the configured range then it decorates the event (i.e. it processes the add\_tag / add\_field / remove\_tag / remove\_field options). If you only want one event each period to not be decorated then configure before\_count and after\_count to have the same value. ---
**Author:** ![holobolo0815](https://avatars.discourse-cdn.com/v4/letter/h/ecc23a/32.png) [@holobolo0815](https://discuss.elastic.co/u/holobolo0815)\ **Post date:** [January 22, 2021, 7:11pm UTC](https://discuss.elastic.co/t/throttle-filter-notify-on-throttle/261861/3 "2021-01-22T19:11:08Z")
Okay, let me rephrase: I want to 1. Drop all events that do not fit into the throttle parameters (e.g. if there are more than 5 events of a certain type during an interval of 1 minute, I want the rest to be dropped) 2. Additionally, I would like to know that this was triggered, i.e. I would like to know if stuff gets dropped that way. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [January 22, 2021, 7:21pm UTC](https://discuss.elastic.co/t/throttle-filter-notify-on-throttle/261861/4 "2021-01-22T19:21:24Z")
OK, so use two throttle filters. One to tag all the events that you want to drop, and another (with before\_count and after\_count set to the same value) that you can conditionally route to a destination that tells you events have been throttled. ---
**Author:** ![holobolo0815](https://avatars.discourse-cdn.com/v4/letter/h/ecc23a/32.png) [@holobolo0815](https://discuss.elastic.co/u/holobolo0815)\ **Post date:** [January 22, 2021, 11:48pm UTC](https://discuss.elastic.co/t/throttle-filter-notify-on-throttle/261861/5 "2021-01-22T23:48:00Z")
I'm not sure what you mean. Once I've drop{}ed the event I can't use it any longer. Do you have a tangible example of how to do this? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [January 23, 2021, 12:06am UTC](https://discuss.elastic.co/t/throttle-filter-notify-on-throttle/261861/6 "2021-01-23T00:06:53Z")
I have not tested it but you could try ``` filter { throttle { period => 60 after_count => 5 add_tag => ["dropMe"] } throttle { period => 60 before_count => 5 after_count => 5 add_tag => ["importantWhenNotPresent"] } if "importantWhenNotPresent" not in [tags] { mutate { remove_tag => ["importantWhenNotPresent", "dropMe"] add_tag => ["sendAlert"] } } if "dropMe" in [tags] { drop {} } } output { if "sendAlert" in [tags] { ... } } ``` ---
**Author:** ![holobolo0815](https://avatars.discourse-cdn.com/v4/letter/h/ecc23a/32.png) [@holobolo0815](https://discuss.elastic.co/u/holobolo0815)\ **Post date:** [January 23, 2021, 12:45am UTC](https://discuss.elastic.co/t/throttle-filter-notify-on-throttle/261861/7 "2021-01-23T00:45:52Z")
Double negation... oh thanks, I hate it. 😉 Your idea boils down to "negatively" tagging the "interesting" event in addition to it needing to be dropped. 1: importantWhenNotPresent 2: importantWhenNotPresent 3: importantWhenNotPresent 4: importantWhenNotPresent 5: importantWhenNotPresent 6: throttle 7: throttle importantWhenNotPresent 8: throttle importantWhenNotPresent ... Maybe the following could work. It still looks like pain. ``` filter { throttle { # decorates event if OUTSIDE bounds key => "%{srcip}-%{dstip}-%{port}" before_count => -1 after_count => 12 period => 60 max_age => 300 add_tag => "throttle" } throttle { # decorates event if OUTSIDE bounds key => "%{srcip}-%{dstip}-%{port}" before_count => 12 # same as after_count of previous throttle filter after_count => 12 # same as before period => 60 max_age => 300 add_tag => "importantWhenNotPresent" } if "throttle" in [tags] and "importantWhenNotPresent" in [tags] { drop {} } # what remains are the events before throttle kicks in # and the first event that is throttled (which does NOT contain the "important" tag) } ``` Now we should be able to filter using tag "throttle" to get the first of each set of events that was throttled. Of course you could also send an email in output{} section and whatnot. (Not tested yet. Did I mention it looks painful?) ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [January 23, 2021, 1:06am UTC](https://discuss.elastic.co/t/throttle-filter-notify-on-throttle/261861/8 "2021-01-23T01:06:58Z")
> [@holobolo0815](#): > > (Not tested yet. Did I mention it looks painful?) I do not think there is going to be a pretty solution to this problem. You want to drop events but also have an event that tells you those events were dropped. Another approach would be to have a metrics filter that counts to number of events with a "throttle" tag. Another approach would be to use pipeline-to-pipeline with a forked path pattern. Send the events to a pipeline that applies the simple throttle to drop events, and also to a pipeline that generates the alerts. Each pipeline might be easier to follow but I suspect this is over complicating things. ---
**Author:** ![holobolo0815](https://avatars.discourse-cdn.com/v4/letter/h/ecc23a/32.png) [@holobolo0815](https://discuss.elastic.co/u/holobolo0815)\ **Post date:** [January 24, 2021, 4:51pm UTC](https://discuss.elastic.co/t/throttle-filter-notify-on-throttle/261861/9 "2021-01-24T16:51:15Z")
Just some remarks for the rest of the world. Actually you do not need to tag every event up to the limit. Also you need to use an offset of one in after\_count between filters, otherwise you don't get the message that "sticks out". It then becomes this: ``` filter { # prevent DoS # "negatively" tag the "interesting" event in addition to it needing to be dropped # example: before_count=>-1, after_count=>5 and 6, respectively #1: NOOP #2: NOOP #3: NOOP #4: NOOP #5: NOOP #6: throttle #7: throttle throttleDrop #8: throttle throttleDrop throttle { # decorates event if OUTSIDE bounds key => "%{srcip}-%{dstip}-%{port}" before_count => -1 after_count => 499 # same as below - 1 period => 60 max_age => 300 add_tag => "throttle" } throttle { # decorates event if OUTSIDE bounds key => "%{srcip}-%{dstip}-%{port}" before_count => -1 after_count => 500 # same as above + 1 period => 60 max_age => 300 add_tag => "throttleDrop" } if "throttleDrop" in [tags] { drop {} } } output { if "throttle" in [tags] { email { from => 'holobolo@example.com' to => 'holobolo@example.com' subject => 'Throttle Alert - %{srcip}-%{dstip}-%{port}' } } } ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 21, 2021, 4:51pm UTC](https://discuss.elastic.co/t/throttle-filter-notify-on-throttle/261861/10 "2021-02-21T16:51:19Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.