Aggregate only working when timeout = 0

Elastic Stack Logstash
1

I have an Elasticsearch input that collects list of fails that have not been handled. Then aggregate the false count and output the count to an index.

It only works when the timeout = 0, if i change the timeout to anything but 0 it doesn't aggregate the result. I'm only using 20 rows to test ,

what will happen when I put it to production with 100k+ rows? what does the timeout do in this situation? or what is it doing in the = 0 situation? Thanks

so the input gets

    {
        was_handled: "false",
        was_handled: "false",
        was_handled: "false",
        was_handled: "false",
        ..........
    }

....... so on and so on

my aggregate filter looks like this

filter{
    aggregate {
        task_id => "%{was_handled}"
        code => "map['fail_count'] ||= 0; map['fail_count'] += 1;"
        push_map_as_event_on_timeout => true
        timeout_task_id_field => "was_handled"
        timeout => 0 
        timeout_tags => ['_aggregatetimeout']
        timeout_code => "event.set('multiple_fails', event.get('fail_count') > 1)"
    }

    if "_aggregatetimeout" not in [tags] {
        drop { }
    }
}

then it outputs to the new index only if timeout is set to 0 anything else the index doesn't get created

{
      fail_count: 20,
      @timestamp: ....
      @version: .....
}
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.