Aggregate filter only works sometimes, randomly doesn't work

Elastic Stack Logstash
1

Hi,

I'm using the aggregate function to sum together groups of lines coming from a CSV. I am using push_previous_map_as_event => true to handle the rows. All rows are grouped by a unique name, and are in the correct format required for this method (Rows are ordered in csv by unique name).

aggregate {
task_id => "%{name}"
code => "
map['name'] = event.get('name')
map['aggusablecapacity'] ||=0; map['aggusablecapacity'] += event.get('usablecapacity')
map['aggconsumedcapacity'] ||=0; map['aggconsumedcapacity'] += event.get('consumedcapacity')
map['aggavailablecapacity'] ||=0; map['aggavailablecapacity'] += event.get('availablecapacity')
event.cancel()"
push_previous_map_as_event => true
timeout=> 5
}

So here is the issue, this section works maybe 70-90% of the time, seemingly randomly. Every time I run it, some events are grouped correctly, and some are not. There doesn't seem to be any pattern as to which groups are incorrectly passed through to the output each time I run it. When a group is incorrectly passed through, it returns multiple rows of the same unique name, as they appear in the CSV.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.