Aggregate push_previous_map_as_event seems not work


(Rayburn) #1

I followed the sample 4# of logstash aggregation guide which url is https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html#plugins-filters-aggregate-example4
The log output json is right, data have been aggregated , but nothing happen in elasticsearch, no index created. Does anyone encounter this occurrence either? Is this a bug to aggregate filter plugin? I doubt whether "aggregate push_previous_map_as_event => true" is worked.
Logstash is the newest version which is 6.1.1


(Rayburn) #2

I have figured out this issue by myself. Cause I index multiple types and use type condition to insure the input data to specific elasticsearch index, like

output {
if([type] == "town_info") {
elasticsearch {
hosts => "192.168.5.100:9200"
index => "town_info"
document_type => "data"
template => "/home/es/logstash-6.1.1/conf/aggregation/town_template.json"
template_name => "town_template.json"
template_overwrite => true
}
}
stdout {
codec => json_lines
}
}

But in the sample 4#, the original event has been cancelled. Therefore the "type" didn't get from the input configuration. In the "aggregate" section "map" has been instead of event, So I assign the "type" value to map, like

aggregate {
task_id => "%{country_name}"
code => "
map['country_name'] = event.get('country_name')
map['type'] = event.get('type')
map['towns'] ||= []
map['towns'] << {'town_name' => event.get('town_name')}
event.cancel()
"
push_previous_map_as_event => true
timeout => 3
}

Then it works. But the blemish is there will be a field named "type" in this type index. Does anybody can tell me how to do it perfect!


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.