I followed the sample 4# of logstash aggregation guide which url is https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html#plugins-filters-aggregate-example4
The log output json is right, data have been aggregated , but nothing happen in elasticsearch, no index created. Does anyone encounter this occurrence either? Is this a bug to aggregate filter plugin? I doubt whether "aggregate push_previous_map_as_event => true" is worked.
Logstash is the newest version which is 6.1.1
I have figured out this issue by myself. Cause I index multiple types and use type condition to insure the input data to specific elasticsearch index, like
output {
if([type] == "town_info") {
elasticsearch {
hosts => "192.168.5.100:9200"
index => "town_info"
document_type => "data"
template => "/home/es/logstash-6.1.1/conf/aggregation/town_template.json"
template_name => "town_template.json"
template_overwrite => true
}
}
stdout {
codec => json_lines
}
}
But in the sample 4#, the original event has been cancelled. Therefore the "type" didn't get from the input configuration. In the "aggregate" section "map" has been instead of event, So I assign the "type" value to map, like
aggregate {
task_id => "%{country_name}"
code => "
map['country_name'] = event.get('country_name')
map['type'] = event.get('type')
map['towns'] ||=
map['towns'] << {'town_name' => event.get('town_name')}
event.cancel()
"
push_previous_map_as_event => true
timeout => 3
}
Then it works. But the blemish is there will be a field named "type" in this type index. Does anybody can tell me how to do it perfect!
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.