Logstash aggregation filter not working properly

Elastic Stack Logstash
1

I tried aggregating events ,it gets agrregated ,but in my output I get all lines displayed ,aggregatted lines and also the other lines. I want just the aggregated lines alone in my index.
input {
file {
path => "/opt/logs_trial/gst_session_log.log*"
#start_position => "beginning"
type => "gst_session"
}
}
filter {
if [type] == "gst_session" {
if [Status] == "start" {
aggregate {
task_id => "%{SessionId}"
code => "map['bitrate1']=0"
map_action => "create"
add_tag => [ "aggregateStart" ]
}
}
if [Status] == "play" or [Status] == "success" or [Status] == "bitrate_shift"
{
aggregate {
task_id => "%{SessionId}"
code => "map['bitrate1'] += event['bitrate']"
map_action => "update"
add_tag => [ "aggregateMiddle" ]
}
}
if [Status] == "close" {
aggregate {
task_id => "%{SessionId}"
code => "event['bitrate1'] = map['bitrate1']"
map_action => "update"
end_of_task => true
add_tag => [ "aggregateEnd" ]
timeout => 120
}
}
}

}
output {
if [type] == "gst_session"{
elasticsearch{
hosts => "10.126.250.45"
#port => "9200"
index => "gst_summary"
#protocol => "http"
}
}}

Did I miss any configuration?

1 Like
2

This is the normal aggregate filter behaviour.
aggregate filter doesn't aim to delete existing events, but to enrich the final event of each "task".
If you don't want other lines, you can use drop filter as last filter.

For example :

if "aggregateEnd" not in [tags] {
drop {}
}

1 Like
3

Thanks fbaligand . Its working fine