Aggregation task_id

Elastic Stack Logstash
1

I am trying to project the task_id to be part of my index name or to have it as fields in the aggregated events.

I try to have it as %{[task_id]} or some other forms, but without success.

Any idea how to be able to expose it?

My Grok filter is:

            grok {
                  break_on_match => true
                  keep_empty_captures => false
                  match => {
                       message => [
                             "%{NUMBER:threadid};%{TIMESTAMP_ISO8601:eventutctime};%{DATA:username};%{JAVACLASS:apiname};%{WORD:apireturnstatus};%{WORD:componentname};%{BASE10NUM:customisation-level};%{BASE10NUM:cputimeinmillisec};%{WORD:runmode};%{WORD:errorcodeiffailure}",
                             "%{NUMBER:threadid};%{TIMESTAMP_ISO8601:eventutctime};%{DATA:username};%{JAVACLASS:apiname};%{WORD:apireturnstatus};%{WORD:componentname};%{BASE10NUM:customisation-level};%{BASE10NUM:cputimeinmillisec};%{WORD:runmode};\(%{GREEDYDATA:errorcodeiffailure}\)\s*%{GREEDYDATA:error_description}"
                       ]
                  }
                 patterns_dir => "C:\DoLense\Patterns\patterns"
            }

My aggregation filter is:

                  aggregate {
                          task_id => "%{inputserver}_%{exceptiontype}_%{apiname}"
                          code => " map['totalProcessTime'] ||= 0;
                                    map['counter'] ||= 0;
                                    map['counter'] += 1;
                                    map['totalProcessTime'] += event.get('cputimeinmillisec') 
                          "
                          push_map_as_event_on_timeout  => true
                          timeout => 50
                          timeout_tags => ['_aggregatetimeout']
                          timeout_code => "event.set('avgProcessTime' , ( event.get('totalProcessTime') / event.get('counter') ) );
                                           event.set('aggregation' , 'true');
                                           event.set('entity' , 'cmserver');

event.set('aggregationfields' , '(event.get('inputserver')_event.get('exceptiontype')_event.get('apiname') )' );

                                          "
                  } # aggregate

My aggregation output is:

     if [aggregation] == "true" {
           elasticsearch  {
                 hosts => ["localhost:9200"]
                 index => "logstashaggregation%{[entity]}%{+YYYY.MM.dd}"
           }
           stdout { codec => rubydebug }
     }

Thanks
Sharon.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.