Understanding the aggregation filter:


(tomer zaks) #1

well I am able to aggregate that my last log has all the terms, but I can't know who will be my last log, therefore I did "push_map_as_event_on_timeout => true" (I kept all config that is not repetitive since maybe some part that I didn't think of destroys the push map.

This is supposed to print out a new log after 120 seconds isn't it? (for me it is not working)

Here is my LS config:

input {
 ...
}

filter {
  json{
        source => "message"
  }

  aggregate {
		task_id => "%{transactionId}"
		code => "
			
			if (map['authCBGot'])
				event.set('authCBGot', (map['authCBGot']))
			else 
				if (event.get('authCBGot').eql? '0')
					# do nothing
				else
					map['authCBGot'] = event.get('authCBGot')
				end
			end
			... (many more as the "authCBGot")
			"
		push_map_as_event_on_timeout => true
		timeout_task_id_field => "transactionId"
		timeout => 120
		timeout_code => "event.set('cdrType', 'aggregated')"
    
   }
   
  if [ASR]{
	  mutate {
		convert => { "ASR" => "float" }
	  }
  }
  
  date {
    match => [ "eventTimestamp", "UNIX_MS" ]
    target => "eventTimestamp"
  }
  ...(many more Date filters)
  
 (some ruby)
  if [medGotAck_MT] and [MedGotMsgFromApi] {
    ruby {
      init => "require 'time'"
      code => "
					medGotAck_MT = Time.iso8601(event.get('medGotAck_MT').to_s).to_i;
					MedGotMsgFromApi = Time.iso8601(event.get('MedGotMsgFromApi').to_s).to_i;
					event.set('delay' , medGotAck_MT - MedGotMsgFromApi);
					if (event.get('cdrType').eql? 'dlr')
						if (event.get('MedGotMsgFromApi').eql? '0')
							event.set('delay' , 120);
						else
							event.set('delay' , medGotAck_MT - MedGotMsgFromApi);
						end
					else
						event.set('delay' , medGotAck_MT - MedGotMsgFromApi);
					end
					event.set('epoch_received_at_in_seconds' , medGotAck_MT);
					event.set('epoch_timestamp_in_seconds' , MedGotMsgFromApi);
				"
			
    }
  }
  
  
	 
}

output {
    elasticsearch {
      hosts => ["192.168.1.116:9200"]
      manage_template => false
      index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
      document_type => "%{[@metadata][type]}"
  }
  


}

(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.