Correlate log using logstash aggregate plugin

Elastic Stack Logstash
1

Hi all
I'm trying to correlate multiple log file into 1 log message send to elastic using aggregate plugin in logstash.

2022-02-18T09:51:25.528Z,mail-srv\Client mail-srv,08D9DD5FF4988FD5,37,192.168.1.1:587,192.168.2.2:60095,*,,User Name: NULL
2022-02-18T09:51:25.528Z,mail-srv\Client mail-srv,08D9DD5FF4988FD5,19,192.168.1.1:587,192.168.2.2:60095,>,235 2.7.0 Authentication successful,

Now i want to from same id 08D9DD5FF4988FD5 i can get this two log messge into 1 log message like this

2022-02-18T09:51:25.528Z,mail-srv\Client mail-srv,08D9DD5FF4988FD5,37,192.168.1.1:587,192.168.2.2:60095,User Name: NULL,235 2.7.0 Authentication successful,

Or somethings like that.

I have tried this config

filter {
  aggregate  {
     task_id => "%{id}"
     code => "
       map['mail_data'] ||= event.get('mail.data')
       map['mail_context'] ||= event.get('mail.context')
     "
    push_map_as_event_on_timeout => true
    timeout_task_id_field => "id"
    timeout => 30
    timeout_tags => ['_aggregatetimeout']
  }
}

the mail.data field is User Name: NULL and the mail.context is 235 2.7.0 Authentication successful

Thanks for your time.

2

When you use this the only data in the event that aggregate creates is what you added to the map. If you want additional parts of the original message in that event then you must add them to the map.

3

Ok but is that why my config does not working like at all.

4

The current problems for me is that i have tried the config multiple time and it is not working for me. It is not aggregated, notthing happened.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.