# Correlate log using logstash aggregate plugin

**URL:** https://discuss.elastic.co/t/correlate-log-using-logstash-aggregate-plugin/297589
**Category:** Logstash
**Created:** [February 18, 2022, 10:19am UTC](https://discuss.elastic.co/t/correlate-log-using-logstash-aggregate-plugin/297589 "2022-02-18T10:19:40Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![lusynda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusynda/32/53557_2.png) [@lusynda](https://discuss.elastic.co/u/lusynda)
#### Post date: [February 18, 2022, 10:19am UTC](https://discuss.elastic.co/t/correlate-log-using-logstash-aggregate-plugin/297589/1 "2022-02-18T10:19:40Z")

</div>

Hi all  
I'm trying to correlate multiple log file into 1 log message send to elastic using aggregate plugin in logstash.

```auto
2022-02-18T09:51:25.528Z,mail-srv\Client mail-srv,08D9DD5FF4988FD5,37,192.168.1.1:587,192.168.2.2:60095,*,,User Name: NULL
2022-02-18T09:51:25.528Z,mail-srv\Client mail-srv,08D9DD5FF4988FD5,19,192.168.1.1:587,192.168.2.2:60095,>,235 2.7.0 Authentication successful,

```

Now i want to from same id `08D9DD5FF4988FD5` i can get this two log messge into 1 log message like this

```auto
2022-02-18T09:51:25.528Z,mail-srv\Client mail-srv,08D9DD5FF4988FD5,37,192.168.1.1:587,192.168.2.2:60095,User Name: NULL,235 2.7.0 Authentication successful,

```

Or somethings like that.

I have tried this config

```auto
filter {
  aggregate {
     task_id => "%{id}"
     code => "
       map['mail_data'] ||= event.get('mail.data')
       map['mail_context'] ||= event.get('mail.context')
     "
    push_map_as_event_on_timeout => true
    timeout_task_id_field => "id"
    timeout => 30
    timeout_tags => ['_aggregatetimeout']
  }
}

```

the `mail.data` field is `User Name: NULL` and the `mail.context` is `235 2.7.0 Authentication successful`

Thanks for your time.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [February 18, 2022, 5:29pm UTC](https://discuss.elastic.co/t/correlate-log-using-logstash-aggregate-plugin/297589/2 "2022-02-18T17:29:08Z")

</div>

> [@lusynda](#):
>
> `push_map_as_event_on_timeout`

When you use this the only data in the event that aggregate creates is what you added to the map. If you want additional parts of the original message in that event then you must add them to the map.

---

<div class="post-metadata">

### Author: ![lusynda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusynda/32/53557_2.png) [@lusynda](https://discuss.elastic.co/u/lusynda)
#### Post date: [February 23, 2022, 7:39am UTC](https://discuss.elastic.co/t/correlate-log-using-logstash-aggregate-plugin/297589/3 "2022-02-23T07:39:36Z")

</div>

Ok but is that why my config does not working like at all.

---

<div class="post-metadata">

### Author: ![lusynda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusynda/32/53557_2.png) [@lusynda](https://discuss.elastic.co/u/lusynda)
#### Post date: [February 23, 2022, 9:33am UTC](https://discuss.elastic.co/t/correlate-log-using-logstash-aggregate-plugin/297589/4 "2022-02-23T09:33:34Z")

</div>

The current problems for me is that i have tried the config multiple time and it is not working for me. It is not aggregated, notthing happened.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 23, 2022, 9:34am UTC](https://discuss.elastic.co/t/correlate-log-using-logstash-aggregate-plugin/297589/5 "2022-03-23T09:34:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
