# Combining two events in one

**URL:** <https://discuss.elastic.co/t/combining-two-events-in-one/43336>\
**Category:** Logstash\
**Created:** [March 3, 2016, 5:44am UTC](https://discuss.elastic.co/t/combining-two-events-in-one/43336 "2016-03-03T05:44:39Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![vsadokhin](https://avatars.discourse-cdn.com/v4/letter/v/ec9cab/32.png) [@vsadokhin](https://discuss.elastic.co/u/vsadokhin)\
**Post date:** [March 3, 2016, 5:44am UTC](https://discuss.elastic.co/t/combining-two-events-in-one/43336/1 "2016-03-03T05:44:39Z")

</div>

Hello over there!

I'm looking for a way to combine two logs into one event. Say I have log lines that look like:

> [timestamp] [unique id] [some message]  
> ... // different format/type log messages goes here  
> [timestamp] [unique id] [a message]

I need to make one event instead of 2 combining first and last events if their unique ids matches. Other log lines between has to be processed their own way and it's something I can do with [if](https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html) statement. I wonder if [aggregate](https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html) + [drop](https://www.elastic.co/guide/en/logstash/current/plugins-filters-drop.html) filters is something I actually need to use.

Thanks in advance,  
Vasiliy

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 3, 2016, 6:33am UTC](https://discuss.elastic.co/t/combining-two-events-in-one/43336/2 "2016-03-03T06:33:57Z")

</div>

Probably the aggregate filter.

---

<div class="post-metadata">

**Author:** ![vsadokhin](https://avatars.discourse-cdn.com/v4/letter/v/ec9cab/32.png) [@vsadokhin](https://discuss.elastic.co/u/vsadokhin)\
**Post date:** [March 7, 2016, 3:18am UTC](https://discuss.elastic.co/t/combining-two-events-in-one/43336/3 "2016-03-07T03:18:48Z")

</div>

Aggregate + drop filters resolved my issue.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 7, 2016, 3:28am UTC](https://discuss.elastic.co/t/combining-two-events-in-one/43336/4 "2016-03-07T03:28:19Z")

</div>

What does the config look like? It might be helpful for someone else in the future 🙂

---

<div class="post-metadata">

**Author:** ![vsadokhin](https://avatars.discourse-cdn.com/v4/letter/v/ec9cab/32.png) [@vsadokhin](https://discuss.elastic.co/u/vsadokhin)\
**Post date:** [March 7, 2016, 3:38am UTC](https://discuss.elastic.co/t/combining-two-events-in-one/43336/5 "2016-03-07T03:38:35Z")

</div>

Consider "requestMethod" and "responseStatus" is something coming from "message". "message" is a rest part of log event after timestamp and unique id like [timestamp] [unique id] [message].

```
  if [uniqueId] =~ /.+/ {
    if [requestMethod] =~ /.+/ {
     aggregate {
       task_id => "%{uniqueId}"
       code => "map['requestContent'] = event['message']"
       map_action => "create"
       timeout => 120
     }
     drop {}
    } else if [responseStatus] =~ /.+/ {
     aggregate {
       task_id => "%{uniqueId}"
       code => "event['requestContent'] = map['requestContent']"
       map_action => "update"
       end_of_task => true
     }
    }
  }
```

---

<div class="post-metadata">

**Author:** ![Nikolay\_Shushkin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikolay_shushkin/32/13809_2.png) [@Nikolay\_Shushkin](https://discuss.elastic.co/u/Nikolay_Shushkin)\
**Post date:** [March 14, 2017, 6:40pm UTC](https://discuss.elastic.co/t/combining-two-events-in-one/43336/6 "2017-03-14T18:40:25Z")

</div>

There is a limitation in **aggregate** usage: `You should be very careful to set logstash filter workers to 1 (-w 1 flag) for this filter to work correctly otherwise events may be processed out of sequence and unexpected results will occur`

But reduce filter workers to 1 impacts performance.  
But there is additional detail is known about the input files: all unique IDs needed for aggregation are in the same file.  
Is it possible to setup logstash processing somehow with multiple workers (e.g. equal to the input files amount) so that every worker processed only one file?

If no then any advise is appreciated.

---

<div class="post-metadata">

**Author:** ![rocavalcante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rocavalcante/32/18648_2.png) [@rocavalcante](https://discuss.elastic.co/u/rocavalcante)\
**Post date:** [May 31, 2017, 9:51pm UTC](https://discuss.elastic.co/t/combining-two-events-in-one/43336/7 "2017-05-31T21:51:49Z")

</div>

Hi! can you help me please, I'm newbie in LS and FB.  
I'm using filebeat do send a log.txt to logstash and I'll save some value fields, for example:  
FB sending two events, event one contains a value field XPO and I save a variable "word1",  
event2: sending two values "NG, A000" and capture "word2=NG" and "word3=A000" , But I wish to put together the 3 information captured in a single event, how can I do it, could you show me an example please.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 31, 2017, 9:52pm UTC](https://discuss.elastic.co/t/combining-two-events-in-one/43336/8 "2017-05-31T21:52:37Z")

</div>

It would be better if you start your own thread for this request 🙂

---

<div class="post-metadata">

**Author:** ![vsadokhin](https://avatars.discourse-cdn.com/v4/letter/v/ec9cab/32.png) [@vsadokhin](https://discuss.elastic.co/u/vsadokhin)\
**Post date:** [June 1, 2017, 1:23am UTC](https://discuss.elastic.co/t/combining-two-events-in-one/43336/9 "2017-06-01T01:23:40Z")

</div>

Aggregate plugin would help you if you have something common in all 3 log messages. See my example above in original topic. I have some unique id spread throw few log messages. Using this data I can combine few logs in one.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:26am UTC](https://discuss.elastic.co/t/combining-two-events-in-one/43336/10 "2017-07-06T04:26:08Z")

</div>


