# Compare current event against previously one

**URL:** https://discuss.elastic.co/t/compare-current-event-against-previously-one/43176
**Category:** Logstash
**Created:** [March 1, 2016, 11:26pm UTC](https://discuss.elastic.co/t/compare-current-event-against-previously-one/43176 "2016-03-01T23:26:59Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![Thales\_Fernando](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thales_fernando/32/8193_2.png) [@Thales\_Fernando](https://discuss.elastic.co/u/Thales_Fernando)
#### Post date: [March 1, 2016, 11:26pm UTC](https://discuss.elastic.co/t/compare-current-event-against-previously-one/43176/1 "2016-03-01T23:26:59Z")

</div>

Hello Guys!

We are processing some MSSQL tables via JDBC input plugin. Some times, 3 rolls will be almost equal. Our goal is merge these lines together and transform the values that are different in an array. So, given this 3 events

`
event1 = {
  "id" => 1 ,
  "name" => "John",
  "country" => "Texas"
}
event2 = {
"id" => 1 ,
"name" => "John",
"state" => "Alaska"
}
event3 = {
"id" => 1 ,
"name" => "John",
"state" => "Oregon"
}
`

It need to be merged in something such as:

`
finalEvent = {
  "id" => 1 ,
  "name" => "John",
  "state" => ["Texas", "Alaske", "Oregon"]
}
`

And finally saved into ES.

It's possible? Another question, how to achieve two stages processing into LogStash, something as:

Event -\> UnWindKeys () -\> MergeObjectsWithSameId () -\> Save to ES?

Thanks! 🙂

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [March 2, 2016, 5:43am UTC](https://discuss.elastic.co/t/compare-current-event-against-previously-one/43176/2 "2016-03-02T05:43:17Z")

</div>

Have you seen [https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html](https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html)?

---

<div class="post-metadata">

### Author: ![Thales\_Fernando](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thales_fernando/32/8193_2.png) [@Thales\_Fernando](https://discuss.elastic.co/u/Thales_Fernando)
#### Post date: [March 2, 2016, 9:08pm UTC](https://discuss.elastic.co/t/compare-current-event-against-previously-one/43176/3 "2016-03-02T21:08:27Z")

</div>

@warkolm Wow! It's awesome! Can you help-me with some piece of code, to achieve the finalEvent? I don't know Ruby, also I'm new to LogStash. Thanks!

---

<div class="post-metadata">

### Author: ![fbaligand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fbaligand/32/5657_2.png) [@fbaligand](https://discuss.elastic.co/u/fbaligand)
#### Post date: [March 6, 2016, 2:57pm UTC](https://discuss.elastic.co/t/compare-current-event-against-previously-one/43176/4 "2016-03-06T14:57:03Z")

</div>

Unfortunaly, aggregate plugin can't help you as it works today.  
It needs a clearly identified "task end event" to push aggregate map into end task event.

In that case, we can clearly detect "task start event" : it is the first one with id=1.

To fill the need, I plan to add a new option in aggregate plugin :  
push\_previous\_map\_as\_event =\> true

When this option is activated, each time a new task is detected (in your case, each time a new id is detected), it automatically push previous aggregate map as a new event in logstash.

Using that feature in your case, you can enrich aggregate map with each db line result, and when aggregate detects a new id, it push map with all aggregated data info as new event.

---

<div class="post-metadata">

### Author: ![Thales\_Fernando](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thales_fernando/32/8193_2.png) [@Thales\_Fernando](https://discuss.elastic.co/u/Thales_Fernando)
#### Post date: [March 7, 2016, 9:01pm UTC](https://discuss.elastic.co/t/compare-current-event-against-previously-one/43176/5 "2016-03-07T21:01:19Z")

</div>

Thanks for you answer, I also answered you via email 😉 Cheers!

---

<div class="post-metadata">

### Author: ![fbaligand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fbaligand/32/5657_2.png) [@fbaligand](https://discuss.elastic.co/u/fbaligand)
#### Post date: [July 14, 2016, 6:15pm UTC](https://discuss.elastic.co/t/compare-current-event-against-previously-one/43176/6 "2016-07-14T18:15:23Z")

</div>

This option is now released in logstash-filter-aggregate version 2.2.0.

You can find an example here :

> **[logstash-plugins/logstash-filter-aggregate](https://github.com/logstash-plugins/logstash-filter-aggregate#example-3)**
>
> logstash-filter-aggregate - The aim of this filter is to aggregate informations available among several events (typically log lines) belonging to a same task, and finally push aggregated informatio...

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:48am UTC](https://discuss.elastic.co/t/compare-current-event-against-previously-one/43176/7 "2017-07-06T04:48:00Z")

</div>


