# Logstash aggregate/mutate

**URL:** <https://discuss.elastic.co/t/logstash-aggregate-mutate/237876>\
**Category:** Logstash\
**Created:** [June 19, 2020, 9:04pm UTC](https://discuss.elastic.co/t/logstash-aggregate-mutate/237876 "2020-06-19T21:04:13Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![pixelinc](https://avatars.discourse-cdn.com/v4/letter/p/51bf81/32.png) [@pixelinc](https://discuss.elastic.co/u/pixelinc)\
**Post date:** [June 19, 2020, 9:04pm UTC](https://discuss.elastic.co/t/logstash-aggregate-mutate/237876/1 "2020-06-19T21:04:13Z")

</div>

Hey all, I'm attempting to combine take 2 logs and add a field from a previous log to the other log.  
I've read around about aggregate and mutate, and all that but I'm not quite understanding how to do this.

I have 2 logs like so:

```auto
{"event":"eventX","id":"123456","name":"abcd"}
{"event":"eventZ","id":"123456","field":"value","otherfield":"another_value"}

```

These two things have an id in common, along with other log lines but I only want to add the `name` field from eventX, to the eventZ log line so it would all come together as:

```auto
{"event":"eventZ","id":"123456","field":"value","otherfield":"another_value", "name": "abcd"}

```

I figure I could use memorize, or maybe aggregate/mutate and add\_field to do this but I can't figure out how to do so correctly.

Any help is appreciated, thanks!

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 19, 2020, 9:29pm UTC](https://discuss.elastic.co/t/logstash-aggregate-mutate/237876/2 "2020-06-19T21:29:34Z")

</div>

You can do that with an aggregate filter. If the two lines are always in that order then do something like [example 1](https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html#plugins-filters-aggregate-example1).

If the order can vary then something similar to example 3. Note that only things you put into the map are added to the event that is created after the timeout is triggered.

Make sure pipeline.workers set to 1 and you may need to disable [java\_execution](https://github.com/elastic/logstash/issues/10938).

---

<div class="post-metadata">

**Author:** ![pixelinc](https://avatars.discourse-cdn.com/v4/letter/p/51bf81/32.png) [@pixelinc](https://discuss.elastic.co/u/pixelinc)\
**Post date:** [June 19, 2020, 10:39pm UTC](https://discuss.elastic.co/t/logstash-aggregate-mutate/237876/3 "2020-06-19T22:39:50Z")

</div>

Righto, so I've got this so far.

```auto
filter {
   json {
     source => "context"
     target => "json_data"
   }

   if [json_data.event] == "eventX" {
     aggregate {
       task_id => "%{json_data.id}"
         code => "
           if json_data['name'].is_a?(String)
             map['name'] = json_data['name']
           end
         "
         map_action => "create"
     }
   }

   if [json_data.event] == "eventZ" {
     aggregate {
       task_id => "%{json_data.id}"
         code => "
         if map['name'].is_a?(String)
           event.set('name', map['name'])
         end
         "
         map_action => "create"
         end_of_task => true
         timeout => 10
     }
   }
 }

```

For reference, `context` is an inner json object, contains all the data from the above post.  
So,  
`{"field": "value", "context": { "name": "abcd", ...`  
In this case, I get `class org.jruby.RubyHash cannot be cast to class org.jruby.RubyIO` which I know what it means, but don't know why it's happening.

(EDIT: I've also tried `json.name` as well, to no avail in case that might've been an issue.

Sorry for the ignorance.

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [June 20, 2020, 1:59am UTC](https://discuss.elastic.co/t/logstash-aggregate-mutate/237876/4 "2020-06-20T01:59:00Z")

</div>

Your syntax to access your field is wrong: [https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#logstash-config-field-references](https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#logstash-config-field-references)  
And just like you use `event.set(…)` to create/update field values with ruby, you need to use `event.get(…)` to read them.  
And your map\_action cannot be "create" if you want to work with an already existing map.  
(And if you don't want to keep both the enriched eventZ and eventX, you'll need to cancel eventX)

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 20, 2020, 3:51pm UTC](https://discuss.elastic.co/t/logstash-aggregate-mutate/237876/5 "2020-06-20T15:51:31Z")

</div>

> [@pixelinc](#):
>
> class org.jruby.RubyHash cannot be cast to class org.jruby.RubyIO

What produces that exception?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 18, 2020, 3:51pm UTC](https://discuss.elastic.co/t/logstash-aggregate-mutate/237876/6 "2020-07-18T15:51:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.