# Different values in new runs of logstash with Aggregate filter

**URL:** <https://discuss.elastic.co/t/different-values-in-new-runs-of-logstash-with-aggregate-filter/347451>\
**Category:** Logstash\
**Created:** [November 18, 2023, 3:36pm UTC](https://discuss.elastic.co/t/different-values-in-new-runs-of-logstash-with-aggregate-filter/347451 "2023-11-18T15:36:25Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ilia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ilia/32/60042_2.png) [@Ilia](https://discuss.elastic.co/u/Ilia)\
**Post date:** [November 18, 2023, 3:36pm UTC](https://discuss.elastic.co/t/different-values-in-new-runs-of-logstash-with-aggregate-filter/347451/1 "2023-11-18T15:36:25Z")

</div>

I've used jdbc input plugin to extract financial transactions from database. But each logical transaction is composed from multiple transactions so I used aggregate filter to merge them in one event. Here is the code of my filter:

```auto
filter {
  if [type] == 'BUY' {
    aggregate {
      task_id => "%{id}"
      push_map_as_event_on_timeout => true
      timeout_task_id_field => "id"
      timeout => 5
      timeout_tags => ['_aggregatetimeout']
      code => "
# map.merge!(event)
        if (event.get('reason') == 'NET_AMOUNT' && event.get('type') == 'DEBIT' && event.get('account') == event.get('issuer_account'))
            map[' _____ m_amount'] = event.get('amount')
            event.cancel
        elsif (event.get('reason') == 'NET_AMOUNT' && event.get('type') == 'CREDIT' && event.get('account') == event.get('issuer_account'))
            map[' _____ r_amount'] = event.get('amount')
            event.cancel
        elsif (event.get('reason') == 'COMMISSION' && event.get('type') == 'DEBIT' && event.get('account') == 1)
            map[' _____ m_commission'] = event.get('amount')
            event.cancel
        elsif (event.get('reason') == 'COMMISSION' && event.get('type') == 'CREDIT' && event.get('account') == event.get('issuer_account'))
            map[' _____ r_commission'] = event.get('amount')
            event.cancel
        end
        event.cancel
      "
      timeout_code => "event.set('testttttttttt', event.get('id'))"
    }
  }
}

```

The problem is that the ` _____ m_commission` field is never created despite its if condition is true for one case (at least one transaction met its condition). Also the values of other fields vary in new runs of logstash! It sounds the order of incoming events affect the results of the if conditions despite there is only one event related to each if statement!!

I've set `pipeline.workers: 1` and `pipeline.java_execution: false` in /etc/logstash/pipelines.yml file but nothing changed. I was suspected to the if statements and searched a lot for ruby syntax but it sounds is true .

What is wrong in this filter?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [November 18, 2023, 5:40pm UTC](https://discuss.elastic.co/t/different-values-in-new-runs-of-logstash-with-aggregate-filter/347451/2 "2023-11-18T17:40:10Z")

</div>

> [@Ilia](#):
>
> It sounds the order of incoming events affect the results of the if conditions

No, but if the order changes map entries may get overwritten with different values. I did not know that pipeline.java\_execution was still supported, but I think what you want to set is pipeline.ordered.

Without seeing the source data it is impossible to say why \_\_\_\_\_m\_commission is not created.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 16, 2023, 5:40pm UTC](https://discuss.elastic.co/t/different-values-in-new-runs-of-logstash-with-aggregate-filter/347451/3 "2023-12-16T17:40:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.