# Question about filter effects on event handling order

**URL:** <https://discuss.elastic.co/t/question-about-filter-effects-on-event-handling-order/185223>\
**Category:** Logstash\
**Created:** [June 11, 2019, 2:55pm UTC](https://discuss.elastic.co/t/question-about-filter-effects-on-event-handling-order/185223 "2019-06-11T14:55:35Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![jdmcalee](https://avatars.discourse-cdn.com/v4/letter/j/90ced4/32.png) [@jdmcalee](https://discuss.elastic.co/u/jdmcalee)\
**Post date:** [June 11, 2019, 2:55pm UTC](https://discuss.elastic.co/t/question-about-filter-effects-on-event-handling-order/185223/1 "2019-06-11T14:55:35Z")

</div>

I am trying to make a config that does a batch of indexing, then an elasticsearch filter query to find documents that need an update to be marked as stale (via a new event created in a subsequent ruby filter). It's important that my filter runs after all indexing is complete.

What I have found is that applying a filter (any filter, it seems) disrupts the expected order the events are processed. At least, the order they are output is disrupted. A simple config that demonstrates the behavior is the following:

```
input {
    generator {
        count => 1
        lines => ["1", "2"]
    }
}

filter {
    if [message] == "2" {
        mutate {
            add_field => { "foo" => "blah" }
        }
    }
}

output {
    stdout { codec => rubydebug }
}

```

With the filter active, the event for the second line (the only one the filter is applied to) is output first. With the filter disabled, the events are output in the expected order. Can anyone explain what is going on?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 11, 2019, 4:00pm UTC](https://discuss.elastic.co/t/question-about-filter-effects-on-event-handling-order/185223/2 "2019-06-11T16:00:58Z")

</div>

> [@jdmcalee](#):
>
> With the filter disabled, the events are output in the expected order. Can anyone explain what is going on?

This is one of the effects of javafication. Events can get re-ordered even when you use a single pipeline worker. You could get the old behaviour using

```
pipeline.java_execution: false

```

but I would expect that workaround to disappear in a future release.

Over on github I have seen explicit statements from elastic developers that logstash does not guarantee event order, so I don't think they will accept that this is a bug.

---

<div class="post-metadata">

**Author:** ![jdmcalee](https://avatars.discourse-cdn.com/v4/letter/j/90ced4/32.png) [@jdmcalee](https://discuss.elastic.co/u/jdmcalee)\
**Post date:** [June 11, 2019, 4:21pm UTC](https://discuss.elastic.co/t/question-about-filter-effects-on-event-handling-order/185223/3 "2019-06-11T16:21:27Z")

</div>

does that mean that the aggregate filter can't be trusted either?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 11, 2019, 4:25pm UTC](https://discuss.elastic.co/t/question-about-filter-effects-on-event-handling-order/185223/4 "2019-06-11T16:25:43Z")

</div>

That's a really good question, one which I have been wondering about for a while.

---

<div class="post-metadata">

**Author:** ![jdmcalee](https://avatars.discourse-cdn.com/v4/letter/j/90ced4/32.png) [@jdmcalee](https://discuss.elastic.co/u/jdmcalee)\
**Post date:** [June 11, 2019, 4:26pm UTC](https://discuss.elastic.co/t/question-about-filter-effects-on-event-handling-order/185223/5 "2019-06-11T16:26:51Z")

</div>

Excellent! I was going to need that filter as well for this 🙃

Thank you for clarifying this, though. It's better that I know now than find out later.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 9, 2019, 4:26pm UTC](https://discuss.elastic.co/t/question-about-filter-effects-on-event-handling-order/185223/6 "2019-07-09T16:26:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.