# Ruby filter processes events in incorrect order **URL:** \ **Category:** Logstash\ **Created:** [March 3, 2018, 11:36am UTC](https://discuss.elastic.co/t/ruby-filter-processes-events-in-incorrect-order/122369 "2018-03-03T11:36:28Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Heckler\_GlobalOperat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/heckler_globaloperat/32/115896_2.png) [@Heckler\_GlobalOperat](https://discuss.elastic.co/u/Heckler_GlobalOperat)\ **Post date:** [March 3, 2018, 11:36am UTC](https://discuss.elastic.co/t/ruby-filter-processes-events-in-incorrect-order/122369/1 "2018-03-03T11:36:29Z")
Here is my test configuration I create a simple list of events like below, the events are numbered according to their order of appearance > 2016-12-06 07:00:00,251 event1 > 2016-12-06 07:00:10,651 event2 > 2016-12-06 07:02:00,251 event3 > 2016-12-06 07:05:00,451 event4 In the ruby filter, I define a global variable $messageNo to keep track the number of event > ruby { > code =\> " > unless defined? $messageNo; > $messageNo = 0 > end > $messageNo += 1; > event.set('messageNo', $messageNo); > " > } In the end, I insert the processed events in Elasticsearch > output { > elasticsearch { > hosts =\> ["localhost:9200"] > index =\> "message\_index" > } > } But looking at the results in Elasticsearch (pictures attached), the events are not processed in order in Ruby. In the pictures, you can see 'event1' has messageNo 4 while 'event3' has messageNo 1. ![event1](https://us1.discourse-cdn.com/elastic/original/3X/b/f/bf29fba47a65969c238e1bf7702943ed90f48e82.JPG) ![event2](https://us1.discourse-cdn.com/elastic/original/3X/7/1/7130017f591cad88abff5081904a590022d2b177.JPG) ---
**Author:** ![eperry](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eperry/32/551_2.png) [@eperry](https://discuss.elastic.co/u/eperry)\ **Post date:** [March 3, 2018, 11:52am UTC](https://discuss.elastic.co/t/ruby-filter-processes-events-in-incorrect-order/122369/2 "2018-03-03T11:52:36Z")
This is correct, Logstash filters are batch oriented and threaded, there is no way to guarantee order in which they are processed ---
**Author:** ![Heckler\_GlobalOperat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/heckler_globaloperat/32/115896_2.png) [@Heckler\_GlobalOperat](https://discuss.elastic.co/u/Heckler_GlobalOperat)\ **Post date:** [March 3, 2018, 12:54pm UTC](https://discuss.elastic.co/t/ruby-filter-processes-events-in-incorrect-order/122369/3 "2018-03-03T12:54:54Z")
This is a problem to me ☹ I must find a way to process them in sequence ---
**Author:** ![eperry](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eperry/32/551_2.png) [@eperry](https://discuss.elastic.co/u/eperry)\ **Post date:** [March 3, 2018, 1:21pm UTC](https://discuss.elastic.co/t/ruby-filter-processes-events-in-incorrect-order/122369/4 "2018-03-03T13:21:48Z")
Why not just use the time stamp? As an order mechanism? ---
**Author:** ![Heckler\_GlobalOperat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/heckler_globaloperat/32/115896_2.png) [@Heckler\_GlobalOperat](https://discuss.elastic.co/u/Heckler_GlobalOperat)\ **Post date:** [March 3, 2018, 3:02pm UTC](https://discuss.elastic.co/t/ruby-filter-processes-events-in-incorrect-order/122369/5 "2018-03-03T15:02:42Z")
yes, the @timestamp field is in order, but I'm thinking how to control the order in which Ruby script processes those events. ---
**Author:** ![eperry](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eperry/32/551_2.png) [@eperry](https://discuss.elastic.co/u/eperry)\ **Post date:** [March 4, 2018, 1:00am UTC](https://discuss.elastic.co/t/ruby-filter-processes-events-in-incorrect-order/122369/6 "2018-03-04T01:00:26Z")
What are you trying to accomplish? you will have to write some other tool as Logstash is not going to be able to do what you want. I think you may have to rethink how to get what you need. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [March 4, 2018, 7:00pm UTC](https://discuss.elastic.co/t/ruby-filter-processes-events-in-incorrect-order/122369/7 "2018-03-04T19:00:58Z")
Have you tried setting the number of pipeline workers to one? ---
**Author:** ![Heckler\_GlobalOperat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/heckler_globaloperat/32/115896_2.png) [@Heckler\_GlobalOperat](https://discuss.elastic.co/u/Heckler_GlobalOperat)\ **Post date:** [March 5, 2018, 3:03pm UTC](https://discuss.elastic.co/t/ruby-filter-processes-events-in-incorrect-order/122369/8 "2018-03-05T15:03:08Z")
Thank you, it works !! Reducing the number of workers to 1 allows it to process the events in sequence 🙂 I also find a good tutorial about creating a custom filter to keep track of the event order - [https://stackoverflow.com/questions/23920655/include-monotonically-increasing-value-in-logstash-field/23921517#23921517](https://stackoverflow.com/questions/23920655/include-monotonically-increasing-value-in-logstash-field/23921517#23921517) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 2, 2018, 3:03pm UTC](https://discuss.elastic.co/t/ruby-filter-processes-events-in-incorrect-order/122369/9 "2018-04-02T15:03:11Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.