# Logstash 6.0 breaks ruby code if running more than 1 worker

**URL:** <https://discuss.elastic.co/t/logstash-6-0-breaks-ruby-code-if-running-more-than-1-worker/109083>\
**Category:** Logstash\
**Created:** [November 24, 2017, 9:34pm UTC](https://discuss.elastic.co/t/logstash-6-0-breaks-ruby-code-if-running-more-than-1-worker/109083 "2017-11-24T21:34:18Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![vb4t](https://avatars.discourse-cdn.com/v4/letter/v/a698b9/32.png) [@vb4t](https://discuss.elastic.co/u/vb4t)\
**Post date:** [November 24, 2017, 9:34pm UTC](https://discuss.elastic.co/t/logstash-6-0-breaks-ruby-code-if-running-more-than-1-worker/109083/1 "2017-11-24T21:34:18Z")

</div>

I upgraded Logstash to 6.0. After the upgrade a lot of ruby code stopped to work. Following code worked in Logstash 5, but in Logstash 6 it ofter causes these errors:  
Ruby exception occurred: org.jruby.RubyNil cannot be cast to org.jruby.RubyMatchData

Code:

```
  # fixes JSON before processing in LS and indexing in ES ...
  # lowercase json keys, replace invalid characters in json keys, in case of object value, add _o suffix to json keys
  ruby {
    code => '
      nlog_str_inarr = event.get("json_message").split("")
      nlog_str_outarr = []
      nlog_str_outarr_buff = []
      nlog_str_outarr_buffenabled = false
      nlog_key_prefixmatch = ["", "", ""]
      nlog_inside_key = false
      nlog_str_inarr.each { |c|
        if c !~ /^\s$/
          nlog_key_prefixmatch[0] = nlog_key_prefixmatch[1]
          nlog_key_prefixmatch[1] = nlog_key_prefixmatch[2]
          nlog_key_prefixmatch[2] = c
       end
        # detect key values
        if ! nlog_inside_key
          if nlog_key_prefixmatch[2] == "\"" && (nlog_key_prefixmatch[1] == "{" || nlog_key_prefixmatch[1] == "," || nlog_key_prefixmatch[1] == "")
            nlog_inside_key = true
          end
        else
          if nlog_key_prefixmatch[2] == "\"" && nlog_key_prefixmatch[1] != "\\"
            nlog_inside_key = false
            nlog_str_outarr_buffenabled = true
         else
            c.downcase!
            if c !~ /^[a-zA-Z0-9_-]$/
              c = "_"
            end
          end
        end
        if ! nlog_str_outarr_buffenabled
          nlog_str_outarr.push(c)
        else
          # buffering until reach the value, after that determine if the value is object and in this case add to the key "_o" suffix
          nlog_str_outarr_buff.push(c)
          if nlog_key_prefixmatch[1] == ":"
            if nlog_key_prefixmatch[2] == "{"
              nlog_str_outarr.push("_o")
            end
            nlog_str_outarr_buff.each { |bc|
              nlog_str_outarr.push(bc)
            }
            nlog_str_outarr_buff = []
            nlog_str_outarr_buffenabled = false
          end
        end
      }
      event.set("json_message_parsed", nlog_str_outarr.join())
    '
  }

```

Am I doing something wrong? Why is this code not running in LS 6.0 with more than 1 worker? Is that a ruby bug?

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [November 25, 2017, 9:26am UTC](https://discuss.elastic.co/t/logstash-6-0-breaks-ruby-code-if-running-more-than-1-worker/109083/2 "2017-11-25T09:26:40Z")

</div>

It is not a Ruby bug, its a bug in Logstash.

Please file an issue in [https://github.com/elastic/logstash/issues](https://github.com/elastic/logstash/issues)  
Please add the info you have here plus the full error stacktrace. You should label the issue with v6.0.0

---

<div class="post-metadata">

**Author:** ![vb4t](https://avatars.discourse-cdn.com/v4/letter/v/a698b9/32.png) [@vb4t](https://discuss.elastic.co/u/vb4t)\
**Post date:** [November 25, 2017, 11:15am UTC](https://discuss.elastic.co/t/logstash-6-0-breaks-ruby-code-if-running-more-than-1-worker/109083/3 "2017-11-25T11:15:08Z")

</div>

I created issue [https://github.com/elastic/logstash/issues/8727](https://github.com/elastic/logstash/issues/8727), I have no permission to set any label on it.

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [November 25, 2017, 3:07pm UTC](https://discuss.elastic.co/t/logstash-6-0-breaks-ruby-code-if-running-more-than-1-worker/109083/4 "2017-11-25T15:07:35Z")

</div>

Ok. Thanks, I added the label just now. 🙂

As `MatchData` is associated with regular expressions, as an attempted workaround, you might like to try reversing the regex tests - its valid ruby to do so.  
e.g.

```auto
if /^\s$/ !~ c

```

and

```auto
if /^[a-zA-Z0-9_-]$/ !~ c

```

Post back here if that helps.

Also are you able to patch the ruby filter code to get it to print the backtrace with the error?  
The file is here:  
`logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-ruby-3.0.4/lib/logstash/filters/ruby.rb`  
Find line 46 and change it to:

```auto
@logger.error("Ruby exception occurred", "exception" => e.message, "backtrace" => e.backtrace)

```

If you decide to do this, please add the backtrace and your full config to the issue.  
I'm not sure we can recreate the bug without sufficient sample data i.e. enough for two batches so multiple workers will actually invoke the ruby filter.

---

<div class="post-metadata">

**Author:** ![vb4t](https://avatars.discourse-cdn.com/v4/letter/v/a698b9/32.png) [@vb4t](https://discuss.elastic.co/u/vb4t)\
**Post date:** [November 27, 2017, 6:41am UTC](https://discuss.elastic.co/t/logstash-6-0-breaks-ruby-code-if-running-more-than-1-worker/109083/5 "2017-11-27T06:41:39Z")

</div>

Reversing the regex did not help.  
I modified the ruby.rb and added the trace to the [https://github.com/elastic/logstash/issues/8727](https://github.com/elastic/logstash/issues/8727) comment,  
What configuration files do you need - logstash.yml or full pipeline conf files? Pipeline contains multiple config files (15 filters) and some configuration contain confidental information. Also the sample data is confidental - I could provide one anonymized input message but probably it would not be very helpful.

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [November 27, 2017, 10:36am UTC](https://discuss.elastic.co/t/logstash-6-0-breaks-ruby-code-if-running-more-than-1-worker/109083/6 "2017-11-27T10:36:49Z")

</div>

I can recreate the bug - no need for more info from you.

Thanks for the bug report.

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [November 27, 2017, 4:13pm UTC](https://discuss.elastic.co/t/logstash-6-0-breaks-ruby-code-if-running-more-than-1-worker/109083/7 "2017-11-27T16:13:33Z")

</div>

@vb4t

See my comment - [https://github.com/elastic/logstash/issues/8727#issuecomment-347225889](https://github.com/elastic/logstash/issues/8727#issuecomment-347225889)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 25, 2017, 4:13pm UTC](https://discuss.elastic.co/t/logstash-6-0-breaks-ruby-code-if-running-more-than-1-worker/109083/8 "2017-12-25T16:13:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
