Ruby exception occurred: -1

Elastic Stack Logstash
1

I'm running Elastic stack 5.6.3. Since this version or 5.6.2. I'm getting random LS error:
[2017-11-09T07:50:24,829][ERROR][logstash.filters.ruby ] Ruby exception occurred: -1

I've tried to recreate it, but as it looks like it appears at random.

Is there a way to get more a wider error message?
Has anyone an idea what could cause this?

2

What does your ruby filter look like?

3 
ruby {
		code => "
			a = event.get('message').split('|').delete_if{|x| !x.match(/=/)}
			a.each {|y| b = y.split('=', 2)
				event.set(b[0].strip, b[1])
			}
			event.set('acronym', event.get('acronym').upcase)"
	}

To explain my ruby code:

4

And what does an event that this ruby filter has problems with look like?

5

I've couldn't pin point it. I'm leaning towards that this happens on random.
I had created a new index and inserted data from a day that I found in the logs and nothing happened.
In the temp index I've altered the ruby filter to this:

ruby {
		code => "
			a = event.get('message').split('|').delete_if{|x| !x.match(/=/)}
			a.each {|y| b = y.split('=', 2)
				event.set(b[0].strip, b[1])
			}
			begin
				event.set('acronym', event.get('acronym').upcase)
			rescue
				event.set('acronym', 'ERROR')
			end"
	}

Acronym is in my case a required parameter, so it will allways be there.

Here's an example of my logline:
|cir=C2|date=13.11.2017 15:26:00|acronym=MKL|libraryCode=55851|user=someuser|type=11|transactionHostDepartment=45|membIdentificNumb=4200699|patronId=4200699|inventoryNo=90171047239|cobissId=292179456|note=W|patronCategory=006|patronEducation=7|gender=2|busStopId=18|district=023|lastVisitDate=13.11.2017|libraryDept=42|firstsignUpDate=07.03.2016|bibl001c=m|biblUDK675s=82|biblLanguage101a=slv|biblType001b=a|biblTargetAudienceCode100e=m|parentDepartment=45|materialType=01|loanDate=13.11.2017|returnDate=04.12.2017

Should I run my altered ruby code on my primary index or is there another way to catch this.

6

If you get rid of the leading |, why not parse this with a kv filter instead?

kv {
  field_split => "|"
}
7

Leading | was not the problem with kv filter.
In my second post, there's a link to the topic which explains my problem with kv.
To sum up:
The problem was, that my value could contain = and kv is not smart enough to just take the first = and after that take all for the value part.

8

OK, missed that thread.

9

For the example log you mentioned doesn't the following work

kv {
                    source => "message"
                    field_split => "|"
                    value_split => "="
    }
10

As mentioned above, you cannot split this with kv:
cir=C3|date=18.07.2012 09:05:57|acronym=BS|… |firstsignUpDate=30.07.2007|bibl001c=m|biblUDK675s=(038)33=111=163.6|…

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.