Logstash 5 beta 1: Ruby exception occurred: undefined method `[]'

asp · October 6, 2016, 12:20pm

Hi,
I am currently playing around with logstash 5 beta 1.

When I use my filter settings, which are fine with 2.3.4 I get now a ruby exception:

Ruby exception occurred: undefined method `[]'...

my filter code is as followed:

filter
{
        ruby
        {
            code =>
            "
                m = event['message'].to_s
                puts m
                event['added_field'] = 'hallo'.to_s
            "
        }
}

I used event[] to add, update or read from fields of the event.
What do I need to use in Logstash 5? Is there a syntax which is working in logstash 1.5 to 5.0 ?

asp · October 6, 2016, 12:30pm

ok, I shortly after I posted I found the solution:

github.com/elastic/logstash

New Event API proposal

opened 08:40PM - 19 Apr 16 UTC

closed 09:58PM - 13 May 16 UTC

colinsurprenant

breaking change discuss v5.0.0-beta1

As per #5140 we have decided to remove Ruby hash-like API and expose new getter …and setter in 5.0. This is a **WIP proposal** and is open for discussions. The identified **undefined behaviours** will probably be the most discussed. Lets keep in mind that this is only the new getter and setter API, other new API proposals should be in their own issue which #5140 will track. #### Ruby Event API Proposal - A **field reference** is a string using the field reference syntax which is either a bare field name string like `"foo"` or using the nested syntax `"[foo]"` or `"[foo][bar]"`. ##### Setter ``` ruby # @param fieldref [String] field reference string # @param value [Object] the value to set in that field reference # @return [Event] this event or self for chainable calls event.set(fieldref, value) ``` - Values are either string, numeric or timestamp _scalar_ values, for example: ``` ruby event.set("foo", "baz") event.set("[foo]", "zab") event.set("[foo][bar]", 1) event.set("[foo][bar]", 1.0) event.set("[@metadata][foo]", "baz") ``` - Values can be arrays or hashes or nested ``` ruby event.set("[foo][bar]", [1, 2, 3]) event.set("[foo][bar]", {"a" => 1, "b" => 2}) event.set("[foo][bar]", {"a" => 1, "b" => 2, "c" => [1, 2]}) ``` - When setting hash collections, the hash keys will become fields accessible as fieldref ``` ruby event.set("[foo][bar]", {"a" => 1, "b" => 2, "c" => [1, 2]}) event.get("[foo][bar][a]") # => 1 event.get("[foo][bar][c]") # => [1, 2] ``` - Mutating a collections after setting it in the Event has an **undefined behaviour** ``` ruby h = {"a" => 1, "b" => 2, "c" => [1, 2]} event.set("[foo][bar]", h) h["c"] = [3, 4] event.get("[foo][bar][c]") # => ????? but most probably [1, 2] ``` This behaviour is certainly up for discussion, the main idea is that if you want to set or update a value in the event you have to use the explicit setter it cannot be assumed that mutating an object that was set in the event will also update it in the event, like this: ``` ruby h = {"a" => 1, "b" => 2, "c" => [1, 2]} event.set("[foo][bar]", h) h["c"] = [3, 4] event.set("[foo][bar]", h) # or better event.set("[foo][bar][c]", [3, 4]) ``` We say it is **undefined** because it is implementation specific and could change at any time so any observed behaviour is not an API contract. ##### Getter ``` ruby # @param fieldref [String] field reference string # @return [Object] the value at this field reference or nil if none event.get(fieldref) ``` - Returned values are either string, numeric or timestamp _scalar_ values, for example: ``` ruby event.get("foo" ) # => "baz" event.get("[foo]") # => "zab" event.get("[foo][bar]") # => 1 event.get("[foo][bar]") # => 1.0 event.get("[@metadata][foo]") # => "baz" ``` - Returned values can be arrays or hashes or nested ``` ruby event.get("[foo][bar]") # => [1, 2, 3] event.get("[foo][bar]") # => {"a" => 1, "b" => 2} event.get("[foo][bar]") # => {"a" => 1, "b" => 2, "c" => [1, 2]} ``` - Mutating a collections after getting it from the Event has an **undefined behaviour** ``` ruby h = event.get("[foo][bar]") # => {"a" => 1, "b" => 2, "c" => [1, 2]} h["c"] = [3, 4] h = event.get("[foo][bar][c]) # => ????? but most probably [1, 2] ``` One way to avoid an **undefined behaviour** here would be to always return deep clones of the value at _fieldref_. This has obviously a rather high cost and I am unsure if this is a cost worth paying?

So I need to use the following code, then it works:

filter
{
        ruby
        {
            code =>
            "
                m = event.get('message').to_s
                puts m
                event.set('added_field', 'hallo'.to_s)
            "
        }
}