Logstash Ruby plugin code block not raising Exception

[Colin_K]

I have a ruby block. The "happy" path works where I see the expected map output. When I provide an invalid scheme for an Array with a log line and an element that does not contain an object with field, I do not throw the exception.

For example an input log line that contains body = [{"field" => {"key1": => "value1","key2": => "value2"}]
returns mapped as expected. But an input line missing field
body = [{"notField" => "other_value"}
does not show any exception raised in logstash logs or output.

json{
source =>['body']  
target => "some"
}

    ruby {
       code => "
           if (event.get('some').kind_of?(Array))           
             if (!event.get('some')[0].nil?)
               if (!event.get('some')[0]['field'].nil?)
                  mapped = event.get('some').map {
                    |h|
                      { 
                       'some_value' => h['field']['key1']
                       'another_value' => h['field']['key2'']
                     }
                  }; 
                  event.set('mapped',mapped)
                else 
                  raise Exception.new 'Non-empty Array type without expected schema:'+ event.get('some').to_s
                end
              end  
            end
        "
    }

[Badger]

There are syntax errors in your ruby code that will cause logstash to crash. If I fix them

input { generator { count => 1 lines => [ '' ] } }
filter {
    #mutate { add_field => { "body" => '[{"field": {"key1": "value1", "key2": "value2"}}]' } }
    mutate { add_field => { "body" => '[{"field": null }]' } }
    json{ source => "body"  target => "some" }
    ruby {
        code => "
            if (event.get('some').kind_of?(Array))
                if (!event.get('some')[0].nil?)
                    if (!event.get('some')[0]['field'].nil?)
                        mapped = event.get('some').map { |h| { 'some_value' => h['field']['key1'], 'another_value' => h['field']['key2'] } };
                        event.set('mapped',mapped)
                    else
                        raise Exception.new 'Non-empty Array type without expected schema:'+ event.get('some').to_s
                    end
                end
            end
        "
    }
}
output { stdout { codec => rubydebug { metadata => false } } }

then I do get an error logged

[ERROR][logstash.filters.ruby ][main][ae9d0609338ac65d4f73d4911a57e44fce57dd39876d923c3802769de68bf55a] Ruby exception occurred: Non-empty Array type without expected schema:[{"field"=>nil}]

[Colin_K]

@Colin_K

           if (event.get('some').kind_of?(Array))           
             if (!event.get('some')[0].nil?)
               if (!event.get('some')[0]['field'].nil?)
                  mapped = event.get('some').map {
                    |h|
                      { 
                       'some_value' => h['field']['key1']
                       'another_value' => h['field']['key2'']
                     }
                  }; 
                  event.set('mapped',mapped)
                else 
                  raise Exception.new 'Non-empty Array type without expected schema:'+ event.get('some').to_s
                end
              end  
            end

Perhaps I have an issue transcribing the code here but I'm not getting a compilation error and the logstash filter is running.

Are you showing that it's required to place the map block on a single line? Can you describe in english what the Syntax Error is?

[Badger]

@Colin_K

               { 
                       'some_value' => h['field']['key1']
                       'another_value' => h['field']['key2'']
                     }

ruby requires a comma between the two entries in the hash, and you have an unmatched quote in 'key2''

Newlines within the code block are a matter of personal preference.

[Colin_K]

I agree that the comma is required. I missed that when transcribing from production code. However the issue is the exception Non-empty Array type without expected schema, and NOT a ruby syntax error.

[ERROR][logstash.filters.ruby ][main][ae9d0609338ac65d4f73d4911a57e44fce57dd39876d923c3802769de68bf55a] Ruby exception occurred: Non-empty Array type without expected schema:[{"field"=>nil}]