Logstash Crashes on Ruby filter when I change a field name

[ChrisW]

Logstash version: 6.4.2

I have this code for a ruby in filter {...}:

    ruby {
            code => 'event.set("logstash_time_difference", (((event.get("[received_at]").to_f - event.get("[@timestamp]").to_f).abs)*1000).round(0));'

}

This code works just fine. I have no problem with it.

I wanted the change the name of the field to: "log_received_time_diff_ms"

However when i do that and change the above code to:
ruby {
code => 'event.set("log_recieved_time_diff_ms", (((event.get("[received_at]").to_f - event.get("[@timestamp]").to_f).abs)*1000).round(0));'
}

Logstash fails to process anything and keeps crashing filling the logs with the following:

[2019-01-30T17:40:19,651][ERROR][logstash.pipeline ] Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash. {:pipeline_id=>"main", "exception"=>"undefined method >=' for nil:NilClass", "backtrace"=>["(eval):759:inblock in initialize'", "org/jruby/RubyArray.java:1734:in each'", "(eval):757:inblock in initialize'", "(eval):307:in block in filter_func'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:341:infilter_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:320:in worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:286:inblock in start_workers'"], :thread=>"#<Thread:0x4ac4fc87 sleep>"}

The above repeats for about a few dozen times and then ends with this:

[2019-01-30T17:40:19,795][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<NoMethodError: undefined method >=' for nil:NilClass>, :backtrace=>["(eval):759:inblock in initialize'", "org/jruby/RubyArray.java:1734:in each'", "(eval):757:inblock in initialize'", "(eval):307:in block in filter_func'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:341:infilter_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:320:in worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:286:inblock in start_workers'"]}
[2019-01-30T17:40:19,854][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

At which point logstash restarts.

If I change that original line to any other field name but "logstash_time_difference" even just deleting a single letter or adding a letter, this problem occurs.

If I change back to that field it works again.

Also, I can change the math in the line and it seems to have no problem, it appears to only keep failing changing the field name.

I can't tell what is wrong based on these entries.
Can anyone give some advice?

Edit:

I also tried making the new field with add_field above this line, but it didn't change anything.

Thanks.

[Badger]

If you run with the following as the complete configuration do you get the error?

input { generator { count => 1 message => '' } }
filter {
    mutate { add_field => { "received_at"=> 1548972119 } }
    ruby {
        code => '
            event.set("log_received_time_diff_ms",
                (((event.get("[received_at]").to_f - event.get("[@timestamp]").to_f).abs)*1000).round(0));
        '
    }
}
output { stdout {} }

[ChrisW]

I tried the config you provided and it worked with no errors. I got the output on stdout with the calculated field.

Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties
{
"log_received_time_diff_ms" => 99741054,
"@timestamp" => 2019-01-30T18:19:37.946Z,
"host" => "host1-log",
"message" => "",
"@version" => "1",
"received_at" => "1548972119",
"sequence" => 0
}

[Badger]

OK, so you need to find the minimal configuration that can reproduce the issue and show that to us.

[ChrisW]

Slowly removing things to try and find the problem resolved it. Thank you.

The problem ended up being the field was being referenced later on in the filter config and then again in the output config.

And the name change wasn't initially reflected everywhere and this is why we were getting the errors.

I got confused with all the ruby references in the error listing; and that we have experienced a misnamed field name before but didn't get this error.

Thanks.