Logstash undefined local variable

Elastic Stack Logstash
1

Hello
I'm trying to transform timestamp to specific format YYYY-MM and occured some trouble.

I want to convert timestamp date to format YYYY-MM in specific string crea.

Here my test

echo "1524227744"|logstash -e 'filter {date { match => ["message", "UNIX"] target => crea }}' -e 'filter {ruby {code => "event.set('newfield', event.get('crea').strftime('%Y-%m').to_i)" }}'

Settings: Default pipeline workers: 4
Pipeline main started
Ruby exception occurred: undefined local variable or method `crea' for #LogStash::Filters::Ruby:0x54686cb7 {:level=>:error}
{
"message" => "1524227744",
"@version" => "1",
"@timestamp" => "2018-06-09T22:40:16.838Z",
"type" => "stdin",
"host" => "tech",
"tags" => [
[0] "_rubyexception"
]
}
Pipeline main has been shutdown
stopping pipeline {:id=>"main"}

Not working.
Any help to troubleshoot

Thank you

2

You are using single-quotes to quote a string that contains both single-quotes and double-quotes. You could insert 4 spaces before that line to preserve the quoting. But it would be easier to put the filters in a file and use -f.

3

@Badger Thank you for your feed back
but i have the same issue.

waly@tech:/etc/logstash/conf.d$ echo "1524227744"|logstash -e 'filter {date { match => ["message", "UNIX"] target => crea }}'
Settings: Default pipeline workers: 4
Pipeline main started
{
"message" => "1524227744",
"@version" => "1",
"@timestamp" => "2018-06-10T22:06:53.655Z",
"type" => "stdin",
"host" => "tech",
"crea" => "2018-04-20T12:35:44.000Z"
}

The first filter is well but when i add ruby code it failed

echo "1524227744"|logstash -e 'filter {date { match => ["message", "UNIX"] target => crea }}' -e 'filter {ruby {code => "event.set('newfield', event.get('crea').strftime('%Y-%m').to_i)" }}'

Settings: Default pipeline workers: 4
SyntaxError: (ruby filter code):1: unknown type of %string
@codeblock = lambda { |event| event.set(newfield, event.get(crea).strftime(%Y-%m).to_i) }
^
eval at org/jruby/RubyKernel.java:1079
register at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-ruby-2.0.5/lib/logstash/filters/ruby.rb:29
start_workers at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:182
each at org/jruby/RubyArray.java:1613
start_workers at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:182
run at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:136
start_pipeline at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/agent.rb

I'm not undertand your feedback with the use of single-quote
Could you help for comprehension ?

4

What I meant was that if you put this into a file called foo.conf

input { stdin {} }
filter {
    date { match => ["message", "UNIX"] target => "crea" }
    ruby {code => "event.set('newfield', event.get('crea').strftime('%Y-%m').to_i)" }
}
output { stdout { codec => rubydebug } }

Then the command

echo 1524227744 | /usr/share/logstash/bin/logstash -f foo.conf

will result in

       "message" => "1524227744",
          "tags" => [
        [0] "_rubyexception"
    ],
          "crea" => 2018-04-20T12:35:44.000Z,

Note that crea is a timestamp. However, a Logstash timestamp does not have a strftime method.

Ruby exception occurred: undefined method `strftime' for 2018-04-20T12:35:44.000Z:LogStash::Timestamp

You might be better off not using a date filter and just doing the strptime in a Ruby filter to get a datetime that does have a strftime.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.