Ruby filter plugin does not read new field values

Elastic Stack Logstash
1

How to get correct value of the field inside ruby code in logstash pipeline?
sample 1:

input { 
    elasticsearch {
		hosts 				=> "http://localhost:9200"
		index 				=> "test1"
	}
}
filter {
	mutate {
		add_field 			=> { "yearsdiff" => "10" }
		add_field			=> { "timestampdiff1" => "" }
	}
	ruby {
		code => '
			event.set("[timestampdiff1]", event.get("[yearsdiff]"));
		'
	}
}
output {
	elasticsearch {
		hosts => "http://localhost:9200/"
		index => "test1"
		action => "update"
		document_id => "%{docid}"
		doc_as_upsert => true
	}
}

Output :
"timestampdiff1" : 0

sample 2:
Same as above. Only used add_field inside ruby instead of mutate.
Output:
"timestampdiff1" : null

Expected output:
"timestampdiff1" : 10

2

Where are you planning on getting yearsdiff value from? Or is it always 10?

3

@aaron-nimocks ,
Yes, its always 10.
Actually I want to calculate ( currentYear - yearsDiff ) in timestampdiff1 field but I always get 2021.

4

If I read correctly and your goal is to get current year and subtract 10 then this how I would do it.

ruby {
 code => '
  event.set("year_diff", ((Time.now().to_s[0..3]).to_i) - 10)
 '      
}

This takes the current system time -> transforms to a string -> using substring function extract the first 4 characters which is the year -> convert back to an integer so you can do math functions -> subtract 10 -> save in new field called year_diff.

Not sure if this is the most efficient but that's just the first solution I thought of.

5

I answered a closely related question from the same poster here.

1 Like
6

@Badger , actually this question is different. I am still not able to access add_field values inside ruby code. Please suggest a way.

7

The configuration

input { generator { count => 1 lines => [ '' ] } }
filter {
    mutate {
        add_field           => { "yearsdiff" => "10" }
        add_field           => { "timestampdiff1" => "" }
    }
    ruby { code => ' event.set("[timestampdiff1]", event.get("[yearsdiff]")) ' }
}
output { stdout { codec => rubydebug { metadata => false } } }

produces

     "yearsdiff" => "10",
       "message" => "",
"timestampdiff1" => "10"
8

I dont know why but in my case exact same code is giving output as "timestampdiff1" : "0".
I am using ELK 7.12. Is this bug ?

9

I cannot think of any reason why that would happen.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.