Ruby filter to convert date

Elastic Stack Logstash
1

Hi there.

In my Logstash configuration I need a Ruby filter to convert date from a given collection of strings.

My collection uses an structure with 3 values containing a field name, it's value and it's data type:

"Colecs": [
		{
			"Value": {
				"Value": "Tester"
			},
			"FieldName": {
				"Value": "Name"
			},
			"Type": {
				"Value": "String"
			}
		},
		{
			"Value": {
				"Value": "7/17/2019 3:58:31 PM"
			},
			"FieldName": {
				"Value": "DateOfBirth"
			},
			"Type": {
				"Value": "DateTime"
			}
		}
	],

This is my Ruby filter:

ruby{
	code => "

		require 'date'
		require 'time'

		event.get('[Colecs]').each do |item|

			varValue = item['Value']['Value']
			varFieldName = item['FieldName']['Value']
			varType = item['Type']['Value']

			if varType == 'Integer'
				event.set( varFieldName, varValue.to_i)
			elsif varType == 'Float'			
				event.set( varFieldName, varValue.to_f)

			elsif varType == 'DateTime'

				event.set( varFieldName, { 'date' => varValue} )    <--- This is where I convert dates

			else
				event.set( varFieldName, varValue.to_s)
			end
		end
	"
}

Leaving the code as this will produce me the date as string. I could use the Logstash date filter to convert it to date but I don't know how I can do it because after running the Ruby filter what I get is a field with the FieldName, but it must be generic, I'll never know it's name.
Just to clarify, the above code will produce a field in the root of my JSON as:

"DateOfBirth": {
		"date": "2019-07-17T15:58:31.000Z"
	},

I don't know the "DateOfBirth" name to use it on date filter.



In the Ruby filter, I have tried to convert date as the following:

.to_datetime

event.set( varFieldName, { 'date' => varValue.to_datetime} )

Result: I get this following error on Logstash log

[ERROR][logstash.filters.ruby    ] Ruby exception occurred: undefined method `to_datetime' for "7/17/2019 3:58:31 PM":String

Using time conversion

varDateStringToDate = Time.strptime(varValue, '%m/%d/%Y %H:%M:%S %P')
event.set( varFieldName, { 'date' => varDateStringToDate.to_datetime} )

Result: I get this following error on Logstash log

[ERROR][logstash.filters.ruby    ] Ruby exception occurred: Missing Converter handling for full class name=org.jruby.RubyObjectVar3, simple name=RubyObjectVar3

Can someone help me please?

2

I do not understand what you are trying to do with .to_datetime. If you want it to be a LogStash::Timestamp you can just

event.set( varFieldName, { 'date' => varDateStringToDate} )
1 Like
3

Thank you @Badger , it was just that simple.
Now it's working :slightly_smiling_face:

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.