Compute duration between two ISO8601 fields in the same event

Elastic Stack Logstash
1

I have an oracle ETL table that I am polling with the JDBC input and wanted to add a field as a calculation between the start and end time of the event which is logged in the same event.

Here is what I've tried:

filter {
	if [type] == "etl" {
		mutate {
			id => "add_etl_index"
			add_field => { "[@metadata][index]" => "active-etl" }
		}
		date {
			id => "parse_etl_last_dttm"
			match => [ "last_dttm", "ISO8601" ]
		}
		date {
			id => "parse_etl_end_date"
			match => [ "end_date", "ISO8601" ]
			target => "end_date"
		}
		ruby {
			id => "add_etl_duration"
			code => "event.set('[duration]', event.get('[end_date]') - event.get('[last_dttm]'))"
		}
	}
}

But this produces the following Logstash error per event ingested:

[ERROR][logstash.filters.ruby    ] Ruby exception occurred: no implicit conversion to rational from string

The event is still pushed to elasticsearch but without the desired "duration" field. Any help is most apprecieated.

2

last_dttm is still a string. You're missing target => "last_dttm" in your first date filter.

3

Thanks @magnusbaeck. I had actually noticed that but my last_dttm was still getting indexed as a date type so I didn't think that was the cause but now I see thats only because I set it to the date type in the index template.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.