Date filter issue String to ISO Timestamp

Elastic Stack Logstash
1

Hi all,

I've got another issue with the conversion of dates using the date filter.
The main goal i'm trying to achieve is to put the date into the @timestamp field, but it seems that i'm doing something wrong.

I've got this date:

"date_created" => "2019-05-09 12:42:52.483023",

and i want to put it into the @timestamp field.

i've done a lot of thins to try this but nothing worked.
Ex:

date {
match => ["%{temp_date_created}", "YYYY-MM-dd HH:mm:ss.SSS", "ISO8601"]
target => "@timestamp"
}

mutate {
add_field => { "temp_date_created" => "%{date_created}" }
convert => { "temp_date_created" => "string" }
}

date {
match => [ "temp_date_created", "", "ISO8601"]
target => "@timestamp"
}

Any ideas?

Cheers,

2

If the field you are parsing has six digits after the . then your pattern has to have SSSSSS. Your pattern has to consume the entire field.

Also, you can remove the

, "ISO8601"
3

Hi,

I've done your recomendation, but the @timestamp remains with its own value:

       "date_created" => "2019-05-09 14:18:52.700813",

         "@timestamp" => 2019-05-09T13:21:52.997Z,

Any reason why the @timestamp remains unchanged.

Cheers,

4 
input { generator { count => 1 lines => [ '' ] } }

filter {
    mutate { add_field => { "date_created" => "2019-05-09 12:42:52.483023" } }
    date { match => ["date_created", "YYYY-MM-dd HH:mm:ss.SSSSSS"] }
}

will set @timestamp appropriately.

5

Hi Badger,

I'm doing exactly that:

mutate { 
	add_field => { "temp_date_created" => "%{date_created}" }
	convert => { "temp_date_created" => "string" }
}

date {
	match => ["%{temp_date_created}", "YYYY-MM-dd HH:mm:ss.SSSSS"]
	target => "@timestamp"
}

I've in this approach try to convert into a string, but the @timestamp remains untouchable.

     "date_created" => "2019-05-09 16:24:04.837336"
     "@timestamp" => 2019-05-09T15:24:44.054Z,

Cheers,

6

No you are not doing exactly that. You have SSSSS, you need to have SSSSSS. The pattern has to match the field it is parsing exactly.

7

Your are absolutely correct, i'm sorry for that.
But, besides this error, i still have the problem:

"temp_date_created" => "2019-05-09 16:36:52.751031",
"date_created" => "2019-05-09 16:36:52.751031",
"@timestamp" => 2019-05-09T15:37:02.512Z,

mutate { 
	add_field => { "temp_date_created" => "%{date_created}" }
}

date {
	match => ["%{temp_date_created}", "YYYY-MM-dd HH:mm:ss.SSSSSS"]
	target => "@timestamp"
}
8

You do not need a sprintf reference there. Use

 match => ["temp_date_created", "YYYY-MM-dd HH:mm:ss.SSSSSS"]
9

That's the problem.
Thank you very much for the help.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.