Getting _dateparsefailure on date filter

Elastic Stack Logstash
1

Hi,

I'm trying to do this stuff:

date {
match => [ "date_created", "ISO8601" ]
target => "@timestamp"
}
and
date {
match => [ "date_created", "yyyy-MM-dd HH:mm:ss,SSS" ]
target => "@timestamp"
}

But everything fails.

The date_created in Elastic has this pattern:
"date_created": "2019-02-22T11:19:59.802Z"

Anyone knows what is the problem?

Cheers

2

The second one does not match, but the first one (ISO8601) should set @timestamp correctly. What exactly do you mean by "everything fails"?

3

Hi,

Like i said.
I put the ISO8601 pattern and i also get the error _dateparsefailure.

Cheers

4

It works for me.

input { generator { count => 1 message => '' } }
filter {
    mutate { add_field => { "date_created" => "2019-02-22T11:19:59.802Z" } }
    date { match => [ "date_created", "ISO8601" ] }
}
output { stdout { codec => rubydebug { metadata => false } } }

gets me

  "@timestamp" => 2019-02-22T11:19:59.802Z,
"date_created" => "2019-02-22T11:19:59.802Z",
5

This is very strange, i redirect the ouput to a file and the date_created has the same pattern as you:

{
"system_timestamp" => 2019-02-22T14:41:30.294Z,
"operation_id" => "f10c4906-36af-11e9-ba43-0a030bd70000",
"lang_code" => "",
"application_id" => 195,
"ext_origin" => "",
"message_type" => "Req",
"status_code" => "X0",
"api_organization_id" => "",
"client_id" => 1,
"@timestamp" => 2019-02-22T14:46:01.918Z,
"channel" => "",
"@version" => "1",
"approach" => "C",
"num_retries" => "",
"status_msg" => "",
"transaction_id" => "f108a7c4-36af-11e9-ba43-0a030bd70000",
"ext_host" => "",
"ext_session_id" => "",
"api_client_id" => "",
"ext_transaction_id" => "",
"application" => "",
"tags" => [
[0] "_dateparsefailure",
[1] "taskStarted"
],
"date_created" => 2019-02-22T14:41:30.280Z,
"api_transaction_id" => "",
"operation_version" => "1.0",
"communication_id" => 51453625
}

The difference from your example is that your date is presented like a string "" and mine no "date_created" => 2019-02-22T14:41:30.280Z.

This makes any sense? Can be that this difference is the cause of this behavior?

Cheers.

6

Yes, that's critical. In logstash your date_created field is already a LogStash::timestamp. A date filter cannot parse that.

The traditional workaround is

mutate { convert { "date_created" => "string" } }
7

That's it.
You solve it.

Thanks.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.