# \_dateparsefailure, can't debug further
**URL:** https://discuss.elastic.co/t/dateparsefailure-cant-debug-further/198119
**Category:** Logstash
**Created:** [September 4, 2019, 11:14pm UTC](https://discuss.elastic.co/t/dateparsefailure-cant-debug-further/198119 "2019-09-04T23:14:12Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@marinipete](https://discuss.elastic.co/u/marinipete)
#### Post date: [September 4, 2019, 11:14pm UTC](https://discuss.elastic.co/t/dateparsefailure-cant-debug-further/198119/1 "2019-09-04T23:14:13Z")
So I've been creating logstash pipelines for a while now and this is the first time I came across an issue I couldn't find a solution online or debug further, so I decided to post here.
I'm getting '\_dateparsefailure' in my output tags while trying to parse a simple **ISO8601** timestamp from a json I'm receiving from AWS Kinesis, that's my pipeline:
```
input {
kinesis {
kinesis_stream_name => "es-tractor-osb-data-prd"
application_name => "logstash-kinesis-poc-v2"
region => "us-east-1"
profile => "default"
codec => json { }
}
}
filter {
date {
match => ["@timestamp", "ISO8601"]
locale => "en-US"
timezone => "America/Sao_Paulo"
target => "newtimestamp"
}
}
output {
if "erro" in [metricType] {
if [component] == "CheckoutV2/CheckoutSOAPV2" {
file {
path => "/home/ec2-user/logstash/newosb-output-kinesis-all-to-emr-error"
enable_metric => false
}
}
}
if [component] == "CheckoutV2/CheckoutSOAPV2" {
file {
path => "/home/ec2-user/logstash/newosb-output-kinesis-all-to-emr"
enable_metric => false
}
}
}
```
Sample input:
`{"component":"CheckoutV2/CheckoutSOAPV2","@timestamp":"2019-09-04T22:16:00.230Z"}`
What I find strange is that this works:
```
[root]$echo "2019-09-04T22:16:00.230Z" | bin/logstash -e 'input { stdin {} } filter { date { match => ["message", "ISO8601"] locale => "en-US" timezone => "America/Sao_Paulo" target => "newtimestamp" } }'
{
"@version" => "1",
"host" => "ip-10-2-9-142.ec2.internal",
"newtimestamp" => 2019-09-04T22:16:00.230Z,
"message" => "2019-09-04T22:16:00.230Z",
"@timestamp" => 2019-09-04T23:10:17.979Z
}
```
I've already set logstash's log do _trace_ but there is not a single dateparsefailure hint in the logs.
Trying to match "YYYY-MM-dd'T'H:mm:ss.SSS'Z'" results in the same problem. No 'newtimestamp' field comes up in my output file and I get the dateparsefailre in the tags.
Any help would be greatly appreciated.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [September 4, 2019, 11:37pm UTC](https://discuss.elastic.co/t/dateparsefailure-cant-debug-further/198119/2 "2019-09-04T23:37:08Z")
A date filter parses a string. It [cannot](https://github.com/logstash-plugins/logstash-filter-date/issues/95) parse a LogStash::Timestamp. The @timestamp field is special. If a json filter sees a field called @timestamp it will try to parse it into a LogStash::Timestamp...
```
input { generator { count => 1 lines => ['{"component":"CheckoutV2/CheckoutSOAPV2","@timestamp":"2019-09-04T22:16:00.230Z"}'] } }
filter { json { source => "message" } }
output { stdout { codec => rubydebug } }
```
results in
```
"@timestamp" => 2019-09-04T22:16:00.230Z,
```
Note that there are no double quotes around the value of @timestamp, which is how you can tell it is a LogStash::Timestamp and not a string.
---
### Author:  [@marinipete](https://discuss.elastic.co/u/marinipete)
#### Post date: [September 5, 2019, 8:54pm UTC](https://discuss.elastic.co/t/dateparsefailure-cant-debug-further/198119/3 "2019-09-05T20:54:21Z")
Hello Badger, thanks for the insight. I did consider this before, so I tried to copying (with mutate filter) the value of @timestamp to another field, then applying the date parse on that field, but got the same result.
Also, the json I'm receiving from Kinesis has another field called timestamp (without the '@'), the value of this field has no double quotes, so I guess this is the field Logstash _considering_ as a timestamp ?
Input json:
`{"flowId":"data","host":"data","metadata":"data","componentType":"data","metricType":null,"messageType":"data","component":"data","@timestamp":"2019-09-04T22:57:19.233Z","payload":null,"way":"data","instanceId":"data","timestamp":1567637837587,"@version":"1"}`
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [September 5, 2019, 8:56pm UTC](https://discuss.elastic.co/t/dateparsefailure-cant-debug-further/198119/4 "2019-09-05T20:56:30Z")
> [@marinipete](#):
>
> I tried to copying (with mutate filter) the value of @timestamp to another field, then applying the date parse on that field, but got the same result.
It's type will not change when you copy it. You could mutate+convert it to be a string, and then parse it, but why would you want to that? It is already parsed!
---
### Author:  [@marinipete](https://discuss.elastic.co/u/marinipete)
#### Post date: [September 5, 2019, 9:41pm UTC](https://discuss.elastic.co/t/dateparsefailure-cant-debug-further/198119/5 "2019-09-05T21:41:52Z")
I tried using the mutate+convert strategy but still got \_dateparsefailure:
```
filter {
mutate {
copy => {"@timestamp" => "timestamp" }
convert => { "timestamp" => "string" }
}
date {
match => ["timestamp", "ISO8601"]
locale => "en-US"
timezone => "America/Sao_Paulo"
target => "timestamp"
}
}
```
> but why would you want to that? It is already parsed!
Basically, I'm sending this output to a Big Data Platform that is expecting to receiving this input:
`{"@timestamp":"2019-08-26T23:48:31.931Z","timestamp":"2019-08-26T20:48:30.347-03:00"}`
So I'm trying to convert the bellow, into the above:
`{"@timestamp":"2019-08-26T23:48:31.931Z","timestamp":1567637837587}`
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [September 5, 2019, 10:37pm UTC](https://discuss.elastic.co/t/dateparsefailure-cant-debug-further/198119/6 "2019-09-05T22:37:07Z")
A mutate filter does operations in a [fixed order](https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-proc_order) and convert comes before copy, so the timestamp field would not exist when the convert executes, so it is a no-op. Split the mutate into two filters.
---
### Author:  [@marinipete](https://discuss.elastic.co/u/marinipete)
#### Post date: [September 5, 2019, 11:04pm UTC](https://discuss.elastic.co/t/dateparsefailure-cant-debug-further/198119/7 "2019-09-05T23:04:49Z")
Wow, didn't know about the fixed order of the operations.
Thanks a lot for the help Badger, that did it! 🥂
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 3, 2019, 11:04pm UTC](https://discuss.elastic.co/t/dateparsefailure-cant-debug-further/198119/8 "2019-10-03T23:04:50Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.