# Date can not parse date formated field

**URL:** <https://discuss.elastic.co/t/date-can-not-parse-date-formated-field/105218>\
**Category:** Logstash\
**Created:** [October 25, 2017, 10:16am UTC](https://discuss.elastic.co/t/date-can-not-parse-date-formated-field/105218 "2017-10-25T10:16:59Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![stevie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stevie/32/23394_2.png) [@stevie](https://discuss.elastic.co/u/stevie)\
**Post date:** [October 25, 2017, 10:17am UTC](https://discuss.elastic.co/t/date-can-not-parse-date-formated-field/105218/1 "2017-10-25T10:17:00Z")

</div>

Version: logstash 5.6.3

We have 2 date filters in our logstash config.  
The first parses the timestamp field we matched in our logs. We want to have always the same timestamp format so we use date to do this.  
Second date is used to set @timestamp to timestamp for each event, if we have a timestamp parsed. But this fails if we altered date with timestamp.

I created a little test case to reproduce the issue.

> input { stdin {} }  
> filter {  
> grok {  
> match =\> ["message", "%{TIMESTAMP\_ISO8601:timestamp} %{WORD}"]  
> }  
> # trying to fix timestamp parsing  
> date {  
> match =\> ["timestamp", "ISO8601", "UNIX", "UNIX\_MS", "TAI64N", "yyyy-MM-dd HH:mm:ss", "MMM dd YYYY HH:mm:ss", "YYYY/MM/dd HH:mm:ss", "dd/MMM/YYYY:HH:mm:ss Z"]  
> target =\> "timestamp"  
> }  
> # if we have a valid timestamp, set is as @timestamp instead of parsed time of logstash  
> date {  
> match =\> ["timestamp", "ISO8601"]  
> target =\> "@timestamp"  
> }
> 
> }  
> output { stdout { codec =\> rubydebug } }

```
# echo '2011-04-19T03:44:01.103Z blabla' | /usr/share/logstash/bin/logstash -f test_logstash.config
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
{
      "@version" => "1",
          "host" => "test-host",
    "@timestamp" => 2017-10-25T10:06:16.527Z,
       "message" => "2011-04-19T03:44:01.103Z blabla",
     "timestamp" => 2011-04-19T03:44:01.103Z,
          "tags" => [
        [0] "_dateparsefailure"
    ]
}

```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 25, 2017, 10:45am UTC](https://discuss.elastic.co/t/date-can-not-parse-date-formated-field/105218/2 "2017-10-25T10:45:15Z")

</div>

The immediate reason for the failure is that `timestamp` already is a timestamp and the date filter can only parse strings. You could use a mutate filter to convert `timestamp` to a string.

But I don't understand the difference between `@timestamp` and `timestamp`. And why do you want to copy from `timestamp` to `@timestamp`?

---

<div class="post-metadata">

**Author:** ![stevie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stevie/32/23394_2.png) [@stevie](https://discuss.elastic.co/u/stevie)\
**Post date:** [October 25, 2017, 12:30pm UTC](https://discuss.elastic.co/t/date-can-not-parse-date-formated-field/105218/3 "2017-10-25T12:30:01Z")

</div>

Yes, you are right, we should only use @timestamp.

But if we have both we need to have two date entrys for this to not run into Elasticsearch exceptions:

> [2017-10-25T12:25:37,686][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"logstash-2017.10.25\_12", :\_type=\>"rg-filter", :\_routing=\>nil}, 2017-10-25T12:25:37.665Z [node1.domain.net](http://node1.domain.net) Updating cluster description to {type=SHARDED, servers=[{address=node1.domain:port, type=SHARD\_ROUTER, roundTripTime=1.1 ms, state=CONNECTED}]  
> ], :response=\>{"index"=\>{"\_index"=\>"logstash-2017.10.25\_12", "\_type"=\>"rg-filter", "\_id"=\>"AV9TfsSSUQ4zGMlkXWni", "status"=\>400, "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse [timestamp]", "caused\_by"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"Invalid format: "1508934337.665" is malformed at "934337.665""}}}}}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 25, 2017, 1:59pm UTC](https://discuss.elastic.co/t/date-can-not-parse-date-formated-field/105218/4 "2017-10-25T13:59:10Z")

</div>

That error means that `timestamp` has been mapped as something that isn't compatible with the value 1508934337.665. What does that field contain? A timestamp? Then use the date filter to parse it.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 22, 2017, 1:59pm UTC](https://discuss.elastic.co/t/date-can-not-parse-date-formated-field/105218/5 "2017-11-22T13:59:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.