# My timestamp can't get converted to a date type **URL:** \ **Category:** Logstash\ **Created:** [May 29, 2017, 3:50pm UTC](https://discuss.elastic.co/t/my-timestamp-cant-get-converted-to-a-date-type/87476 "2017-05-29T15:50:06Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![gregbk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gregbk/32/80289_2.png) [@gregbk](https://discuss.elastic.co/u/gregbk)\ **Post date:** [May 29, 2017, 3:50pm UTC](https://discuss.elastic.co/t/my-timestamp-cant-get-converted-to-a-date-type/87476/1 "2017-05-29T15:50:06Z")
Hello, I have some working timestamp detected as date in elasticsearch. But can't make this one works. Grok is matching well, and logstash is parsing well the line: `2017-05-29 10:27:41,756 | soft-1 | user1 | | 10.x.x.x | INFO | intranet | Order 9979a662-b42a-414d-9d5c-5cxxxxxxxx creation TESTING request is finished` ``` filter { match => { "message" => [ "^%{TIMESTAMP_ISO8601:[log][timestamp]}\s*\|\s*%{GREEDYDATA:[log][message]}$" ] } } mutate { convert => { "[log][timestamp]" => "date" } } date { match => ["[log][timestamp]", "yyyy-MM-dd HH:mm:ss,SSS" ] } } ``` For some reason, [log][timestamp] appears as a string, even after deleting everything, or forcing to mutate to date. How to troubleshoot that? Using logstash debug doesn't show me the resulting type of data (I tried in debug/verbose mode): `output { stdout { codec => rubydebug } }` Thank you for any help! Greg. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 29, 2017, 6:59pm UTC](https://discuss.elastic.co/t/my-timestamp-cant-get-converted-to-a-date-type/87476/2 "2017-05-29T18:59:02Z")
Please show what `stdout { codec => rubydebug }` produces. What do the mappings of your ES index look like? ---
**Author:** ![gregbk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gregbk/32/80289_2.png) [@gregbk](https://discuss.elastic.co/u/gregbk)\ **Post date:** [May 30, 2017, 7:26am UTC](https://discuss.elastic.co/t/my-timestamp-cant-get-converted-to-a-date-type/87476/3 "2017-05-30T07:26:42Z")
Thank you for taking time helping me! Here is the output of stdout: ![](https://us1.discourse-cdn.com/elastic/original/3X/b/6/b63949a08bdc65ab752c8fab13d266b92b6f937c.png) From this output, is there a way to see the format of the field log.timestamp? Only test I can do is to send to elastic and check the field type over there: ``` { "filebeat-2017.05.29": { "aliases": {}, "mappings": { "log": { "properties": { "@timestamp": { "type": "date" }, "@version": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, ... "log": { "properties": { "timestamp": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } ... } ``` - Seems like I am having problem (detected as a string) with this timestamp specifically `"2017-05-29 10:27:41,756"` and `date { match => ["[log][timestamp]", "yyyy-MM-dd HH:mm:ss,SSS" ] }` - `"2017-05-29 10:27:41Z"` is detecting fine as a date with`date { match => ["[log][timestamp]", "ISO8601" ] }` Thank you! ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 30, 2017, 7:44am UTC](https://discuss.elastic.co/t/my-timestamp-cant-get-converted-to-a-date-type/87476/4 "2017-05-30T07:44:47Z")
You didn't include the mapping of the `timestamp` field when you redacted the mappings. The date filter worked just fine and was able to update the `@timestamp` field. If you expect it to update `[log][timestamp]` you need to configure it accordingly. Why keep the fields under `log`? They appear to be duplicates of the corresponding root-level fields. ---
**Author:** ![gregbk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gregbk/32/80289_2.png) [@gregbk](https://discuss.elastic.co/u/gregbk)\ **Post date:** [May 30, 2017, 7:56am UTC](https://discuss.elastic.co/t/my-timestamp-cant-get-converted-to-a-date-type/87476/5 "2017-05-30T07:56:13Z")
Sorry, I updated/edited the mapping on top. I didn't write in advance the [log][timestamp] mapping, it was automatic. I will remove this `[log][timestamp]` you are right, no need for duplication. ---
**Author:** ![gregbk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gregbk/32/80289_2.png) [@gregbk](https://discuss.elastic.co/u/gregbk)\ **Post date:** [May 30, 2017, 9:14am UTC](https://discuss.elastic.co/t/my-timestamp-cant-get-converted-to-a-date-type/87476/6 "2017-05-30T09:14:23Z")
Trying this pattern: ``` filter { grok { match => { "message" => [ "^%{MYTIME:mytimestamp:date}\s*\|\s*%{GREEDYDATA:msg}$" ] } } date { match => ["mytimestamp", "yyyy-MM-dd HH:mm:ss"] } } } # log example: 2017-05-29 10:26:41 | test1 # Added patterns : MYTIME %{YEAR}-%{MONTHNUM}-%{MONTHDAY}\s%{HOUR}:?%{MINUTE}(?::?%{SECOND}) ``` Still, `mytimestamp` is recognized as a string. I can't see what is missing... ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 30, 2017, 9:53am UTC](https://discuss.elastic.co/t/my-timestamp-cant-get-converted-to-a-date-type/87476/7 "2017-05-30T09:53:08Z")
> ``` > %{MYTIME:mytimestamp:date} > > ``` "date" isn't a valid type here. See the docs. Why do you need `mytimestamp` to be a date? Your date filter (as configured) stores the result in `@timestamp`, a field you can't delete. Why not just use `@timestamp` and delete `mytimestamp` after the date filter? ---
**Author:** ![gregbk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gregbk/32/80289_2.png) [@gregbk](https://discuss.elastic.co/u/gregbk)\ **Post date:** [May 30, 2017, 10:01am UTC](https://discuss.elastic.co/t/my-timestamp-cant-get-converted-to-a-date-type/87476/8 "2017-05-30T10:01:09Z")
Thx Magnus for your answer! I got 2 sources of `mytimestamp`: - 2017-05-23 17:14:16,125 --\> recognized as string - 2017-05-23T17:13:16.925Z --\> recognized as date These makes conflict in ES so I wanted both as date. I managed to fix my issue with forcing `target => "mytimestamp"`, but it is a bit ugly. ``` filter { match => { "message" => [ "^%{TIMESTAMP_ISO8601:mytimestamp:date}\s*\|\s*%{GREEDYDATA:msg}$" ] } } date { match => ["mytimestamp", "yyyy-MM-dd HH:mm:ss,SSS"] target => "@timestamp" timezone => "Europe/Zurich" } date { match => ["mytimestamp", "yyyy-MM-dd HH:mm:ss,SSS"] target => "mytimestamp" timezone => "Europe/Zurich" } } } ``` You are right, for now I will remove this temp field (mytimestamp) as @timestamp is right. Thank you for your help! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 27, 2017, 10:01am UTC](https://discuss.elastic.co/t/my-timestamp-cant-get-converted-to-a-date-type/87476/9 "2017-06-27T10:01:17Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.