# Parsing timestamp from file and put in @timestamp

**URL:** <https://discuss.elastic.co/t/parsing-timestamp-from-file-and-put-in-timestamp/62788>\
**Category:** Logstash\
**Created:** [October 12, 2016, 8:43am UTC](https://discuss.elastic.co/t/parsing-timestamp-from-file-and-put-in-timestamp/62788 "2016-10-12T08:43:40Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Artyom\_Davydov](https://avatars.discourse-cdn.com/v4/letter/a/8dc957/32.png) [@Artyom\_Davydov](https://discuss.elastic.co/u/Artyom_Davydov)\
**Post date:** [October 12, 2016, 8:43am UTC](https://discuss.elastic.co/t/parsing-timestamp-from-file-and-put-in-timestamp/62788/1 "2016-10-12T08:43:40Z")

</div>

Hi all.  
I use folowing scheme to collect logs: filebeat -\> logstash -\> graylog  
So i have problem with parsing date and put it to the @timestamp

example of log:

> 20161012T082829Z|urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST|a54ca74edj5cajdb208i7e6bgf08846|[https://xxx.ru|http](https://xxx.ru%7Chttp)://shibboleth.net/ns/profiles/saml2/sso/browser|[https://xxx.ru/idp|urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST|\_4675f53ddc3239b44bbff0e10a5539d7|salnikova-oy-130408|urn:oasis:names:tc:SAML:2.0:ac:classes:PasswordProtectedTransport|smsAuth,phone,principalId,group|salnikova-oy-130408|\_b088e4ce70c78c59979fa0e8fe98e41f](https://xxx.ru/idp%7Curn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST%7C_4675f53ddc3239b44bbff0e10a5539d7%7Csalnikova-oy-130408%7Curn:oasis:names:tc:SAML:2.0:ac:classes:PasswordProtectedTransport%7CsmsAuth,phone,principalId,group%7Csalnikova-oy-130408%7C_b088e4ce70c78c59979fa0e8fe98e41f)

in logstash i use this config:

> filter {  
> if [type] == "idp" {  
> grok {  
> match =\> { "message" =\> "%{MY2\_TIMESTAMP\_ISO8601:idp\_timestamp}Z|%{IDP:method}|%{IDP:hash1}|%{IDP:domain}|%{IDP}|%{IDP}|%{IDP}|%{IDP:hash2}|%{IDP:login}|%{IDP}|%{IDP}|%{IDP}|%{GREEDYDATA:hash3}" }  
> overwrite =\> ["short\_message"]  
> }  
> date {  
> match =\> ["idp\_timestamp", MY2\_TIMESTAMP\_ISO8601]  
> target =\> "@timestamp"  
> }

patterns file:

> IDP [^|]+  
> MY2\_TIMESTAMP\_ISO8601 %{YEAR}%{MONTHNUM}%{MONTHDAY}T%{HOUR}%{MINUTE}%{SECOND}

so timestamp from log parsed to idp\_timestamp field , but dont get into @timestamp  
Where is my mistake? Or what i do wrong.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 12, 2016, 9:05am UTC](https://discuss.elastic.co/t/parsing-timestamp-from-file-and-put-in-timestamp/62788/2 "2016-10-12T09:05:42Z")

</div>

The date filter doesn't use grok patterns. This probably works:

```nohighlight
date {
  match => ["idp_timestamp", "yyyyMMdd'T'HHmmss'Z'"]
}

```

Why not use a csv filter to parse each line?

---

<div class="post-metadata">

**Author:** ![Artyom\_Davydov](https://avatars.discourse-cdn.com/v4/letter/a/8dc957/32.png) [@Artyom\_Davydov](https://discuss.elastic.co/u/Artyom_Davydov)\
**Post date:** [October 12, 2016, 10:44am UTC](https://discuss.elastic.co/t/parsing-timestamp-from-file-and-put-in-timestamp/62788/3 "2016-10-12T10:44:59Z")

</div>

thx Magnus!  
it works but in another case i use this

> ```
> date {
> match => ["idp_timestamp", ISO8601]
> target => "@timestamp"
> 
> ```
> 
> }

and it also works, but you said that data filter doesn't use grok patterns.

one more question i parsed log which use UTC time , how can i add +3 hours ?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 12, 2016, 10:48am UTC](https://discuss.elastic.co/t/parsing-timestamp-from-file-and-put-in-timestamp/62788/4 "2016-10-12T10:48:00Z")

</div>

The `@timestamp` field is always UTC.

The date filter's `timezone` option is useful if the dates being parsed don't include a timezone and their actual timezone is different from the timezone of the machine where Logstash runs.

---

<div class="post-metadata">

**Author:** ![Artyom\_Davydov](https://avatars.discourse-cdn.com/v4/letter/a/8dc957/32.png) [@Artyom\_Davydov](https://discuss.elastic.co/u/Artyom_Davydov)\
**Post date:** [October 12, 2016, 11:01am UTC](https://discuss.elastic.co/t/parsing-timestamp-from-file-and-put-in-timestamp/62788/5 "2016-10-12T11:01:10Z")

</div>

do i need to add  
timezone =\> "Europe/Moscow"  
in date ?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 12, 2016, 11:15am UTC](https://discuss.elastic.co/t/parsing-timestamp-from-file-and-put-in-timestamp/62788/6 "2016-10-12T11:15:14Z")

</div>

Not if the system timezone of the machine where Logstash runs is Europe/Moscow.

---

<div class="post-metadata">

**Author:** ![Artyom\_Davydov](https://avatars.discourse-cdn.com/v4/letter/a/8dc957/32.png) [@Artyom\_Davydov](https://discuss.elastic.co/u/Artyom_Davydov)\
**Post date:** [October 12, 2016, 11:49am UTC](https://discuss.elastic.co/t/parsing-timestamp-from-file-and-put-in-timestamp/62788/7 "2016-10-12T11:49:07Z")

</div>

thank you ! we have changed timezone log timestamp in jetty ))

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:34am UTC](https://discuss.elastic.co/t/parsing-timestamp-from-file-and-put-in-timestamp/62788/8 "2017-07-06T04:34:34Z")

</div>