# How I can modify timestamp in log message to UTC **URL:** \ **Category:** Logstash\ **Created:** [March 30, 2016, 4:36pm UTC](https://discuss.elastic.co/t/how-i-can-modify-timestamp-in-log-message-to-utc/45830 "2016-03-30T16:36:59Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![nagesh](https://avatars.discourse-cdn.com/v4/letter/n/7ba0ec/32.png) [@nagesh](https://discuss.elastic.co/u/nagesh)\ **Post date:** [March 30, 2016, 4:36pm UTC](https://discuss.elastic.co/t/how-i-can-modify-timestamp-in-log-message-to-utc/45830/1 "2016-03-30T16:36:59Z")
Hi, Our log messages contain timestamp in EST format. How I can modify the timestamp to UTC before displaying logs into Kibana. Example message: 2016-03-26T01:55:47.78-0500 [App/0] OUT [AUDIT] CWWKE0001I: The server defaultServer has been launched Looking forward for help on this. Thanks, Nagesh Mandava. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [March 30, 2016, 4:53pm UTC](https://discuss.elastic.co/t/how-i-can-modify-timestamp-in-log-message-to-utc/45830/2 "2016-03-30T16:53:54Z")
Use the grok filter to parse the log lines and extract the timestamp to a separate field. Feed that field to the date filter. It'll parse and store the timestamp in the `@timestamp` field which always is UTC. ---
**Author:** ![nagesh](https://avatars.discourse-cdn.com/v4/letter/n/7ba0ec/32.png) [@nagesh](https://discuss.elastic.co/u/nagesh)\ **Post date:** [March 30, 2016, 5:05pm UTC](https://discuss.elastic.co/t/how-i-can-modify-timestamp-in-log-message-to-utc/45830/3 "2016-03-30T17:05:31Z")
We have below pattern defined to parse the log message CF\_DIRECT\_PULL\_FORMAT %{TIMESTAMP\_ISO8601:timestamp} [%{CF\_LOG\_TYPE}]%{SPACE}%{WORD}%{SPACE}([%{CF\_WLP\_LOG\_LEVEL:loglevel}%{SPACE}])?%{GREEDYDATA:details} We have two fields in the viewlet to display the logs in Kibana dashboard. First field is @timestamp and second is message. I want to modify the timestamp in message filed to UTC format. ---
**Author:** ![kirill\_polishchuk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kirill_polishchuk/32/6489_2.png) [@kirill\_polishchuk](https://discuss.elastic.co/u/kirill_polishchuk)\ **Post date:** [March 30, 2016, 7:23pm UTC](https://discuss.elastic.co/t/how-i-can-modify-timestamp-in-log-message-to-utc/45830/4 "2016-03-30T19:23:04Z")
Hello I think you can do something like this: 1. Split original message field to `timestamp` and `sub_message` with grok: grok { match =\> ["message", "%{TIMESTAMP\_ISO8601:timestamp} %{GREEDYDATA:sub\_message}"] remove\_field =\> ["message"] } 2. Parse `timestamp` field with `date` filter. 3. Create new field `message` from `@timestamp` and `sub_message`: if [sub\_message] { mutate { add\_field =\> { "message" =\> "%{@timestamp} %{sub\_message}" } } } ---
**Author:** ![nagesh](https://avatars.discourse-cdn.com/v4/letter/n/7ba0ec/32.png) [@nagesh](https://discuss.elastic.co/u/nagesh)\ **Post date:** [March 30, 2016, 10:07pm UTC](https://discuss.elastic.co/t/how-i-can-modify-timestamp-in-log-message-to-utc/45830/5 "2016-03-30T22:07:36Z")
Can we have multiple grok statements in conf file, because I already have below grok { match =\> {"details" =\> "[DWPerf-%{WORD}][UN-%{GREEDYDATA:userid}][SI-%{GREEDYDATA:sessionid}]%{GREEDYDATA:details}%{NUMBER:elapsed\_time}s"} overwrite =\> ["details"] } mutate { convert =\> { "elapsed\_time" =\> "float" } } ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [March 31, 2016, 5:31am UTC](https://discuss.elastic.co/t/how-i-can-modify-timestamp-in-log-message-to-utc/45830/6 "2016-03-31T05:31:45Z")
> Can we have multiple grok statements in conf file Yes, of course. ---
**Author:** ![nagesh](https://avatars.discourse-cdn.com/v4/letter/n/7ba0ec/32.png) [@nagesh](https://discuss.elastic.co/u/nagesh)\ **Post date:** [March 31, 2016, 1:43pm UTC](https://discuss.elastic.co/t/how-i-can-modify-timestamp-in-log-message-to-utc/45830/7 "2016-03-31T13:43:36Z")
Can you please provide me an example ? Thanks, Nagesh Mandava. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [March 31, 2016, 1:56pm UTC](https://discuss.elastic.co/t/how-i-can-modify-timestamp-in-log-message-to-utc/45830/8 "2016-03-31T13:56:43Z")
There's nothing special about having multiple grok filters. ```auto grok { match => ["field1", "expression1"] } grok { match => ["field2", "expression2"] } ``` ---
**Author:** ![nagesh](https://avatars.discourse-cdn.com/v4/letter/n/7ba0ec/32.png) [@nagesh](https://discuss.elastic.co/u/nagesh)\ **Post date:** [March 31, 2016, 4:57pm UTC](https://discuss.elastic.co/t/how-i-can-modify-timestamp-in-log-message-to-utc/45830/9 "2016-03-31T16:57:36Z")
Thanks for the example. I have updated my conf file as below grok { match =\> {"details" =\> "[DWPerf-%{WORD}][UN-%{GREEDYDATA:userid}][SI-%{GREEDYDATA:sessionid}]%{GREEDYDATA:details}%{NUMBER:elapsed\_time}s"} overwrite =\> ["details"] } mutate { convert =\> { "elapsed\_time" =\> "float" } } grok{ match =\> ["message", "%{TIMESTAMP\_ISO8601:timestamp} [%{CF\_LOG\_TYPE}]%{SPACE}%{WORD}%{SPACE}([%{CF\_WLP\_LOG\_LEVEL:loglevel}%{SPACE}])?%{GREEDYDATA:details}" ] remove\_field =\> ["message"] } ``` if[CF_DIRECT_PULL_FORMAT2]{ mutate { add_field => { "message" => "%{@timestamp} %{CF_DIRECT_PULL_FORMAT2}" } } } ``` Patterns file updated as below CF\_DIRECT\_PULL\_FORMAT2 [%{CF\_LOG\_TYPE}]%{SPACE}%{WORD}%{SPACE}([%{CF\_WLP\_LOG\_LEVEL:loglevel}%{SPACE}])?%{GREEDYDATA:details} I am not seeing any difference after above changes for the timestamp in message. ![](https://us1.discourse-cdn.com/elastic/original/2X/8/8b0e07bdbe4e6fd4a82fe052875449e5e3277415.jpg) ---
**Author:** ![nagesh](https://avatars.discourse-cdn.com/v4/letter/n/7ba0ec/32.png) [@nagesh](https://discuss.elastic.co/u/nagesh)\ **Post date:** [March 31, 2016, 5:40pm UTC](https://discuss.elastic.co/t/how-i-can-modify-timestamp-in-log-message-to-utc/45830/10 "2016-03-31T17:40:38Z")
Hi Kirill, Can you please check my previous message and let me know where I am wrong. Thanks, Nagesh. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [March 31, 2016, 6:01pm UTC](https://discuss.elastic.co/t/how-i-can-modify-timestamp-in-log-message-to-utc/45830/11 "2016-03-31T18:01:46Z")
Comments: - Where's your date filter? That's what converts your UTC-5 timestamp into UTC so that you can put it back into `message`. - `CF_DIRECT_PULL_FORMAT2` is the name of a grok pattern but you're treating it like the name of a field. - Don't post screenshots from Kibana. Post the output from a `stdout { codec => rubydebug }` output. ES and Kibana will only distract you. Reintroduce them when you've verified that the events looks as expected. - When you use `add_field` on the `message` field you'll actually turn that field into an array that'll contain both the original value and the new value. Use `replace` instead. Follow Kirill's example more closely. Except for the `add_field` thing his example looks perfectly correct. ---
**Author:** ![nagesh](https://avatars.discourse-cdn.com/v4/letter/n/7ba0ec/32.png) [@nagesh](https://discuss.elastic.co/u/nagesh)\ **Post date:** [March 31, 2016, 6:32pm UTC](https://discuss.elastic.co/t/how-i-can-modify-timestamp-in-log-message-to-utc/45830/12 "2016-03-31T18:32:51Z")
Thanks Magnus. I have updated as below based on your review comments. grok{ patterns\_dir =\> "./patterns" remove\_field =\> ["message"] match =\> {"sub\_message" =\> "%{CF\_DIRECT\_PULL\_FORMAT2}" } } ``` if[sub_message]{ date { match => ["timestamp", "yyyy-MM-dd'T'HH:mm:ss.SSSSSSZZ", "ISO8601"] target => "@timestamp" } mutate { replace => { "message" => "%{@timestamp} %{sub_message}" } } } ``` There is still some issue in above snippet. Please help me to fix it. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [March 31, 2016, 6:42pm UTC](https://discuss.elastic.co/t/how-i-can-modify-timestamp-in-log-message-to-utc/45830/13 "2016-03-31T18:42:29Z")
> I have updated as below based on your review comments. No you haven't. What you're trying is quite far from Kirill's example. ---
**Author:** ![nagesh](https://avatars.discourse-cdn.com/v4/letter/n/7ba0ec/32.png) [@nagesh](https://discuss.elastic.co/u/nagesh)\ **Post date:** [March 31, 2016, 7:02pm UTC](https://discuss.elastic.co/t/how-i-can-modify-timestamp-in-log-message-to-utc/45830/14 "2016-03-31T19:02:47Z")
Earlier I was trying to add\_filed and remove\_filed for the message. I think it is not correct, so updated as below grok { patterns\_dir =\> "./patterns" match =\> {"sub\_message" =\> "%{CF\_DIRECT\_PULL\_FORMAT2}" } } if[sub\_message]{ ``` date { match => ["timestamp", "yyyy-MM-dd'T'HH:mm:ss.SSSSSSZZ", "ISO8601"] target => "@timestamp" } mutate { add_field => { "message" => "%{@timestamp} %{sub_message}" } } } ``` Is this fine ? Thanks, Nagesh Mandava. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [March 31, 2016, 8:31pm UTC](https://discuss.elastic.co/t/how-i-can-modify-timestamp-in-log-message-to-utc/45830/15 "2016-03-31T20:31:45Z")
I have twice asked you to follow the previously posted example, and yet you are not doing that. I don't have time for this anymore. ---
**Author:** ![nagesh](https://avatars.discourse-cdn.com/v4/letter/n/7ba0ec/32.png) [@nagesh](https://discuss.elastic.co/u/nagesh)\ **Post date:** [April 6, 2016, 4:37pm UTC](https://discuss.elastic.co/t/how-i-can-modify-timestamp-in-log-message-to-utc/45830/16 "2016-04-06T16:37:33Z")
I have made following changes and it worked for me. grok { patterns\_dir =\> "./patterns" match =\> {"message" =\> "%{CF\_DIRECT\_PULL\_FORMAT3}" } } mutate{ convert =\> ["timestamp", "string"] } date { locale =\> "en" match =\> ["timestamp", "ISO8601"] timezone =\> "Europe/Vienna" target =\> "timestamp" } grok { patterns\_dir =\> "./patterns" match =\> {"sub\_message" =\> "%{CF\_DIRECT\_PULL\_FORMAT2}" } } mutate { replace =\> { "message" =\> "%{timestamp} %{sub\_message}" } } ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:03am UTC](https://discuss.elastic.co/t/how-i-can-modify-timestamp-in-log-message-to-utc/45830/17 "2017-07-06T05:03:27Z")