# How to correctly parse and index dates into timestamp that have no offset? (Post title renamed)

**URL:** <https://discuss.elastic.co/t/how-to-correctly-parse-and-index-dates-into-timestamp-that-have-no-offset-post-title-renamed/52535>\
**Category:** Logstash\
**Created:** [June 12, 2016, 11:55am UTC](https://discuss.elastic.co/t/how-to-correctly-parse-and-index-dates-into-timestamp-that-have-no-offset-post-title-renamed/52535 "2016-06-12T11:55:01Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![niceguybob](https://avatars.discourse-cdn.com/v4/letter/n/50afbb/32.png) [@niceguybob](https://discuss.elastic.co/u/niceguybob)\
**Post date:** [June 12, 2016, 11:55am UTC](https://discuss.elastic.co/t/how-to-correctly-parse-and-index-dates-into-timestamp-that-have-no-offset-post-title-renamed/52535/1 "2016-06-12T11:55:01Z")

</div>

Hi,

Hopefully this is a simple one.

I have a rather basic Logstash config to read nginx apache and error logs and push them to an index. It's all working OK except the date is incorrect for my error log file.

access log settings for timestamp which is working fine:  
\<\<  
date {  
match =\> ["time", "dd/MMM/YYYY:HH:mm:ss Z"]  
locale =\> en

> >

error log settings for timestamp which gives incorrect timestamp.  
\<\<  
date {  
match =\> ["err\_time" , 'YYYY/MM/dd HH:mm:ss']  
locale =\> en  
}

> >

Input from logfile:  
2016/05/03 23:02:47

Creates the following in table view  
@timestamp = May 4th 2016, 00:02:47.000  
err\_time = May 4th 2016, 00:02:47.000

And the following in the JSON document for the log entry.  
\<\<  
"@timestamp": "2016-05-03T23:02:47.000Z",  
"err\_time": "2016/05/03 23:02:47",

"fields": {  
"err\_time": [  
1462316567000  
],  
"@timestamp": [  
1462316567000  
]  
},

> >

As you can see, the hour is being stripped off/ignored and the minutes are being used as the hour.

I'd really appreciate a steer on this, apologies in advance if I've missed something obvious in the documentation.

Cheers,

Mark.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [June 12, 2016, 12:20pm UTC](https://discuss.elastic.co/t/how-to-correctly-parse-and-index-dates-into-timestamp-that-have-no-offset-post-title-renamed/52535/2 "2016-06-12T12:20:56Z")

</div>

The date filter creates UTC timestamps, which is the standard in Elasticsearch. Depending on which time zone you are in, an offset is therefore expected.

---

<div class="post-metadata">

**Author:** ![niceguybob](https://avatars.discourse-cdn.com/v4/letter/n/50afbb/32.png) [@niceguybob](https://discuss.elastic.co/u/niceguybob)\
**Post date:** [June 12, 2016, 12:42pm UTC](https://discuss.elastic.co/t/how-to-correctly-parse-and-index-dates-into-timestamp-that-have-no-offset-post-title-renamed/52535/3 "2016-06-12T12:42:06Z")

</div>

Thanks for the pointer. Before reading your reply I tried extending my mapping to support the date format.  
\<\<  
properties": {  
"@timestamp": {  
"type": "date",  
"format": "YYYY/MM/dd HH:mm:ss||strict\_date\_optional\_time||epoch\_millis"

> >

I also added the following the timezone thinking it might help

\<\<  
date {  
match =\> ["err\_time" , 'YYYY/MM/dd HH:mm:ss']  
locale =\> en  
timezone =\> "UTC"  
}

> >

Should I use add\_field for the offset being one is not included in the source logfile? Are you able to provide example syntax for inserting the offset for dates that do not contain them?

Many thanks in advance.

Mark.

---

<div class="post-metadata">

**Author:** ![niceguybob](https://avatars.discourse-cdn.com/v4/letter/n/50afbb/32.png) [@niceguybob](https://discuss.elastic.co/u/niceguybob)\
**Post date:** [June 12, 2016, 2:24pm UTC](https://discuss.elastic.co/t/how-to-correctly-parse-and-index-dates-into-timestamp-that-have-no-offset-post-title-renamed/52535/4 "2016-06-12T14:24:35Z")

</div>

Post renamed to be more relevant to the problem as this was nothing to do with handling of multiple date formats and related to missing offset.

Thanks,

Mark.

---

<div class="post-metadata">

**Author:** ![niceguybob](https://avatars.discourse-cdn.com/v4/letter/n/50afbb/32.png) [@niceguybob](https://discuss.elastic.co/u/niceguybob)\
**Post date:** [June 13, 2016, 1:16pm UTC](https://discuss.elastic.co/t/how-to-correctly-parse-and-index-dates-into-timestamp-that-have-no-offset-post-title-renamed/52535/5 "2016-06-13T13:16:21Z")

</div>

Anyone able to help on this?

Thanks,

Mark.

---

<div class="post-metadata">

**Author:** ![niceguybob](https://avatars.discourse-cdn.com/v4/letter/n/50afbb/32.png) [@niceguybob](https://discuss.elastic.co/u/niceguybob)\
**Post date:** [June 14, 2016, 11:41am UTC](https://discuss.elastic.co/t/how-to-correctly-parse-and-index-dates-into-timestamp-that-have-no-offset-post-title-renamed/52535/6 "2016-06-14T11:41:14Z")

</div>

Moved to Logstash from Kibana as advised by folk in IRC. Any pointers would be appreciated.

Thanks.

---

<div class="post-metadata">

**Author:** ![niceguybob](https://avatars.discourse-cdn.com/v4/letter/n/50afbb/32.png) [@niceguybob](https://discuss.elastic.co/u/niceguybob)\
**Post date:** [June 15, 2016, 11:49am UTC](https://discuss.elastic.co/t/how-to-correctly-parse-and-index-dates-into-timestamp-that-have-no-offset-post-title-renamed/52535/7 "2016-06-15T11:49:49Z")

</div>

Solved.

The fix was a simple as adding 'timezone =\> "Europe/London"' into my date config. Thanks for the help all, especially the guys in #logstash IRC channel (bjorn\_ and kartwheel'.

Cheers,

Mark.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:52am UTC](https://discuss.elastic.co/t/how-to-correctly-parse-and-index-dates-into-timestamp-that-have-no-offset-post-title-renamed/52535/8 "2017-07-06T04:52:43Z")

</div>