# Recommended way to force date fields to be UTC **URL:** \ **Category:** Logstash\ **Created:** [August 1, 2019, 8:29pm UTC](https://discuss.elastic.co/t/recommended-way-to-force-date-fields-to-be-utc/193404 "2019-08-01T20:29:47Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![jcaballero](https://avatars.discourse-cdn.com/v4/letter/j/278dde/32.png) [@jcaballero](https://discuss.elastic.co/u/jcaballero)\ **Post date:** [August 1, 2019, 8:29pm UTC](https://discuss.elastic.co/t/recommended-way-to-force-date-fields-to-be-utc/193404/1 "2019-08-01T20:29:48Z")
Hello, using LogStash and ElasticSearch **5.6.10**. My documents have a few fields of type "date", besides @timestamp. The template includes, for each one of them, a section like this ``` "my_datefield_1": { "type": "date", "format": "MM/dd/yy HH:mm:ss||MM/dd/yy HH:mm:ss.SSS" }, ``` I have just discovered that ElasticSearch expects them to be in UTC. As they are not, they are in local time, when I visualize the data with Kibana, all dates are shifted 4 hours -the different between my local time and UTC-. So I was wondering what is the best way to convert the content of those fields to be in UTC when dumping the data into ElasticSearch: 1. Should I use the **LogStash Filter plugin "Alter"** to change their values? Maybe the **Ruby plugin**? 2. Is there a way to use the **LogStash Filter plugin "Date"** to change them from local to UTC? 3. Or is it actually possible to do it thru the **ElasticSearch Template** , telling it that the dates are coming in Local Time format and letting ElasticSearch to do the conversion to UTC itself? I don't feel 2. and 3. are actually possible, and I should go for 1. But no harm in asking, in case I am missing something. Thanks a lot in advance, Jose ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 1, 2019, 8:32pm UTC](https://discuss.elastic.co/t/recommended-way-to-force-date-fields-to-be-utc/193404/2 "2019-08-01T20:32:01Z")
> [@jcaballero](#): > > Is there a way to use the **LogStash Filter plugin "Date"** to change them from local to UTC? That is the function of the date filter. ---
**Author:** ![jcaballero](https://avatars.discourse-cdn.com/v4/letter/j/278dde/32.png) [@jcaballero](https://discuss.elastic.co/u/jcaballero)\ **Post date:** [August 1, 2019, 8:43pm UTC](https://discuss.elastic.co/t/recommended-way-to-force-date-fields-to-be-utc/193404/3 "2019-08-01T20:43:24Z")
Hi @Badger, thanks a lot for such a prompt response. I read this in the documentation ``` The date filter is used for parsing dates from fields, and then using that date or timestamp as the logstash timestamp for the event. ``` The part I am concerned is **"and then using that date or timestamp as the logstash timestamp for the event."** I already have another field acting as timestamp. I don't want any of the new ones, after converting them to UTC using filter plugin Date, to suddenly become the timestamp. Or that can be avoid simply by using option **target**? Would something like this work? ``` filter{ date { match => ["my_datefield_1", "MM/dd/yy HH:mm:ss.SSS", "MM/dd/yy HH:mm:ss"] target => ["my_datefield_1"] } } ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 1, 2019, 8:54pm UTC](https://discuss.elastic.co/t/recommended-way-to-force-date-fields-to-be-utc/193404/4 "2019-08-01T20:54:03Z")
> [@jcaballero](#): > > Or that can be avoid simply by using option **target**? > > Would something like this work? Yes, and yes. ---
**Author:** ![jcaballero](https://avatars.discourse-cdn.com/v4/letter/j/278dde/32.png) [@jcaballero](https://discuss.elastic.co/u/jcaballero)\ **Post date:** [August 1, 2019, 8:57pm UTC](https://discuss.elastic.co/t/recommended-way-to-force-date-fields-to-be-utc/193404/5 "2019-08-01T20:57:45Z")
In that case, the documentation is a little bit misleading, as it makes you believe that **my\_datefield\_1** would become the new timestamp for the document in ElasticSearch... ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 1, 2019, 9:01pm UTC](https://discuss.elastic.co/t/recommended-way-to-force-date-fields-to-be-utc/193404/6 "2019-08-01T21:01:35Z")
One thing I should mention is that the target of the date filter is of type LogStash::Timestamp ``` "my_datefield_1" => 2019-08-01T20:55:48.719Z, ``` So elasticsearch will (perhaps once you roll to a new index) store it as a date type, and Kibana will give you the option (using a dropdown, if I recall correctly) of using it as a timestamp, but will default to using @timestamp. ---
**Author:** ![jcaballero](https://avatars.discourse-cdn.com/v4/letter/j/278dde/32.png) [@jcaballero](https://discuss.elastic.co/u/jcaballero)\ **Post date:** [August 2, 2019, 9:53pm UTC](https://discuss.elastic.co/t/recommended-way-to-force-date-fields-to-be-utc/193404/8 "2019-08-02T21:53:31Z")
Just one final question. I am not sure how to handle the transition. As I mentioned, the current Template for ElasticSearch includes a section like ``` "my_datefield_1": { "type": "date", "format": "MM/dd/yy HH:mm:ss||MM/dd/yy HH:mm:ss.SSS" }, ``` That works currently since "my\_datefield\_1", as created by LogStash, is a string with one of those formats. Once I start using the new LogStash configuration, that converts "my\_datefield\_1" into a LogStash::Timestamp type, I will need a new Template, without the format, right? ``` "my_datefield_1": { "type": "date" }, ``` But the new Template will start being used when a new index is created. As usual, it will be the next day. Indices are like "my\_index-\ I am not sure how to make ElasticSearch to accept both the old type (string with a given format) and new type (timestamp) for "my\_datefield\_1" during the day of the transition. Is it possible? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 2, 2019, 10:22pm UTC](https://discuss.elastic.co/t/recommended-way-to-force-date-fields-to-be-utc/193404/9 "2019-08-02T22:22:05Z")
> [@jcaballero](#): > > Is it possible? I do not know. That is an elasticsearch question, not a logstash question. ---
**Author:** ![jcaballero](https://avatars.discourse-cdn.com/v4/letter/j/278dde/32.png) [@jcaballero](https://discuss.elastic.co/u/jcaballero)\ **Post date:** [August 3, 2019, 12:54am UTC](https://discuss.elastic.co/t/recommended-way-to-force-date-fields-to-be-utc/193404/10 "2019-08-03T00:54:53Z")
🙂 true. I will post the question at the ES forum. Thanks a lot for everything !! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 31, 2019, 12:55am UTC](https://discuss.elastic.co/t/recommended-way-to-force-date-fields-to-be-utc/193404/11 "2019-08-31T00:55:02Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.