# Logstash date offset issue and indicies **URL:** \ **Category:** Logstash\ **Created:** [October 16, 2018, 6:09am UTC](https://discuss.elastic.co/t/logstash-date-offset-issue-and-indicies/152588 "2018-10-16T06:09:45Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Pirate\_Trkmn](https://avatars.discourse-cdn.com/v4/letter/p/977dab/32.png) [@Pirate\_Trkmn](https://discuss.elastic.co/u/Pirate_Trkmn)\ **Post date:** [October 16, 2018, 6:09am UTC](https://discuss.elastic.co/t/logstash-date-offset-issue-and-indicies/152588/1 "2018-10-16T06:09:45Z")
Hi guys i have two problem cant solve. 1.i have log\_time field and want to load it @timestamp with date filter. but there is time difference after i load it to @timestamp example: log\_time is 09:34, but @timestamp is 05:34. i know @timestamp work on utc timezone, now how can solve this problem and load correct time to @timestamp 2.I have very big indicies nearly 3000, and it is working very slow how can i customize it. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 16, 2018, 6:16am UTC](https://discuss.elastic.co/t/logstash-date-offset-issue-and-indicies/152588/2 "2018-10-16T06:16:09Z")
> [@Pirate\_Trkmn](#): > > 1.i have log\_time field and want to load it @timestamp with date filter. but there is time difference after i load it to @timestamp example: log\_time is 09:34, but @timestamp is 05:34. i know @timestamp work on utc timezone, now how can solve this problem and load correct time to @timestamp As you pointed out, the @timestamp field has to be in UTC timezone, so I am not sure what you mean by correcting it. Which timezone is the `log_time` field in? What would you expect `@timestamp` to be? > [@Pirate\_Trkmn](#): > > 2.I have very big indicies nearly 3000, and it is working very slow how can i customize it. Having very large number of indices and/or shards can be very inefficient and cause performance problems. Please read [this blog post](https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster) for some practical guidelines. ---
**Author:** ![Pirate\_Trkmn](https://avatars.discourse-cdn.com/v4/letter/p/977dab/32.png) [@Pirate\_Trkmn](https://discuss.elastic.co/u/Pirate_Trkmn)\ **Post date:** [October 16, 2018, 6:30am UTC](https://discuss.elastic.co/t/logstash-date-offset-issue-and-indicies/152588/3 "2018-10-16T06:30:34Z")
Thanks for reply. log\_time is in Australia/Sydney and now how can i load it to @timestamp ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 16, 2018, 6:32am UTC](https://discuss.elastic.co/t/logstash-date-offset-issue-and-indicies/152588/4 "2018-10-16T06:32:15Z")
Are you specifying the correct [timezone](https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html#plugins-filters-date-timezone) in the Logstash date filter? What does your config look like? ---
**Author:** ![Pirate\_Trkmn](https://avatars.discourse-cdn.com/v4/letter/p/977dab/32.png) [@Pirate\_Trkmn](https://discuss.elastic.co/u/Pirate_Trkmn)\ **Post date:** [October 16, 2018, 6:33am UTC](https://discuss.elastic.co/t/logstash-date-offset-issue-and-indicies/152588/5 "2018-10-16T06:33:41Z")
yes date { match =\> ["log\_timestamp", "yyyy-MM-dd HH:mm:ss"] timezone =\> "Australia/Sydney" } ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 16, 2018, 6:39am UTC](https://discuss.elastic.co/t/logstash-date-offset-issue-and-indicies/152588/6 "2018-10-16T06:39:01Z")
Can you show an example of a full event? ---
**Author:** ![Pirate\_Trkmn](https://avatars.discourse-cdn.com/v4/letter/p/977dab/32.png) [@Pirate\_Trkmn](https://discuss.elastic.co/u/Pirate_Trkmn)\ **Post date:** [October 16, 2018, 6:43am UTC](https://discuss.elastic.co/t/logstash-date-offset-issue-and-indicies/152588/8 "2018-10-16T06:43:49Z")
yes please: { "\_index": "iis-2017.08.01", "\_type": "doc", "\_id": "HOJce2YBnfWJqk2v0LFB", "\_version": 1, "\_score": null, "\_source": { "path": "/etc/logstash/logs/prod\_user\_login/user\_logins\_prod.csv", "Login Date": "2/08/2017 09:11", "type": "prod\_user\_login", "@timestamp": "2017-08-01T23:11:00.000Z", "tags": [ "\_grokparsefailure", "\_geoip\_lookup\_failure", "PROD\_USER\_LOGIN" ], "UserID": "1857", "User Type": "HLRA\_Client", "@version": "1", "host": "elk-on-ubuntu", "Role Name": "Diversa Superannuation Services" }, "fields": { "@timestamp": [ "2017-08-01T23:11:00.000Z" ] }, "sort": [ 1501629060000 ] } ---
**Author:** ![Pirate\_Trkmn](https://avatars.discourse-cdn.com/v4/letter/p/977dab/32.png) [@Pirate\_Trkmn](https://discuss.elastic.co/u/Pirate_Trkmn)\ **Post date:** [October 16, 2018, 6:44am UTC](https://discuss.elastic.co/t/logstash-date-offset-issue-and-indicies/152588/9 "2018-10-16T06:44:50Z")
and it is filter which i use filter { if [type] == "prod\_user\_login" { csv { separator =\> "," columns =\> ["UserID","User Type","Role Name","Login Date"] add\_tag =\> ["PROD\_USER\_LOGIN"] remove\_field =\> ["message"] } date { match =\> ["Login Date", "dd/MM/yyyy HH:mm"] } } } ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 16, 2018, 6:50am UTC](https://discuss.elastic.co/t/logstash-date-offset-issue-and-indicies/152588/10 "2018-10-16T06:50:56Z")
That looks like the correct UTC timestamp corresponding to the Sydney timezone timestamp. As timestamps in Elasticsearch has to be in UTC, I do not see what the problem is. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 13, 2018, 6:54am UTC](https://discuss.elastic.co/t/logstash-date-offset-issue-and-indicies/152588/11 "2018-11-13T06:54:03Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.