Logstash Date Result in one day late

Elastic Stack Logstash
1

my field "sdatetime" value is "2016 Aug 31 00:47:17Aug 31 00:47:17"

I did the following

date
{
match => [ "sdatetime", "YYYY MMM dd HH:mm:ss" ]
target => "sdatetime2"

}

The result is
"sdatetime2" => "2016-08-30T16:47:17.000Z"

Why is the result 30 Aug instead of 31 Aug? I have tried applying the timezone but it does not work.

2

What's the timezone of the timestamp in the sdatetime field?

3

Tried with no timezone specified and

tried with
timezone => "Asia/Hong_Kong" in the date{}

The result is the same

4

That's not what I asked. Anyway, the @timestamp field is always UTC and if you're in Hong Kong is appears to be doing the right thing.

5

below is my timestamp

"@timestamp" => "2016-10-07T08:57:59.136Z",

6

How do I make
"sdatetime2" => "2016-08-31T00:47:17.000Z"

7

I have tried putting an UCT setting

date
{
timezone => "UCT"
.............................................
}

It works. What will happen if it is being sent to elastic search? Will be just as it is? or timezone will be silently added to mess up my data?

8

Timestamps in Elasticsearch must be in UTC, so if you are in Hong Kong and the timestamps are in local time what you initially saw is the correct behaviour. Kibana and Elasticsearch assumes this is the case, so by setting it to something that is not UTC will cause problems.

9

My purpose of retaining the datetime as it is is because field in Elasticsearch is not time zone sensitive and need it to be in this time exactly.

10

field in Elasticsearch is not time zone sensitive

What do you mean by this?