Data filter not working properly


(idrees) #1

Dear All,
i have the following date format

6/1/2016 0:00

the date filter of my logstash file is below

date {
match => [ "TMSTAMP", "MM/dd/yyyy hh:mm" ]
target => "@timestamp"

but when i query it in the dev tools in kibana, the @timestamp is like below

"@timestamp": "2018-10-17T10:59:36.297Z"

its taking the system date instead.
how can i fix this?

thanks


#2

It probably doesn't work because your date is actually "M/d/yyyy H:mm". Your pattern does not match, so the @timestamp is not changed and a "_dateparsefailure" tag is added instead.


(idrees) #3

its working now, but the time is wrong in the timestamp. can you guide please?

"@timestamp": "2014-11-17T19:49:00.000Z",

"message": "11/17/2014 23:49

you see, in the message, the origional time is 23:49, while in the @timestamp, i see 19:49


why?


(idrees) #4

may be i am not providing timezone? how would i know my timezone? and how to add in the below logstash file

date {
match => [ "TMSTAMP", "M/d/yyyy H:mm" ]
target => "@timestamp"
}

because i see all the dates are correct, but all the times are difference in kibana dev tools


#5

I think right now IT IS using your timezone. The default for the "timezone" setting is the platform default (UTC + 4 ?) and the dev tools show you the corresponding Zulu time (because ES always uses UTC). If your logs' timezone and your server settings are different from each other, you'll have to provide the timezone in your logstash settings.
For example: If your logs are already UTC, you'd add timezone => "UTC"

https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html#plugins-filters-date-timezone


(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.