# @timestamp is not getting updated through date filter. getting \_dateparsefailure **URL:** \ **Category:** Logstash\ **Created:** [June 28, 2018, 12:06pm UTC](https://discuss.elastic.co/t/timestamp-is-not-getting-updated-through-date-filter-getting-dateparsefailure/137784 "2018-06-28T12:06:46Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Honda\_fred\_elk](https://avatars.discourse-cdn.com/v4/letter/h/919ad9/32.png) [@Honda\_fred\_elk](https://discuss.elastic.co/u/Honda_fred_elk)\ **Post date:** [June 28, 2018, 12:06pm UTC](https://discuss.elastic.co/t/timestamp-is-not-getting-updated-through-date-filter-getting-dateparsefailure/137784/1 "2018-06-28T12:06:46Z")
I have following logstash configuration for Websphere App server SystemErr logs. As per the grok pattern, I do see the logs are getting parsed, but Overriding @timestamp is not working. I'm getting \_dateparsefailure. here is the configuration ``` input { file { path => ["/Path/System*.log"] start_position => "beginning" sincedb_path => "/Path/sincedbfile_WAS.txt" codec => multiline { pattern => "^\[" negate => true what => "previous" } exclude => ["*.gz","native_std*.log"] type => "WAS_logs" tags => "WAS_logs" } } filter{ grok { patterns_dir => ["./appl/grokpattern"] match => ["message", "\[%{TimeDate:WAS_TimeStamp}\] %{BASE16NUM:WAS_ThreadId} %{WORD:WAS_EventType}%{SPACE}%{WORD:WAS_LogLevel}%{SPACE}%{GREEDYDATA:WAS_LogMessage}"] overwrite => ["message"] } date { match => ["WAS_TimeStamp", "M/dd/YY HH:mm:ss:SSS ZZZ", "MM/d/YY HH:mm:ss:SSS ZZZ", "M/d/YY HH:mm:ss:SSS ZZZ", "MM/dd/YY H:mm:ss:SSS ZZZ", "M/d/YY H:mm:ss:SSS ZZZ", "MM/d/YY H:mm:ss:SSS ZZZ", "M/dd/YY H:mm:ss:SSS ZZZ"] target => "@timestamp" } } output { elasticsearch { hosts => ["HOST1:9200","HOST2:9200"] index => "WAS-logs-%{+YYYY.MM.dd}" } file { path => "/appl/logstash/LogStash_WAS_log_output.log" } } ``` I use custom patterns too. TimeDate (?(%{DATE} %{TIME} %{TIMEZONE})) DATE (?(\d{1,2}/\d{1,2}/\d{2})) TIME (?(\d{1,2}:\d{1,2}:\d{1,2}:\d{1,3})) TIMEZONE (?(\S{3})) This is the log [6/22/18 1:01:23:615 EDT] 00000078 SystemErr R at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.initializeState(JDBCStoreManager.java:322) below is the processed string by logstash { "TimeDate": "6/22/18 1:01:23:615 EDT", **"WAS\_TimeStamp": "6/22/18 1:01:23:615 EDT"** **"@timestamp": "2018-06-27T19:17:35.304Z",** "TIME": "1:01:23:615", "WAS\_EventType": "SystemErr", "WAS\_LogLevel": "R", "host": "Host1", "type": "WAS\_logs", "DATE": "6/22/18", "tags": [ "WAS\_logs", "\_dateparsefailure" ], "WAS\_LogMessage": "at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.initializeState(JDBCStoreManager.java:322)\r", "TIMEZONE": "EDT", "message": "[6/22/18 1:01:23:615 EDT] 00000078 SystemErr R \tat org.apache.openjpa.jdbc.kernel.JDBCStoreManager.initializeState(JDBCStoreManager.java:322)\r", "@version": "1", "path": "/appl/logfiles/SystemOut\_Test.log", "WAS\_ThreadId": "00000078", } To me it looks like .. code wise everything is correct. But not sure why @timestamp is not getting updated with WAS\_TimeStamp. Could you check what's wrong this config? Thanks in Advance. Fredrick ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 29, 2018, 6:39am UTC](https://discuss.elastic.co/t/timestamp-is-not-getting-updated-through-date-filter-getting-dateparsefailure/137784/2 "2018-06-29T06:39:28Z")
If the date filter fails to parse a string it'll give you details about the failure in the Logstash log. The problem might be the "EDT" timezone name. I don't think Logstash can parse those. Unless the timezone is _always_ EST/EDT you may have to extract it into a field of its own and use a translate filter to translate it into an UTC offset that the date filter accepts. (The reason it can't parse timezone names is that they're ambiguous. Take CST for example, what's the UTC offset for that?) ---
**Author:** ![Honda\_fred\_elk](https://avatars.discourse-cdn.com/v4/letter/h/919ad9/32.png) [@Honda\_fred\_elk](https://discuss.elastic.co/u/Honda_fred_elk)\ **Post date:** [July 9, 2018, 2:39pm UTC](https://discuss.elastic.co/t/timestamp-is-not-getting-updated-through-date-filter-getting-dateparsefailure/137784/3 "2018-07-09T14:39:38Z")
Magnus, For one another logs, I have been usisg below configuration to parse date stuffs. That also has the time zone. that is getting parsed and works great. Below is the configuration and JSON log string to be parsed ``` filter { date{ locale => "en" match => ["loggingTime","EEE d MMM yyyy HH-mm-ss SSS z"] target => "@timestamp" } } { "loggingTime":"Mon 9 Jul 2018 05-40-15 135 EDT", "applicationName":"EVENTS", "loglevel":"INFO", "threadName":"SchedulerWorkManager.Alarm Pool : 0", "logMessage":"method - process(taskStatus)" } ``` But my first post is from WAS logs.. still fails. ---
**Author:** ![Honda\_fred\_elk](https://avatars.discourse-cdn.com/v4/letter/h/919ad9/32.png) [@Honda\_fred\_elk](https://discuss.elastic.co/u/Honda_fred_elk)\ **Post date:** [July 9, 2018, 3:14pm UTC](https://discuss.elastic.co/t/timestamp-is-not-getting-updated-through-date-filter-getting-dateparsefailure/137784/4 "2018-07-09T15:14:09Z")
I solved the date issue using below cofig. ``` date { match => ["WAS_TimeStamp", "M/dd/yy HH:mm:ss:SSS z", "MM/d/yy HH:mm:ss:SSS z", "M/d/yy HH:mm:ss:SSS z", "MM/dd/yy H:mm:ss:SSS z", "M/d/yy H:mm:ss:SSS z", "MM/d/yy H:mm:ss:SSS z", "M/dd/yy H:mm:ss:SSS z"] target => "@timestamp" } ``` I changed year from Y to y and time zone from ZZZ to z ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 6, 2018, 3:14pm UTC](https://discuss.elastic.co/t/timestamp-is-not-getting-updated-through-date-filter-getting-dateparsefailure/137784/5 "2018-08-06T15:14:10Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.