# Date Filter - \_dateparsefailure **URL:** \ **Category:** Logstash\ **Created:** [June 14, 2022, 9:12pm UTC](https://discuss.elastic.co/t/date-filter-dateparsefailure/307199 "2022-06-14T21:12:16Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![yquirion](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yquirion/32/107068_2.png) [@yquirion](https://discuss.elastic.co/u/yquirion)\ **Post date:** [June 14, 2022, 9:12pm UTC](https://discuss.elastic.co/t/date-filter-dateparsefailure/307199/1 "2022-06-14T21:12:16Z")
Dear all, I'm running into a strange problem converting syslogtimestamp field into @timestamp. After reading the thread: [https://discuss.elastic.co/t/date-filter--dateparsefailure-solved/64692](https://discuss.elastic.co/t/date-filter--dateparsefailure-solved/64692), I can't make it to work. When I launch this simple command, it failed: ```auto echo "Nov 2 10:40:49" | logstash -e 'input { stdin {} } filter { date { match => ["message", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"] } }' { "@version" => "1", "message" => "Nov 2 10:40:49", "@timestamp" => 2022-06-14T21:05:58.605Z, "tags" => [ [0] "_dateparsefailure" ], "host" => "logstash-dev03.elk.usherbrooke.ca" } ``` I've also tried to use the `MMMd HH:mm:ss`, but logstash failed with this error: ```auto [ERROR] 2022-06-14 17:02:54.415 [Converge PipelineAction::Create
] agent - Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: Illegal pattern component: p", :backtrace=>["org.logstash.config.ir.CompiledPipeline.(CompiledPipeline.java:120)", "org.logstash.execution.JavaBasePipelineExt.initialize(JavaBasePipelineExt.java:86)", "org.logstash.execution.JavaBasePipelineExt$INVOKER$i$1$0$initialize.call(JavaBasePipelineExt$INVOKER$i$1$0$initialize.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:837)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1169)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuperSplatArgs(IRRuntimeHelpers.java:1156)", "org.jruby.ir.targets.InstanceSuperInvokeSite.invoke(InstanceSuperInvokeSite.java:39)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$initialize$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:48)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:333)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:87)", "org.jruby.RubyClass.newInstance(RubyClass.java:939)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0(/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0$ __VARARGS__ (/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:50)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "usr.share.logstash.logstash_minus_core.lib.logstash.agent.RUBY$block$converge_state$2(/usr/share/logstash/logstash-core/lib/logstash/agent.rb:388)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)", "org.jruby.runtime.Block.call(Block.java:139)", "org.jruby.RubyProc.call(RubyProc.java:318)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)", "java.base/java.lang.Thread.run(Thread.java:829)"]} warning: thread "Converge PipelineAction::Create
" terminated with exception (report_on_exception is true): LogStash::Error: Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create
` create at org/logstash/execution/ConvergeResultExt.java:135 add at org/logstash/execution/ConvergeResultExt.java:60 converge_state at /usr/share/logstash/logstash-core/lib/logstash/agent.rb:401 [ERROR] 2022-06-14 17:02:54.421 [Agent thread] agent - An exception happened when converging configuration {:exception=>LogStash::Error, :message=>"Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create
`"} [FATAL] 2022-06-14 17:02:54.438 [LogStash::Runner] runner - An unexpected error occurred! {:error=>#`>, :backtrace=>["org/logstash/execution/ConvergeResultExt.java:135:in `create'", "org/logstash/execution/ConvergeResultExt.java:60:in `add'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:401:in `block in converge_state'"]} [FATAL] 2022-06-14 17:02:54.445 [LogStash::Runner] Logstash - Logstash stopped processing because of an error: (SystemExit) exit org.jruby.exceptions.SystemExit: (SystemExit) exit at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.20.1.jar:?] at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.20.1.jar:?] at usr.share.logstash.lib.bootstrap.environment.
(/usr/share/logstash/lib/bootstrap/environment.rb:94) ~[?:?] ``` So I'm a bit confused on how to make this syslog timestamp date into `@timestamp` field. Any idea? I'm runniong logstash v. 7.17.4. Regards! Yanick ---
**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\ **Post date:** [June 14, 2022, 10:03pm UTC](https://discuss.elastic.co/t/date-filter-dateparsefailure/307199/2 "2022-06-14T22:03:44Z")
It works for me. Only thing you are doing differently is starting it from the command line so I would look at that first. ```auto input { stdin {} } filter { date { match => ["message", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"] } } output { stdout { codec => "json_lines" } } ``` or ```auto input { generator { count => 1 lines => ['{ "message": "Nov 2 10:40:49" }'] codec => json } } filter { date { match => ["message", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"] } } output { stdout { codec => "json_lines" } } ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [June 14, 2022, 10:22pm UTC](https://discuss.elastic.co/t/date-filter-dateparsefailure/307199/3 "2022-06-14T22:22:27Z")
It works for me too. > [@yquirion](#): > > ``` > "message" => "Nov 2 10:40:49", > > ``` There is only a single space between the month and the day, but that is fine since the dd in "MMM dd HH:mm:ss" will match one or two digits. > [@](#): > > I've also tried to use the `MMMd HH:mm:ss`, but logstash failed with "Unable to configure plugins: Illegal pattern component: p", You cannot use `` to represent a space. Alphabetic characters in the pattern have to be valid Joda time specifiers. The p in "space" is causing this exception, if you removed that it would complain about the c. If you removed that your pattern would match `Nov<59am3>2 10:40:49` which has seconds, am/pm, and numeric day of the week between the \<\>. ---
**Author:** ![yquirion](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yquirion/32/107068_2.png) [@yquirion](https://discuss.elastic.co/u/yquirion)\ **Post date:** [June 15, 2022, 12:46pm UTC](https://discuss.elastic.co/t/date-filter-dateparsefailure/307199/4 "2022-06-15T12:46:00Z")
Hi there, I tried the following: ```auto input { generator { codec => json count => 1 lines => ['{"message":"Nov 2 10:40:49"}'] } } filter { date { match => ["message", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"] } } output { stdout { codec => rubydebug { metadata => true } } } ``` To launch it, I'm using` logstash -f pipeline-test6.conf`. I'm still running into the same error: ```auto { "sequence" => 0, "@timestamp" => 2022-06-15T12:39:35.631Z, "host" => "logstash-dev03.elk.usherbrooke.ca", "@version" => "1", "message" => "Nov 2 10:40:49", "tags" => [ [0] "_dateparsefailure" ] } ``` Do you think it may be related to the LANG of my server? It has` LANG=en_CA.UTF-8` as value. Don't understand why it is working on your system, but mine... πŸ˜” Regards, Yanick ---
**Author:** ![yquirion](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yquirion/32/107068_2.png) [@yquirion](https://discuss.elastic.co/u/yquirion)\ **Post date:** [June 15, 2022, 12:49pm UTC](https://discuss.elastic.co/t/date-filter-dateparsefailure/307199/5 "2022-06-15T12:49:48Z")
Hi Badger! My bad! I tought the '``' was something like we have into grok filter such as `%{SPACE}`. But like I answered to aaron, even uysing his script will continue to fail with `_dateparsefailure`. Regards, Yanick ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [June 15, 2022, 4:32pm UTC](https://discuss.elastic.co/t/date-filter-dateparsefailure/307199/6 "2022-06-15T16:32:24Z")
> [@yquirion](#): > > Do you think it may be related to the LANG of my server? It has` LANG=en_CA.UTF-8` as value. Yes, that is the problem. Someone else had this problem [last week](https://discuss.elastic.co/t/logstash-filter-date-match-not-replacing-timestamp/306364/22) and solved it by using mutate+gsub to change the abbreviated month name to a month number. The problem is that that particular locale requires a period at the end of the month abbreviation. So `May 2 10:40:49` works, and `Nov. 2 10:40:49` works, but `Nov 2 10:40:49` cannot be parsed. ---
**Author:** ![yquirion](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yquirion/32/107068_2.png) [@yquirion](https://discuss.elastic.co/u/yquirion)\ **Post date:** [June 15, 2022, 5:49pm UTC](https://discuss.elastic.co/t/date-filter-dateparsefailure/307199/7 "2022-06-15T17:49:38Z")
Ho wow! You exactly right! Having this `May 14 15:05:01` will parse correctly, while having `Feb 14 15:05:01` will failed with `_dateparsefailure` tag πŸ™„ When I tried with a date such as `Feb. 14 15:05:01` this time, my grok parser will failed `_grokparsefailure_sysloginput`.The grok pattern %{SYSLOGTIMESTAMP} doesn't seems to understand the '.' after the month name. I will try to implement to solution from the other guy. I also open a case at Elastic letting them know this issue. From their documentation for date parser we can read the following: > The date filter is used for parsing dates from fields, and then using that date or timestamp as the logstash timestamp for the event. > > For example, syslog events usually have timestamps like this: > > ``` > `"Apr 17 09:32:01"` > > ``` > > You would use the date format MMM dd HH:mm:ss to parse this. > > The date filter is especially important for sorting events and for backfilling old data. If you don’t get the date correct in your event, then searching for them later will likely sort out of order. So I'm expecting this to work! Thanks again Badger, your help is very appreciated! ---
**Author:** ![yquirion](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yquirion/32/107068_2.png) [@yquirion](https://discuss.elastic.co/u/yquirion)\ **Post date:** [June 15, 2022, 6:04pm UTC](https://discuss.elastic.co/t/date-filter-dateparsefailure/307199/8 "2022-06-15T18:04:53Z")
Hi there! Here it is how I solved it and it is working fine. I have to play a bit because syslogs doesn't send the year into the date field. So it look like this: ```auto mutate { gsub => ["syslog.timestamp", "Jan ", "01-"] gsub => ["syslog.timestamp", "Feb ", "02-"] gsub => ["syslog.timestamp", "Mar ", "03-"] gsub => ["syslog.timestamp", "Apr ", "04-"] gsub => ["syslog.timestamp", "May ", "05-"] gsub => ["syslog.timestamp", "Jun ", "06-"] gsub => ["syslog.timestamp", "Jul ", "07-"] gsub => ["syslog.timestamp", "Aug ", "08-"] gsub => ["syslog.timestamp", "Sep ", "09-"] gsub => ["syslog.timestamp", "Oct ", "10-"] gsub => ["syslog.timestamp", "Nov ", "11-"] gsub => ["syslog.timestamp", "Dec ", "12-"] } date { match => ["syslog.timestamp","MM-dd HH:mm:ss", "MM- d HH:mm:ss"] timezone => "America/Toronto" target => "@timestamp" remove_field => ["syslog.timestamp"] } ``` Thank you so much @Badger, @yhache and @aaron-nimocks! ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [June 15, 2022, 6:11pm UTC](https://discuss.elastic.co/t/date-filter-dateparsefailure/307199/9 "2022-06-15T18:11:46Z")
> [@yquirion](#): > > I also open a case at Elastic letting them know this issue. It is not really something Elastic can address (except in terms of documentation). The problem is way down in the Unicode CLDR data that the JVM includes. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 13, 2022, 6:12pm UTC](https://discuss.elastic.co/t/date-filter-dateparsefailure/307199/10 "2022-07-13T18:12:43Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.