Date Filter - _dateparsefailure

yquirion · June 14, 2022, 9:12pm

Dear all,

I'm running into a strange problem converting syslogtimestamp field into @timestamp.

After reading the thread: https://discuss.elastic.co/t/date-filter--dateparsefailure-solved/64692, I can't make it to work. When I launch this simple command, it failed:

echo "Nov  2 10:40:49" | logstash -e 'input { stdin {} } filter { date { match => [ "message", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss"] } }'

{
      "@version" => "1",
       "message" => "Nov 2 10:40:49",
    "@timestamp" => 2022-06-14T21:05:58.605Z,
          "tags" => [
        [0] "_dateparsefailure"
    ],
          "host" => "logstash-dev03.elk.usherbrooke.ca"
}

I've also tried to use the MMM<space><space>d HH:mm:ss, but logstash failed with this error:

[ERROR] 2022-06-14 17:02:54.415 [Converge PipelineAction::Create<main>] agent - Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: Illegal pattern component: p", :backtrace=>["org.logstash.config.ir.CompiledPipeline.<init>(CompiledPipeline.java:120)", "org.logstash.execution.JavaBasePipelineExt.initialize(JavaBasePipelineExt.java:86)", "org.logstash.execution.JavaBasePipelineExt$INVOKER$i$1$0$initialize.call(JavaBasePipelineExt$INVOKER$i$1$0$initialize.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:837)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1169)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuperSplatArgs(IRRuntimeHelpers.java:1156)", "org.jruby.ir.targets.InstanceSuperInvokeSite.invoke(InstanceSuperInvokeSite.java:39)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$initialize$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:48)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:333)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:87)", "org.jruby.RubyClass.newInstance(RubyClass.java:939)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0(/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0$__VARARGS__(/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:50)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "usr.share.logstash.logstash_minus_core.lib.logstash.agent.RUBY$block$converge_state$2(/usr/share/logstash/logstash-core/lib/logstash/agent.rb:388)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)", "org.jruby.runtime.Block.call(Block.java:139)", "org.jruby.RubyProc.call(RubyProc.java:318)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)", "java.base/java.lang.Thread.run(Thread.java:829)"]}
warning: thread "Converge PipelineAction::Create<main>" terminated with exception (report_on_exception is true):
LogStash::Error: Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create<main>`
          create at org/logstash/execution/ConvergeResultExt.java:135
             add at org/logstash/execution/ConvergeResultExt.java:60
  converge_state at /usr/share/logstash/logstash-core/lib/logstash/agent.rb:401
[ERROR] 2022-06-14 17:02:54.421 [Agent thread] agent - An exception happened when converging configuration {:exception=>LogStash::Error, :message=>"Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create<main>`"}
[FATAL] 2022-06-14 17:02:54.438 [LogStash::Runner] runner - An unexpected error occurred! {:error=>#<LogStash::Error: Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create<main>`>, :backtrace=>["org/logstash/execution/ConvergeResultExt.java:135:in `create'", "org/logstash/execution/ConvergeResultExt.java:60:in `add'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:401:in `block in converge_state'"]}
[FATAL] 2022-06-14 17:02:54.445 [LogStash::Runner] Logstash - Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.20.1.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.20.1.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:94) ~[?:?]

So I'm a bit confused on how to make this syslog timestamp date into @timestamp field.

Any idea?

I'm runniong logstash v. 7.17.4.

Regards!
Yanick

aaron-nimocks · June 14, 2022, 10:03pm

It works for me. Only thing you are doing differently is starting it from the command line so I would look at that first.

input { stdin {} }
filter {
  date { match => [ "message", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss"] } 
}
output { stdout { codec => "json_lines" } }

or

input { generator { count => 1 lines => [ '{ "message": "Nov 2 10:40:49" }' ] codec => json } }
filter {
  date { match => [ "message", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss"] } 
}
output { stdout { codec => "json_lines" } }

Badger · June 14, 2022, 10:22pm

It works for me too.

There is only a single space between the month and the day, but that is fine since the dd in "MMM dd HH:mm:ss" will match one or two digits.

You cannot use <space> to represent a space. Alphabetic characters in the pattern have to be valid Joda time specifiers. The p in "space" is causing this exception, if you removed that it would complain about the c. If you removed that your pattern would match Nov<59am3>2 10:40:49 which has seconds, am/pm, and numeric day of the week between the <>.

yquirion · June 15, 2022, 12:46pm

Hi there,

I tried the following:

input { generator { codec => json count => 1 lines => [ '{"message":"Nov  2 10:40:49"}' ] } }

filter {
  date { match => [ "message", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss"] } 
}

output {
  stdout { 
    codec => rubydebug { metadata => true } 
  }
}

To launch it, I'm using logstash -f pipeline-test6.conf. I'm still running into the same error:

{
      "sequence" => 0,
    "@timestamp" => 2022-06-15T12:39:35.631Z,
          "host" => "logstash-dev03.elk.usherbrooke.ca",
      "@version" => "1",
       "message" => "Nov  2 10:40:49",
          "tags" => [
        [0] "_dateparsefailure"
    ]
}

Do you think it may be related to the LANG of my server? It has LANG=en_CA.UTF-8 as value.

Don't understand why it is working on your system, but mine...

Regards,
Yanick

yquirion · June 15, 2022, 12:49pm

Hi Badger!

My bad! I tought the '<space>' was something like we have into grok filter such as %{SPACE}.

But like I answered to aaron, even uysing his script will continue to fail with _dateparsefailure.

Regards,
Yanick

Badger · June 15, 2022, 4:32pm

Yes, that is the problem. Someone else had this problem last week and solved it by using mutate+gsub to change the abbreviated month name to a month number. The problem is that that particular locale requires a period at the end of the month abbreviation. So May 2 10:40:49 works, and Nov. 2 10:40:49 works, but Nov 2 10:40:49 cannot be parsed.

yquirion · June 15, 2022, 5:49pm

Ho wow! You exactly right!

Having this May 14 15:05:01 will parse correctly, while having Feb 14 15:05:01 will failed with _dateparsefailure tag

When I tried with a date such as Feb. 14 15:05:01 this time, my grok parser will failed _grokparsefailure_sysloginput.The grok pattern %{SYSLOGTIMESTAMP} doesn't seems to understand the '.' after the month name.

I will try to implement to solution from the other guy. I also open a case at Elastic letting them know this issue. From their documentation for date parser we can read the following:

The date filter is used for parsing dates from fields, and then using that date or timestamp as the logstash timestamp for the event.

For example, syslog events usually have timestamps like this:
`"Apr 17 09:32:01"`
You would use the date format MMM dd HH:mm:ss to parse this.

The date filter is especially important for sorting events and for backfilling old data. If you don’t get the date correct in your event, then searching for them later will likely sort out of order.

So I'm expecting this to work!

Thanks again Badger, your help is very appreciated!

yquirion · June 15, 2022, 6:04pm

Hi there!

Here it is how I solved it and it is working fine. I have to play a bit because syslogs doesn't send the year into the date field. So it look like this:

  mutate {
    gsub => ["syslog.timestamp", "Jan ", "01-"]
    gsub => ["syslog.timestamp", "Feb ", "02-"]
    gsub => ["syslog.timestamp", "Mar ", "03-"]
    gsub => ["syslog.timestamp", "Apr ", "04-"]
    gsub => ["syslog.timestamp", "May ", "05-"]
    gsub => ["syslog.timestamp", "Jun ", "06-"]
    gsub => ["syslog.timestamp", "Jul ", "07-"]
    gsub => ["syslog.timestamp", "Aug ", "08-"]
    gsub => ["syslog.timestamp", "Sep ", "09-"]
    gsub => ["syslog.timestamp", "Oct ", "10-"]
    gsub => ["syslog.timestamp", "Nov ", "11-"]
    gsub => ["syslog.timestamp", "Dec ", "12-"]    
  }

   date {
     match => ["syslog.timestamp","MM-dd HH:mm:ss", "MM- d HH:mm:ss"]
     timezone => "America/Toronto"
     target => "@timestamp"
     remove_field => [ "syslog.timestamp" ]
   }

Thank you so much @Badger, @yhache and @aaron-nimocks!

Badger · June 15, 2022, 6:11pm

It is not really something Elastic can address (except in terms of documentation). The problem is way down in the Unicode CLDR data that the JVM includes.

system · July 13, 2022, 6:12pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
_dateparsefailure again Logstash	3	1148	June 27, 2017
Syslog filter issue with timestamp Logstash	13	1854	January 3, 2022
Data parse error ["_dateparsefailure"] Logstash	9	1031	August 15, 2018
Dateparse failure When trying to get date from xml file Logstash	8	657	May 14, 2020
Error using date match filter Logstash	5	1969	July 6, 2017

Date Filter - _dateparsefailure

Related topics