GROK Filter fails and not able to find out the solution

Hi Team,
I'm trying to use the grok filter in the logstash configuration pipeline. The grok pattern seem to work in the kibana provided grok debugger. But using the pattern in the filter throws the error that logstash configuration error.

Logstash Config filter
filter {
if [path] == "/media/sf_ELK/ATG01log.txt" {
grok {
match => { "message" => "(?%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}) (?(?:[A-Z]+)) (?(?:[[A-Za-z.]+])) (?(?:[a-zA-Z0-9]+)) (?(?:[a-zA-Z0-9]{7}-[a-zA-Z0-9]{18}-[a-zA-Z0-9]{3}-[a-zA-Z0-9]{9}-[a-zA-Z0-9]{11}!-[0-9]{9}![0-9]{13})) (?(?:[a-z0-9]+)) (?(?:AEO_(CA|INTL|US))) (?(?:(en|fr)_(US|CA))) (?<request_type>(?:[A-Z]+)) (?<request_path>(?:[/a-zA-Z0-9]+)) (?<active_thread>(?:([.'))) (?<thread_info>(?:.],)) %{GREEDYDATA:body}"}
tag_on_failure => ["grok-parse-failed"]
}
kv {
trimkey => "\s"
field_split => ",?"
}
}
}

Sample Log
2019-07-24 08:29:54,838 INFO [nucleusNamespace.aeo.service.jaxrs.LoggingResponseFilter] 80f4dd01fbed4e688ab24e822866c08a 87Yj9Zv-GlrPJkesCNodmLHg3S-VgY-rbdwfebRc-C2cSDL0JSkX!-211948433!1563971394558 ugp417017865 XXX_CA en_US POST /public/v1/currentUser/Login ([ACTIVE] ExecuteThread: '16' for queue: 'weblogic.kernel.Default (self-tuning)') {thread=Thread[[ACTIVE] ExecuteThread: '16' for queue: 'weblogic.kernel.Default (self-tuning)',5,Pooled Threads], threadId=91, sessionId=null, profileId=ugp417017816, userEmail=test@test.com, customerIPAddress=127.0.0.1, User-Agent=Mozilla/6.3.0.1365-Develop (iPhone; iOS 11.2.5; Scale/3.0), x-forwarded-for=127.0.0.1} Authentication status=200, loginID=test@test.com CAP Route End: route=/public/v1/currentUser/Login#POST, totalTime=273 ms

Logstash Exception
Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, => at line 11, column 542 (byte 734) after filter { \n if [path] == "/media/sf_ELK/ATG01log.txt" {\n\tgrok {\n\t\tmatch => { "message" => {'(?%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}) (?(?:[A-Z]+)) (?(?:\[[A-Za-z\.]+\])) (?(?:[a-zA-Z0-9]+)) (?(?:[a-zA-Z0-9]{7}\-[a-zA-Z0-9]{18}\-[a-zA-Z0-9]{3}\-[a-zA-Z0-9]{9}\-[a-zA-Z0-9]{11}\!\-[0-9]{9}\![0-9]{13})) (?(?:[a-z0-9]+)) (?(?:AEO_(CA|INTL|US))) (?(?:(en|fr)_(US|CA))) (?<request_type>(?:[A-Z]+)) (?<request_path>(?:[\/a-zA-Z0-9]+)) (?<active_thread>(?:\(\[.\'\))) (?<thread_info>(?:.\]\,)) %{GREEDYDATA:body}'", :backtrace=>["/media/sf_ELK/logstash-7.0.1/logstash-core/lib/logstash/compiler.rb:41:in compile_imperative'", "/media/sf_ELK/logstash-7.0.1/logstash-core/lib/logstash/compiler.rb:49:incompile_graph'", "/media/sf_ELK/logstash-7.0.1/logstash-core/lib/logstash/compiler.rb:11:in block in compile_sources'", "org/jruby/RubyArray.java:2577:inmap'", "/media/sf_ELK/logstash-7.0.1/logstash-core/lib/logstash/compiler.rb:10:in compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:151:ininitialize'", "org/logstash/execution/JavaBasePipelineExt.java:47:in initialize'", "/media/sf_ELK/logstash-7.0.1/logstash-core/lib/logstash/java_pipeline.rb:23:ininitialize'", "/media/sf_ELK/logstash-7.0.1/logstash-core/lib/logstash/pipeline_action/create.rb:36:in execute'", "/media/sf_ELK/logstash-7.0.1/logstash-core/lib/logstash/agent.rb:325:inblock in converge_state'"]}

@magnusbaeck Would you be able to help out on this.

I have fixed the issue myself.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.