Grok filter compile error

Hi,

I have the following log pattern

2019-01-31 01:27:48 10.24.32.4 POST 200 932 0.059 "394" - - - - - - /api/tbapi/services/oe/productinstances/findProductInstances?customerId=9151637854613445859&distributionChannelId=9150626251313798593&lightInit=false - - - - - - - - - -

And the following grok filter:

"%{DATE:date}%{SPACE}%{TIME:logtime}%{SPACE}%{IP:c-ip}%{SPACE}%{WORD:cs-method}%{SPACE}%{NUMBER:sc-status}%{SPACE}%{NUMBER:bytes:int}%{SPACE}%{NUMBER:duration:float}%{SPACE}(-|"(%{NUMBER:ContentLength})")%{SPACE}(-|%{WORD:x-nc.user})%{SPACE}(-|%{WORD:x-nctid})%{SPACE}(-|%{WORD:x-ncparentspan})%{SPACE}(-|%{WORD:x-ncspan})%{SPACE}(-|%{WORD:x-ncsampled})%{SPACE}(-|%{WORD:sXBSOICOMSUSER})%{SPACE}(-|%{URIPATHPARAM:cs-uri})%{SPACE}(-|%{WORD:x-ncsession})%{SPACE}(-|%{WORD:x-ncid})%{SPACE}(-|%{WORD:x-nctab})%{SPACE}(-|%{WORD:x-ncaction})%{SPACE}(-|%{WORD:x-details})%{SPACE}(-|%{WORD:cRequestID})%{SPACE}(-|%{WORD:cOperationID})%{SPACE}(-|%{WORD:sXBSOPROCESSID})%{SPACE}(-|%{URIPATHPARAM:cpageurl})%{SPACE}(-|%{WORD:cReferer})"

The issue is It compiles fine in grok debug sites like https://grokdebug.herokuapp.com/ but not in my logstash:

[ERROR] 2019-01-31 12:48:44.118 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] agent - Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, {, } at line 11, column 206 (byte 268) after filter {\n\n grok {\n\n match => { "message" => "%{DATE:date}%{SPACE}%{TIME:logtime}%{SPACE}%{IP:c-ip}%{SPACE}%{WORD:cs-method}%{SPACE}%{NUMBER:sc-status}%{SPACE}%{NUMBER:bytes:int}%{SPACE}%{NUMBER:duration:float}%{SPACE}(-|"", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:42:in compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:incompile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:12:in block in compile_sources'", "org/jruby/RubyArray.java:2486:inmap'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in compile_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:51:ininitialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:169:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:inexecute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:315:in block in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:inwith_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:312:in block in converge_state'", "org/jruby/RubyArray.java:1734:ineach'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:299:in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:166:inblock in converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:164:inconverge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:348:inblock in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}

Any idea?

You have double quotes embedded in the pattern. It is getting an error where it finds the first one. Either escape them using backslash or use single quotes around the entire pattern instead of double quotes.

It works! I'm using single quotes. Thanks

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.