COMMONAPACHELOG vs Explicit replacement


(JM) #1

gi,
can some1 help with this one ? ( ES 7, LS 7)

if i use COMMONAPACHELOG logstash works ...
if i replace it for its "long version" gives me an error...
( i only use 1 at a time, cant edit the coments)

full pattern taken from
#https://github.com/elastic/logstash/blob/v1.4.2/patterns/grok-patterns

grok { 
	match => { 
		"message" => "%{COMMONAPACHELOG} %{NUMBER:timetaken:int}"
		"message" => "%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{NUMBER:timetaken:int}"
	} 
}

error is :slightly_frowning_face:
[2019-04-17T15:36:34,298][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.0.0"}
[2019-04-17T15:36:35,986][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:apache, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, {, } at line 11, column 93 (byte 269) after filter{\n\tgrok { \n\t\tmatch => { \n#\t\t\t"message" => "%{COMMONAPACHELOG} %{NUMBER:timetaken:int}"\n\t\t\t"message" => "%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "", :backtrace=>["C:/Kibana_ElasticSearch/logstash/logstash-core/lib/logstash/compiler.rb:41:in compile_imperative'", "C:/Kibana_ElasticSearch/logstash/logstash-core/lib/logstash/compiler.rb:49:incompile_graph'", "C:/Kibana_ElasticSearch/logstash/logstash-core/lib/logstash/compiler.rb:11:in block in compile_sources'", "org/jruby/RubyArray.java:2577:inmap'", "C:/Kibana_ElasticSearch/logstash/logstash-core/lib/logstash/compiler.rb:10:in compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:151:ininitialize'", "org/logstash/execution/JavaBasePipelineExt.java:47:in initialize'", "C:/Kibana_ElasticSearch/logstash/logstash-core/lib/logstash/java_pipeline.rb:23:ininitialize'", "C:/Kibana_ElasticSearch/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:36:in execute'", "C:/Kibana_ElasticSearch/logstash/logstash-core/lib/logstash/agent.rb:325:inblock in converge_state'"]}

any toughts?

JM


(JM) #2

from


tryed with
"message" => "%{IPORHOST:clientip} %{HTTPDUSER:ident} %{HTTPDUSER:auth} [%{HTTPDATE:timestamp}] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{NUMBER:timetaken:int}"

same problem...


#3

If you have double quotes in your pattern then surround it using single quotes

match => {  "message" => '%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{NUMBER:timetaken:int}' }

(JM) #4

Had to be something like tat !
1/2 a day lost and own sanity in check and it was a devilish detail.

Thanks a lot !