Hello, I'm starting to use ELK and I'm attempting to start logstash with a custom logstash.conf file but I've an error :
Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, \", ', -, [, { at line 10, column 37 (byte 149) after filter {\n if \"HTTP\" in [message] {\n grok {\n mapping => { \"message\" => ", :backtrace=>["/opt/logstash/logstash-core/lib/logstash/compiler.rb:41:in `compile_imperative'", "/opt/logstash/logstash-core/lib/logstash/compiler.rb:49:in `compile_graph'", "/opt/logstash/logstash-core/lib/logstash/compiler.rb:11:in `block in compile_sources'", "org/jruby/RubyArray.java:2486:in `map'", "/opt/logstash/logstash-core/lib/logstash/compiler.rb:10:in `compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:149:in `initialize'", "/opt/logstash/logstash-core/lib/logstash/pipeline.rb:22:in `initialize'", "/opt/logstash/logstash-core/lib/logstash/pipeline.rb:90:in `initialize'", "/opt/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:42:in `block in execute'", "/opt/logstash/logstash-core/lib/logstash/agent.rb:92:in `block in exclusive'", "org/jruby/ext/thread/Mutex.java:148:in `synchronize'", "/opt/logstash/logstash-core/lib/logstash/agent.rb:92:in `exclusive'", "/opt/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:38:in `execute'", "/opt/logstash/logstash-core/lib/logstash/agent.rb:317:in `block in converge_state'"
Here my configuration file :
input {
beats {
port => 5044
codec => "json"
}
}
filter {
if [message] =~ "HTTP" {
grok {
mapping => { "message" => %{TIMESTAMP_ISO8601:timestamp} %{WORD:type} %{LOGLEVEL:level} "%{WORD:method} %{URIPATHPARAM:url}" %{INT:code} %{INT:bytes} - %{GREEDYDATA:response_time} }
}
}
else if [message] =~ "APP" {
grok {
mapping => { "message" => %{TIMESTAMP_ISO8601:timestamp} %{WORD:type} %{LOGLEVEL:level} %{GREEDYDATA:jsonstring} }
}
json {
source => "jsonstring"
target => "doc"
}
mutate {
add_field => {
"code" => "%{[doc][code]}"
"message" => "%{[doc][message]}"
}
}
}
}
output {
elasticsearch {
hosts => ["localhost"]
}
}
Logs sample I want to get back (docker stdout logs) :
-
HTTP request logs
2019-01-29T18:35:15.423Z HTTP INFO "POST /myroute/?param1=test" 201 41 - 44.014 ms -
APP logs
2019-01-29T18:48:19.657Z APP ERROR : {"code":201,"message":"ok"}
discriminator : APP or HTTP
What is wrong with this config ? I tried a change a lot things specially around line 37 but i don't understand why it doesn't work.
Thank you very much for your help