Hi Logstash community,
I'm trying to parse a log through logstash but there seems to be an issue with my grok pattern, however I have tried a lot of different things now and always get the exact same error message.
The Log I'm trying to parse
[
{"name":"mmc.exe","pid":"6052","start_time":"1585318707"},
{"name":"notepad++.exe","pid":"5888","start_time":"1585318573"},
{"name":"firefox.exe","pid":"7456","start_time":"1585318484"},
{"name":"firefox.exe","pid":"5560","start_time":"1585318360"},
{"name":"firefox.exe","pid":"7668","start_time":"1585318348"}
]
My pipeline.conf
input {
beats {
port => "5044"
ssl => true
ssl_certificate_authorities => ["/usr/share/logstash/config/certs/ca.crt"]
ssl_certificate => "/usr/share/logstash/config/certs/ls-server.crt"
ssl_key => "/usr/share/logstash/config/certs/ls-server-pkcs8.key"
ssl_verify_mode => "none"
}
}
filter {
if "demo" in [tags]{
mutate {
gsub => [ "message", "\[", "","message", "\]", "" ]
}
grok {
match => { "message" => \{"name":%{QUOTEDSTRING:name},"pid":"%{NUMBER:pid}","start_time":"%{NUMBER:date}"\} }
}
}
}
Logstash wont boot and always throws this error:
Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \\t\\r\\n], \"#\", [A-Za-z0-9_-], '\"', \"'\", [A-Za-z_], \"-\", [0-9], \"[\", \"{\" at line 18, column 27 (byte 459) after filter {\n if \"demo\" in [tags]{\n mutate {\n gsub => [ \"message\", \"\\[\", \"\",\"message\", \"\\]\", \"\" ]\n }\n grok {\n match => { \"message\" => ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:58:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:66:in `compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:28:in `block in compile_sources'", "org/jruby/RubyArray.java:2577:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:27:in `compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:181:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:67:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:43:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:342:in `block in converge_state'"]}
The gsub part seems to be working fine, at least when I commented out the grok part the logstash booted without a problem.
Here is a link to the official documentation for the grok match plugin, maybe I'm reading it wrong but I cant seem to find any syntax error with my grok filter.
I debugged the GROK Pattern with the heroku debugger and it worked fine. I have also uploaded the log manually to Elasticsearchs Machine Learning tool and tried that grok filter out. Also to no avail.