Unable to parse specific log with logstash 6.3

Elastic Stack Logstash
1

Dear all,

I'm new on logstash, and I'm trying to parse this log with logstash:

2018-07-10T10:46:25+0200 coll_bddf [3] [0] [6] [24] [2] [5]

so, I've create the logstash configuration file below:

input {
file {
path => "/var/log/logstash/SOLR_LOGS/Solr_generate_1.log"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}

filter {
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{USERNAME:collection-name} [%{INT:global_time}] [%{INT:num_docs}] [%{INT:creation_time}] [%{INT:commit_time}] [%{INT:search_time}] [%{INT:delete_time}]" }
overwrite => [“message”]
}
}

output {
elasticsearch{
hosts => ["192.168.186.120:9200"]
cluster => "elasticsearch"
protocol => "http"
index => "solrcollbddf-%{+YYYY.MM.dd}"
stdout { codec => rubydebug }
}
}

but, when I trying to run logstash, I receive this kind of error:

[2018-07-10T10:26:17,981][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, ", ', -, [, {, ] at line 12, column 15 (byte 393) after filter {\ngrok {\nmatch => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{USERNAME:collection-name} \[%{INT:global_time}\] \[%{INT:num_docs}\] \[%{INT:creation_time}\] \[%{INT:commit_time}\] \[%{INT:search_time}\] \[%{INT:delete_time}\]" }\noverwrite => [", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:42:in compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:incompile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:12:in block in compile_sources'", "org/jruby/RubyArray.java:2486:inmap'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in compile_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:49:ininitialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:167:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:inexecute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:305:in `block in converge_state'"]}
[2018-07-10T10:26:18,519][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

Could you Help me to resolve this issue?

Many Thanks for your help

Kind regards

2

These two parameters are no longer supported. See the docs for a list of supported configuration parameters.

This is a separate plugin and can not be specified within the elasticsearch plugin.

3

Christian,

Many thanks for your feed back.

So, I removed the parameters, and now my conf is like that:

input {
file {
path => "/var/log/logstash/SOLR_LOGS/Solr_generate_1.log"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}

filter {
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{USERNAME:collection-name} [%{INT:global_time}] [%{INT:num_docs}] [%{INT:creation_time}] [%{INT:commit_time}] [%{INT:search_time}] [%{INT:delete_time}]" }
overwrite => [“message”]
}
}

output {
elasticsearch{
hosts => ["192.168.186.120:9200"]
index => "solrcollbddf-%{+YYYY.MM.dd}"
}
}

But, I'm still have an issue, the error is:

[2018-07-10T12:05:54,634][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, ", ', -, [, {, ] at line 12, column 15 (byte 393) after filter {\ngrok {\nmatch => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{USERNAME:collection-name} \[%{INT:global_time}\] \[%{INT:num_docs}\] \[%{INT:creation_time}\] \[%{INT:commit_time}\] \[%{INT:search_time}\] \[%{INT:delete_time}\]" }\noverwrite => [", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:42:in compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:incompile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:12:in block in compile_sources'", "org/jruby/RubyArray.java:2486:inmap'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in compile_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:49:ininitialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:167:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:inexecute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:305:in `block in converge_state'"]}
[2018-07-10T12:05:55,416][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

The problem seems to be at the filter step, but I did it with an inline grok debugger and it give a correct response.

4

Check that you have proper quotation marks here. The config isn't formatted properly, so it is hard to tell what is wrong.

5

Christian, many thanks it's working fine!! the quotation marks were wrong!! I replaced them, an now is working properly!!
The topic can be closed!!

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.