LogStash ConfigurationError

Elastic Stack Logstash
1

Log.conf

1524206424.034   19395 207.96.0.0 TCP_MISS/304 15363 GET http://elastic.co/android-chrome-192x192.gif - DIRECT/10.0.5.120 -
1524206424.145     106 207.96.0.0 TCP_HIT/200 68247 GET http://elastic.co/guide/en/logstash/current/images/logstash.gif - NONE/- image/gif

logstash.conf

input

{

 file{

   path=> "/home/arun1297/POC/logs/log.conf"

   start_position=> "beginning"

   sincedb_path => "/dev/null"

 }

}



filter {
 dissect {
   mapping => {
     "message" => "%{timestamp->} %{duration} %{client_address} %{cache_result}/%{status_code} %{bytes} %{request_method} %{url} %{user} %{hierarchy_code}/%{server} %{content_type}"
    }
   remove_field => [“message”]
  }
}

output {
  stdout {
    codec => rubydebug
  }
}

I'm getting below error when I run bin/logstash -f logstash.conf

[2018-11-09T13:55:13,466][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, \", ', -, [, {, ] at line 15, column 21 (byte 388) after filter {\r\n dissect {\r\n   mapping => {\r\n     \"message\" => \"%{timestamp->} %{duration} %{client_address} %{cache_result}/%{status_code} %{bytes} %{request_method} %{url} %{user} %{hierarchy_code}/%{server} %{content_type}\"\r\n    }\r\n   remove_field => [", :backtrace=>["/home/arun1297/Downloads/logstash-6.4.2/logstash-core/lib/logstash/compiler.rb:41:in `compile_imperative'", "/home/arun1297/Downloads/logstash-6.4.2/logstash-core/lib/logstash/compiler.rb:49:in `compile_graph'", "/home/arun1297/Downloads/logstash-6.4.2/logstash-core/lib/logstash/compiler.rb:11:in `block in compile_sources'", "org/jruby/RubyArray.java:2486:in `map'", "/home/arun1297/Downloads/logstash-6.4.2/logstash-core/lib/logstash/compiler.rb:10:in `compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:149:in `initialize'", "/home/arun1297/Downloads/logstash-6.4.2/logstash-core/lib/logstash/pipeline.rb:22:in `initialize'", "/home/arun1297/Downloads/logstash-6.4.2/logstash-core/lib/logstash/pipeline.rb:90:in `initialize'", "/home/arun1297/Downloads/logstash-6.4.2/logstash-core/lib/logstash/pipeline_action/create.rb:38:in `execute'", "/home/arun1297/Downloads/logstash-6.4.2/logstash-core/lib/logstash/agent.rb:309:in `block in converge_state'"]}
2

If I remove remove_field => [“message”] from filter it works.. but this code was given in elastic search tutorial.

3

In which tutorial was this example given?

4

yes correct! from https://www.elastic.co/blog/a-practical-introduction-to-logstash

5

It worked fine when I wrote that blog post. It seems the issue is with the quotation marks in the remove_field statement. If I replace them with "or copy it straight from the blog post it all works though. I therefore wonder if you have had them replaced while copying and pasting them somehow?

6

If I copied from blog the code is this remove_field => [“message”] and if I manually enter and the code is this remove_field => ["message"] which is correct double quotation mark.. some problem with keyboard layout I guess.. Thanks! Now it is working

7

Am facing the same issue ...... can someone pls help ... below is my config file

input {
file{
path => ["/etc/data/dept_subsystem.csv"]
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
csv {
separator => ","
columns => ["Universal Id","State","Sv","Age","CQ Pri","CMVC ID","CMVC Track","CR Fix Date","Platform","Subsys","Reported Release","Release","Fix Potential","Component","Phase Found","Submitter","Owner","Headline","Symptoms","Keywords","Reference","LastUpdate","DaysSinceOpen","Dept","Projected Date","Action","Subm Dept","CI"]
}

}

output{
elasticsearch{
hosts => "localhost"
index => "defects_new1"
document_type => "defects_temp1"
}
stdout{}
}

8

Please open a new thread and post any errors you are getting.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.