Syntactic error on logstash config

Elastic Stack Logstash
1 
[2017-06-30T19:06:06,714][DEBUG][logstash.agent           ] Reading config file {:config_file=>"/etc/logstash/conf.d/logstash.conf"}
[2017-06-30T19:06:06,902][ERROR][logstash.agent           ] Cannot create pipeline {:reason=>"Expected one of #, => at line 24, column 7 (byte 633) after output {\n  stdout { codec => rubydebug }\n\n  elasticsearch {\n    hosts => [\"localhost:9200\"]\n    manage_template => false\n\n    if", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:50:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:145:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:286:in `create_pipeline'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:95:in `register_pipeline'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:274:in `execute'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:185:in `run'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132:in `run'", "/usr/share/logstash/lib/bootstrap/environment.rb:71:in `(root)'"]}

Is the error I get when trying to run logstash with my new configuration, which looks like this:
input {
beats{
port => 5044
}
}

filter{
    if[message] =~ "^#" { drop{} }

    if[server] == "alkistis"{
        grok{
match => ["message", "%{TIMESTAMP_ISO8601:log_timestamp} %{WORD:iisSite} %{IPORHOST:site} %{WORD:method} %{URIPATH:page} %{NOTSPACE:querystring} %{NUMBER:port} %{NOTSPACE:username} %{IPORHOST:clienthost} %{NOTSPACE:useragent} %{NOTSPACE:referer} %{NUMBER:response} %{NUMBER:subresponse} %{NUMBER:scstatus} %{NUMBER:bytes:int} %{NUMBER:timetaken:int}"]

        }
    }
}

output {
stdout { codec => rubydebug }

elasticsearch {
    hosts => ["localhost:9200"]
    manage_template => false

    if[server] == "alkistis"{
        index => "alkistis-%{+YYYY.MM.dd}"
        document_type => "iis-log"
    }else{
        index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
        document_type => "%{[@metadata][type]}"
    }
}
}

The only thing I added from the previous version is the filter block and these if-alkistis blocks.

2

You can not have conditionals within the elasticsearch output config. Instead use conditionals in the filter block to set index prefix and type in the metadata and then refer to this in the elasticsearch output.

3

Do I need to use mutate in the filter tho?

4

Yes, something like this would allow you to use [@metadata][beat] and [@metadata][type] to specify the index prefix and type for all types of logs, which will simplify your Elasticsearch output config. You can naturally choose other metadata variables to hold this if you prefer that.

if[server] == "alkistis"{
  mutate {
    add_field => {
      "[@metadata][beat]" => "alkistis"
      "[@metadata][type]" => "iis-logs"
    }
  }
}
1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.