Expected one of #, { in logstash configuration parsing

Elastic Stack Logstash
1

Hi,
I am receiving the below error of checking logstash parsing:

Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, { at line 28, column 22 (byte 951) after filter {\n  if \"onm-server\" in [tags] {\n    grok {\n      keep_empty_captures => true\n      patterns_dir => [\"/etc/logstash/conf.d/patterns\"]\n      match => { \"message\" => \"%{DATE:date}%{SPACE}%{MONTH:month}%{SPACE}%{YEAR:year}%{SPACE}%{TIME:time}%{SPACE}%{SPACE}%{LOGLEVEL:log_level}%{SPACE}%{START_END:msg_onm}\" }\n      tag_on_failure => [\"grok1\"]\n      remove_tag => [ \"_grokparsefailure\" ]\n    }\n\n    grok {\n      patterns_dir => [\"/etc/logstash/conf.d/patterns\"]\n      match => { \"message\" => \"%{DATE:date}%{SPACE}%{MONTH:month}%{SPACE}%{YEAR:year}%{SPACE}%{TIME:time}%{SPACE}%{SPACE}%{LOGLEVEL:log_level}%{SPACE}%{START_END:msg_onm}\\s+%{LOGLEVEL:log_err}%{SPACE}%{NUMBER:error_no}\\s+\" }\n      tag_on_failure => [\"grok2\"]\n      remove_tag => [ \"_grokparsefailure\" ]\n    }\n\n    if \"grok2\" in [tags] {\n      tag_on_failure ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:42:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:in `compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:12:in `block in compile_sources'", "org/jruby/RubyArray.java:2486:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in `compile_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:51:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:169:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:315:in `block in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:312:in `block in converge_state'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:299:in `converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:166:in `block in converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:164:in `converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:348:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}

Here's my config:

input {
  beats {
    port => 5044
  }
}

# The filter part of this file is commented out to indicate that it is
# optional.

filter {
  if "onm-server" in [tags] {
    grok {
      keep_empty_captures => true
      patterns_dir => ["/etc/logstash/conf.d/patterns"]
      match => { "message" => "%{DATE:date}%{SPACE}%{MONTH:month}%{SPACE}%{YEAR:year}%{SPACE}%{TIME:time}%{SPACE}%{SPACE}%{LOGLEVEL:log_level}%{SPACE}%{START_END:msg_onm}" }
      tag_on_failure => ["grok1"]
      remove_tag => [ "_grokparsefailure" ]
    }

    grok {
      patterns_dir => ["/etc/logstash/conf.d/patterns"]
      match => { "message" => "%{DATE:date}%{SPACE}%{MONTH:month}%{SPACE}%{YEAR:year}%{SPACE}%{TIME:time}%{SPACE}%{SPACE}%{LOGLEVEL:log_level}%{SPACE}%{START_END:msg_onm}\s+%{LOGLEVEL:log_err}%{SPACE}%{NUMBER:error_no}\s+" }
      tag_on_failure => ["grok2"]
      remove_tag => [ "_grokparsefailure" ]
    }

    if "grok2" in [tags] {
      tag_on_failure => []
    } else if "grok1" in [tags] {
      remove_tag => [ "grok1" ]
    }
    mutate {
      convert => { "error_no" => "integer" }
    }


  }

I've been looking at the if condition but I was not able to track down the issue.

2

You do not have any filters, e.g. mutate, defined in this block.

3

Tried by adding mutate { remove_tag => ["grok2"] }, but still didn't work.

Update:

This worked. Just a question, why is that tag_on_failure requires mutate filter and remove_tag doesn't? I only added mutate filter in if condition only.

4

This worked. Just a question, why is that tag_on_failure requires mutate filter and remove_tag doesn't? I only added mutate filter in if condition only.

It's very hard to understand what you're asking. An example might help.

5

Okay. Here's an example:

if "grok2" in [tags] {
      tag_on_failure => []
} else if "grok1" in [tags] {
      mutate { remove_tag => ["grok1"] }
}

If you can see we're using mutate for remove_tag. But tag_on_failure works without it (at least I think it does, I might be wrong). I just wanted to know if there's a difference as I am trying to explain between these two.

6

Putting tag_on_failure => [] alone inside an if block doesn't work. tag_on_failure is an option to the grok filter.

7

Okay. Thanks for the information.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.