Input / Filter problems with 6.3.2

[DanielE (Daniel Ebling)]

Hi,

i´m having a problem with my new 6.3.2 stack. I am using three different conf files for Input, filter and ouput. At the Moment i can´t add a field in my Input to start using my filter. Also my experiements to use tags as a filter option didn´t work.

Input.conf

input {
  syslog {
      port => 5000
      syslog_field => "syslog"
      grok_pattern => "<%{POSINT:priority}>%{SYSLOGTIMESTAMP:timestamp} CUSTOM GROK HERE"
      tags => "syslog"
  }
  add_field => {
      origin => "syslog"
  }
}

filter.conf

filter {
  if [tags] == "syslog" {
      grok {
         match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
         add_field => [ "received_at", "%{@timestamp}" ]
         add_field => [ "received_from", "%{host}" ]
      }
      date {
         match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
      }
      mutate {
         add_field => {
            retention => "medium"
         }
      }
  }
}

Output.conf

output {
    elasticsearch {
            index => "logstash-%{retention}-%{+YYYY.MM.dd}"
        }
    }

Logstash Error

[2018-08-02T10:50:25,917][ERROR][logstash.agent ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Expected one of #, { at line 26, column 13 (byte 749) after input {\n syslog {\n port => 5000\n syslog_field => "syslog"\n grok_pattern => "<%{POSINT:priority}>%{SYSLOGTIMESTAMP:timestamp} CUSTOM GROK HERE"\n tags => "syslog"\n }\n add_field ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:42:in compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:incompile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:12:in block in compile_sources'", "org/jruby/RubyArray.java:2486:inmap'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in compile_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:49:ininitialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/reload.rb:38:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:305:inblock in converge_state'"]}

[magnusbaeck (Magnus Bäck)]

The add_field option in your input block needs to go inside the syslog input (rather than be nested directly under "input").

[DanielE (Daniel Ebling)]

Thank you Magnus, it now works!

I needed to put origin in double quotes. The final input looks like this:

input {
  syslog {
      port => 5000
      syslog_field => "syslog"
      tags => "syslog"
      add_field => {
          "origin" => "syslog"
      }
  }
}