Custom grok_pattern on syslog input

[A_B]

Hi,

I am trying to ingest logs from our Juniper switshes and I'm trying to configure a syslog input. Manyally editing the Logstash input config I got this to work.

  # Juniper  syslog input
  syslog {
    port => 5518
    grok_pattern => "<%{POSINT:priority}>1 %{SYSLOGLINE}"
    }
  }

Unfortunately I can use this with Puppet. The input config file is an .erb template and ruby can not parse all the special characters...

I I created my own pattern to simplify the syntax

# cat /etc/logstash/patterns/extra_patterns
JUNIPER_INPUT <%{POSINT:priority}>1 %{SYSLOGLINE}

The I changed the input to

  # Juniper  syslog input
  syslog {
    port => 5518
    grok_pattern => %{JUNIPER_INPUT}"
    }
  }

Puppet would be fine with that but Logstash does not find the pattern JUNIPER_INPUT. If it was a grok filter I could set patterns_dir but that does not seem to be possible to do on the syslog input.

Any suggestions on how to get around this? I guess I could move everything to the filter section but I was very happy to see that it was possible to do some grok stuff already on the input...

-AB

[A_B]

I am still very interested in hearing suggestions and ideas on this. I did manage to find a working solution for now by looking through and choosing patterns already known to Logstash

  # Juniper log JSON input
  syslog {
    port => 5518
    grok_pattern => "%{SYSLOG5424PRI}1 %{SYSLOGLINE}"
    type => "juniper"
    add_field => {
      "[@metadata][index]" => "juniper"
      "[@metadata][log_prefix]" => "dc"
    }
  }

[waqark3389]

In the Puppet ERB file you need to escape the special characters... From first looks you have

grok_pattern => "<%{POSINT:priority}>1 %{SYSLOGLINE}"

Where as % is a puppet character so use:

grok_pattern => "<%%{POSINT:priority}>1 %{SYSLOGLINE}"

With regard to the extra patterns - you have to set patterns_dir inside the "filter" section and create a grok pattern there.
You can make the filter conditional by using if [type] == "juniper"

Here you are trying to filter inside the syslog input...

[A_B]

Thanks @waqark3389, I'll keep that in mind