Adding optional match pattern to grok for syslog messages

Elastic Stack Logstash
1

Currently I am using the following logstash configuration to match syslog messages in the form of:

Message:

Aug 10 01:09:56.071 (info) smtpd: [178cea3e-9c3a-11e8-93fa-f3ebd9db2b94] [SMTP] [EHLO] EHLO SEKO-SBS.sekonorthampton.local

Configuration:

input {
  tcp {
    port => 5000
    type => syslog
  }
}

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: \[%{UUID:msgid}\] %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
    }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss.SSS", "MMM dd HH:mm:ss.SSS" ]
    }
    mutate {
      remove_field => [ "syslog_timestamp", "message", "host" ]
    }
    if "_grokparsefailure" in [tags] {
      drop { }
    }
  }
}

output {
  elasticsearch { hosts => ["${ELASTICSEARCH_ENDPOINT}"] }
}

However, I need to add an optional match for the following slightly different message format:

Aug 10 01:07:56.371 (info) smtpd: <hg4892ty> [d0b9edad-9c39-11e8-93fa-f3ebd9db2b94] [SMTP] [EHLO] EHLO [contc1.ore.domain.com](http://contc1.ore.domain.com/)

Notice, it includes an additional string <hg4892ty>. Essentially I need help updating grok match to optionally accept <at least one character up to N characters of letters and numbers>

2 
(?<foo>[a-zA-Z0-9]+)

If you know what N is then you could replace the +. For example, if N is 10 then replace it with {1,10}

3

So the full additional to the grok string would be:

%{<[a-zA-Z0-9]+>:h_id}

Where the variable name is h_id and should optionally match a-zA-Z0-9 one or more.

4

No, the syntax should be what I shewed. This is not using %{} for pattern name expansion. It's a bare pattern.

(?<h_id>[a-zA-Z0-9]+)
5

I think using the following seems to work as well:

<?%{USERNAME:h_id}?>? ?

Note, need to option the space at the end and the opening and closing brackets.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.