# Adding optional match pattern to grok for syslog messages

**URL:** <https://discuss.elastic.co/t/adding-optional-match-pattern-to-grok-for-syslog-messages/144250>\
**Category:** Logstash\
**Created:** [August 13, 2018, 10:51pm UTC](https://discuss.elastic.co/t/adding-optional-match-pattern-to-grok-for-syslog-messages/144250 "2018-08-13T22:51:06Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![nodesocket](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nodesocket/32/27023_2.png) [@nodesocket](https://discuss.elastic.co/u/nodesocket)\
**Post date:** [August 13, 2018, 10:51pm UTC](https://discuss.elastic.co/t/adding-optional-match-pattern-to-grok-for-syslog-messages/144250/1 "2018-08-13T22:51:06Z")

</div>

Currently I am using the following logstash configuration to match syslog messages in the form of:

**Message** :

`Aug 10 01:09:56.071 (info) smtpd: [178cea3e-9c3a-11e8-93fa-f3ebd9db2b94] [SMTP] [EHLO] EHLO SEKO-SBS.sekonorthampton.local`

**Configuration:**

```
input {
  tcp {
    port => 5000
    type => syslog
  }
}

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: \[%{UUID:msgid}\] %{GREEDYDATA:syslog_message}" }
      add_field => ["received_at", "%{@timestamp}"]
    }
    date {
      match => ["syslog_timestamp", "MMM d HH:mm:ss.SSS", "MMM dd HH:mm:ss.SSS"]
    }
    mutate {
      remove_field => ["syslog_timestamp", "message", "host"]
    }
    if "_grokparsefailure" in [tags] {
      drop { }
    }
  }
}

output {
  elasticsearch { hosts => ["${ELASTICSEARCH_ENDPOINT}"] }
}

```

However, I need to add an optional match for the following slightly different message format:

`Aug 10 01:07:56.371 (info) smtpd: <hg4892ty> [d0b9edad-9c39-11e8-93fa-f3ebd9db2b94] [SMTP] [EHLO] EHLO [contc1.ore.domain.com](http://contc1.ore.domain.com/)`

Notice, it includes an additional string `<hg4892ty>`. Essentially I need help updating grok match to optionally accept `<at least one character up to N characters of letters and numbers>`

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 14, 2018, 12:22am UTC](https://discuss.elastic.co/t/adding-optional-match-pattern-to-grok-for-syslog-messages/144250/2 "2018-08-14T00:22:14Z")

</div>

> [@nodesocket](#):
>
> \<at least one character up to N characters of letters and numbers\>

```
(?<foo>[a-zA-Z0-9]+)

```

If you know what N is then you could replace the +. For example, if N is 10 then replace it with {1,10}

---

<div class="post-metadata">

**Author:** ![nodesocket](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nodesocket/32/27023_2.png) [@nodesocket](https://discuss.elastic.co/u/nodesocket)\
**Post date:** [August 21, 2018, 10:07pm UTC](https://discuss.elastic.co/t/adding-optional-match-pattern-to-grok-for-syslog-messages/144250/3 "2018-08-21T22:07:30Z")

</div>

So the full additional to the grok string would be:

```auto
%{<[a-zA-Z0-9]+>:h_id}

```

Where the variable name is `h_id` and should optionally match `a-zA-Z0-9` one or more.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 21, 2018, 10:40pm UTC](https://discuss.elastic.co/t/adding-optional-match-pattern-to-grok-for-syslog-messages/144250/4 "2018-08-21T22:40:42Z")

</div>

> [@Badger](#):
>
> (?\<foo\>[a-zA-Z0-9]+)

No, the syntax should be what I shewed. This is not using %{} for pattern name expansion. It's a bare pattern.

```
(?<h_id>[a-zA-Z0-9]+)

```

---

<div class="post-metadata">

**Author:** ![nodesocket](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nodesocket/32/27023_2.png) [@nodesocket](https://discuss.elastic.co/u/nodesocket)\
**Post date:** [August 21, 2018, 11:25pm UTC](https://discuss.elastic.co/t/adding-optional-match-pattern-to-grok-for-syslog-messages/144250/5 "2018-08-21T23:25:16Z")

</div>

I think using the following seems to work as well:

```auto
<?%{USERNAME:h_id}?>? ?

```

Note, need to option the space at the end and the opening and closing brackets.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 18, 2018, 11:25pm UTC](https://discuss.elastic.co/t/adding-optional-match-pattern-to-grok-for-syslog-messages/144250/6 "2018-09-18T23:25:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.