# Help with grok patterns for custom Postfix logs

**URL:** <https://discuss.elastic.co/t/help-with-grok-patterns-for-custom-postfix-logs/153835>\
**Category:** Logstash\
**Created:** [October 24, 2018, 2:57pm UTC](https://discuss.elastic.co/t/help-with-grok-patterns-for-custom-postfix-logs/153835 "2018-10-24T14:57:13Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![decibel83](https://avatars.discourse-cdn.com/v4/letter/d/7feea3/32.png) [@decibel83](https://discuss.elastic.co/u/decibel83)\
**Post date:** [October 24, 2018, 2:57pm UTC](https://discuss.elastic.co/t/help-with-grok-patterns-for-custom-postfix-logs/153835/1 "2018-10-24T14:57:14Z")

</div>

Hi,

I am parsing Postfix logs using the good work from [https://github.com/topmedia/logstash-postfix](https://github.com/topmedia/logstash-postfix).  
Now I'm trying to customise the grok patterns because I have to parse some custom logs created with the Postfix **header\_checks** feature (example at [https://serverfault.com/questions/392838/how-to-log-extension-header-in-postfix-log](https://serverfault.com/questions/392838/how-to-log-extension-header-in-postfix-log)).

The use case is to parse and import custom headers name and value which are written in Postfix logs like these:

```
Oct 24 15:18:09 server postfix/cleanup[30021]: 7D5F140AAA: warning: header X-Header-Custom1: Bla bla bla from unknown[123.123.123.123]; from=<user@domain.com> to=<user@domain.com> proto=ESMTP helo=<client.domain.com>
Oct 24 15:18:09 server postfix/cleanup[30021]: 7D5F140AAA: warning: header X-Header-Custom2: Test test test from unknown[123.123.123.123]; from=<user@domain.com> to=<user@domain.com> proto=ESMTP helo=<client.domain.com>
Oct 24 15:18:09 server postfix/cleanup[30021]: 7D5F140AAA: warning: header Subject: Mail test with custom headeer from unknown[123.123.123.123]; from=<user@domain.com> to=<user@domain.com> proto=ESMTP helo=<client.domain.com>

```

These logs are parsed with the following grok patterns (you can see full patterns at [https://github.com/topmedia/logstash-postfix/blob/master/etc/logstash/patterns.d/postfix.grok](https://github.com/topmedia/logstash-postfix/blob/master/etc/logstash/patterns.d/postfix.grok)):

```
common postfix patterns
POSTFIX_QUEUEID ([0-9A-F]{6,}|[0-9a-zA-Z]{12,})
POSTFIX_WARNING_LEVEL (warning|fatal|info)
POSTFIX_KEYVALUE_DATA [\w-]+=[^;]*

# warning patterns
POSTFIX_WARNING_WITH_KV (%{POSTFIX_QUEUEID:postfix_queueid}: )?%{POSTFIX_WARNING_LEVEL:postfix_message_level}: %{GREEDYDATA:postfix_message}; %{POSTFIX_KEYVALUE_DATA:postfix_keyvalue_data}
POSTFIX_WARNING_WITHOUT_KV (%{POSTFIX_QUEUEID:postfix_queueid}: )?%{POSTFIX_WARNING_LEVEL:postfix_message_level}: %{GREEDYDATA:postfix_message}
POSTFIX_WARNING %{POSTFIX_WARNING_WITH_KV}|%{POSTFIX_WARNING_WITHOUT_KV}

```

With this patterns I get the following result on the first log (I'm debugging at [http://grokdebug.herokuapp.com/](http://grokdebug.herokuapp.com/)):

```
{
  "POSTFIX_WARNING_WITHOUT_KV": [
    [
      null
    ]
  ],
  "postfix_queueid": [
    [
      null,
      "7D5F140AAA"
    ]
  ],
  "postfix_message_level": [
    [
      null,
      "warning"
    ]
  ],
  "postfix_message": [
    [
      null,
      "header X-Header-Custom1: Bla bla bla from unknown[123.123.123.123]"
    ]
  ],
  "POSTFIX_WARNING_WITH_KV": [
    [
      "7D5F140AAA: warning: header X-Header-Custom1: Bla bla bla from unknown[123.123.123.123]; from=<user@domain.com> to=<user@domain.com> proto=ESMTP helo=<client.domain.com>"
    ]
  ],
  "postfix_keyvalue_data": [
    [
      "from=<user@domain.com> to=<user@domain.com> proto=ESMTP helo=<client.domain.com>"
    ]
  ]
}

```

What I want to get is two new metadata:

- **postfix\_header\_name:** X-Header-Custom1
- **postfix\_header\_value:** Bla bla bla

I tried to add two new custom patterns:

```
POSTFIX_HEADER_NAME Subject|X-[A-z\-]+
POSTFIX_HEADER_VALUE .+

```

and tried to change the POSTFIX\_WARNING\_WITH\_KV pattern with the following:

```
POSTFIX_WARNING_WITH_KV (%{POSTFIX_QUEUEID:postfix_queueid}: )?%{POSTFIX_WARNING_LEVEL:postfix_message_level}: (header %{POSTFIX_HEADER_NAME:postfix_header_name}: {POSTFIX_HEADER_VALUE:postfix_header_value} from)? %{GREEDYDATA:postfix_message}; %{POSTFIX_KEYVALUE_DATA:postfix_keyvalue_data}}

```

but this don't work.

Could you help me to understand where I'm wrong, please?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 21, 2018, 2:57pm UTC](https://discuss.elastic.co/t/help-with-grok-patterns-for-custom-postfix-logs/153835/2 "2018-11-21T14:57:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.