# SMTP logs from Windows Server can't be parsed

**URL:** <https://discuss.elastic.co/t/smtp-logs-from-windows-server-cant-be-parsed/229916>\
**Category:** Logs\
**Created:** [April 27, 2020, 7:07am UTC](https://discuss.elastic.co/t/smtp-logs-from-windows-server-cant-be-parsed/229916 "2020-04-27T07:07:14Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vadym\_Mykolaichuk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vadym_mykolaichuk/32/66239_2.png) [@Vadym\_Mykolaichuk](https://discuss.elastic.co/u/Vadym_Mykolaichuk)\
**Post date:** [April 27, 2020, 7:07am UTC](https://discuss.elastic.co/t/smtp-logs-from-windows-server-cant-be-parsed/229916/1 "2020-04-27T07:07:14Z")

</div>

Hi!

I'm using filebeat to send IIS log files from Windows Server machine. It parses logs from IIS (web logs) just fine, but it can't parse SMTP logs which have the same format.  
On Kibana I see this error:  
Provided Grok expressions do not match field value:  
[2020-04-27 06:43:55 127.0.0.1 _hostname_ SMTPSVC1 _hostname_ 127.0.0.1 0 EHLO - +_hostname_ 250 0 185 16 0 SMTP - - - -]

Please advise what I supposed to do to see parsed SMTP logs. Let me know if any extra info is needed.

---

<div class="post-metadata">

**Author:** ![Kerry](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kerry/32/40330_2.png) [@Kerry](https://discuss.elastic.co/u/Kerry)\
**Post date:** [April 27, 2020, 9:40am UTC](https://discuss.elastic.co/t/smtp-logs-from-windows-server-cant-be-parsed/229916/2 "2020-04-27T09:40:09Z")

</div>

Hi @Vadym_Mykolaichuk,

Sorry to hear you're having problems. Could you please provide the following information:

- Are you using Filebeat "as-is" (using the Filebeat index templates, using the IIS module etc)? Or do you have something like an Ingest pipeline in between?

- What is the Grok expression that you're using?

- What version of Filebeat are you using?

---

<div class="post-metadata">

**Author:** ![Vadym\_Mykolaichuk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vadym_mykolaichuk/32/66239_2.png) [@Vadym\_Mykolaichuk](https://discuss.elastic.co/u/Vadym_Mykolaichuk)\
**Post date:** [April 27, 2020, 10:29am UTC](https://discuss.elastic.co/t/smtp-logs-from-windows-server-cant-be-parsed/229916/3 "2020-04-27T10:29:43Z")

</div>

Hi @Kerry!  
I'm using IIS module for filebeat 7.6.1 version "as-is" with default grok patterns.  
Here are these patterns from this file:  
C:\Program Files\Elastic\Beats\7.6.1\filebeat\module\iis\access\ingest  
default.json

```auto
"%{TIMESTAMP_ISO8601:iis.access.time} %{IPORHOST:destination.address} %{WORD:http.request.method} %{NOTSPACE:url.path} %{NOTSPACE:url.query} %{NUMBER:destination.port:long} %{NOTSPACE:user.name} %{IPORHOST:source.address} %{NOTSPACE:user_agent.original} %{NOTSPACE:http.request.referrer} %{NUMBER:http.response.status_code:long} %{NUMBER:iis.access.sub_status:long} %{NUMBER:iis.access.win32_status:long} %{NUMBER:temp.duration:long}",

"%{TIMESTAMP_ISO8601:iis.access.time} %{NOTSPACE:iis.access.site_name} %{WORD:http.request.method} %{NOTSPACE:url.path} %{NOTSPACE:url.query} %{NUMBER:destination.port:long} %{NOTSPACE:user.name} %{IPORHOST:source.address} %{NOTSPACE:user_agent.original} %{NOTSPACE:iis.access.cookie} %{NOTSPACE:http.request.referrer} %{NOTSPACE:destination.domain} %{NUMBER:http.response.status_code:long} %{NUMBER:iis.access.sub_status:long} %{NUMBER:iis.access.win32_status:long} %{NUMBER:http.response.body.bytes:long} %{NUMBER:http.request.body.bytes:long} %{NUMBER:temp.duration:long}",

"%{TIMESTAMP_ISO8601:iis.access.time} %{NOTSPACE:iis.access.site_name} %{NOTSPACE:iis.access.server_name} %{IPORHOST:destination.address} %{WORD:http.request.method} %{NOTSPACE:url.path} %{NOTSPACE:url.query} %{NUMBER:destination.port:long} %{NOTSPACE:user.name} %{IPORHOST:source.address} HTTP/%{NUMBER:http.version} %{NOTSPACE:user_agent.original} %{NOTSPACE:iis.access.cookie} %{NOTSPACE:http.request.referrer} %{NOTSPACE:destination.domain} %{NUMBER:http.response.status_code:long} %{NUMBER:iis.access.sub_status:long} %{NUMBER:iis.access.win32_status:long} %{NUMBER:http.response.body.bytes:long} %{NUMBER:http.request.body.bytes:long} %{NUMBER:temp.duration:long}",

"%{TIMESTAMP_ISO8601:iis.access.time} \\[%{IPORHOST:destination.address}\\]\\(http://%{IPORHOST:destination.address}\\) %{WORD:http.request.method} %{NOTSPACE:url.path} %{NOTSPACE:url.query} %{NUMBER:destination.port:long} %{NOTSPACE:user.name} \\[%{IPORHOST:source.address}\\]\\(http://%{IPORHOST:source.address}\\) %{NOTSPACE:user_agent.original} %{NUMBER:http.response.status_code:long} %{NUMBER:iis.access.sub_status:long} %{NUMBER:iis.access.win32_status:long} %{NUMBER:temp.duration:long}",

"%{TIMESTAMP_ISO8601:iis.access.time} %{IPORHOST:destination.address} %{WORD:http.request.method} %{NOTSPACE:url.path} %{NOTSPACE:url.query} %{NUMBER:destination.port:long} %{NOTSPACE:user.name} %{IPORHOST:source.address} %{NOTSPACE:user_agent.original} %{NUMBER:http.response.status_code:long} %{NUMBER:iis.access.sub_status:long} %{NUMBER:iis.access.win32_status:long} %{NUMBER:temp.duration:long}"

```

Here are some example of logs (data has been modified):

```
#Software: Microsoft Internet Information Services 10.0
#Version: 1.0
#Date: 2020-04-27 06:43:55
#Fields: date time c-ip cs-username s-sitename s-computername s-ip s-port cs-method cs-uri-stem cs-uri-query sc-status sc-win32-status sc-bytes cs-bytes time-taken cs-version cs-host cs(User-Agent) cs(Cookie) cs(Referer) 
2020-04-27 06:43:55 127.0.0.1 server-01 SMTPSVC1 server-01 127.0.0.1 0 EHLO - +server-01 250 0 185 16 0 SMTP - - - -
2020-04-27 06:43:55 127.0.0.1 server-01 SMTPSVC1 server-01 127.0.0.1 0 MAIL - +FROM:<test@example.com> 250 0 36 23 0 SMTP - - - -
2020-04-27 06:43:55 127.0.0.1 server-01 SMTPSVC1 server-01 127.0.0.1 0 RCPT - +TO:<recipient@example.com> 250 0 42 39 15 SMTP - - - -
2020-04-27 06:43:55 127.0.0.1 server-01 SMTPSVC1 server-01 127.0.0.1 0 DATA - <server-01YRofCD2h00000001@server-01> 250 0 124 291 16 SMTP - - - -
2020-04-27 06:43:55 8.8.8.8 OutboundConnectionResponse SMTPSVC1 server-01 - 25 - - 220+MS+ESMTP+service+ready+at+ismtpd0002p1lon1.mailservice.com 0 0 59 0 15 SMTP - - - -
2020-04-27 06:43:55 8.8.8.8 OutboundConnectionCommand SMTPSVC1 server-01 - 25 EHLO - server-01 0 0 4 0 15 SMTP - - - -
2020-04-27 06:43:55 8.8.8.8 OutboundConnectionResponse SMTPSVC1 server-01 - 25 - - 250-smtp.mailservice.com 0 0 21 0 31 SMTP - - - -
2020-04-27 06:43:55 8.8.8.8 OutboundConnectionCommand SMTPSVC1 server-01 - 25 AUTH - - 0 0 4 0 31 SMTP - - - -
```

---

<div class="post-metadata">

**Author:** ![Kerry](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kerry/32/40330_2.png) [@Kerry](https://discuss.elastic.co/u/Kerry)\
**Post date:** [April 27, 2020, 3:39pm UTC](https://discuss.elastic.co/t/smtp-logs-from-windows-server-cant-be-parsed/229916/4 "2020-04-27T15:39:20Z")

</div>

Great, thank you.

Those Grok patterns are definitely incompatible with the log examples you've provided. It looks like you'll need to add some extra patterns to cover the SMTP scenario.

We have a Grok debugger available at `app/kibana#/dev_tools/grokdebugger` which you can use to help with the creation of patterns.

[Using the fields listed for the IIS module](https://www.elastic.co/guide/en/beats/filebeat/current/exported-fields-iis.html) you'll be able to create a suitable pattern that follows  
`date time c-ip cs-username s-sitename s-computername s-ip s-port cs-method cs-uri-stem cs-uri-query sc-status sc-win32-status sc-bytes cs-bytes time-taken cs-version cs-host cs(User-Agent) cs(Cookie) cs(Referer)`

Something like this should get you started:

 ![Screenshot 2020-04-27 16.36.57](https://us1.discourse-cdn.com/elastic/original/3X/1/4/149bbb404f6be14bd87c1a63d0b7804d45ebc5c4.png)

`%{TIMESTAMP_ISO8601:iis.access.time} %{IPORHOST:iis.access.remote_ip} %{NOTSPACE:iis.access.user_name} %{NOTSPACE:iis.access.site_name} %{NOTSPACE:iis.access.server_name} %{IPORHOST:iis.access.server_ip} %{INT:iis.access.port}`

Hopefully that helps. If you need further help the [Beats board](https://discuss.elastic.co/c/beats/28) will be able to advise you further on the Filebeat configuration.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 25, 2020, 3:39pm UTC](https://discuss.elastic.co/t/smtp-logs-from-windows-server-cant-be-parsed/229916/5 "2020-05-25T15:39:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
