# What is the best way to extract Syslog Fields with Beats Inputs

**URL:** <https://discuss.elastic.co/t/what-is-the-best-way-to-extract-syslog-fields-with-beats-inputs/44447>\
**Category:** Beats\
**Created:** [March 15, 2016, 3:08pm UTC](https://discuss.elastic.co/t/what-is-the-best-way-to-extract-syslog-fields-with-beats-inputs/44447 "2016-03-15T15:08:06Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![marius](https://avatars.discourse-cdn.com/v4/letter/m/ed655f/32.png) [@marius](https://discuss.elastic.co/u/marius)\
**Post date:** [March 15, 2016, 3:08pm UTC](https://discuss.elastic.co/t/what-is-the-best-way-to-extract-syslog-fields-with-beats-inputs/44447/1 "2016-03-15T15:08:06Z")

</div>

Hi,

I setup ELK and Logstash with a Beats input:

```
input {
  beats {
    port => 5044
    ssl => false
    # ssl_certificate => "/etc/pki/tls/certs/logstash-beats.crt"
    # ssl_key => "/etc/pki/tls/private/logstash-beats.key"
    # codec => syslog
  }
}

```

I experimented with assigning a codec, but this doesn't work. Also the SSL stuff is inactive because this is a test environment.

Now I have filebeat, and I forward logs:

```
output:
  logstash:
    enabled: true
    hosts:
      - elk:5044
    # tls:
    # certificate_authorities:
    # - /etc/pki/tls/certs/logstash-beats.crt
    timeout: 15

filebeat:
  prospectors:
    -
      paths:
        - /var/log/syslog
        - /var/log/auth.log
      document_type: syslog

```

I would like to get fields extracted in Kibana from messages like this:

`Mar 15 15:56:30 mjo dnsmasq-dhcp[4818]: DHCPREQUEST(virbr0) 192.168.100.225 52:54:00:d7:47:49`

In Kibana the application field (dnsmasq-dhcp) does not get extracted:

 ![](https://us1.discourse-cdn.com/elastic/original/2X/2/2fe93e8a7c2d793650d5e72b86d9fe3629b6ad35.png)

Is there an easy way to perform the field extractions to benefit from the visualizations? I would like to use the Beats agent to forward Syslog as well as files, because the integration is central and easy to manage. Of course I can use Rsyslog and forward into Logstash directly, but I'd like to do all the Forwarding with Filebeat here.

Thanks ;),  
Marius

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [March 15, 2016, 4:02pm UTC](https://discuss.elastic.co/t/what-is-the-best-way-to-extract-syslog-fields-with-beats-inputs/44447/2 "2016-03-15T16:02:05Z")

</div>

Hi @marius, you don't need a codec here. You need grok and date filters. See the example filter in this Logstash [configuration example](https://www.elastic.co/guide/en/logstash/current/config-examples.html#_processing_syslog_messages).

If you need to debug grok patterns try this handy site: [https://grokdebug.herokuapp.com/](https://grokdebug.herokuapp.com/)

---

<div class="post-metadata">

**Author:** ![marius](https://avatars.discourse-cdn.com/v4/letter/m/ed655f/32.png) [@marius](https://discuss.elastic.co/u/marius)\
**Post date:** [March 15, 2016, 5:33pm UTC](https://discuss.elastic.co/t/what-is-the-best-way-to-extract-syslog-fields-with-beats-inputs/44447/3 "2016-03-15T17:33:23Z")

</div>

Ok, I get that.

- The type of the received syslog entry via Beats is "log".

So I would apply Grok filters from the Syslog example on the type "log"?

The problem with Beats is that this is a very generic type. It would be better if we could set types like "app log" "sys log" or "foo log" and match Grok patterns specifically per prospector this way.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 15, 2016, 8:15pm UTC](https://discuss.elastic.co/t/what-is-the-best-way-to-extract-syslog-fields-with-beats-inputs/44447/5 "2016-03-15T20:15:24Z")

</div>

> The problem with Beats is that this is a very generic type. It would be better if we could set types like "app log" "sys log" or "foo log" and match Grok patterns specifically per prospector this way.

Your `document_type: syslog` should take care of that.

The configuration you posted isn't consistent with the Kibana screenshot; the configuration reads syslog and auth.log but the event in Kibana is from daemon.log.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 9:54pm UTC](https://discuss.elastic.co/t/what-is-the-best-way-to-extract-syslog-fields-with-beats-inputs/44447/6 "2017-07-05T21:54:37Z")

</div>