# Parse rancher text logs using filebeat

**URL:** <https://discuss.elastic.co/t/parse-rancher-text-logs-using-filebeat/207243>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [November 10, 2019, 2:06pm UTC](https://discuss.elastic.co/t/parse-rancher-text-logs-using-filebeat/207243 "2019-11-10T14:06:15Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![tru64gurus](https://avatars.discourse-cdn.com/v4/letter/t/c0e974/32.png) [@tru64gurus](https://discuss.elastic.co/u/tru64gurus)\
**Post date:** [November 10, 2019, 2:06pm UTC](https://discuss.elastic.co/t/parse-rancher-text-logs-using-filebeat/207243/1 "2019-11-10T14:06:15Z")

</div>

Hi,

I have serveral k8s clusters running on rancher 2.3.1 sending several GB of logs per second and causing disk pressure on source side .

To solve source bottleneck , logs are being sent to syslog server and get written to text files, configure syslog program variable to be cluster name , so each cluster logs get written to separate files on syslog side .

I would like to use filebeat or logstash on syslog server to parse logs and send them to elasticsearch 7.x . I can't get grok to capture k8s meta data like pod name , cluster name , deployment and any other relevant information which requires extraction of variable number of key/value pairs.. Also I would like to create separate index per cluster ( log file)

Any feed back on rancher logs parsing is appreciated

Thanks

---

<div class="post-metadata">

**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\
**Post date:** [November 13, 2019, 2:06pm UTC](https://discuss.elastic.co/t/parse-rancher-text-logs-using-filebeat/207243/2 "2019-11-13T14:06:18Z")

</div>

Hi @tru64gurus,

Did you try using the system syslog fileset?[https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-system.html#\_syslog\_fileset\_settings](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-system.html#_syslog_fileset_settings) It should help you reading logs from syslog.

Once they are in, probably you can make use of `add_kubernetes_metadata` processor to enrich them.

Best regards

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 11, 2019, 2:06pm UTC](https://discuss.elastic.co/t/parse-rancher-text-logs-using-filebeat/207243/3 "2019-12-11T14:06:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.