# Kubernetes - Logstash or Fluentd?

**URL:** <https://discuss.elastic.co/t/kubernetes-logstash-or-fluentd/223532>\
**Category:** Logstash\
**Created:** [March 13, 2020, 3:01pm UTC](https://discuss.elastic.co/t/kubernetes-logstash-or-fluentd/223532 "2020-03-13T15:01:20Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mauricio\_Borges](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mauricio_borges/32/54964_2.png) [@Mauricio\_Borges](https://discuss.elastic.co/u/Mauricio_Borges)\
**Post date:** [March 13, 2020, 3:01pm UTC](https://discuss.elastic.co/t/kubernetes-logstash-or-fluentd/223532/1 "2020-03-13T15:01:21Z")

</div>

Hi Team!  
I have started working with ELK since last year and has been using a lot Beats, Elasticsearch, Kibana and a bit about Logstash.  
I have started POC case using ELK on-premise supporting OCP 4.x and customer asked about use Fluentd instead Logstash.  
Then I have research about it and found some articles saying it's better for orchestration cases.... However, since I don't know much about both one, I'd like hear some extra information from experts or someone already worked with both to get more feedback about it.  
If could get a matrix comparing features between both would be great.

Below articles I have read about it:  
[https://platform9.com/blog/kubernetes-logging-comparing-fluentd-vs-logstash/](http://comparing-fluentd-vs-logstash/)  
[https://medium.com/tensult/the-log-battle-logstash-and-fluentd-c65f2f7c24b4](http://battle-logstash-and-fluentd)

Thanks, Mauricio

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 14, 2020, 10:06am UTC](https://discuss.elastic.co/t/kubernetes-logstash-or-fluentd/223532/2 "2020-03-14T10:06:47Z")

</div>

I would recommend looking at [Filebeat](https://www.elastic.co/blog/monitoring-kubernetes-and-docker-containers-with-beats-logs-metrics-and-metadata) as well.

---

<div class="post-metadata">

**Author:** ![Mauricio\_Borges](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mauricio_borges/32/54964_2.png) [@Mauricio\_Borges](https://discuss.elastic.co/u/Mauricio_Borges)\
**Post date:** [March 17, 2020, 7:32pm UTC](https://discuss.elastic.co/t/kubernetes-logstash-or-fluentd/223532/3 "2020-03-17T19:32:01Z")

</div>

Thanks Christian!

---

<div class="post-metadata">

**Author:** ![ananbela](https://avatars.discourse-cdn.com/v4/letter/a/e495f1/32.png) [@ananbela](https://discuss.elastic.co/u/ananbela)\
**Post date:** [March 18, 2020, 6:45am UTC](https://discuss.elastic.co/t/kubernetes-logstash-or-fluentd/223532/4 "2020-03-18T06:45:01Z")

</div>

**LogStash** is part of the popular ELK stack. **Fluentd** is built by Treasure Data and is part of the CNCF. **Fluentd** also has excellent support for Elastic. For CNCF hosted project (e.g. **Kubernetes** , OpenTracing or Prometheus), **Fluentd** could be a better choice.

---

<div class="post-metadata">

**Author:** ![Veeresh\_Reddy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/veeresh_reddy/32/58906_2.png) [@Veeresh\_Reddy](https://discuss.elastic.co/u/Veeresh_Reddy)\
**Post date:** [March 23, 2020, 3:18am UTC](https://discuss.elastic.co/t/kubernetes-logstash-or-fluentd/223532/5 "2020-03-23T03:18:44Z")

</div>

When an elastic index moves into a read only state, it can prevent additional data being written to the cluster. Typically a read only state occurs when an index in on a node that reaches 85% disk space used.

would you please help me out with the below challenge ?  
The cluster does have an API to get details for each index, so it may be possible to scan each index and check if it's in read only state.

- how to know if one or more indexes are marked as read only, would you please let me know?
- how can we generate an alert somewhere - likely grafana - when this happens  
Note: we use cerebro ES admin tool as well

---

<div class="post-metadata">

**Author:** ![ananbela](https://avatars.discourse-cdn.com/v4/letter/a/e495f1/32.png) [@ananbela](https://discuss.elastic.co/u/ananbela)\
**Post date:** [March 24, 2020, 12:30pm UTC](https://discuss.elastic.co/t/kubernetes-logstash-or-fluentd/223532/6 "2020-03-24T12:30:39Z")

</div>

> [@ananbela](#):
>
> **LogStash** is part of the popular ELK stack. **Fluentd** is built by Treasure Data and is part of the CNCF. **Fluentd** also has excellent support for Elastic. For CNCF hosted project (e.g. **Kubernetes** , OpenTracing or Prometheus), **Fluentd** could be a better choice. [Walgreens Survey](https://www.walgreenlistens.one/)

**LogStash** is part of the popular ELK stack. **Fluentd** is built by Treasure Data and is part of the CNCF. **Fluentd** also has excellent support for Elastic. For CNCF hosted project (e.g. **Kubernetes** , OpenTracing or Prometheus), **Fluentd** could be a better choice.

---

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [March 24, 2020, 1:18pm UTC](https://discuss.elastic.co/t/kubernetes-logstash-or-fluentd/223532/7 "2020-03-24T13:18:38Z")

</div>

Hi @Mauricio_Borges,

> better for orchestration cases

That probably depends on what orchestration tools you use.

In my organization we use Puppet and Elastic provides official Puppet modules for e.g. Logstash. That being said. Logstash only needs a config file and JAVA (plus jvm.options conig) which could be easily orchestrated with any tool. I would expect Fluentd to be similar...

Elastic have ever expanding support for Kubernetes. We use Filebeat, running as a DaemonSet inside of Kubernetes to send logs via Logstash to Elasticsearch. You could cut out Logstash in the middle if you do not need extra log parsing. If you logs are well tokenized JSON then you could probably go straight to Elasticsearch.

For shipping logs from Kubernetes [fluent-bit](https://fluentbit.io/) is another option.

We decided to go with Filebeat -\> Logstash because the protocol they use to communicate lets Filebeat know if Logstash is under pressure and Filebeat can back down etc.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 21, 2020, 1:19pm UTC](https://discuss.elastic.co/t/kubernetes-logstash-or-fluentd/223532/8 "2020-04-21T13:19:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
