# Logstash HA **URL:** \ **Category:** Logstash\ **Created:** [September 9, 2020, 5:29am UTC](https://discuss.elastic.co/t/logstash-ha/247956 "2020-09-09T05:29:11Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![aksshm](https://avatars.discourse-cdn.com/v4/letter/a/50afbb/32.png) [@aksshm](https://discuss.elastic.co/u/aksshm)\ **Post date:** [September 9, 2020, 5:29am UTC](https://discuss.elastic.co/t/logstash-ha/247956/1 "2020-09-09T05:29:11Z")
Hi, We're trying to install/achieve logstash HA. Is logstash support HA? or Is there any way to achieve the same? ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [September 9, 2020, 4:03pm UTC](https://discuss.elastic.co/t/logstash-ha/247956/2 "2020-09-09T16:03:57Z")
Logstash does not natively support HA, you need to use other tools to implement a HA Logstash deployment, like message queues Kafka, Virtual IPs, load balancers like HAProxy or NGINX, it depends on what do you need. What is your use case? ---
**Author:** ![d.silwon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/d.silwon/32/65853_2.png) [@d.silwon](https://discuss.elastic.co/u/d.silwon)\ **Post date:** [September 9, 2020, 6:01pm UTC](https://discuss.elastic.co/t/logstash-ha/247956/3 "2020-09-09T18:01:15Z")
You can use load balance in case of filebeats: ```auto output.logstash: hosts: ["node1:5044", "node2:5044", "node3:5044"] loadbalance: true ``` more info: [https://www.elastic.co/guide/en/beats/filebeat/current/load-balancing.html](https://www.elastic.co/guide/en/beats/filebeat/current/load-balancing.html) ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [September 9, 2020, 6:57pm UTC](https://discuss.elastic.co/t/logstash-ha/247956/4 "2020-09-09T18:57:22Z")
Lots of good information [here](https://www.elastic.co/guide/en/logstash/current/deploying-and-scaling.html) ---
**Author:** ![aksshm](https://avatars.discourse-cdn.com/v4/letter/a/50afbb/32.png) [@aksshm](https://discuss.elastic.co/u/aksshm)\ **Post date:** [September 10, 2020, 5:30am UTC](https://discuss.elastic.co/t/logstash-ha/247956/5 "2020-09-10T05:30:27Z")
Hi, Right now!! we have single instance of logstash whose data is mounted on glusterfs. Now, we are trying to remove glusterfs, so we are looking a way for logstash data to be available during k8s node failure. for ex: if logstash is enabled with persistent queue , we need to have a way for disk replication like glusterfs,ceph etc. if logstash is not enable with persistent queue i.e with in memory db, then if nodes fails then k8s will schedule logstash to other worker node, where logstash will come up but he will not have old data. so figuring out how can we achieve the scenario!!!. ---
**Author:** ![aksshm](https://avatars.discourse-cdn.com/v4/letter/a/50afbb/32.png) [@aksshm](https://discuss.elastic.co/u/aksshm)\ **Post date:** [September 10, 2020, 11:35am UTC](https://discuss.elastic.co/t/logstash-ha/247956/6 "2020-09-10T11:35:01Z")
thanks @stephenb , can you help me with below query? we have single instance of logstash whose data is mounted on glusterfs. Now, we are trying to remove glusterfs, so we are looking a way for logstash data to be available during k8s node failure. for ex: if logstash is enabled with persistent queue , we need to have a way for disk replication like glusterfs,ceph etc. if logstash is not enable with persistent queue i.e with in memory db, then if nodes fails then k8s will schedule logstash to other worker node, where logstash will come up but he will not have old data. Note: we are using input plugin as kafka. ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [September 10, 2020, 12:41pm UTC](https://discuss.elastic.co/t/logstash-ha/247956/7 "2020-09-10T12:41:04Z")
I would suggest you to use a Kafka cluster as a message queue, but it seems that you are already doing that. WIth Kafka you can have multiple logstash as consumers, if one node fail, you can spin up another node and start consuming from where the other node stopped, you just need to configure the [group\_id](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-kafka.html#plugins-inputs-kafka-group_id) in the input as the same in the nodes. With the same `group_id` you can also have multiple nodes running at the same time, or if you start to get lag in your topics, you can start other nodes temporarily to help empty the queue faster. ---
**Author:** ![aksshm](https://avatars.discourse-cdn.com/v4/letter/a/50afbb/32.png) [@aksshm](https://discuss.elastic.co/u/aksshm)\ **Post date:** [September 10, 2020, 1:50pm UTC](https://discuss.elastic.co/t/logstash-ha/247956/8 "2020-09-10T13:50:03Z")
Node, you mean kubernetes worker node ? Start other nodes temporarily, I didn't get it? can you please elaborate it. if one node fail, you can spin up another node, what do you mean? ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [September 10, 2020, 2:07pm UTC](https://discuss.elastic.co/t/logstash-ha/247956/9 "2020-09-10T14:07:21Z")
Logstash node, not Kubernetes, it doesn't matter where your Logstash is running, you just need to have the logstashs that consume from your Kafka with the same `group_id`. If one of your logstash fails, you can start a new one and it will start consuming from where the last one stopped. If your queue in your Kafka is getting too big and start giving you lag, you can start a new logstash to help consume the queue, and then stop it later after things went back to normal. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 8, 2020, 2:07pm UTC](https://discuss.elastic.co/t/logstash-ha/247956/10 "2020-10-08T14:07:55Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.