# Kafka logstash plugin on a clustered zookeeper setup

**URL:** <https://discuss.elastic.co/t/kafka-logstash-plugin-on-a-clustered-zookeeper-setup/36255>\
**Category:** Logstash\
**Created:** [December 3, 2015, 8:04am UTC](https://discuss.elastic.co/t/kafka-logstash-plugin-on-a-clustered-zookeeper-setup/36255 "2015-12-03T08:04:46Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![shivam\_singh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shivam_singh/32/4225_2.png) [@shivam\_singh](https://discuss.elastic.co/u/shivam_singh)\
**Post date:** [December 3, 2015, 8:04am UTC](https://discuss.elastic.co/t/kafka-logstash-plugin-on-a-clustered-zookeeper-setup/36255/1 "2015-12-03T08:04:46Z")

</div>

Hi,

I have a clustered setup for zookeeper and kafka. I wanted to parse the kafka topic (multiple topic) logs to logstash.

1. What is the best way to do so?
2. Will kafka-logstash plugin be able to solve my problem?
3. Do we have to install logstash on the server where zookeeper is running? I want logstash to be installed on a seprate instance.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 3, 2015, 8:27am UTC](https://discuss.elastic.co/t/kafka-logstash-plugin-on-a-clustered-zookeeper-setup/36255/2 "2015-12-03T08:27:34Z")

</div>

1. That's a very broad question.
2. Yes, Logstash with the kafka input plugin can pull messages from Kafka.
3. You can run Logstash on any machine that can connect to Kafka.

---

<div class="post-metadata">

**Author:** ![shivam\_singh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shivam_singh/32/4225_2.png) [@shivam\_singh](https://discuss.elastic.co/u/shivam_singh)\
**Post date:** [December 3, 2015, 11:56am UTC](https://discuss.elastic.co/t/kafka-logstash-plugin-on-a-clustered-zookeeper-setup/36255/3 "2015-12-03T11:56:12Z")

</div>

kafka input plugin will read messages from the kafka topic right..? I don't want to use kafka as a messaging queue for logstash. Kafka is used for some other component message queuing. I just want to parse the **logs** of each topic to ELK stack.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 3, 2015, 12:03pm UTC](https://discuss.elastic.co/t/kafka-logstash-plugin-on-a-clustered-zookeeper-setup/36255/4 "2015-12-03T12:03:03Z")

</div>

> kafka input plugin will read messages from the kafka topic right..?

Yes.

> I don't want to use kafka as a messaging queue for logstash. Kafka is used for some other component message queuing. I just want to parse the logs of each topic to ELK stack.

Sure. Logstash doesn't know if the topic is Logstash-specific or if there's some other piece of software that publishes messages to the topic.

---

<div class="post-metadata">

**Author:** ![shivam\_singh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shivam_singh/32/4225_2.png) [@shivam\_singh](https://discuss.elastic.co/u/shivam_singh)\
**Post date:** [December 3, 2015, 1:47pm UTC](https://discuss.elastic.co/t/kafka-logstash-plugin-on-a-clustered-zookeeper-setup/36255/5 "2015-12-03T13:47:41Z")

</div>

Great, but is thr a way by which we can get the **logs** of topics in kafka using this plugin or any-other plugin?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 3, 2015, 1:57pm UTC](https://discuss.elastic.co/t/kafka-logstash-plugin-on-a-clustered-zookeeper-setup/36255/6 "2015-12-03T13:57:43Z")

</div>

To me "logs" in a Kafka context means the message store where messages published to a topic end up, and from which consumers (like Logstash) can consume messages. Are you talking about something else?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:20am UTC](https://discuss.elastic.co/t/kafka-logstash-plugin-on-a-clustered-zookeeper-setup/36255/7 "2017-07-06T05:20:07Z")

</div>


