# How to configure Logstash to output to dynamic list of kafka bootstrap servers?

**URL:** https://discuss.elastic.co/t/how-to-configure-logstash-to-output-to-dynamic-list-of-kafka-bootstrap-servers/203002
**Category:** Logstash
**Created:** [October 10, 2019, 10:43am UTC](https://discuss.elastic.co/t/how-to-configure-logstash-to-output-to-dynamic-list-of-kafka-bootstrap-servers/203002 "2019-10-10T10:43:29Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![furiabhavesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/furiabhavesh/32/52498_2.png) [@furiabhavesh](https://discuss.elastic.co/u/furiabhavesh)
#### Post date: [October 10, 2019, 10:43am UTC](https://discuss.elastic.co/t/how-to-configure-logstash-to-output-to-dynamic-list-of-kafka-bootstrap-servers/203002/1 "2019-10-10T10:43:29Z")

</div>

We need to configure logstash to output to multiple kafka brokers whose list needs to be dynamic.

We need to pass this list of kafka hosts as follows:

```
docker run -e BOOTSTRAP_SERVERS="host1:port1,host2:port2,hostn:portn"

```

and my output block is configured as below:

```
output {
  kafka {
    bootstrap_servers => "${BOOTSTRAP_SERVERS}"
    topic_id => "${KAFKA_TOPIC_ID}"
  }
}

```

I noticed that Logstash config does not accept env variable if it contains colon ( : ) and fallbacks to default value of env variable.

Is it expected that we create N env variables for adding N no. of hosts or is there any other work around for this ?

---

<div class="post-metadata">

### Author: ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)
#### Post date: [October 11, 2019, 3:37pm UTC](https://discuss.elastic.co/t/how-to-configure-logstash-to-output-to-dynamic-list-of-kafka-bootstrap-servers/203002/2 "2019-10-11T15:37:20Z")

</div>

Others might run into the same issue, so quickly writing down the answer for the problem would be great 🙂

---

<div class="post-metadata">

### Author: ![furiabhavesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/furiabhavesh/32/52498_2.png) [@furiabhavesh](https://discuss.elastic.co/u/furiabhavesh)
#### Post date: [October 13, 2019, 6:40am UTC](https://discuss.elastic.co/t/how-to-configure-logstash-to-output-to-dynamic-list-of-kafka-bootstrap-servers/203002/3 "2019-10-13T06:40:21Z")

</div>

It was not the case that logstash does not accept : in environment variable's values. It was just that I had a default value defined in keystore file which was always taking preference.

I am able to use only 1 env variable to pass multiple kafka bootstrap servers as string.

---

<div class="post-metadata">

### Author: ![ethrbunny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ethrbunny/32/34603_2.png) [@ethrbunny](https://discuss.elastic.co/u/ethrbunny)
#### Post date: [October 13, 2019, 3:27pm UTC](https://discuss.elastic.co/t/how-to-configure-logstash-to-output-to-dynamic-list-of-kafka-bootstrap-servers/203002/4 "2019-10-13T15:27:14Z")

</div>

Another approach would be a round-robin DNS entry that encompasses all of your kafka brokers.

We're using powerdns which supports LUA scripts. This gives us a nice spread between brokers.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 20, 2019, 5:10am UTC](https://discuss.elastic.co/t/how-to-configure-logstash-to-output-to-dynamic-list-of-kafka-bootstrap-servers/203002/6 "2019-11-20T05:10:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
