# How to use environment variable for logstash output elasticsearch plugin for multiple elasticsearch hosts?

**URL:** <https://discuss.elastic.co/t/how-to-use-environment-variable-for-logstash-output-elasticsearch-plugin-for-multiple-elasticsearch-hosts/317169>\
**Category:** Logstash\
**Created:** [October 21, 2022, 6:54am UTC](https://discuss.elastic.co/t/how-to-use-environment-variable-for-logstash-output-elasticsearch-plugin-for-multiple-elasticsearch-hosts/317169 "2022-10-21T06:54:58Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![asp](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@asp](https://discuss.elastic.co/u/asp)\
**Post date:** [October 21, 2022, 6:54am UTC](https://discuss.elastic.co/t/how-to-use-environment-variable-for-logstash-output-elasticsearch-plugin-for-multiple-elasticsearch-hosts/317169/1 "2022-10-21T06:54:58Z")

</div>

Hi,

I am deploying logstash and its pipelines via ansible. I want to use the same pipeline on different installations. So I don't want to modify the elasticsearch output plugin in each pipeline. I want to use an environment Variable (configured for each logstash service in `/etc/sysconfig/<LOGSTASH_SERVICE_NAME>`.

For debugging purpose I exported the environment variable via shell and run a logstash config-test with -t option.

My demanded output configuration looks like this, where I want to set hosts via environment variable:

```auto
output
{
  elasticsearch
  {
    hosts => ["host1:9200","host2:9200"]
    ssl => "${USE_ES_SSL}"
    cacert => "${ES_CA_CERT_PATH}"
    ssl_certificate_verification => "${USE_ES_OUTPUT_SSL_CERT_VERIFICATION}"

    # credentials are fetched from envrionment or logstash-keystore

    user => "${LOGSTASH_USER}"
    password => "${LOGSTASH_PASSWORD}"

    index => "%{[@metadata][indexName]}"
  }
}

```

I tried the following:

Got it working for a **single** output host by setting array and **doublequotes inside** the pipeline configuration

```auto
bash-4.4$ export ES_HOSTS="host1:9200"
bash-4.4$ echo $ES_HOSTS
host1:9200

```

```auto
hosts => ["${ES_HOSTS}"]

```

  

* * *

**Outsourcing the doublequotes** to the environment variable does **not work**.

```auto
hosts => [${ES_HOSTS}]

```

```auto
export ES_HOSTS='"host1:9200"'
bash-4.4$ echo $ES_HOSTS
"host1:9200"

```

a config\_test is throwing following error:

```auto
[2022-10-21T08:50:07,135][FATAL][logstash.runner] The given configuration is invalid. Reason: Expected one of [\t\r\n], "#", [A-Za-z0-9_-], '"', "'", [A-Za-z_], "-", [0-9], "[", "{", "]" at line 6, column 16 (byte 89) after output
{
        elasticsearch
        {
                hosts => [
[2022-10-21T08:50:07,137][FATAL][org.logstash.Logstash] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:790) ~[jruby.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:753) ~[jruby.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:91) ~[?:?]

```

  

* * *

My target is to set something like this as environment, but I don't get it working:

```auto
bash-4.4$ export ES_HOSTS='"host1:9200","host2:9200"'
bash-4.4$ echo $ES_HOSTS
"host1:9200","host2:9200"

```

```auto
[2022-10-21T08:49:07,213][FATAL][logstash.runner] The given configuration is invalid. Reason: Expected one of [\t\r\n], "#", [A-Za-z0-9_-], '"', "'", [A-Za-z_], "-", [0-9], "[", "{", "]" at line 6, column 16 (byte 89) after output
{
        elasticsearch
        {
                hosts => [
[2022-10-21T08:49:07,214][FATAL][org.logstash.Logstash] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:790) ~[jruby.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:753) ~[jruby.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:91) ~[?:?]

```

  

* * *

Also this pattern does **not work** (leaving the "outside quotes" in output config and having the "inside quotes" in the variable. Still failing but another error message:

```auto
 hosts => ["${ES_HOSTS}"]

```

```auto
bash-4.4$ export ES_HOSTS='host1:9200","host2:9200'
bash-4.4$ echo $ES_HOSTS
host1:9200","host2:9200

```

```auto
[2022-10-21T08:53:48,821][FATAL][logstash.runner] The given configuration is invalid. Reason: Unable to configure plugins: Illegal character in opaque part at index 10: host1:9200","host2:9200
[2022-10-21T08:53:48,823][FATAL][org.logstash.Logstash] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:790) ~[jruby.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:753) ~[jruby.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:91) ~[?:?]

```

How can I solve this issue?

Thanks a lot, Andreas

---

<div class="post-metadata">

**Author:** ![asp](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@asp](https://discuss.elastic.co/u/asp)\
**Post date:** [October 21, 2022, 6:57am UTC](https://discuss.elastic.co/t/how-to-use-environment-variable-for-logstash-output-elasticsearch-plugin-for-multiple-elasticsearch-hosts/317169/2 "2022-10-21T06:57:39Z")

</div>

I also tried this pattern with no luck (no array or quotes in config, but full array with inside quotes in environment):

```auto
 hosts => ${ES_HOSTS}

```

```auto
bash-4.4$ export ES_HOSTS='["host1:9200","host2:9200"]'
bash-4.4$ echo $ES_HOSTS
["host1:9200","host2:9200"]

```

```auto
[2022-10-21T08:56:57,718][FATAL][logstash.runner] The given configuration is invalid. Reason: Expected one of [\t\r\n], "#", [A-Za-z0-9_-], '"', "'", [A-Za-z_], "-", [0-9], "[", "{" at line 6, column 14 (byte 87) after output
{
        elasticsearch
        {
                hosts =>
[2022-10-21T08:56:57,720][FATAL][org.logstash.Logstash] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:790) ~[jruby.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:753) ~[jruby.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:91) ~[?:?]

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 21, 2022, 3:15pm UTC](https://discuss.elastic.co/t/how-to-use-environment-variable-for-logstash-output-elasticsearch-plugin-for-multiple-elasticsearch-hosts/317169/3 "2022-10-21T15:15:54Z")

</div>

From my reading of the [PR](https://github.com/elastic/logstash/pull/12051) used to support this, you should be using `export ES_HOSTS="host1:9200 host2:9200"` and `hosts => "${ES_HOSTS}"`.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [October 21, 2022, 8:40pm UTC](https://discuss.elastic.co/t/how-to-use-environment-variable-for-logstash-output-elasticsearch-plugin-for-multiple-elasticsearch-hosts/317169/4 "2022-10-21T20:40:44Z")

</div>

As Badger said, you need to put the hosts separated by spaces.

I use the environment variable for this in `/etc/sysconfig/logstash` in this way:

```auto
ES_NODES="https://host-01:9200 https://host-02:9200 https://host-03:9200"

```

---

<div class="post-metadata">

**Author:** ![asp](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@asp](https://discuss.elastic.co/u/asp)\
**Post date:** [November 1, 2022, 3:07pm UTC](https://discuss.elastic.co/t/how-to-use-environment-variable-for-logstash-output-elasticsearch-plugin-for-multiple-elasticsearch-hosts/317169/5 "2022-11-01T15:07:54Z")

</div>

thanks, space between the nodes did the trick

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 29, 2022, 3:08pm UTC](https://discuss.elastic.co/t/how-to-use-environment-variable-for-logstash-output-elasticsearch-plugin-for-multiple-elasticsearch-hosts/317169/6 "2022-11-29T15:08:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.