How to use environment variable for logstash output elasticsearch plugin for multiple elasticsearch hosts?

Elastic Stack Logstash
1

Hi,

I am deploying logstash and its pipelines via ansible. I want to use the same pipeline on different installations. So I don't want to modify the elasticsearch output plugin in each pipeline. I want to use an environment Variable (configured for each logstash service in /etc/sysconfig/<LOGSTASH_SERVICE_NAME>.

For debugging purpose I exported the environment variable via shell and run a logstash config-test with -t option.

My demanded output configuration looks like this, where I want to set hosts via environment variable:

output
{
  elasticsearch
  {
    hosts           => ["host1:9200","host2:9200"]
    ssl             => "${USE_ES_SSL}"
    cacert          => "${ES_CA_CERT_PATH}"
    ssl_certificate_verification    =>      "${USE_ES_OUTPUT_SSL_CERT_VERIFICATION}"

    # credentials are fetched from envrionment or logstash-keystore

    user           => "${LOGSTASH_USER}"
    password       => "${LOGSTASH_PASSWORD}"

    index          => "%{[@metadata][indexName]}"
  }
}

I tried the following:

Got it working for a single output host by setting array and doublequotes inside the pipeline configuration

bash-4.4$ export ES_HOSTS="host1:9200"
bash-4.4$ echo $ES_HOSTS
host1:9200

hosts           => ["${ES_HOSTS}"]

Outsourcing the doublequotes to the environment variable does not work.

hosts           => [ ${ES_HOSTS} ]

export ES_HOSTS='"host1:9200"'
bash-4.4$ echo $ES_HOSTS
"host1:9200"

a config_test is throwing following error:

[2022-10-21T08:50:07,135][FATAL][logstash.runner          ] The given configuration is invalid. Reason: Expected one of [ \t\r\n], "#", [A-Za-z0-9_-], '"', "'", [A-Za-z_], "-", [0-9], "[", "{", "]" at line 6, column 16 (byte 89) after output
{
        elasticsearch
        {
                hosts           => [
[2022-10-21T08:50:07,137][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:790) ~[jruby.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:753) ~[jruby.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:91) ~[?:?]

My target is to set something like this as environment, but I don't get it working:

bash-4.4$ export ES_HOSTS='"host1:9200","host2:9200"'
bash-4.4$ echo $ES_HOSTS
"host1:9200","host2:9200"


[2022-10-21T08:49:07,213][FATAL][logstash.runner          ] The given configuration is invalid. Reason: Expected one of [ \t\r\n], "#", [A-Za-z0-9_-], '"', "'", [A-Za-z_], "-", [0-9], "[", "{", "]" at line 6, column 16 (byte 89) after output
{
        elasticsearch
        {
                hosts           => [
[2022-10-21T08:49:07,214][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:790) ~[jruby.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:753) ~[jruby.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:91) ~[?:?]

Also this pattern does not work (leaving the "outside quotes" in output config and having the "inside quotes" in the variable. Still failing but another error message:

 hosts           => [ "${ES_HOSTS}" ]

bash-4.4$ export ES_HOSTS='host1:9200","host2:9200'
bash-4.4$ echo $ES_HOSTS
host1:9200","host2:9200


[2022-10-21T08:53:48,821][FATAL][logstash.runner          ] The given configuration is invalid. Reason: Unable to configure plugins: Illegal character in opaque part at index 10: host1:9200","host2:9200
[2022-10-21T08:53:48,823][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:790) ~[jruby.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:753) ~[jruby.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:91) ~[?:?]

How can I solve this issue?

Thanks a lot, Andreas

2

I also tried this pattern with no luck (no array or quotes in config, but full array with inside quotes in environment):

 hosts           => ${ES_HOSTS}

bash-4.4$ export ES_HOSTS='["host1:9200","host2:9200"]'
bash-4.4$ echo $ES_HOSTS
["host1:9200","host2:9200"]

[2022-10-21T08:56:57,718][FATAL][logstash.runner          ] The given configuration is invalid. Reason: Expected one of [ \t\r\n], "#", [A-Za-z0-9_-], '"', "'", [A-Za-z_], "-", [0-9], "[", "{" at line 6, column 14 (byte 87) after output
{
        elasticsearch
        {
                hosts           =>
[2022-10-21T08:56:57,720][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:790) ~[jruby.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:753) ~[jruby.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:91) ~[?:?]
3

From my reading of the PR used to support this, you should be using export ES_HOSTS="host1:9200 host2:9200" and hosts => "${ES_HOSTS}".

4

As Badger said, you need to put the hosts separated by spaces.

I use the environment variable for this in /etc/sysconfig/logstash in this way:

ES_NODES="https://host-01:9200 https://host-02:9200 https://host-03:9200"
5

thanks, space between the nodes did the trick

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.