Multipe host output using keystore passwords

Elastic Stack Logstash
#1

Hi All,

I need to configure logstash (ver 7.7) to write to two elastisearch. I have two separate output blocks, one for each elasticsearch but how do I configure password retrieval from keystore, since documentation refer:

  • In configuration files, use: output { elasticsearch {...password => "${ES_PWD}" } } }
  • In logstash.yml , use: xpack.management.elasticsearch.password: ${ES_PWD}

I can use different variables, like {ES_PWD1} and {ES_PWD2}, for each output block but how do I refer both in logstash.yml xpack.management.elasticsearch.password line?

Thank you

#2

xpack.management.elasticsearch.password is the password of your elasticsearch cluster where you want to use centralized management of pipelines.
You will need to choose one cluster for that

#3

Thank you for your reply.

I don't have centralized pipelines (pipeline management is done by editing files on /etc/logstash/conf.d directory) but I have a requirement for writing in one of two elasticsearch, depending on a if condition of output block.

I know that I can also use user and password explicitly on each output block but that's not very secure/elegant solution (hardcoded user/pass) compared with password retrieval from keystore.

I guess I'm not the first to have the need to write into multiple elasticsearch, from one logstash without centralized pipeline management.

Is it possible to not have xpack.management.elasticsearch.password: {ES_PWD} in logstash.yml and still use output { elasticsearch {...password => "{ES_PWD}" } } } on each output block?

#4

Why you need xpack.management.elasticsearch.password ? it's only used for monitoring the pipelines and if that's you requirement you can monitor them only in one cluster

Now you can output data to mulitple elasticsearch cluster and for each one you need to store the password in the keystore with a specific key

SO basically you can have :

  • output { elasticsearch {...password => "${ES_PWD_ES1}" } } }
  • output { elasticsearch {...password => "${ES_PWD_ES2}" } } }
  • output { elasticsearch {...password => "${ES_PWD_ES3}" } } }
  • output { elasticsearch {...password => "${ES_PWD_ES4}" } } }
  • output { elasticsearch {...password => "${ES_PWD_ES5}" } } }
1 Like
#5

Great!
Thank you and keep safe :mask: