SIngle URL for elastic cluster

[Bryce_Fernandes]

Hi,

Can we create a single URL for elastic cluster if we have a cluster with 10 12 nodes so we can use a single URL in beats or logstash output.

Regards,
Bryce Fernandes.

[leandrojmp (Leandro Pereira)]

You can, but you need to use a load balancer tool like NGINX or HAProxy, or use environment variables, at least for Logstash, I'm not sure if the environment variables for hosts will work with the beats.

For logstash you can have a variable named, for example ES_HOSTS with all your hosts in the following format.

ES_HOSTS="https://es1:9200 https://es2:9200 https://esN:9200"

Then in your output you will have:

elasticsearch {
    hosts => ["${ES_HOSTS}"]
}

You can have this variable in the environment for the user running logstash, in the logstash.keystore file or in the /etc/sysconfig/logstash file.

I'm not sure if the same approach will work in the beats family, never tested.

The other way would be setting up a load balancer with NGINX, HAProxy or any other similar tool.

[Bryce_Fernandes]

Hi @leandrojmp ,

Thanks for quick reply, I think environment variables is good option for now
had 2 doubts:

1.In the hosts you mentioned will it be comma separated or as it is.
2.Can we do it in windows as well if yes which file to edit.

Thanks,
Bryce Fernandes.

[leandrojmp (Leandro Pereira)]

The variable with the hosts for logstash it is as it is, space separated.

ES_HOSTS="https://es1:9200 https://es2:9200 https://esN:9200"

I do not use windows, but you can use logstash.keystore in Windows as well or set an Environment Variable in your system, you need to make sure that this variable is available for the logstash user.