Elasticsearch logstash output plugin production configuration

Elastic Stack Logstash
1

Hello All,

I have prototype working for me with Devices sending syslogs and then logstash parsing it and putting into elasticsearch.

In my logstash config file I have this :

elasticsearch {
hosts => ["hostIP:9200"]
}

I only have one node elastic search cluster.

Now Question is :

I will be productizing this solution. For simplicity assume that I have one Cluster and I have right now 5 nodes inside that cluster.
SO I know I can give array of 5 nodes IP / Hostname in elasticsearch output plugin and then it will round robin to distribute data. How can I avoid putting all my node IP / hostnames into logstash config file. As system goes into production I dont want to manually go into each logstash instance and update these hosts.

What are the best practices one should follow in this case ?
My requirement is :
I want to run my ES cluster and I want to add / remove / update any number of node at any time. I need all of my logstash instances send data irrespective of changes at ES side.

Appreciate your time.

Thanks,
Gaurav

2

Route all traffic through a load balancer or use a DNS alias that points to one or more servers to use.

1 Like