# Working with elasticsearch cluster **URL:** \ **Category:** Elasticsearch\ **Created:** [August 22, 2018, 9:08am UTC](https://discuss.elastic.co/t/working-with-elasticsearch-cluster/145520 "2018-08-22T09:08:57Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![rgaduput](https://avatars.discourse-cdn.com/v4/letter/r/a5b964/32.png) [@rgaduput](https://discuss.elastic.co/u/rgaduput)\ **Post date:** [August 22, 2018, 9:08am UTC](https://discuss.elastic.co/t/working-with-elasticsearch-cluster/145520/1 "2018-08-22T09:08:57Z")
Hi, we use elasticsearch for storing some metadata of CI, we have automation's to collect metadata and push to elasticsearch. Our elasticsearch cluster has below configuration, 2 - master nodes 3 - data nodes all nodes are individual VM's with different IP's connected into 1 cluster We configure one of the active master node IP in metadata collectors to push data to ES, But our question is what if that node goes down for some reason ? We don't want to reconfigure our collectors with new master node IP. Is there any best way to handle such scenarios ? I guess its quite common scenario when working with clusters Thanks in advance. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 22, 2018, 9:10am UTC](https://discuss.elastic.co/t/working-with-elasticsearch-cluster/145520/2 "2018-08-22T09:10:46Z")
A few important things; - Elasticsearch is not a database - You need 3 master eligible nodes - [https://www.elastic.co/guide/en/elasticsearch/guide/2.x/important-configuration-changes.html#\_minimum\_master\_nodes](https://www.elastic.co/guide/en/elasticsearch/guide/2.x/important-configuration-changes.html#_minimum_master_nodes) Does your collector not support load balancing with multiple hosts? ---
**Author:** ![rgaduput](https://avatars.discourse-cdn.com/v4/letter/r/a5b964/32.png) [@rgaduput](https://discuss.elastic.co/u/rgaduput)\ **Post date:** [August 22, 2018, 9:14am UTC](https://discuss.elastic.co/t/working-with-elasticsearch-cluster/145520/3 "2018-08-22T09:14:18Z")
Thanks for correcting 🙂 as of now no, collectors doesn't support load balancing or multiple hosts configuration. We are looking if this can be done in some other way than collector code change (this we want to handle as a last option) ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 22, 2018, 9:15am UTC](https://discuss.elastic.co/t/working-with-elasticsearch-cluster/145520/4 "2018-08-22T09:15:01Z")
What is the collector, can you share more details on that? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 22, 2018, 9:17am UTC](https://discuss.elastic.co/t/working-with-elasticsearch-cluster/145520/5 "2018-08-22T09:17:33Z")
Why not add a load balancer between your collector and the cluster? ---
**Author:** ![rgaduput](https://avatars.discourse-cdn.com/v4/letter/r/a5b964/32.png) [@rgaduput](https://discuss.elastic.co/u/rgaduput)\ **Post date:** [August 22, 2018, 9:18am UTC](https://discuss.elastic.co/t/working-with-elasticsearch-cluster/145520/6 "2018-08-22T09:18:56Z")
Collects data from our various CI sources, One is maven plugin: to collect data of Maven artifacts generated in maven build. Jenkins plugin: to collect non-maven CI data. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 22, 2018, 9:19am UTC](https://discuss.elastic.co/t/working-with-elasticsearch-cluster/145520/7 "2018-08-22T09:19:27Z")
Right, but does it do that by collecting logs, or something else? ---
**Author:** ![rgaduput](https://avatars.discourse-cdn.com/v4/letter/r/a5b964/32.png) [@rgaduput](https://discuss.elastic.co/u/rgaduput)\ **Post date:** [August 22, 2018, 9:19am UTC](https://discuss.elastic.co/t/working-with-elasticsearch-cluster/145520/8 "2018-08-22T09:19:59Z")
We tried it, same question arises there also, no ? Like what if that LB is down (correct me if i am not wrong , it is also single point of failure) ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 22, 2018, 9:21am UTC](https://discuss.elastic.co/t/working-with-elasticsearch-cluster/145520/9 "2018-08-22T09:21:30Z")
Then I think a code change in the collector may be required. How does your collector handle redundancy? ---
**Author:** ![rgaduput](https://avatars.discourse-cdn.com/v4/letter/r/a5b964/32.png) [@rgaduput](https://discuss.elastic.co/u/rgaduput)\ **Post date:** [August 22, 2018, 9:21am UTC](https://discuss.elastic.co/t/working-with-elasticsearch-cluster/145520/10 "2018-08-22T09:21:31Z")
No, not by parsing by logs. they have different logic, example maven plugin uses maven-core API's and loop through all the projects built and generate report. ---
**Author:** ![rgaduput](https://avatars.discourse-cdn.com/v4/letter/r/a5b964/32.png) [@rgaduput](https://discuss.elastic.co/u/rgaduput)\ **Post date:** [August 22, 2018, 10:10am UTC](https://discuss.elastic.co/t/working-with-elasticsearch-cluster/145520/11 "2018-08-22T10:10:51Z")
ok. for redundancy it follows different way, it writes data into a file and upload to binary repo. When ES is UP later, We have automation which reads these files and push data to ES. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 22, 2018, 10:25am UTC](https://discuss.elastic.co/t/working-with-elasticsearch-cluster/145520/12 "2018-08-22T10:25:31Z")
Doesn't that make your collector a single-point-of-failure? ---
**Author:** ![rgaduput](https://avatars.discourse-cdn.com/v4/letter/r/a5b964/32.png) [@rgaduput](https://discuss.elastic.co/u/rgaduput)\ **Post date:** [August 22, 2018, 10:39am UTC](https://discuss.elastic.co/t/working-with-elasticsearch-cluster/145520/13 "2018-08-22T10:39:47Z")
No, because binary repo is always available. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 19, 2018, 10:40am UTC](https://discuss.elastic.co/t/working-with-elasticsearch-cluster/145520/14 "2018-09-19T10:40:01Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.