ElasticSearch Load Balancer

Elastic Stack Elasticsearch
1

Hi,

I got a chance to see a Architecture of ElasticSearch in a following manner, and got few doubts regarding it.

ElasticSearch Server - cluster with 9 nodes (Windows Server)

4 Client Application (.Net Web) to consume the ElasticSearch Service.

Out of 9 node, in one of the node a DNS was hosted (in IIS) lets say https://Elastic

All the 4 client applications has 2 urls, one for "Query" and other for "Index" as follows, which will the same https://Elastic

  1. Https://Elastic/Query/Application1
  2. Https://Elastic/index/Application1

in the DNS, config file the rewrite URL was defined to point to one of the specific node by specifying the IP as follows,

   <rule name="Elastic Search Application1" stopProcessing="true">
      <match url="query/Application1(.*)" />
      <action type="Rewrite" url="http://11.11.11.11:9200/{R:1}" />
    </rule>

in the above rewrite, they are pointing to the specific node in the cluster.

So my doubt is, here by pointing to the specific node in the cluster is correct ?

For example if the node with ip "22.22.22.22" was shutdown or corrupted then there will be an issue ? How to overcome this issue?

Since we are pointing to the specific IP then is the default ElasticSearch Load Balancer will work?

Please Suggest.

Thanks,
Kannan

2

Don't point to a specific node is pretty much the only way.

3

Thanks Warkolm,

My doubt is, if we are having 9 nodes with 5 Replica and 2 master nodes.

from any of the client machine using, "Sense/.net Nest Wrapper" if I call the URL http://Hostname:9200 to search/index the data then, which of the 9 nodes will receive this request ?

If anyone of the master node receive the http request, do we need to configure the URL in the master nodes or automatically the ElasticSearch server has this option once we mark it as master node ?

4

The one you call, ie Hostname.

What do you mean by this? You shouldn't send queries to masters, that is what you have client nodes for.

5

I mean the Hostname as the URL, like localhost:9200

the URL which we are using to call the ElasticSearch Server, from any of the client system to get the data using sense.

from marvel sense, when we are trying to search/insert a data - then out of nine nodes in the cluster - which node will receive this request ?

6

Whichever host you have pointed Sense to.

7

Mark,

In Elastic Search version 0.90.13 ,

We are having 9 nodes in Amzon Cloud.

Please let us know how to setup the Host in one of these nodes (out of 9)

We are using windows server, in one of the node we want to setup the host something like "http://myES:9200"

Once we created the host in one of the ES node, how to integrate the ElasticSearch Windows Service to this Host URL ? i.e. from one of the client system - in sense if some user pointing to the host "http://myES:9200" then how it will get/insert the data from the node (9 nodes)

So the Host entry "http://myES:9200" in one node (out of nine) how this will push and get data from other nodes (note: the cluster name of all the 9 nodes are same... my query is regarding the host URL)

Please share your lync/skype ID I can explain it in detail if required.

Thanks,
Kannan

8

Why are you using such an old version? There are so many fixes and improvements since then.

I think I'm not 100% following you here.
You want to know how you will get data from the other nodes when using a client if you only point to one of the 9 nodes in the cluster? If so you'd have to read this - Distributed Search Execution | Elasticsearch: The Definitive Guide [2.x] | Elastic

If you'd like custom support, I can ask one of our sales people to contact you to arrange a subscription :slight_smile:

9

Thanks Mark,

This system was built when the 0.90.13 was a stable version :slight_smile: ,yes it was pretty old one.

Recently I joined this team, and find a single point of failure chance was there in the implementation.

Steps:

  1. From the client system, if the Host http request was initiated to the DNS/Host Server.

  2. The DNS/Host server receives the Request. In the IIS Web.Config file - the Rewrite URL was mentioned if the http request is for "Search" then redirect to this IP.

  3. In case, if the redirected IP node was down, then it will be failure.

So, is there is any option in the ElasticSearch 0.90.13 to overcome this issue - can you suggest us any other option than using this rewrite URL in the Web.config in the DNS/Host server.

Regarding getting Support,
I am from a company called "Accenture", already my leadership was in progress, to get the ElasticSearch support and licensed version of the same.

Thanks,
Kannan

10

Can you use something else that's more intelligent as the load balancer?