Queries without routing


(Ashwin Sathya) #1

I have set up a 3-node cluster, with the following endpoint configuration.

These are Windows Azure VMs running Windows Server.

There is no multicast support so unicast support with config.yml calling out
their internal IPs. They all sit inside a single cloud service, which
provides the network boundary for the items.

Machine Public Port Private Port

Machine1 9200 9200

Machine2 9200

Machine3 9200

They all have the ports from 9300 to 9399 open for both in/out in the
firewall.

All these machines are configured as part of the same machine.

All our writes and search queries are redirected to the machine1:9200

Now, when a search query is performed on the Machine1:9200 port specified as
the endpoint, which of the nodes are touched for serving that query ? (i.e)
Is only Machine1 involved in serving that query ? I don't want this machine
alone to be hot for serving queries, I want them all to participate taking
turns serving a query, or else I will have to move to a coordinator
node/data node model. If not, then then how do I find out which machine is
serving the search query ?

Thanks,

Ashwin Sathya

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Ashwin Sathya) #2

Any help on this folks ?

Thanks,
Ashwin Sathya

From: ashwin.sathya@outlook.com
To: elasticsearch@googlegroups.com
Subject: Queries without routing
Date: Fri, 13 Sep 2013 17:31:19 +0530

I have set up a 3-node cluster, with the following endpoint configuration.These are Windows Azure VMs running Windows Server.There is no multicast support so unicast support with config.yml calling out their internal IPs. They all sit inside a single cloud service, which provides the network boundary for the items. Machine Public Port Private PortMachine1 9200 9200Machine2 9200Machine3 9200 They all have the ports from 9300 to 9399 open for both in/out in the firewall.All these machines are configured as part of the same machine.All our writes and search queries are redirected to the machine1:9200 Now, when a search query is performed on the Machine1:9200 port specified as the endpoint, which of the nodes are touched for serving that query ? (i.e) Is only Machine1 involved in serving that query ? I don’t want this machine alone to be hot for serving queries, I want them all to participate taking turns serving a query, or else I will have to move to a coordinator node/data node model. If not, then then how do I find out which machine is serving the search query ? Thanks,Ashwin Sathya

--

You received this message because you are subscribed to the Google Groups "elasticsearch" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #3

So, you are using REST network layer.
Machine1 will get the REST request and will identify which shards need to be queried.
It could happen that Machine1 hold one of those. Probably, your request will go to all machines depending on what you are doing.

Some designs you can think about:

  • your client side could connect on public ports 9200, 9201, 9202 for example
  • you can add a nginx on top of elasticsearch or may be something like azure traffic manager could help
  • you can have one elasticsearch non data node which will get all ingoing requests. It will rebalance requests to "backend nodes". Note that this non data node does not need to be a "big" instance.

Does it help?

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 13 sept. 2013 à 14:01, R Ashwin Sathya ashwin.sathya@outlook.com a écrit :

I have set up a 3-node cluster, with the following endpoint configuration.
These are Windows Azure VMs running Windows Server.
There is no multicast support so unicast support with config.yml calling out their internal IPs. They all sit inside a single cloud service, which provides the network boundary for the items.

Machine Public Port Private Port
Machine1 9200 9200
Machine2 9200
Machine3 9200

They all have the ports from 9300 to 9399 open for both in/out in the firewall.
All these machines are configured as part of the same machine.
All our writes and search queries are redirected to the machine1:9200

Now, when a search query is performed on the Machine1:9200 port specified as the endpoint, which of the nodes are touched for serving that query ? (i.e) Is only Machine1 involved in serving that query ? I don’t want this machine alone to be hot for serving queries, I want them all to participate taking turns serving a query, or else I will have to move to a coordinator node/data node model. If not, then then how do I find out which machine is serving the search query ?

Thanks,
Ashwin Sathya

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Ashwin Sathya) #4

Thanks David for the responses. My comments inline.

From: david@pilato.fr
Subject: Re: Queries without routing
Date: Sat, 14 Sep 2013 10:12:52 +0200
To: elasticsearch@googlegroups.com

So, you are using REST network layer.Machine1 will get the REST request and will identify which shards need to be queried.It could happen that Machine1 hold one of those. Probably, your request will go to all machines depending on what you are doing.
Some designs you can think about:- your client side could connect on public ports 9200, 9201, 9202 for example
This is very doable, but the dependency would be on my client side code to push data/search from ES cluster. If I were to add/decommission one of the nodes, it would become a trouble.

  • you can add a nginx on top of elasticsearch or may be something like azure traffic manager could help
    I will try out these.
  • you can have one elasticsearch non data node which will get all ingoing requests. It will rebalance requests to "backend nodes". Note that this non data node does not need to be a "big" instance.
    This is a pretty good option, basically one of the nodes can perform automatic load balancing, and if it all i need lower latency than this, i can always opt for routing. I have one question on this option. Does this mean that all the read and write requests will go this (non data load balancer) node but the actual operation is not performed on this machine ? How does this work ?

Does it help?

--
David Pilato | Technical Advocate | Elasticsearch.com@dadoonet | @elasticsearchfr | @scrutmydocs

Le 13 sept. 2013 à 14:01, R Ashwin Sathya ashwin.sathya@outlook.com a écrit :I have set up a 3-node cluster, with the following endpoint configuration.These are Windows Azure VMs running Windows Server.There is no multicast support so unicast support with config.yml calling out their internal IPs. They all sit inside a single cloud service, which provides the network boundary for the items. Machine Public Port Private PortMachine1 9200 9200Machine2 9200Machine3 9200 They all have the ports from 9300 to 9399 open for both in/out in the firewall.All these machines are configured as part of the same machine.All our writes and search queries are redirected to the machine1:9200 Now, when a search query is performed on the Machine1:9200 port specified as the endpoint, which of the nodes are touched for serving that query ? (i.e) Is only Machine1 involved in serving that query ? I don’t want this machine alone to be hot for serving queries, I want them all to participate taking turns serving a query, or else I will have to move to a coordinator node/data node model. If not, then then how do I find out which machine is serving the search query ? Thanks,Ashwin Sathya--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

You received this message because you are subscribed to the Google Groups "elasticsearch" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #5