ElasticSerach Cluster behind loadbalancer

Elastic Stack Elasticsearch
1

hi guys,

Scenario1.

  1. I have 3 es nodes.
  2. I have not installed nginx,tomcat on elasticsearch nodes.
  3. These are behind amazon elb.
  4. cluster is detecting nodes automatically.

My need.

  1. elb start putting queries on node when it is actually ready to take traffic. Usually shards reallocation takes time depending on the size of the shard.
2

This is something you would need to co-ordinate outside of ES, there is
nothing native that could do it.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 24 June 2014 20:29, Tarun Jangra tarun@izap.in wrote:

hi guys,

Scenario1.

  1. I have 3 es nodes.
  2. I have not installed nginx,tomcat on elasticsearch nodes.
  3. These are behind amazon elb.
  4. cluster is detecting nodes automatically.

My need.

  1. elb start putting queries on node when it is actually ready to take
    traffic. Usually shards reallocation takes time depending on the size of
    the
    shard.

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/ElasticSerach-Cluster-behind-loadbalancer-tp4058369.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1403605780346-4058369.post%40n3.nabble.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624ZGrss3kyubo2qQzrYk1NOZ4C_1u8d68S1TGxavZUVW4Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Thanks mark,

how if node starts getting queries during this reallocating period. Is there anything i am suppose to loose? Like write can also occur during reallocations.

4

Yes they can both occur in parallel, performance may drop a little but it
will still respond to queries and indexing.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 24 June 2014 20:39, Tarun Jangra tarun@izap.in wrote:

Thanks mark,

how if node starts getting queries during this reallocating period. Is
there
anything i am suppose to loose? Like write can also occur during
reallocations.

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/ElasticSerach-Cluster-behind-loadbalancer-tp4058369p4058372.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1403606364842-4058372.post%40n3.nabble.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624aHie7k8fWFEqeMNtAH8i3SaHF5sfPEE3WE1Z5koNR2Pg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

5

In that case, i don't think i should bother if node is up and reallocation is done.

6

Hi Tarun,

You can use ELB ping functionality along with a custom elasticsearch plugin
to achieve this functionality. You can configure your ping path to be
something like /_plugin/<custom_pluginname> and write your plugin in such a
way that it returns HTTP 200 only when you want your node to serve the
traffic and HTTP 503/500 when you don't want the node to serve traffic.

See

for more details on ELB pings and

for troubleshooting.

Thanks,
Himanshu.

On Tue, Jun 24, 2014 at 4:14 PM, Tarun Jangra tarun@izap.in wrote:

In that case, i don't think i should bother if node is up and reallocation
is
done.

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/ElasticSerach-Cluster-behind-loadbalancer-tp4058369p4058375.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1403606669657-4058375.post%40n3.nabble.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CACTSS%3D57q3W35%2BeUZZrpLar5D1hvs7F5%2BrTa%2BXWH6B-j8MiLxw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

7

hi himanshu,

Well currently i've used /_cluster/health for this. And since elb suppose to execute as per IP. And once i start getting response. I means it is up. But it is not identifying reallocating shard process.

I like your idea. that plugin can be programmed in such a way. that it should start sending HTTP 200 only when node has done sharding etc.

Thank you.