Elasticsearch with azure cloud plugin

Elastic Stack Elasticsearch
1

Hello,
We are using ES with the azure cloud plugin for node communication. Our current set-up is :
2 data node (hosted on azure ubuntu 14.04 VM) 3.5 giga RAM, 2 Cores (AKA medium VM) ES version 1.0.0.0. 5 shards and 2 replicas.
The storing and query are done by Kibana and logstash which are hosted on separate machines.
Logstash connects to the cluster via the node protocol.
The problem we are having is that for some unknown reason one of the data nodes leaves the cluster, no errors are reported in the log of the machine that left and the node keeps running independently (wired, I know) when restarting the elasticsearch service on the lonely node it rejoins the cluster. After the node rejoins the cluster everything seems ok until it or the other data node will leave the cluster again (Meaning that there is no specific node that have the problem, each one of the nodes can suddenly leave the cluster).
Did anyone encounter this problem using the azure plugin for node discovery? Any input on how to approach this issue?
Thanks
Neta.

2

So you are saying that when a node suddenly disappear for whatever reason (network, GC…), he can't rejoin again the cluster automatically so you have to restart it?

If so, could you open an issue in cloud-azure plugin repo and if possible attach logs from the both nodes?

Thanks

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 7 juillet 2014 à 09:35:34, NetaN (neta@biocatch.com) a écrit:

Hello,
We are using ES with the azure cloud plugin for node communication. Our
current set-up is :
2 data node (hosted on azure ubuntu 14.04 VM) 3.5 giga RAM, 2 Cores (AKA
medium VM) ES version 1.0.0.0. 5 shards and 2 replicas.
The storing and query are done by Kibana and logstash which are hosted on
separate machines.
Logstash connects to the cluster via the node protocol.
The problem we are having is that for some unknown reason one of the data
nodes leaves the cluster, no errors are reported in the log of the machine
that left and the node keeps running independently (wired, I know) when
restarting the elasticsearch service on the lonely node it rejoins the
cluster. After the node rejoins the cluster everything seems ok until it or
the other data node will leave the cluster again (Meaning that there is no
specific node that have the problem, each one of the nodes can suddenly
leave the cluster).
Did anyone encounter this problem using the azure plugin for node discovery?
Any input on how to approach this issue?
Thanks
Neta.

--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Elasticsearch-with-azure-cloud-plugin-tp4059340.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1404718530391-4059340.post%40n3.nabble.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.53ba58f9.440badfc.2fae%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/d/optout.

3

Yes this is the case. I will post

On Mon, Jul 7, 2014 at 11:24 AM, dadoonet [via Elasticsearch Users] <
ml-node+s115913n4059344h2@n3.nabble.com> wrote:

So you are saying that when a node suddenly disappear for whatever reason
(network, GC…), he can't rejoin again the cluster automatically so you have
to restart it?

If so, could you open an issue in cloud-azure plugin repo and if possible
attach logs from the both nodes?

Thanks

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfr
https://twitter.com/elasticsearchfr

Le 7 juillet 2014 à 09:35:34, NetaN ([hidden email]
http://user/SendEmail.jtp?type=node&node=4059344&i=0) a écrit:

Hello,
We are using ES with the azure cloud plugin for node communication. Our
current set-up is :
2 data node (hosted on azure ubuntu 14.04 VM) 3.5 giga RAM, 2 Cores (AKA
medium VM) ES version 1.0.0.0. 5 shards and 2 replicas.
The storing and query are done by Kibana and logstash which are hosted on
separate machines.
Logstash connects to the cluster via the node protocol.
The problem we are having is that for some unknown reason one of the data
nodes leaves the cluster, no errors are reported in the log of the machine
that left and the node keeps running independently (wired, I know) when
restarting the elasticsearch service on the lonely node it rejoins the
cluster. After the node rejoins the cluster everything seems ok until it
or
the other data node will leave the cluster again (Meaning that there is no
specific node that have the problem, each one of the nodes can suddenly
leave the cluster).
Did anyone encounter this problem using the azure plugin for node
discovery?
Any input on how to approach this issue?
Thanks
Neta.

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Elasticsearch-with-azure-cloud-plugin-tp4059340.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to [hidden email]
http://user/SendEmail.jtp?type=node&node=4059344&i=1.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1404718530391-4059340.post%40n3.nabble.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to [hidden email]
http://user/SendEmail.jtp?type=node&node=4059344&i=2.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.53ba58f9.440badfc.2fae%40MacBook-Air-de-David.local
https://groups.google.com/d/msgid/elasticsearch/etPan.53ba58f9.440badfc.2fae%40MacBook-Air-de-David.local?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

If you reply to this email, your message will be added to the discussion
below:

http://elasticsearch-users.115913.n3.nabble.com/Elasticsearch-with-azure-cloud-plugin-tp4059340p4059344.html
To unsubscribe from Elasticsearch with azure cloud plugin, click here
http://elasticsearch-users.115913.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4059340&code=bmV0YUBiaW9jYXRjaC5jb218NDA1OTM0MHwtMTMxODcwNzM2Ng==
.
NAML
http://elasticsearch-users.115913.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html!nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers!nabble%3Aemail.naml-instant_emails!nabble%3Aemail.naml-send_instant_email!nabble%3Aemail.naml

4

This doesn't sound like it's Azure-specific. For one, I'd try to use ES
1.2.1 as there has been a lot of work in that area (of GC and threads).

I'd also try to avoid using the Azure plugin as long as possible and use
Unicast instead - I've just blogged about exactly that, see
http://code972.com/blog/2014/07/74-the-definitive-guide-for-elasticsearch-on-windows-azure

HTH,

--

Itamar Syn-Hershko
http://code972.com | @synhershko https://twitter.com/synhershko
Freelance Developer & Consultant
Author of RavenDB in Action http://manning.com/synhershko/

On Mon, Jul 7, 2014 at 10:35 AM, NetaN neta@biocatch.com wrote:

Hello,
We are using ES with the azure cloud plugin for node communication. Our
current set-up is :
2 data node (hosted on azure ubuntu 14.04 VM) 3.5 giga RAM, 2 Cores (AKA
medium VM) ES version 1.0.0.0. 5 shards and 2 replicas.
The storing and query are done by Kibana and logstash which are hosted on
separate machines.
Logstash connects to the cluster via the node protocol.
The problem we are having is that for some unknown reason one of the data
nodes leaves the cluster, no errors are reported in the log of the machine
that left and the node keeps running independently (wired, I know) when
restarting the elasticsearch service on the lonely node it rejoins the
cluster. After the node rejoins the cluster everything seems ok until it or
the other data node will leave the cluster again (Meaning that there is no
specific node that have the problem, each one of the nodes can suddenly
leave the cluster).
Did anyone encounter this problem using the azure plugin for node
discovery?
Any input on how to approach this issue?
Thanks
Neta.

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Elasticsearch-with-azure-cloud-plugin-tp4059340.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1404718530391-4059340.post%40n3.nabble.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHTr4Zvr1fLPxPky4RQ5a3wRTFH6KrnXJEJxF4aRUw0KQDxRwQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.