Elasticsearch cluster on Azure using ubuntu. The nodes don't see each other


(Pedro Alonso) #1

I just posted this question on Stackoverflow:

I have been setting up a cluster of Elasticsearch in Azure, using Ubuntu
VM, following the tutorial on the plugin page (elasticsearch-cloud-azure)
on github. I've managed to configure everything and I have elasticsearch
running, but I have 3 clusters of 1 Node instead of 1 Cluster of 3 nodes. I
guess that the problem comes from:

cloud: azure: keystore: /path/to/keystore password:
your_password_for_keystore subscription_id: your_azure_subscription_id
service_name: your_azure_cloud_service_name discovery: type: azure

I'm not sure of what "your_azure_cloud_service_name" should be. I have all
my nodes inside a Virtual Network, so they can communicate each other. By
default, on azure each time I create a VM, a new Cloud Service containing
only that VM is created. Should that value be different for each of the
nodes in my cluster?

I'm a bit lost on that one...

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e2968f5d-9f67-421c-a60f-8fd5053317ce%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(David Pilato) #2

You must create each VM under the same cloud service.
azure vm create azure-elasticsearch-cluster ....
Cloud service name is azure-elasticsearch-cluster

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 21 juin 2014 à 03:54, Pedro Alonso pedro.paf@gmail.com a écrit :

I just posted this question on Stackoverflow:

I have been setting up a cluster of Elasticsearch in Azure, using Ubuntu VM, following the tutorial on the plugin page (elasticsearch-cloud-azure) on github. I've managed to configure everything and I have elasticsearch running, but I have 3 clusters of 1 Node instead of 1 Cluster of 3 nodes. I guess that the problem comes from:

cloud:
azure:
keystore: /path/to/keystore
password: your_password_for_keystore
subscription_id: your_azure_subscription_id
service_name: your_azure_cloud_service_name
discovery:
type: azure

I'm not sure of what "your_azure_cloud_service_name" should be. I have all my nodes inside a Virtual Network, so they can communicate each other. By default, on azure each time I create a VM, a new Cloud Service containing only that VM is created. Should that value be different for each of the nodes in my cluster?

I'm a bit lost on that one...

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e2968f5d-9f67-421c-a60f-8fd5053317ce%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/64636299-774A-4C9B-865A-E3FEB85F326B%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.


(Pedro Alonso) #3

Thanks David, it did work. It's a bit confusing that because by default
Azure binds a VM to the port 22 for SSH it creates a new cloud service. I
need to specify the binding to be a different port to have multiple VMs
under the same cloud service. I'll blog about my experience to clarify this
for other people..

On Sat, Jun 21, 2014 at 5:45 AM, David Pilato david@pilato.fr wrote:

You must create each VM under the same cloud service.

azure vm create azure-elasticsearch-cluster ....

Cloud service name is azure-elasticsearch-cluster

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 21 juin 2014 à 03:54, Pedro Alonso pedro.paf@gmail.com a écrit :

I just posted this question on Stackoverflow:

I have been setting up a cluster of Elasticsearch in Azure, using Ubuntu
VM, following the tutorial on the plugin page (elasticsearch-cloud-azure)
on github. I've managed to configure everything and I have elasticsearch
running, but I have 3 clusters of 1 Node instead of 1 Cluster of 3 nodes. I
guess that the problem comes from:

cloud: azure: keystore: /path/to/keystore password:
your_password_for_keystore subscription_id: your_azure_subscription_id
service_name: your_azure_cloud_service_name discovery: type: azure

I'm not sure of what "your_azure_cloud_service_name" should be. I have all
my nodes inside a Virtual Network, so they can communicate each other. By
default, on azure each time I create a VM, a new Cloud Service containing
only that VM is created. Should that value be different for each of the
nodes in my cluster?

I'm a bit lost on that one...

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/e2968f5d-9f67-421c-a60f-8fd5053317ce%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/e2968f5d-9f67-421c-a60f-8fd5053317ce%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Bq_oRG_1l2g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/64636299-774A-4C9B-865A-E3FEB85F326B%40pilato.fr
https://groups.google.com/d/msgid/elasticsearch/64636299-774A-4C9B-865A-E3FEB85F326B%40pilato.fr?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CA%2BwPCA%2BoVCEoth7oePcC3Wc9YW5iJ37LfVzbfWzR3Jh0K6GBAA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(David Pilato) #4

Great! Feel free to send PR to enhance the plugin documentation as well! :slight_smile:

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 23 juin 2014 à 21:36, Pedro Alonso pedro.paf@gmail.com a écrit :

Thanks David, it did work. It's a bit confusing that because by default Azure binds a VM to the port 22 for SSH it creates a new cloud service. I need to specify the binding to be a different port to have multiple VMs under the same cloud service. I'll blog about my experience to clarify this for other people..

On Sat, Jun 21, 2014 at 5:45 AM, David Pilato david@pilato.fr wrote:
You must create each VM under the same cloud service.

azure vm create azure-elasticsearch-cluster ....
Cloud service name is azure-elasticsearch-cluster

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 21 juin 2014 à 03:54, Pedro Alonso pedro.paf@gmail.com a écrit :

I just posted this question on Stackoverflow:

I have been setting up a cluster of Elasticsearch in Azure, using Ubuntu VM, following the tutorial on the plugin page (elasticsearch-cloud-azure) on github. I've managed to configure everything and I have elasticsearch running, but I have 3 clusters of 1 Node instead of 1 Cluster of 3 nodes. I guess that the problem comes from:

cloud:
azure:
keystore: /path/to/keystore
password: your_password_for_keystore
subscription_id: your_azure_subscription_id
service_name: your_azure_cloud_service_name
discovery:
type: azure

I'm not sure of what "your_azure_cloud_service_name" should be. I have all my nodes inside a Virtual Network, so they can communicate each other. By default, on azure each time I create a VM, a new Cloud Service containing only that VM is created. Should that value be different for each of the nodes in my cluster?

I'm a bit lost on that one...

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e2968f5d-9f67-421c-a60f-8fd5053317ce%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/Bq_oRG_1l2g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/64636299-774A-4C9B-865A-E3FEB85F326B%40pilato.fr.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CA%2BwPCA%2BoVCEoth7oePcC3Wc9YW5iJ37LfVzbfWzR3Jh0K6GBAA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/AC1E7F20-6FA4-4FF5-BEBC-99ED13D88F82%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.


(Pedro Alonso) #5

I have blogged about my experience setting this up as part of the whole
architecture. Hope it helps!

On Mon, Jun 23, 2014 at 11:29 PM, David Pilato david@pilato.fr wrote:

Great! Feel free to send PR to enhance the plugin documentation as well!
:slight_smile:

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 23 juin 2014 à 21:36, Pedro Alonso pedro.paf@gmail.com a écrit :

Thanks David, it did work. It's a bit confusing that because by default
Azure binds a VM to the port 22 for SSH it creates a new cloud service. I
need to specify the binding to be a different port to have multiple VMs
under the same cloud service. I'll blog about my experience to clarify this
for other people..

On Sat, Jun 21, 2014 at 5:45 AM, David Pilato david@pilato.fr wrote:

You must create each VM under the same cloud service.

azure vm create azure-elasticsearch-cluster ....

Cloud service name is azure-elasticsearch-cluster

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 21 juin 2014 à 03:54, Pedro Alonso pedro.paf@gmail.com a écrit :

I just posted this question on Stackoverflow:

I have been setting up a cluster of Elasticsearch in Azure, using Ubuntu
VM, following the tutorial on the plugin page (elasticsearch-cloud-azure)
on github. I've managed to configure everything and I have elasticsearch
running, but I have 3 clusters of 1 Node instead of 1 Cluster of 3 nodes. I
guess that the problem comes from:

cloud: azure: keystore: /path/to/keystore password:
your_password_for_keystore subscription_id: your_azure_subscription_id
service_name: your_azure_cloud_service_name discovery: type: azure

I'm not sure of what "your_azure_cloud_service_name" should be. I have
all my nodes inside a Virtual Network, so they can communicate each other.
By default, on azure each time I create a VM, a new Cloud Service
containing only that VM is created. Should that value be different for each
of the nodes in my cluster?

I'm a bit lost on that one...

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/e2968f5d-9f67-421c-a60f-8fd5053317ce%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/e2968f5d-9f67-421c-a60f-8fd5053317ce%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Bq_oRG_1l2g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/64636299-774A-4C9B-865A-E3FEB85F326B%40pilato.fr
https://groups.google.com/d/msgid/elasticsearch/64636299-774A-4C9B-865A-E3FEB85F326B%40pilato.fr?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CA%2BwPCA%2BoVCEoth7oePcC3Wc9YW5iJ37LfVzbfWzR3Jh0K6GBAA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CA%2BwPCA%2BoVCEoth7oePcC3Wc9YW5iJ37LfVzbfWzR3Jh0K6GBAA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Bq_oRG_1l2g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/AC1E7F20-6FA4-4FF5-BEBC-99ED13D88F82%40pilato.fr
https://groups.google.com/d/msgid/elasticsearch/AC1E7F20-6FA4-4FF5-BEBC-99ED13D88F82%40pilato.fr?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CA%2BwPCAKLV-CmYMhKs9po5iYaLwonRE3yZfAOnkjOq8%2BGowrEQA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(David Pilato) #6

Thank you for sharing it! :slight_smile:

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 26 juin 2014 à 00:19, Pedro Alonso pedro.paf@gmail.com a écrit :

I have blogged about my experience setting this up as part of the whole architecture. Hope it helps! http://www.pedroalonso.net/blog/2014/06/25/azure-cloud-services-and-elasticsearch-slash-nosql-cluster-paas

On Mon, Jun 23, 2014 at 11:29 PM, David Pilato david@pilato.fr wrote:
Great! Feel free to send PR to enhance the plugin documentation as well! :slight_smile:

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 23 juin 2014 à 21:36, Pedro Alonso pedro.paf@gmail.com a écrit :

Thanks David, it did work. It's a bit confusing that because by default Azure binds a VM to the port 22 for SSH it creates a new cloud service. I need to specify the binding to be a different port to have multiple VMs under the same cloud service. I'll blog about my experience to clarify this for other people..

On Sat, Jun 21, 2014 at 5:45 AM, David Pilato david@pilato.fr wrote:
You must create each VM under the same cloud service.

azure vm create azure-elasticsearch-cluster ....
Cloud service name is azure-elasticsearch-cluster

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 21 juin 2014 à 03:54, Pedro Alonso pedro.paf@gmail.com a écrit :

I just posted this question on Stackoverflow:

I have been setting up a cluster of Elasticsearch in Azure, using Ubuntu VM, following the tutorial on the plugin page (elasticsearch-cloud-azure) on github. I've managed to configure everything and I have elasticsearch running, but I have 3 clusters of 1 Node instead of 1 Cluster of 3 nodes. I guess that the problem comes from:

cloud:
azure:
keystore: /path/to/keystore
password: your_password_for_keystore
subscription_id: your_azure_subscription_id
service_name: your_azure_cloud_service_name
discovery:
type: azure

I'm not sure of what "your_azure_cloud_service_name" should be. I have all my nodes inside a Virtual Network, so they can communicate each other. By default, on azure each time I create a VM, a new Cloud Service containing only that VM is created. Should that value be different for each of the nodes in my cluster?

I'm a bit lost on that one...

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e2968f5d-9f67-421c-a60f-8fd5053317ce%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/Bq_oRG_1l2g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/64636299-774A-4C9B-865A-E3FEB85F326B%40pilato.fr.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CA%2BwPCA%2BoVCEoth7oePcC3Wc9YW5iJ37LfVzbfWzR3Jh0K6GBAA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/Bq_oRG_1l2g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/AC1E7F20-6FA4-4FF5-BEBC-99ED13D88F82%40pilato.fr.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CA%2BwPCAKLV-CmYMhKs9po5iYaLwonRE3yZfAOnkjOq8%2BGowrEQA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6A5E72DE-360D-4CA2-9F2A-73C0E05E82D6%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.


(system) #7