Another EC2 clustering question

Elastic Stack Elasticsearch
1

Howdy - I cannot get two ec2 servers to connect to one another as a
cluster. The servers are successfully discovering themselves via the
supplied AWS credentials with proper permissions, however I the non-master
server continually connects to and fails when joining the master. I have
verified the ports are open for both TCP and UDP on both servers in the
9200-9400 port range.

The specific error, found in the log below is:

[2015-01-06 22:52:25,954][TRACE][discovery.ec2 ] [es0] failed to send join
request to master
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]] org.elasticsearch.transport.RemoteTransportException:
[es1][inet[/172.31.23.251:9300]][internal:discovery/zen/join] Caused by:
org.elasticsearch.transport.NotSerializableTransportException:
[org.elasticsearch.transport.ConnectTransportException]
[es0][inet[/172.31.29.66:9300]] connect_timeout[30s]; connection timed out:
/172.31.29.66:9300;

I'm stuck after several hours, hoping someone can point me in the correct
direction.

Thanks!

Shane

Log excerpt:
[2015-01-06 22:51:52,618][TRACE][discovery.ec2 ] [es0] starting to ping

[2015-01-06 22:51:52,758][TRACE][discovery.ec2 ] [es0] building dynamic
unicast discovery nodes...

[2015-01-06 22:51:52,758][TRACE][discovery.ec2 ] [es0] adding i-dd78fdd1,
address 172.31.29.66, transport_address inet[/172.31.29.66:9300]

[2015-01-06 22:51:52,758][TRACE][discovery.ec2 ] [es0] adding i-a220ea54,
address 172.31.23.251, transport_address inet[/172.31.23.251:9300]

[2015-01-06 22:51:52,758][DEBUG][discovery.ec2 ] [es0] using dynamic
discovery nodes
[[#cloud-i-dd78fdd1-0][ip-172-31-29-66][inet[/172.31.29.66:9300]],
[#cloud-i-a220ea54-0][ip-172-31-29-66][inet[/172.31.23.251:9300]]]

[2015-01-06 22:51:54,348][TRACE][discovery.ec2 ] [es0] building dynamic
unicast discovery nodes...

[2015-01-06 22:51:54,348][TRACE][discovery.ec2 ] [es0] adding i-dd78fdd1,
address 172.31.29.66, transport_address inet[/172.31.29.66:9300]

[2015-01-06 22:51:54,349][TRACE][discovery.ec2 ] [es0] adding i-a220ea54,
address 172.31.23.251, transport_address inet[/172.31.23.251:9300]

[2015-01-06 22:51:54,349][DEBUG][discovery.ec2 ] [es0] using dynamic
discovery nodes
[[#cloud-i-dd78fdd1-0][ip-172-31-29-66][inet[/172.31.29.66:9300]],
[#cloud-i-a220ea54-0][ip-172-31-29-66][inet[/172.31.23.251:9300]]]

[2015-01-06 22:51:55,913][TRACE][discovery.ec2 ] [es0] building dynamic
unicast discovery nodes...

[2015-01-06 22:51:55,913][TRACE][discovery.ec2 ] [es0] adding i-dd78fdd1,
address 172.31.29.66, transport_address inet[/172.31.29.66:9300]

[2015-01-06 22:51:55,914][TRACE][discovery.ec2 ] [es0] adding i-a220ea54,
address 172.31.23.251, transport_address inet[/172.31.23.251:9300]

[2015-01-06 22:51:55,914][DEBUG][discovery.ec2 ] [es0] using dynamic
discovery nodes
[[#cloud-i-dd78fdd1-0][ip-172-31-29-66][inet[/172.31.29.66:9300]],
[#cloud-i-a220ea54-0][ip-172-31-29-66][inet[/172.31.23.251:9300]]]

[2015-01-06 22:51:55,922][TRACE][discovery.ec2 ] [es0] full ping responses:

--> ping_response{node
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]],
id[51], master
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]],
hasJoinedOnce [true], cluster_name[elasticsearch]}

[2015-01-06 22:51:55,922][DEBUG][discovery.ec2 ] [es0] filtered ping
responses: (filter_client[true], filter_data[false])

--> ping_response{node
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]],
id[51], master
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]],
hasJoinedOnce [true], cluster_name[elasticsearch]}

[2015-01-06 22:51:55,922][TRACE][discovery.ec2 ] [es0] joining master
[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]

[2015-01-06 22:52:25,954][TRACE][discovery.ec2 ] [es0] failed to send join
request to master
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]]

org.elasticsearch.transport.RemoteTransportException:
[es1][inet[/172.31.23.251:9300]][internal:discovery/zen/join]

Caused by: org.elasticsearch.transport.NotSerializableTransportException:
[org.elasticsearch.transport.ConnectTransportException]
[es0][inet[/172.31.29.66:9300]] connect_timeout[30s]; connection timed out:
/172.31.29.66:9300;

And here is the config file I am using on both servers:
cluster.name: elasticsearch
plugin:
mandatory: cloud-aws

cloud.aws.region: us-west-2
cloud:
aws:
access_key: AAA
secret_key: BBB
discovery:
type: ec2
discovery.ec2.tag.stack: staging

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ee6d6225-f383-4e6b-b4d7-464ae82414e7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Are both running the same ES and java versions?

Can you telnet between the data and master nodes on 9300?

On 7 January 2015 at 09:56, shane@gethashed.com wrote:

Howdy - I cannot get two ec2 servers to connect to one another as a
cluster. The servers are successfully discovering themselves via the
supplied AWS credentials with proper permissions, however I the non-master
server continually connects to and fails when joining the master. I have
verified the ports are open for both TCP and UDP on both servers in the
9200-9400 port range.

The specific error, found in the log below is:

[2015-01-06 22:52:25,954][TRACE][discovery.ec2 ] [es0] failed to send join
request to master
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]] org.elasticsearch.transport.RemoteTransportException:
[es1][inet[/172.31.23.251:9300]][internal:discovery/zen/join] Caused by:
org.elasticsearch.transport.NotSerializableTransportException:
[org.elasticsearch.transport.ConnectTransportException] [es0][inet[/
172.31.29.66:9300]] connect_timeout[30s]; connection timed out: /
172.31.29.66:9300;

I'm stuck after several hours, hoping someone can point me in the correct
direction.

Thanks!

Shane

Log excerpt:
[2015-01-06 22:51:52,618][TRACE][discovery.ec2 ] [es0] starting to ping

[2015-01-06 22:51:52,758][TRACE][discovery.ec2 ] [es0] building dynamic
unicast discovery nodes...

[2015-01-06 22:51:52,758][TRACE][discovery.ec2 ] [es0] adding i-dd78fdd1,
address 172.31.29.66, transport_address inet[/172.31.29.66:9300]

[2015-01-06 22:51:52,758][TRACE][discovery.ec2 ] [es0] adding i-a220ea54,
address 172.31.23.251, transport_address inet[/172.31.23.251:9300]

[2015-01-06 22:51:52,758][DEBUG][discovery.ec2 ] [es0] using dynamic
discovery nodes
[[#cloud-i-dd78fdd1-0][ip-172-31-29-66][inet[/172.31.29.66:9300]],
[#cloud-i-a220ea54-0][ip-172-31-29-66][inet[/172.31.23.251:9300]]]

[2015-01-06 22:51:54,348][TRACE][discovery.ec2 ] [es0] building dynamic
unicast discovery nodes...

[2015-01-06 22:51:54,348][TRACE][discovery.ec2 ] [es0] adding i-dd78fdd1,
address 172.31.29.66, transport_address inet[/172.31.29.66:9300]

[2015-01-06 22:51:54,349][TRACE][discovery.ec2 ] [es0] adding i-a220ea54,
address 172.31.23.251, transport_address inet[/172.31.23.251:9300]

[2015-01-06 22:51:54,349][DEBUG][discovery.ec2 ] [es0] using dynamic
discovery nodes
[[#cloud-i-dd78fdd1-0][ip-172-31-29-66][inet[/172.31.29.66:9300]],
[#cloud-i-a220ea54-0][ip-172-31-29-66][inet[/172.31.23.251:9300]]]

[2015-01-06 22:51:55,913][TRACE][discovery.ec2 ] [es0] building dynamic
unicast discovery nodes...

[2015-01-06 22:51:55,913][TRACE][discovery.ec2 ] [es0] adding i-dd78fdd1,
address 172.31.29.66, transport_address inet[/172.31.29.66:9300]

[2015-01-06 22:51:55,914][TRACE][discovery.ec2 ] [es0] adding i-a220ea54,
address 172.31.23.251, transport_address inet[/172.31.23.251:9300]

[2015-01-06 22:51:55,914][DEBUG][discovery.ec2 ] [es0] using dynamic
discovery nodes
[[#cloud-i-dd78fdd1-0][ip-172-31-29-66][inet[/172.31.29.66:9300]],
[#cloud-i-a220ea54-0][ip-172-31-29-66][inet[/172.31.23.251:9300]]]

[2015-01-06 22:51:55,922][TRACE][discovery.ec2 ] [es0] full ping
responses:

--> ping_response{node
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]],
id[51], master
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]],
hasJoinedOnce [true], cluster_name[elasticsearch]}

[2015-01-06 22:51:55,922][DEBUG][discovery.ec2 ] [es0] filtered ping
responses: (filter_client[true], filter_data[false])

--> ping_response{node
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]],
id[51], master
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]],
hasJoinedOnce [true], cluster_name[elasticsearch]}

[2015-01-06 22:51:55,922][TRACE][discovery.ec2 ] [es0] joining master
[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]

[2015-01-06 22:52:25,954][TRACE][discovery.ec2 ] [es0] failed to send join
request to master
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]]

org.elasticsearch.transport.RemoteTransportException:
[es1][inet[/172.31.23.251:9300]][internal:discovery/zen/join]

Caused by: org.elasticsearch.transport.NotSerializableTransportException:
[org.elasticsearch.transport.ConnectTransportException] [es0][inet[/
172.31.29.66:9300]] connect_timeout[30s]; connection timed out: /
172.31.29.66:9300;

And here is the config file I am using on both servers:
cluster.name: elasticsearch
plugin:
mandatory: cloud-aws

cloud.aws.region: us-west-2
cloud:
aws:
access_key: AAA
secret_key: BBB
discovery:
type: ec2
discovery.ec2.tag.stack: staging

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ee6d6225-f383-4e6b-b4d7-464ae82414e7%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/ee6d6225-f383-4e6b-b4d7-464ae82414e7%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-n4aXvneytPc2JQ2HSfaac%3D4BJ9pak0X1aT3vm9ZTdXQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Hi Mark

Thank you for your response.

Yes both have the same ES version [ same AMI with ES pre-installed]. I can
telnet between the two servers on 9300, 9200, 9400. Running netstat shows
the two machines are actively connected on port 9300.

The original machine which becomes the master machine has around 2.8G with
of indexes. The secondary machine I've tried with a complete copy of the
index or starting with an empty index.

Cheers,
Shane

On Tue, Jan 6, 2015 at 3:15 PM, Mark Walkom markwalkom@gmail.com wrote:

Are both running the same ES and java versions?

Can you telnet between the data and master nodes on 9300?

On 7 January 2015 at 09:56, shane@gethashed.com wrote:

Howdy - I cannot get two ec2 servers to connect to one another as a
cluster. The servers are successfully discovering themselves via the
supplied AWS credentials with proper permissions, however I the non-master
server continually connects to and fails when joining the master. I have
verified the ports are open for both TCP and UDP on both servers in the
9200-9400 port range.

The specific error, found in the log below is:

[2015-01-06 22:52:25,954][TRACE][discovery.ec2 ] [es0] failed to send
join request to master
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]] org.elasticsearch.transport.RemoteTransportException:
[es1][inet[/172.31.23.251:9300]][internal:discovery/zen/join] Caused
by: org.elasticsearch.transport.NotSerializableTransportException:
[org.elasticsearch.transport.ConnectTransportException] [es0][inet[/
172.31.29.66:9300]] connect_timeout[30s]; connection timed out: /
172.31.29.66:9300;

I'm stuck after several hours, hoping someone can point me in the correct
direction.

Thanks!

Shane

Log excerpt:
[2015-01-06 22:51:52,618][TRACE][discovery.ec2 ] [es0] starting to ping

[2015-01-06 22:51:52,758][TRACE][discovery.ec2 ] [es0] building dynamic
unicast discovery nodes...

[2015-01-06 22:51:52,758][TRACE][discovery.ec2 ] [es0] adding i-dd78fdd1,
address 172.31.29.66, transport_address inet[/172.31.29.66:9300]

[2015-01-06 22:51:52,758][TRACE][discovery.ec2 ] [es0] adding i-a220ea54,
address 172.31.23.251, transport_address inet[/172.31.23.251:9300]

[2015-01-06 22:51:52,758][DEBUG][discovery.ec2 ] [es0] using dynamic
discovery nodes
[[#cloud-i-dd78fdd1-0][ip-172-31-29-66][inet[/172.31.29.66:9300]],
[#cloud-i-a220ea54-0][ip-172-31-29-66][inet[/172.31.23.251:9300]]]

[2015-01-06 22:51:54,348][TRACE][discovery.ec2 ] [es0] building dynamic
unicast discovery nodes...

[2015-01-06 22:51:54,348][TRACE][discovery.ec2 ] [es0] adding i-dd78fdd1,
address 172.31.29.66, transport_address inet[/172.31.29.66:9300]

[2015-01-06 22:51:54,349][TRACE][discovery.ec2 ] [es0] adding i-a220ea54,
address 172.31.23.251, transport_address inet[/172.31.23.251:9300]

[2015-01-06 22:51:54,349][DEBUG][discovery.ec2 ] [es0] using dynamic
discovery nodes
[[#cloud-i-dd78fdd1-0][ip-172-31-29-66][inet[/172.31.29.66:9300]],
[#cloud-i-a220ea54-0][ip-172-31-29-66][inet[/172.31.23.251:9300]]]

[2015-01-06 22:51:55,913][TRACE][discovery.ec2 ] [es0] building dynamic
unicast discovery nodes...

[2015-01-06 22:51:55,913][TRACE][discovery.ec2 ] [es0] adding i-dd78fdd1,
address 172.31.29.66, transport_address inet[/172.31.29.66:9300]

[2015-01-06 22:51:55,914][TRACE][discovery.ec2 ] [es0] adding i-a220ea54,
address 172.31.23.251, transport_address inet[/172.31.23.251:9300]

[2015-01-06 22:51:55,914][DEBUG][discovery.ec2 ] [es0] using dynamic
discovery nodes
[[#cloud-i-dd78fdd1-0][ip-172-31-29-66][inet[/172.31.29.66:9300]],
[#cloud-i-a220ea54-0][ip-172-31-29-66][inet[/172.31.23.251:9300]]]

[2015-01-06 22:51:55,922][TRACE][discovery.ec2 ] [es0] full ping
responses:

--> ping_response{node
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]],
id[51], master
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]],
hasJoinedOnce [true], cluster_name[elasticsearch]}

[2015-01-06 22:51:55,922][DEBUG][discovery.ec2 ] [es0] filtered ping
responses: (filter_client[true], filter_data[false])

--> ping_response{node
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]],
id[51], master
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]],
hasJoinedOnce [true], cluster_name[elasticsearch]}

[2015-01-06 22:51:55,922][TRACE][discovery.ec2 ] [es0] joining master
[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]

[2015-01-06 22:52:25,954][TRACE][discovery.ec2 ] [es0] failed to send
join request to master
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]]

org.elasticsearch.transport.RemoteTransportException:
[es1][inet[/172.31.23.251:9300]][internal:discovery/zen/join]

Caused by: org.elasticsearch.transport.NotSerializableTransportException:
[org.elasticsearch.transport.ConnectTransportException] [es0][inet[/
172.31.29.66:9300]] connect_timeout[30s]; connection timed out: /
172.31.29.66:9300;

And here is the config file I am using on both servers:
cluster.name: elasticsearch
plugin:
mandatory: cloud-aws

cloud.aws.region: us-west-2
cloud:
aws:
access_key: AAA
secret_key: BBB
discovery:
type: ec2
discovery.ec2.tag.stack: staging

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ee6d6225-f383-4e6b-b4d7-464ae82414e7%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/ee6d6225-f383-4e6b-b4d7-464ae82414e7%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/TpFqBy5GdXA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-n4aXvneytPc2JQ2HSfaac%3D4BJ9pak0X1aT3vm9ZTdXQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-n4aXvneytPc2JQ2HSfaac%3D4BJ9pak0X1aT3vm9ZTdXQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAG8XaFUvz7nmpKcXQc6jJDqsXuCgwSAYg2hb9o-B%3DkYpWV5-KA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

4

Shane,

What's JVM version is ES running under, as Mark asked? That exception
usually indicates that you're running two different JVMs in the cluster,
which unfortunately is not supported due to how Java serializes exceptions.

Ross

On Wednesday, 7 January 2015 10:22:46 UTC+11, shane adams wrote:

Hi Mark

Thank you for your response.

Yes both have the same ES version [ same AMI with ES pre-installed]. I
can telnet between the two servers on 9300, 9200, 9400. Running netstat
shows the two machines are actively connected on port 9300.

The original machine which becomes the master machine has around 2.8G with
of indexes. The secondary machine I've tried with a complete copy of the
index or starting with an empty index.

Cheers,
Shane

On Tue, Jan 6, 2015 at 3:15 PM, Mark Walkom <markw...@gmail.com
<javascript:>> wrote:

Are both running the same ES and java versions?

Can you telnet between the data and master nodes on 9300?

On 7 January 2015 at 09:56, <sh...@gethashed.com <javascript:>> wrote:

Howdy - I cannot get two ec2 servers to connect to one another as a
cluster. The servers are successfully discovering themselves via the
supplied AWS credentials with proper permissions, however I the non-master
server continually connects to and fails when joining the master. I have
verified the ports are open for both TCP and UDP on both servers in the
9200-9400 port range.

The specific error, found in the log below is:

[2015-01-06 22:52:25,954][TRACE][discovery.ec2 ] [es0] failed to send
join request to master
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]] org.elasticsearch.transport.RemoteTransportException:
[es1][inet[/172.31.23.251:9300]][internal:discovery/zen/join] Caused by:
org.elasticsearch.transport.NotSerializableTransportException:
[org.elasticsearch.transport.ConnectTransportException] [es0][inet[/
172.31.29.66:9300]] connect_timeout[30s]; connection timed out: /
172.31.29.66:9300;

I'm stuck after several hours, hoping someone can point me in the
correct direction.

Thanks!

Shane

Log excerpt:
[2015-01-06 22:51:52,618][TRACE][discovery.ec2 ] [es0] starting to ping

[2015-01-06 22:51:52,758][TRACE][discovery.ec2 ] [es0] building dynamic
unicast discovery nodes...

[2015-01-06 22:51:52,758][TRACE][discovery.ec2 ] [es0] adding
i-dd78fdd1, address 172.31.29.66, transport_address inet[/
172.31.29.66:9300]

[2015-01-06 22:51:52,758][TRACE][discovery.ec2 ] [es0] adding
i-a220ea54, address 172.31.23.251, transport_address inet[/
172.31.23.251:9300]

[2015-01-06 22:51:52,758][DEBUG][discovery.ec2 ] [es0] using dynamic
discovery nodes
[[#cloud-i-dd78fdd1-0][ip-172-31-29-66][inet[/172.31.29.66:9300]],
[#cloud-i-a220ea54-0][ip-172-31-29-66][inet[/172.31.23.251:9300]]]

[2015-01-06 22:51:54,348][TRACE][discovery.ec2 ] [es0] building dynamic
unicast discovery nodes...

[2015-01-06 22:51:54,348][TRACE][discovery.ec2 ] [es0] adding
i-dd78fdd1, address 172.31.29.66, transport_address inet[/
172.31.29.66:9300]

[2015-01-06 22:51:54,349][TRACE][discovery.ec2 ] [es0] adding
i-a220ea54, address 172.31.23.251, transport_address inet[/
172.31.23.251:9300]

[2015-01-06 22:51:54,349][DEBUG][discovery.ec2 ] [es0] using dynamic
discovery nodes
[[#cloud-i-dd78fdd1-0][ip-172-31-29-66][inet[/172.31.29.66:9300]],
[#cloud-i-a220ea54-0][ip-172-31-29-66][inet[/172.31.23.251:9300]]]

[2015-01-06 22:51:55,913][TRACE][discovery.ec2 ] [es0] building dynamic
unicast discovery nodes...

[2015-01-06 22:51:55,913][TRACE][discovery.ec2 ] [es0] adding
i-dd78fdd1, address 172.31.29.66, transport_address inet[/
172.31.29.66:9300]

[2015-01-06 22:51:55,914][TRACE][discovery.ec2 ] [es0] adding
i-a220ea54, address 172.31.23.251, transport_address inet[/
172.31.23.251:9300]

[2015-01-06 22:51:55,914][DEBUG][discovery.ec2 ] [es0] using dynamic
discovery nodes
[[#cloud-i-dd78fdd1-0][ip-172-31-29-66][inet[/172.31.29.66:9300]],
[#cloud-i-a220ea54-0][ip-172-31-29-66][inet[/172.31.23.251:9300]]]

[2015-01-06 22:51:55,922][TRACE][discovery.ec2 ] [es0] full ping
responses:

--> ping_response{node
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]],
id[51], master
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]],
hasJoinedOnce [true], cluster_name[elasticsearch]}

[2015-01-06 22:51:55,922][DEBUG][discovery.ec2 ] [es0] filtered ping
responses: (filter_client[true], filter_data[false])

--> ping_response{node
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]],
id[51], master
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]],
hasJoinedOnce [true], cluster_name[elasticsearch]}

[2015-01-06 22:51:55,922][TRACE][discovery.ec2 ] [es0] joining master
[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]

[2015-01-06 22:52:25,954][TRACE][discovery.ec2 ] [es0] failed to send
join request to master
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]]

org.elasticsearch.transport.RemoteTransportException:
[es1][inet[/172.31.23.251:9300]][internal:discovery/zen/join]

Caused by:
org.elasticsearch.transport.NotSerializableTransportException:
[org.elasticsearch.transport.ConnectTransportException] [es0][inet[/
172.31.29.66:9300]] connect_timeout[30s]; connection timed out: /
172.31.29.66:9300;

And here is the config file I am using on both servers:
cluster.name: elasticsearch
plugin:
mandatory: cloud-aws

cloud.aws.region: us-west-2
cloud:
aws:
access_key: AAA
secret_key: BBB
discovery:
type: ec2
discovery.ec2.tag.stack: staging

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ee6d6225-f383-4e6b-b4d7-464ae82414e7%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/ee6d6225-f383-4e6b-b4d7-464ae82414e7%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/TpFqBy5GdXA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-n4aXvneytPc2JQ2HSfaac%3D4BJ9pak0X1aT3vm9ZTdXQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-n4aXvneytPc2JQ2HSfaac%3D4BJ9pak0X1aT3vm9ZTdXQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f6db3827-b544-43c3-872c-8c433698cc5e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

5

Sorry new born at home and I'm tired, missed the JVM version request. I
verified it's the same [same AMI image I created for both instances so I
expected this to be the case]. java version "1.8.0_25"

I've tried reconfiguring to use unicast and I get a similar error but I
need to retest to confirm 100%.

Shane

On Tue, Jan 6, 2015 at 7:43 PM, Ross Simpson simpsora@gmail.com wrote:

Shane,

What's JVM version is ES running under, as Mark asked? That exception
usually indicates that you're running two different JVMs in the cluster,
which unfortunately is not supported due to how Java serializes exceptions.

Ross

On Wednesday, 7 January 2015 10:22:46 UTC+11, shane adams wrote:

Hi Mark

Thank you for your response.

Yes both have the same ES version [ same AMI with ES pre-installed]. I
can telnet between the two servers on 9300, 9200, 9400. Running netstat
shows the two machines are actively connected on port 9300.

The original machine which becomes the master machine has around 2.8G
with of indexes. The secondary machine I've tried with a complete copy of
the index or starting with an empty index.

Cheers,
Shane

On Tue, Jan 6, 2015 at 3:15 PM, Mark Walkom markw...@gmail.com wrote:

Are both running the same ES and java versions?

Can you telnet between the data and master nodes on 9300?

On 7 January 2015 at 09:56, sh...@gethashed.com wrote:

Howdy - I cannot get two ec2 servers to connect to one another as a
cluster. The servers are successfully discovering themselves via the
supplied AWS credentials with proper permissions, however I the non-master
server continually connects to and fails when joining the master. I have
verified the ports are open for both TCP and UDP on both servers in the
9200-9400 port range.

The specific error, found in the log below is:

[2015-01-06 22:52:25,954][TRACE][discovery.ec2 ] [es0] failed to send
join request to master [[es1][VNcvo8b3TSqObDo7nEwkAg]
[ip-172-31-23-251][inet[/172.31.23.251:9300]]] org.
elasticsearch.transport.RemoteTransportException: [es1][inet[/
172.31.23.251:9300]][internal:discovery/zen/join] Caused by:
org.elasticsearch.transport.NotSerializableTransportException:
[org.elasticsearch.transport.ConnectTransportException] [es0][inet[/
172.31.29.66:9300]] connect_timeout[30s]; connection timed out: /
172.31.29.66:9300;

I'm stuck after several hours, hoping someone can point me in the
correct direction.

Thanks!

Shane

Log excerpt:
[2015-01-06 22:51:52,618][TRACE][discovery.ec2 ] [es0] starting to
ping

[2015-01-06 22:51:52,758][TRACE][discovery.ec2 ] [es0] building
dynamic unicast discovery nodes...

[2015-01-06 22:51:52,758][TRACE][discovery.ec2 ] [es0] adding
i-dd78fdd1, address 172.31.29.66, transport_address inet[/
172.31.29.66:9300]

[2015-01-06 22:51:52,758][TRACE][discovery.ec2 ] [es0] adding
i-a220ea54, address 172.31.23.251, transport_address inet[/
172.31.23.251:9300]

[2015-01-06 22:51:52,758][DEBUG][discovery.ec2 ] [es0] using dynamic
discovery nodes [[#cloud-i-dd78fdd1-0][ip-172-31-29-66][inet[/
172.31.29.66:9300]], [#cloud-i-a220ea54-0][ip-172-31-29-66][inet[/
172.31.23.251:9300]]]

[2015-01-06 22:51:54,348][TRACE][discovery.ec2 ] [es0] building
dynamic unicast discovery nodes...

[2015-01-06 22:51:54,348][TRACE][discovery.ec2 ] [es0] adding
i-dd78fdd1, address 172.31.29.66, transport_address inet[/
172.31.29.66:9300]

[2015-01-06 22:51:54,349][TRACE][discovery.ec2 ] [es0] adding
i-a220ea54, address 172.31.23.251, transport_address inet[/
172.31.23.251:9300]

[2015-01-06 22:51:54,349][DEBUG][discovery.ec2 ] [es0] using dynamic
discovery nodes [[#cloud-i-dd78fdd1-0][ip-172-31-29-66][inet[/
172.31.29.66:9300]], [#cloud-i-a220ea54-0][ip-172-31-29-66][inet[/
172.31.23.251:9300]]]

[2015-01-06 22:51:55,913][TRACE][discovery.ec2 ] [es0] building
dynamic unicast discovery nodes...

[2015-01-06 22:51:55,913][TRACE][discovery.ec2 ] [es0] adding
i-dd78fdd1, address 172.31.29.66, transport_address inet[/
172.31.29.66:9300]

[2015-01-06 22:51:55,914][TRACE][discovery.ec2 ] [es0] adding
i-a220ea54, address 172.31.23.251, transport_address inet[/
172.31.23.251:9300]

[2015-01-06 22:51:55,914][DEBUG][discovery.ec2 ] [es0] using dynamic
discovery nodes [[#cloud-i-dd78fdd1-0][ip-172-31-29-66][inet[/
172.31.29.66:9300]], [#cloud-i-a220ea54-0][ip-172-31-29-66][inet[/
172.31.23.251:9300]]]

[2015-01-06 22:51:55,922][TRACE][discovery.ec2 ] [es0] full ping
responses:

--> ping_response{node [[es1][VNcvo8b3TSqObDo7nEwkAg]
[ip-172-31-23-251][inet[/172.31.23.251:9300]]], id[51], master
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]],
hasJoinedOnce [true], cluster_name[elasticsearch]}

[2015-01-06 22:51:55,922][DEBUG][discovery.ec2 ] [es0] filtered ping
responses: (filter_client[true], filter_data[false])

--> ping_response{node [[es1][VNcvo8b3TSqObDo7nEwkAg]
[ip-172-31-23-251][inet[/172.31.23.251:9300]]], id[51], master
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]],
hasJoinedOnce [true], cluster_name[elasticsearch]}

[2015-01-06 22:51:55,922][TRACE][discovery.ec2 ] [es0] joining master
[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.
31.23.251:9300]]

[2015-01-06 22:52:25,954][TRACE][discovery.ec2 ] [es0] failed to send
join request to master [[es1][VNcvo8b3TSqObDo7nEwkAg]
[ip-172-31-23-251][inet[/172.31.23.251:9300]]]

org.elasticsearch.transport.RemoteTransportException: [es1][inet[/
172.31.23.251:9300]][internal:discovery/zen/join]

Caused by: org.elasticsearch.transport.NotSerializableTransportException:
[org.elasticsearch.transport.ConnectTransportException] [es0][inet[/
172.31.29.66:9300]] connect_timeout[30s]; connection timed out: /
172.31.29.66:9300;

And here is the config file I am using on both servers:
cluster.name: elasticsearch
plugin:
mandatory: cloud-aws

cloud.aws.region: us-west-2
cloud:
aws:
access_key: AAA
secret_key: BBB
discovery:
type: ec2
discovery.ec2.tag.stack: staging

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/ee6d6225-f383-4e6b-b4d7-464ae82414e7%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/ee6d6225-f383-4e6b-b4d7-464ae82414e7%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/TpFqBy5GdXA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/CAEYi1X-n4aXvneytPc2JQ2HSfaac%
3D4BJ9pak0X1aT3vm9ZTdXQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-n4aXvneytPc2JQ2HSfaac%3D4BJ9pak0X1aT3vm9ZTdXQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/TpFqBy5GdXA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f6db3827-b544-43c3-872c-8c433698cc5e%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/f6db3827-b544-43c3-872c-8c433698cc5e%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAG8XaFWh5wGXyjriL4FW_zD2s%2BZiJQHNqRKaf-rufVQVKQQJ_Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

6

Figured it out. One of the servers was not a member of the proper security
group and port 9300 was closed. I had misconfigured ES to use 9200 for
cluster transport and that's why it was giving an odd error, attempting a
non-HTTP protocol against the REST API. Sorry to bug anyone.

On Wed, Jan 7, 2015 at 9:55 AM, shane adams shane@gethashed.com wrote:

Sorry new born at home and I'm tired, missed the JVM version request. I
verified it's the same [same AMI image I created for both instances so I
expected this to be the case]. java version "1.8.0_25"

I've tried reconfiguring to use unicast and I get a similar error but I
need to retest to confirm 100%.

Shane

On Tue, Jan 6, 2015 at 7:43 PM, Ross Simpson simpsora@gmail.com wrote:

Shane,

What's JVM version is ES running under, as Mark asked? That exception
usually indicates that you're running two different JVMs in the cluster,
which unfortunately is not supported due to how Java serializes exceptions.

Ross

On Wednesday, 7 January 2015 10:22:46 UTC+11, shane adams wrote:

Hi Mark

Thank you for your response.

Yes both have the same ES version [ same AMI with ES pre-installed]. I
can telnet between the two servers on 9300, 9200, 9400. Running netstat
shows the two machines are actively connected on port 9300.

The original machine which becomes the master machine has around 2.8G
with of indexes. The secondary machine I've tried with a complete copy of
the index or starting with an empty index.

Cheers,
Shane

On Tue, Jan 6, 2015 at 3:15 PM, Mark Walkom markw...@gmail.com wrote:

Are both running the same ES and java versions?

Can you telnet between the data and master nodes on 9300?

On 7 January 2015 at 09:56, sh...@gethashed.com wrote:

Howdy - I cannot get two ec2 servers to connect to one another as a
cluster. The servers are successfully discovering themselves via the
supplied AWS credentials with proper permissions, however I the non-master
server continually connects to and fails when joining the master. I have
verified the ports are open for both TCP and UDP on both servers in the
9200-9400 port range.

The specific error, found in the log below is:

[2015-01-06 22:52:25,954][TRACE][discovery.ec2 ] [es0] failed to send
join request to master [[es1][VNcvo8b3TSqObDo7nEwkAg]
[ip-172-31-23-251][inet[/172.31.23.251:9300]]] org.
elasticsearch.transport.RemoteTransportException: [es1][inet[/
172.31.23.251:9300]][internal:discovery/zen/join] Caused by:
org.elasticsearch.transport.NotSerializableTransportException:
[org.elasticsearch.transport.ConnectTransportException] [es0][inet[/
172.31.29.66:9300]] connect_timeout[30s]; connection timed out: /
172.31.29.66:9300;

I'm stuck after several hours, hoping someone can point me in the
correct direction.

Thanks!

Shane

Log excerpt:
[2015-01-06 22:51:52,618][TRACE][discovery.ec2 ] [es0] starting to
ping

[2015-01-06 22:51:52,758][TRACE][discovery.ec2 ] [es0] building
dynamic unicast discovery nodes...

[2015-01-06 22:51:52,758][TRACE][discovery.ec2 ] [es0] adding
i-dd78fdd1, address 172.31.29.66, transport_address inet[/
172.31.29.66:9300]

[2015-01-06 22:51:52,758][TRACE][discovery.ec2 ] [es0] adding
i-a220ea54, address 172.31.23.251, transport_address inet[/
172.31.23.251:9300]

[2015-01-06 22:51:52,758][DEBUG][discovery.ec2 ] [es0] using dynamic
discovery nodes [[#cloud-i-dd78fdd1-0][ip-172-31-29-66][inet[/
172.31.29.66:9300]], [#cloud-i-a220ea54-0][ip-172-31-29-66][inet[/
172.31.23.251:9300]]]

[2015-01-06 22:51:54,348][TRACE][discovery.ec2 ] [es0] building
dynamic unicast discovery nodes...

[2015-01-06 22:51:54,348][TRACE][discovery.ec2 ] [es0] adding
i-dd78fdd1, address 172.31.29.66, transport_address inet[/
172.31.29.66:9300]

[2015-01-06 22:51:54,349][TRACE][discovery.ec2 ] [es0] adding
i-a220ea54, address 172.31.23.251, transport_address inet[/
172.31.23.251:9300]

[2015-01-06 22:51:54,349][DEBUG][discovery.ec2 ] [es0] using dynamic
discovery nodes [[#cloud-i-dd78fdd1-0][ip-172-31-29-66][inet[/
172.31.29.66:9300]], [#cloud-i-a220ea54-0][ip-172-31-29-66][inet[/
172.31.23.251:9300]]]

[2015-01-06 22:51:55,913][TRACE][discovery.ec2 ] [es0] building
dynamic unicast discovery nodes...

[2015-01-06 22:51:55,913][TRACE][discovery.ec2 ] [es0] adding
i-dd78fdd1, address 172.31.29.66, transport_address inet[/
172.31.29.66:9300]

[2015-01-06 22:51:55,914][TRACE][discovery.ec2 ] [es0] adding
i-a220ea54, address 172.31.23.251, transport_address inet[/
172.31.23.251:9300]

[2015-01-06 22:51:55,914][DEBUG][discovery.ec2 ] [es0] using dynamic
discovery nodes [[#cloud-i-dd78fdd1-0][ip-172-31-29-66][inet[/
172.31.29.66:9300]], [#cloud-i-a220ea54-0][ip-172-31-29-66][inet[/
172.31.23.251:9300]]]

[2015-01-06 22:51:55,922][TRACE][discovery.ec2 ] [es0] full ping
responses:

--> ping_response{node [[es1][VNcvo8b3TSqObDo7nEwkAg]
[ip-172-31-23-251][inet[/172.31.23.251:9300]]], id[51], master
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]],
hasJoinedOnce [true], cluster_name[elasticsearch]}

[2015-01-06 22:51:55,922][DEBUG][discovery.ec2 ] [es0] filtered ping
responses: (filter_client[true], filter_data[false])

--> ping_response{node [[es1][VNcvo8b3TSqObDo7nEwkAg]
[ip-172-31-23-251][inet[/172.31.23.251:9300]]], id[51], master
[[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.31.23.251:9300]]],
hasJoinedOnce [true], cluster_name[elasticsearch]}

[2015-01-06 22:51:55,922][TRACE][discovery.ec2 ] [es0] joining master
[es1][VNcvo8b3TSqObDo7nEwkAg][ip-172-31-23-251][inet[/172.
31.23.251:9300]]

[2015-01-06 22:52:25,954][TRACE][discovery.ec2 ] [es0] failed to send
join request to master [[es1][VNcvo8b3TSqObDo7nEwkAg]
[ip-172-31-23-251][inet[/172.31.23.251:9300]]]

org.elasticsearch.transport.RemoteTransportException: [es1][inet[/
172.31.23.251:9300]][internal:discovery/zen/join]

Caused by: org.elasticsearch.transport.NotSerializableTransportException:
[org.elasticsearch.transport.ConnectTransportException] [es0][inet[/
172.31.29.66:9300]] connect_timeout[30s]; connection timed out: /
172.31.29.66:9300;

And here is the config file I am using on both servers:
cluster.name: elasticsearch
plugin:
mandatory: cloud-aws

cloud.aws.region: us-west-2
cloud:
aws:
access_key: AAA
secret_key: BBB
discovery:
type: ec2
discovery.ec2.tag.stack: staging

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/ee6d6225-f383-4e6b-b4d7-464ae82414e7%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/ee6d6225-f383-4e6b-b4d7-464ae82414e7%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/TpFqBy5GdXA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/CAEYi1X-n4aXvneytPc2JQ2HSfaac%
3D4BJ9pak0X1aT3vm9ZTdXQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-n4aXvneytPc2JQ2HSfaac%3D4BJ9pak0X1aT3vm9ZTdXQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/TpFqBy5GdXA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f6db3827-b544-43c3-872c-8c433698cc5e%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/f6db3827-b544-43c3-872c-8c433698cc5e%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAG8XaFWWf5LWc66Sw1S5x2chRpMQ2TTZ4r0DbbQaA-LPUxSEFA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.