Java Client Connection issue and AWS Security Group

Elastic Stack Elasticsearch
1

On AWS, I am connecting to a ES node using Java Transport Client.

The client has security group A
The search node has security group B.

For security group B, I opened all TCP/UDP ports from security group A.

I confirmed that client machine from security group A can query search node
in security group B.
*
*
*curl
http://ec2-some_elastic_ip.compute-1.amazonaws.com:9200/_cluster/health?pretty=true
*
returns valid result

However when I run the Java Client, I get a no node available exception.

org.elasticsearch.client.transport.NoNodeAvailableException: No node
available

  • at
    org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:202)
  • at
    org.elasticsearch.client.transport.support.InternalTransportClient.execute(InternalTransportClient.java:106)
  • at
    org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:210)
  • at
    org.elasticsearch.client.transport.TransportClient.search(TransportClient.java:388)
  • at
    org.elasticsearch.action.search.SearchRequestBuilder.doExecute(SearchRequestBuilder.java:816)
  • at
    org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:62)
  • at
    org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:57)

The odd thing is that, for security group B (ElasticSearch), if I open all
ports to 0.0.0.0/0 instead of security group A, requests from client will
work, and I no longer get the error. However, this is a security flaw since
it allows connection from anywhere.

I know it's possible to connect from A to B since I can CURL to it.

I think it's probably an issue with my Java Client setup. I have my
configuration below, the address specified in java client is the public dns
of the search node after assigning it an elastic ip.

Any guidance will be appreciated.

Cheers,
Fish

------------- Search Node Config
*cluster.name:
cloud:
aws:
access_key:
secret_key:
region: us-east-1
discovery:
type: ec2
ec2:
groups: ElasticSearch
path:
logs: /var/log/elasticsearch
data: /var/data/elasticsearch

network.publish_host: ec2:publicIp

index.number_of_shards: 8
index.number_of_replicas: 1

bootstrap.mlockall: true*

Java Client

Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name",
"").build();
transportClient = new TransportClient(settings);
transportClient.addTransportAddress(new InetSocketTransportAddress("ec2-.compute-1.amazonaws.com", 9300));

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Have you specified the cluster name in the transport client?

Have you checked port 9300? Note, that curl doesn't help much since you
connect to port 9200, which is HTTP.

Jörg

Am 26.04.13 21:38, schrieb Fish Tastic:

I know it's possible to connect from A to B since I can CURL to it.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Thanks for the reply

On client, I tried port 9300 to telnet into the search node, and it works.

telnet ec2-.compute-1.amazonaws.com 9300
Trying 10.168.14.253...
Connected to ec2-54-225-87-216.compute-1.amazonaws.com.

I also double checked the cluster name I passed in and it's same as the one
on my search node.
Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name",
"").build();

The issue is some kinda of connection/security issue. When I open port to
security group it doesn't connect, when I open port to all sources, it
would work fine.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Thanks for the reply

On client, I tried port 9300 to telnet into the search node, and it works.

telnet ec2-.compute-1.amazonaws.com 9300
Trying 10.168.14.253...
Connected to ec2-.compute-1.amazonaws.comhttp://ec2-54-225-87-216.compute-1.amazonaws.com/
.

I also double checked the cluster name I passed in and it's same as the
one on my search node.
Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name",
"").build();

The issue is some kinda of connection/security issue. When I open port to
security group it doesn't connect, when I open port to all sources, it
would work fine.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

Hi,

We also met the same issue,
But in our case both client and server in the same security group, and we
use internal ip so don't need to open any port.
We use AWS unicast

Here are config

  1. In ES server
    #discovery.zen.ping.multicast.enabled: true
    discovery.zen.ping.unicast.hosts: ["CLIENT_PRIVATE_DNS"]

  2. In TransportClient

Node test setup

cluster.name=logsene-prod

don't specify name

#node.name="ip-10-4-213-56.ec2.internal"

Disable network

discovery.zen.ping.multicast.enabled=false
discovery.zen.ping.unicast.enabled=true
discovery.zen.ping.unicast.hosts=SERVER_IP[9300]

Regards
Tien

On Saturday, April 27, 2013 4:12:40 AM UTC+7, Fish Tastic wrote:

Thanks for the reply

On client, I tried port 9300 to telnet into the search node, and it works.

telnet ec2-.compute-1.amazonaws.com 9300
Trying 10.168.14.253...
Connected to ec2-.compute-1.amazonaws.comhttp://ec2-54-225-87-216.compute-1.amazonaws.com/
.

I also double checked the cluster name I passed in and it's same as the
one on my search node.
Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name",
"").build();

The issue is some kinda of connection/security issue. When I open port to
security group it doesn't connect, when I open port to all sources, it
would work fine.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

6

We also met the same issue,
But in our case both client and server in the same security group, and we
use internal ip so don't need to open any port.
We use AWS unicast

Here are config

  1. In ES server
    #discovery.zen.ping.multicast.
    enabled: true
    discovery.zen.ping.unicast.hosts: ["CLIENT_PRIVATE_DNS"]

  2. In TransportClient

Node test setup

cluster.name=NAME

don't specify name

#node.name="name"

Disable network

discovery.zen.ping.multicast.enabled=false
discovery.zen.ping.unicast.enabled=true
discovery.zen.ping.unicast.hosts=SERVER_IP[9300]

Regards
Tien

On Saturday, April 27, 2013 2:38:59 AM UTC+7, Fish Tastic wrote:

On AWS, I am connecting to a ES node using Java Transport Client.

The client has security group A
The search node has security group B.

For security group B, I opened all TCP/UDP ports from security group A.

I confirmed that client machine from security group A can query search
node in security group B.
*
*
*curl
http://ec2-some_elastic_ip.compute-1.amazonaws.com:9200/_cluster/health?pretty=true
*
returns valid result

However when I run the Java Client, I get a no node available exception.

org.elasticsearch.client.transport.NoNodeAvailableException: No node
available

  • at
    org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:202)
  • at
    org.elasticsearch.client.transport.support.InternalTransportClient.execute(InternalTransportClient.java:106)
  • at
    org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:210)
  • at
    org.elasticsearch.client.transport.TransportClient.search(TransportClient.java:388)
  • at
    org.elasticsearch.action.search.SearchRequestBuilder.doExecute(SearchRequestBuilder.java:816)
  • at
    org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:62)
  • at
    org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:57)

The odd thing is that, for security group B (Elasticsearch), if I open all
ports to 0.0.0.0/0 instead of security group A, requests from client will
work, and I no longer get the error. However, this is a security flaw since
it allows connection from anywhere.

I know it's possible to connect from A to B since I can CURL to it.

I think it's probably an issue with my Java Client setup. I have my
configuration below, the address specified in java client is the public dns
of the search node after assigning it an elastic ip.

Any guidance will be appreciated.

Cheers,
Fish

------------- Search Node Config
*cluster.name:
cloud:
aws:
access_key:
secret_key:
region: us-east-1
discovery:
type: ec2
ec2:
groups: Elasticsearch
path:
logs: /var/log/elasticsearch
data: /var/data/elasticsearch

network.publish_host: ec2:publicIp

index.number_of_shards: 8
index.number_of_replicas: 1

bootstrap.mlockall: true*

Java Client

Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name",
"").build();
transportClient = new TransportClient(settings);
transportClient.addTransportAddress(new InetSocketTransportAddress("ec2-.compute-1.amazonaws.com", 9300));

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

7

My best guess is that it's a Java Client issue. This is because we can
connect to the search node from client machine using curl/telnet... just
not though java code.

If anyone managed to set this up, an example of ElasticSearch/AWS/Java
Client configuration would be great.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

8

Did you get this to work? I've the same problem with the Java Client API.
The port 9200 and 9300 is set up properly, but only http Rest calls to port
9200 works. I've set the cluster.name in both client and node, turned off
multicast etc. Now i'm smashing my head against the wall.

On Friday, April 26, 2013 9:38:59 PM UTC+2, Fish Tastic wrote:

On AWS, I am connecting to a ES node using Java Transport Client.

The client has security group A
The search node has security group B.

For security group B, I opened all TCP/UDP ports from security group A.

I confirmed that client machine from security group A can query search
node in security group B.
*
*
*curl
http://ec2-some_elastic_ip.compute-1.amazonaws.com:9200/_cluster/health?pretty=true
*
returns valid result

However when I run the Java Client, I get a no node available exception.

org.elasticsearch.client.transport.NoNodeAvailableException: No node
available

  • at
    org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:202)
  • at
    org.elasticsearch.client.transport.support.InternalTransportClient.execute(InternalTransportClient.java:106)
  • at
    org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:210)
  • at
    org.elasticsearch.client.transport.TransportClient.search(TransportClient.java:388)
  • at
    org.elasticsearch.action.search.SearchRequestBuilder.doExecute(SearchRequestBuilder.java:816)
  • at
    org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:62)
  • at
    org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:57)

The odd thing is that, for security group B (Elasticsearch), if I open all
ports to 0.0.0.0/0 instead of security group A, requests from client will
work, and I no longer get the error. However, this is a security flaw since
it allows connection from anywhere.

I know it's possible to connect from A to B since I can CURL to it.

I think it's probably an issue with my Java Client setup. I have my
configuration below, the address specified in java client is the public dns
of the search node after assigning it an elastic ip.

Any guidance will be appreciated.

Cheers,
Fish

------------- Search Node Config
*cluster.name:
cloud:
aws:
access_key:
secret_key:
region: us-east-1
discovery:
type: ec2
ec2:
groups: Elasticsearch
path:
logs: /var/log/elasticsearch
data: /var/data/elasticsearch

network.publish_host: ec2:publicIp

index.number_of_shards: 8
index.number_of_replicas: 1

bootstrap.mlockall: true*

Java Client

Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name",
"").build();
transportClient = new TransportClient(settings);
transportClient.addTransportAddress(new InetSocketTransportAddress("ec2-.compute-1.amazonaws.com", 9300));

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

9

In AWS , even connecting to same sec group, u need to allow the traffic
from the same secgroup once u packet exits your instance sec group rules
apply. In VPC you also have acls to deal with( though if u changed the
allow all acl, you'd know )
On 01/05/2013 8:31 PM, tien.nguyenmanh@sematext.com wrote:

Hi,

We also met the same issue,
But in our case both client and server in the same security group, and we
use internal ip so don't need to open any port.
We use AWS unicast

Here are config

  1. In ES server
    #discovery.zen.ping.multicast.enabled: true
    discovery.zen.ping.unicast.hosts: ["CLIENT_PRIVATE_DNS"]

  2. In TransportClient

Node test setup

cluster.name=logsene-prod

don't specify name

#node.name="ip-10-4-213-56.ec2.internal"

Disable network

discovery.zen.ping.multicast.enabled=false
discovery.zen.ping.unicast.enabled=true
discovery.zen.ping.unicast.hosts=SERVER_IP[9300]

Regards
Tien

On Saturday, April 27, 2013 4:12:40 AM UTC+7, Fish Tastic wrote:

Thanks for the reply

On client, I tried port 9300 to telnet into the search node, and it
works.

telnet ec2-.compute-1.amazonaws.com 9300
Trying 10.168.14.253...
Connected to ec2-.compute-1.amazonaws.comhttp://ec2-54-225-87-216.compute-1.amazonaws.com/
.

I also double checked the cluster name I passed in and it's same as the
one on my search node.
Settings settings = ImmutableSettings.settingsBuilder().put("cluster
.name http://cluster.name/", "").build();

The issue is some kinda of connection/security issue. When I open port
to security group it doesn't connect, when I open port to all sources, it
would work fine.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

10

Do you get any actual connection at all? Any info on the node's logs? TCP
dump on both client and node to see what really is going on (I would start
with a filter like "tcp port 9300 or udp port 53"... Then broaden it if
you get no packets.
On 19/08/2013 6:22 AM, "Bjørn Bråthen" bjorbrat88@gmail.com wrote:

Did you get this to work? I've the same problem with the Java Client API.
The port 9200 and 9300 is set up properly, but only http Rest calls to port
9200 works. I've set the cluster.name in both client and node, turned off
multicast etc. Now i'm smashing my head against the wall.

On Friday, April 26, 2013 9:38:59 PM UTC+2, Fish Tastic wrote:

On AWS, I am connecting to a ES node using Java Transport Client.

The client has security group A
The search node has security group B.

For security group B, I opened all TCP/UDP ports from security group A.

I confirmed that client machine from security group A can query search
node in security group B.
*
*
curl http://ec2-some_elastic_ip.compute-1.amazonaws.com:9200/_
cluster/health?pretty=true
returns valid result

However when I run the Java Client, I get a no node available exception.

org.elasticsearch.client.transport.NoNodeAvailableException: No node
available

  • at org.elasticsearch.client.transport.TransportClientNodesService.
    execute(TransportClientNodesService.java:202)*
  • at org.elasticsearch.client.transport.support.InternalTransportClient.
    execute(InternalTransportClient.java:106)*
  • at org.elasticsearch.client.support.AbstractClient.search(
    AbstractClient.java:210)*
  • at org.elasticsearch.client.transport.TransportClient.
    search(TransportClient.java:388)*
  • at org.elasticsearch.action.search.SearchRequestBuilder.doExecute(
    SearchRequestBuilder.java:816)*
  • at org.elasticsearch.action.ActionRequestBuilder.execute(
    ActionRequestBuilder.java:62)*
  • at org.elasticsearch.action.ActionRequestBuilder.execute(
    ActionRequestBuilder.java:57)*

The odd thing is that, for security group B (Elasticsearch), if I open
all ports to 0.0.0.0/0 instead of security group A, requests from client
will work, and I no longer get the error. However, this is a security flaw
since it allows connection from anywhere.

I know it's possible to connect from A to B since I can CURL to it.

I think it's probably an issue with my Java Client setup. I have my
configuration below, the address specified in java client is the public dns
of the search node after assigning it an elastic ip.

Any guidance will be appreciated.

Cheers,
Fish

------------- Search Node Config
*cluster.name:
cloud:
aws:
access_key:
secret_key:
region: us-east-1
discovery:
type: ec2
ec2:
groups: Elasticsearch
path:
logs: /var/log/elasticsearch
data: /var/data/elasticsearch

network.publish_host: ec2:publicIp

index.number_of_shards: 8
index.number_of_replicas: 1

bootstrap.mlockall: true*

Java Client

Settings settings = ImmutableSettings.settingsBuilder().put("cluster
.name http://cluster.name", "").build();
transportClient = new TransportClient(settings);
transportClient.*addTransportAddress(new InetSocketTransportAddress("ec
*2-.compute-1.amazonaws.**comhttp://compute-1.amazonaws.com
", 9300));

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

11

It was a Play2-elasticsearch issue, client.transport.sniff was set to true
on default, turned client.transport.sniff to false and it worked.

On Monday, August 19, 2013 12:04:41 PM UTC+2, Norberto Meijome wrote:

Do you get any actual connection at all? Any info on the node's logs? TCP
dump on both client and node to see what really is going on (I would start
with a filter like "tcp port 9300 or udp port 53"... Then broaden it if
you get no packets.
On 19/08/2013 6:22 AM, "Bjørn Bråthen" <bjorb...@gmail.com <javascript:>>
wrote:

Did you get this to work? I've the same problem with the Java Client API.
The port 9200 and 9300 is set up properly, but only http Rest calls to port
9200 works. I've set the cluster.name in both client and node, turned
off multicast etc. Now i'm smashing my head against the wall.

On Friday, April 26, 2013 9:38:59 PM UTC+2, Fish Tastic wrote:

On AWS, I am connecting to a ES node using Java Transport Client.

The client has security group A
The search node has security group B.

For security group B, I opened all TCP/UDP ports from security group A.

I confirmed that client machine from security group A can query search
node in security group B.
*
*
curl http://ec2-some_elastic_ip.compute-1.amazonaws.com:9200/_
cluster/health?pretty=true
returns valid result

However when I run the Java Client, I get a no node available exception.

org.elasticsearch.client.transport.NoNodeAvailableException: No node
available

  • at org.elasticsearch.client.transport.TransportClientNodesService.
    execute(TransportClientNodesService.java:202)*
  • at org.elasticsearch.client.transport.support.InternalTransportClient.
    execute(InternalTransportClient.java:106)*
  • at org.elasticsearch.client.support.AbstractClient.search(
    AbstractClient.java:210)*
  • at org.elasticsearch.client.transport.TransportClient.
    search(TransportClient.java:388)*
  • at org.elasticsearch.action.search.SearchRequestBuilder.doExecute(
    SearchRequestBuilder.java:816)*
  • at org.elasticsearch.action.ActionRequestBuilder.execute(
    ActionRequestBuilder.java:62)*
  • at org.elasticsearch.action.ActionRequestBuilder.execute(
    ActionRequestBuilder.java:57)*

The odd thing is that, for security group B (Elasticsearch), if I open
all ports to 0.0.0.0/0 instead of security group A, requests from
client will work, and I no longer get the error. However, this is a
security flaw since it allows connection from anywhere.

I know it's possible to connect from A to B since I can CURL to it.

I think it's probably an issue with my Java Client setup. I have my
configuration below, the address specified in java client is the public dns
of the search node after assigning it an elastic ip.

Any guidance will be appreciated.

Cheers,
Fish

------------- Search Node Config
*cluster.name:
cloud:
aws:
access_key:
secret_key:
region: us-east-1
discovery:
type: ec2
ec2:
groups: Elasticsearch
path:
logs: /var/log/elasticsearch
data: /var/data/elasticsearch

network.publish_host: ec2:publicIp

index.number_of_shards: 8
index.number_of_replicas: 1

bootstrap.mlockall: true*

Java Client

Settings settings = ImmutableSettings.settingsBuilder().put("cluster
.name http://cluster.name", "").build();
transportClient = new TransportClient(settings);
transportClient.**addTransportAddress(new InetSocketTransportAddress("ec
**2-.compute-1.amazonaws.**comhttp://compute-1.amazonaws.com
", 9300));

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.