Need help connecting to elastic search cluster using java api


(Upender Gilakathula) #1

Hej all,
I have created a Elastic Search cluster with 2 nodes in EC2 using the
homepage documentation with out using any gateway. And Thats working fine.
I can see that cluster helath and bigdesk all the stuff.

Here is my settings

cluster.name: broadwayrealtime
cloud.aws.region: eu-west-1
cloud:
cloud.aws.region: eu-west-1
aws:
access_key: XXXXXXX
secret_key: XXXXXXX
discovery:
type: ec2

node.master: true
node.name: "Brendan Test"

But when I am trying to access through the JAVA api I am getting exception.

I am using the below settings to connect to this cluster using JAVA api.
final ImmutableSettings.Builder settings =
ImmutableSettings.settingsBuilder();
settings.put("discovery.type", "ec2");
settings.put("cloud.aws.access_key", "access_key");
settings.put("cloud.aws.secret_key", "secret_key");
settings.put("cloud.aws.region", "eu-west-1");
settings.put("discovery.ec2.groups", "elasticsearch");
Node node =
NodeBuilder.nodeBuilder().settings(settings).clusterName("clusterName").client(true).node();
client = node.client();

But I get this exception after 1 minute
18:29:25.997 [New I/O client boss #1-0] WARN
org.elasticsearch.transport.netty - [Mutant X] Exception caught on netty
layer [[id: 0x129a41d2]]
java.net.SocketException: Network is unreachable
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.6.0_31]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
~[na:1.6.0_31]
at
org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:404)
[elasticsearch-0.19.4.jar:na]
at
org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:366)
[elasticsearch-0.19.4.jar:na]
at
org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:282)
[elasticsearch-0.19.4.jar:na]
at
org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102)
[elasticsearch-0.19.4.jar:na]
at
org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[elasticsearch-0.19.4.jar:na]
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
[na:1.6.0_31]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
[na:1.6.0_31]
at java.lang.Thread.run(Thread.java:680) [na:1.6.0_31]

org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [1m]
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(TransportMasterNodeOperationAction.java:169)
at
org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:356)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)

Just to share the info; If I use like this its working fine.
final ImmutableSettings.Builder settings =
ImmutableSettings.settingsBuilder();
settings.put("discovery.type", "ec2");
settings.put("cloud.aws.access_key", "access_key");
settings.put("cloud.aws.secret_key", "secret_key");
settings.put("cloud.aws.region", "eu-west-1");
settings.put("discovery.ec2.groups", "elasticsearch");
final TransportClient transportClient = new TransportClient(settings)
.addTransportAddress(new
InetSocketTransportAddress("hostname", 9300))
.addTransportAddress(new
InetSocketTransportAddress("hostname", 9300))

Do I have to specify the hostnames like the above? which is kind of
annoying to specify the host names? Shouldn't my java code find the cluster
since I have specified everything such as clustername, discovery, access
key, secret key. and region.

Many thanks for your help? I really appreciate your help.


(Eric Jain) #2

On Jun 12, 9:46 am, Upender Gilakathula upenderg...@gmail.com wrote:

But I get this exception after 1 minute
18:29:25.997 [New I/O client boss #1-0] WARN
org.elasticsearch.transport.netty - [Mutant X] Exception caught on netty
layer [[id: 0x129a41d2]]
java.net.SocketException: Network is unreachable
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.6.0_31]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
~[na:1.6.0_31]

Are the machines all in the same security group, and are ports 9300 -
9400 open?


(Upender Gilakathula) #3

Yes they are open.

22 (SSH)0.0.0.0/0Delete80 (HTTP)0.0.0.0/0Delete9200 - 93000.0.0.0/0Delete9300

  • 94000.0.0.0/0Delete930010.0.0.0/8Delete
    On Tuesday, June 12, 2012 7:44:07 PM UTC+2, Eric Jain wrote:

On Jun 12, 9:46 am, Upender Gilakathula upenderg...@gmail.com wrote:

But I get this exception after 1 minute
18:29:25.997 [New I/O client boss #1-0] WARN
org.elasticsearch.transport.netty - [Mutant X] Exception caught on
netty
layer [[id: 0x129a41d2]]
java.net.SocketException: Network is unreachable
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
~[na:1.6.0_31]
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
~[na:1.6.0_31]

Are the machines all in the same security group, and are ports 9300 -
9400 open?


(Upender Gilakathula) #4

They are in the same security group ! Many thanks for a quick reply !

On Tuesday, June 12, 2012 11:29:46 PM UTC+2, Upender Gilakathula wrote:

Yes they are open.

22 (SSH)0.0.0.0/0Delete80 (HTTP)0.0.0.0/0Delete9200 - 93000.0.0.0/0Delete9300

  • 94000.0.0.0/0Delete930010.0.0.0/8Delete
    On Tuesday, June 12, 2012 7:44:07 PM UTC+2, Eric Jain wrote:

On Jun 12, 9:46 am, Upender Gilakathula upenderg...@gmail.com wrote:

But I get this exception after 1 minute
18:29:25.997 [New I/O client boss #1-0] WARN
org.elasticsearch.transport.netty - [Mutant X] Exception caught on
netty
layer [[id: 0x129a41d2]]
java.net.SocketException: Network is unreachable
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
~[na:1.6.0_31]
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
~[na:1.6.0_31]

Are the machines all in the same security group, and are ports 9300 -
9400 open?


(Eric Jain) #5

On Jun 12, 2:30 pm, Upender Gilakathula upenderg...@gmail.com wrote:

They are in the same security group ! Many thanks for a quick reply !

Did you install the cloud-aws plugin?

https://github.com/elasticsearch/elasticsearch-cloud-aws


(David Pilato) #6

Are you making calls from a 10.xxx.xxx.xxx network ?
What is this last rule ?

Le 12 juin 2012 à 23:30, Upender Gilakathula upendergoud@gmail.com a écrit :

They are in the same security group ! Many thanks for a quick reply !

On Tuesday, June 12, 2012 11:29:46 PM UTC+2, Upender Gilakathula wrote:
Yes they are open.

22 (SSH) 0.0.0.0/0 Delete
80 (HTTP) 0.0.0.0/0 Delete
9200 - 9300 0.0.0.0/0 Delete
9300 - 9400 0.0.0.0/0 Delete
9300 10.0.0.0/8 Delete

On Tuesday, June 12, 2012 7:44:07 PM UTC+2, Eric Jain wrote:
On Jun 12, 9:46 am, Upender Gilakathula upenderg...@gmail.com wrote:

But I get this exception after 1 minute
18:29:25.997 [New I/O client boss #1-0] WARN
org.elasticsearch.transport.netty - [Mutant X] Exception caught on netty
layer [[id: 0x129a41d2]]
java.net.SocketException: Network is unreachable
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.6.0_31]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
~[na:1.6.0_31]

Are the machines all in the same security group, and are ports 9300 -
9400 open?


(Upender Gilakathula) #7

I have tried to run a program from my local machine with this ip
starting 172.16.xxx.xxx
And also from a amazon instances. Yes when I am testing from amazon the ip
is 10.248.xx.xx

Either ways it didn't work.

On Wednesday, June 13, 2012 12:05:48 AM UTC+2, David Pilato wrote:

Are you making calls from a 10.xxx.xxx.xxx network ?
What is this last rule ?

They are in the same security group ! Many thanks for a quick reply !

On Tuesday, June 12, 2012 11:29:46 PM UTC+2, Upender Gilakathula wrote:

Yes they are open.

22 (SSH)0.0.0.0/0Delete80 (HTTP)0.0.0.0/0Delete9200 - 93000.0.0.0/0Delete9300

  • 94000.0.0.0/0Delete930010.0.0.0/8Delete
    On Tuesday, June 12, 2012 7:44:07 PM UTC+2, Eric Jain wrote:

On Jun 12, 9:46 am, Upender Gilakathula upenderg...@gmail.com wrote:

But I get this exception after 1 minute
18:29:25.997 [New I/O client boss #1-0] WARN
org.elasticsearch.transport.netty - [Mutant X] Exception caught on
netty
layer [[id: 0x129a41d2]]
java.net.SocketException: Network is unreachable
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
~[na:1.6.0_31]
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
~[na:1.6.0_31]

Are the machines all in the same security group, and are ports 9300 -
9400 open?


(Upender Gilakathula) #8

Yes ! I did install the cloud-aws plugin. I have followed this
page. http://www.elasticsearch.org/tutorials/2011/08/22/elasticsearch-on-ec2.html

but i am not using any S3 Gateway configuration. since I am using EBS
volumes.

On Wednesday, June 13, 2012 12:00:55 AM UTC+2, Eric Jain wrote:

On Jun 12, 2:30 pm, Upender Gilakathula upenderg...@gmail.com wrote:

They are in the same security group ! Many thanks for a quick reply !

Did you install the cloud-aws plugin?

https://github.com/elasticsearch/elasticsearch-cloud-aws


(Eric Jain) #9

On Jun 12, 9:46 am, Upender Gilakathula upenderg...@gmail.com wrote:

cluster.name: broadwayrealtime
[...]
NodeBuilder.nodeBuilder().settings(settings).clusterName("clusterName").client(true).node();

In the code you're showing, you're not using the same cluster name?


(Upender Gilakathula) #10

In my actual code I am using the clustername as "broadwayrealtime"; i just
put as "clustername" when I am writing this post. sorry about that

On Wednesday, June 13, 2012 7:41:42 PM UTC+2, Eric Jain wrote:

On Jun 12, 9:46 am, Upender Gilakathula upenderg...@gmail.com wrote:

cluster.name: broadwayrealtime
[...]

NodeBuilder.nodeBuilder().settings(settings).clusterName("clusterName").client(true).node();

In the code you're showing, you're not using the same cluster name?


(Eric Jain) #11

On Jun 13, 11:17 pm, Upender Gilakathula upenderg...@gmail.com
wrote:

In my actual code I am using the clustername as "broadwayrealtime"; i just
put as "clustername" when I am writing this post. sorry about that

You're using the latest version of elasticsearch and the cloud plugin,
right?

Is the cloud-aws plugin (incl dependencies) in the classpath of the
client application?

I don't set "discovery.ec2.groups", "node.master" or "node.name".
Also, you set "cloud.aws.region" twice, but that shouldn't matter.


(system) #12