Elasticsearch java client on EC2


(Maxime Nay) #1

Hi,

For some reason I can't get the java client working on EC2.

My elasticsearch cluster and the cluster running my java application are in
two different security groups ('elasticsearchrealtime' and 'storm'), in the
same aws region.
I am using elasticsearch version 0.90.2 and using the cloud aws plugin
version 1.12.0. (in my es cluster and in my java application)

I tried lot of combinations for the properties, and none of them worked.
I always get the following warning :

[Elastic Search Realtime Client] waited for 30s and no initial state was
set by the discovery

And after that, each time the java application is trying to interact with
the client the following exception is thrown :
org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [30s]
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(TransportMasterNodeOperationAction.java:169)
at
org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:377)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)

The elasticsearchrealtime security group authorizes storm security group on
every tcp,udp and icmp ports.
The storm security group authorizes elasticsearchrealtime security group on
every tcp,udp and icmp ports.

Here are my current properties for the client :

################################### Cluster
###################################

cluster.name: elasticsearchrealtime

#################################### Index
####################################

index.number_of_shards: 16
index.number_of_replicas: 1

################################### Memory
####################################

bootstrap.mlockall: true

################################## Discovery
##################################

discovery.zen.ping.timeout: 5m
discovery.type=ec2
discovery.ec2.groups: elasticsearchrealtime,storm
discovery.ec2.ping_timeout: 60s
discovery.ec2.any_group: true

################################# AWS Settings
###################################

cloud.aws.region: us-east-1
cloud.aws.access_key: xxxx
cloud.aws.secret_key: xxxx
cloud.aws.s3.endpoint: s3.amazonaws.com
cloud.aws.ec2.endpoint: ec2.us-east-1.amazonaws.com
gateway.s3.region: us-east-1

################################### Client
###################################

node.name: Elastic Search Realtime Client
node.client: true
node.data: false

And here is the elasticsearch.yml file for one of my elasticsearch node :

################################### Cluster
###################################

cluster.name: elasticsearchrealtime

#################################### Index
####################################

index.number_of_shards: 16
index.number_of_replicas: 1

################################### Memory
####################################

bootstrap.mlockall: true

################################## Discovery
##################################

discovery.zen.ping.timeout: 5m
discovery.type: ec2
discovery.ec2.groups: elasticsearchrealtime,storm
discovery.ec2.ping_timeout: 60s
discovery.ec2.any_group: true

####################### AWS Settings
cloud.aws.region: us-east-1
cloud.aws.access_key: xxxxx
cloud.aws.secret_key: xxxxxx
cloud.aws.s3.endpoint: s3.amazonaws.com
cloud.aws.ec2.endpoint: ec2.us-east-1.amazonaws.com
gateway.s3.region: us-east-1

node.name: Elastic Search Realtime 1

Any help would be greatly appreciated !

Maxime

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/58880718-a1cd-4a8f-8bbd-84dbc71d4805%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jörg Prante) #2

Is this a typo?

discovery.type=ec2

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoF8mo64zwmiyYwQ6FQQPLX9iuPT8-G2O50-mSdnAUFVnQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Maxime Nay) #3

Nope, I just made a typo by translating my elasticsearch properties file
here. :slight_smile:
I use spring to inject all my properties into my Client. All the properties
for the client are actually like this :

node.name=Elastic Search Realtime Client
node.client=true
node.data=false
cluster.name=elasticsearchrealtime

#################################### Index
####################################

index.number_of_shards=16
index.number_of_replicas=1

################################### Memory
####################################

ElasticSearch performs poorly when JVM starts swapping=you should ensure

that

it never swaps.

Set this property to true to lock the memory:

bootstrap.mlockall=true

################################## Discovery
##################################

discovery.type=ec2
discovery.ec2.groups=elasticsearchrealtime,storm
discovery.ec2.ping_timeout=60s
discovery.ec2.any_group=true

####################### AWS Settings
cloud.aws.region=us-east-1
cloud.aws.access_key=xxxx
cloud.aws.secret_key=xxxx

Thanks though!

On Tuesday, January 28, 2014 5:38:01 PM UTC-8, Jörg Prante wrote:

Is this a typo?

discovery.type=ec2

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f3c995a0-52c7-4984-b80f-dc0143a4abdc%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #4