Java client unable to connect - ClusterBlockException

pnayak · June 28, 2012, 12:57am

I'd appreciate any help/insight into this problem... Thanks in advance for
your time and help!

My ElasticSearch setup uses a single shard with no replica. I am running
on a Rackspace cloud server. The works fine on my local Mac

index.number_of_shards: 1
index.number_of_replicas: 0

No other changes in the config file.

My cluster state is green after starting es:

curl -XGET http://localhost:9200/_cluster/health
*
{"cluster_name":"elasticsearch","status":"green","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":0,"active_shards":0,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}
*

When I start my app server, which uses the following to get a client...

Node node = nodeBuilder().client(true).node();
client = node.client();

I get the following exception when a search is sought to be executed:

ERROR [2012-06-28 00:37:34,229]
com.yammer.dropwizard.jersey.LoggingExceptionMapper: Error handling a
request: 8f94de3bc1f8f645! org.elasticsearch.cluster.block.ClusterBlockException:
blocked by: [SERVICE_UNAVAILABLE/1/state not recovered /
initialized];[SERVICE_UNAVAILABLE/2/no master];! at
org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException
(ClusterBlocks.java:136)
! at
org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedRaiseException(ClusterBlocks.java:126)
! at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.(TransportSearchTypeAction.java:108)
! at
org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.(TransportSearchDfsQueryThenFetchAction.java:76)
! at
org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.(TransportSearchDfsQueryThenFetchAction.java:65)
! at
org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction.doExecute(TransportSearchDfsQueryThenFetchAction.java:62)
! at
org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction.doExecute(TransportSearchDfsQueryThenFetchAction.java:52)
! at
org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:61)!
at
org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:106)
! at
org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:43)
! at
org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:61)
! at org.elasticsearch.client.node.NodeClient.execute(NodeClient.java:83)
! at
org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:206)

Igor_Motov · June 28, 2012, 1:44am

This exception means that your client cannot find your server. If you are
running elasticsearch and app server on different machines, they, probably,
cannot find each other because of blocked multicast traffic. If this is the
case, try switching to unicast discovery. If they are running on the same
machine, it's possible that firewall is interfering with discovery.
Multicast discovery is using group 224.2.2.4 and port 54328.
After initial multicast broadcast nodes are communicating on ports
9300-9400 (first node on machine is using 9300, next node is using 9301 and
so on.)

On Wednesday, June 27, 2012 8:57:55 PM UTC-4, Prashant wrote:

I'd appreciate any help/insight into this problem... Thanks in advance
for your time and help!

My Elasticsearch setup uses a single shard with no replica. I am running
on a Rackspace cloud server. The works fine on my local Mac

index.number_of_shards: 1
index.number_of_replicas: 0

No other changes in the config file.

My cluster state is green after starting es:

curl -XGET http://localhost:9200/_cluster/health
*
{"cluster_name":"elasticsearch","status":"green","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":0,"active_shards":0,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}
*

When I start my app server, which uses the following to get a client...

Node node = nodeBuilder().client(true).node();
client = node.client();

I get the following exception when a search is sought to be executed:

ERROR [2012-06-28 00:37:34,229]
com.yammer.dropwizard.jersey.LoggingExceptionMapper: Error handling a
request: 8f94de3bc1f8f645! org.elasticsearch.cluster.block.ClusterBlockException:
blocked by: [SERVICE_UNAVAILABLE/1/state not recovered /
initialized];[SERVICE_UNAVAILABLE/2/no master];! at
org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException
(ClusterBlocks.java:136)
! at
org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedRaiseException(ClusterBlocks.java:126)
! at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.(TransportSearchTypeAction.java:108)
! at
org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.(TransportSearchDfsQueryThenFetchAction.java:76)
! at
org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.(TransportSearchDfsQueryThenFetchAction.java:65)
! at
org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction.doExecute(TransportSearchDfsQueryThenFetchAction.java:62)
! at
org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction.doExecute(TransportSearchDfsQueryThenFetchAction.java:52)
! at
org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:61)!
at
org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:106)
! at
org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:43)
! at
org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:61)
! at org.elasticsearch.client.node.NodeClient.execute(NodeClient.java:83)
! at
org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:206)

pnayak · June 28, 2012, 1:04pm

Thanks - it looks like multicast fails on Rackspace - for the time being
I'm going to try unicast discovery...

pnayak · June 28, 2012, 6:10pm

Got it working. In case this helps anyone going forward, this is what I
did to get Unicast working:

Upgraded to ES 0.19.7 (was previously using 0.19.4)
Disabled multicast discovery on my ES server by
setting: discovery.zen.ping.multicast.enabled: false
Note that I also used a single shard with no replicas since all I
wanted for now was a single data node
I did not specify discovery.zen.ping.unicast.hosts

Created my Java client using:

       Settings settings = ImmutableSettings.settingsBuilder()

.put("http.enabled", "false")
.put("transport.tcp.port", "9300-9400")
.put("discovery.zen.ping.multicast.enabled", "false")
.put("discovery.zen.ping.unicast.hosts", "localhost").build();

Node node =
nodeBuilder().client(true).settings(settings).clusterName("elasticsearch").node();
client = node.client();