Hey guys,
We're running some load tests, and we're finding some of our clients are
having issues joining the cluster. We have the following setup running in
EC2.
24 c3.4xlarge ES instances. These instances are our data storage and
search instances.
40 c3.xlarge Tomcat instances. These are our ephemeral webapp instances.
In our tomcat instance, we're using the Node java client. What I'm finding
is that as the cluster comes up, only about 36~38 of the tomcat nodes ever
discover the 24 nodes that are actually storing data. We're using tcp
discovery, and have the following code in our client.
Settings settings = ImmutableSettings.settingsBuilder()
.put( "cluster.name", fig.getClusterName() )
// this assumes that we're using zen for host
discovery. Putting an
// explicit set of bootstrap hosts ensures we
connect to a valid cluster.
.put( "discovery.zen.ping.unicast.hosts", allHosts )
.put( "discovery.zen.ping.multicast.enabled", "false"
).put( "http.enabled", false )
.put( "client.transport.ping_timeout", 2000 ) //
milliseconds
.put( "client.transport.nodes_sampler_interval", 100 )
.put( "network.tcp.blocking", true )
.put( "node.client", true )
.put( "node.name", nodeName )
.build();
log.debug( "Creating ElasticSearch client with settings: " +
settings.getAsMap() );
where allHosts = "ip1:port, ip2:port" etc etc.
Not sure what we're doing wrong. We have retry code so that if a client
fails 20 consecutive times with a TransportException, the node instance is
stopped, and a new instance is created. This has worked in dealing with
network hiccups in the past, however we can just never seem to get past the
38 client mark. Any ideas what we may be doing wrong?
Thanks,
Todd
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/55924250-3013-4f70-b3d2-b5381f8af993%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.