ElasticSearch, huge loadtest


(Алексей Крылов) #1

Hi,

Last week I found great article about elasticsearch as a database and this
week I'm trying to test this case.

Currently I have only one server for Elastic, but this is doesn't matter for
the first-time look.

My test case is an elastic Java-client which is continuously loads the data.
Data is a simple entity, which is serialized to JSON with Gson Java-library,
something like this:

class MsgLog{

private UUID id;

private String text;

private int templateID;

private int templateID

}

The client just inserts the data in infinite loop (no bulks, just usual
client.index(IndexRequest)).

After some irregular periods of this inserts-load I have this exception:

2012-05-22 20:16:51,940 [Timer-0] INFO r.b.elasticsearch.ElasticGenerator -
inserts: [68] ops/sec, avg: [1364] ops/sec

logs count: [21298659], free: [7]

2012-05-22 20:16:52,242 [elasticsearch[generic]-pool-3-thread-1] INFO
org.elasticsearch.client.transport - [Sprite] fa

led to get node info for [#transport#-1][inet[/xxxxxxx:9300]],
disconnecting...

org.elasticsearch.transport.ReceiveTimeoutTransportException:
[][inet[/xxxxxxxxx:9300]][cluster/nodes/info] request

id [21299220] timed out after [5001ms]

    at

org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportSer
vice.java:347) ~[elasticsearch-0

19.4.jar:na]

    at

java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.ja
va:886) [na:1.6.0_25]

    at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:9
08) [na:1.6.0_25]

    at java.lang.Thread.run(Thread.java:662) [na:1.6.0_25]

2012-05-22 20:16:52,250 [elasticsearch[generic]-pool-3-thread-1] ERROR
r.b.elasticsearch.ElasticGenerator - Failed to e

ecute index

org.elasticsearch.client.transport.NoNodeAvailableException: No node
available

    at

org.elasticsearch.client.transport.TransportClientNodesService$RetryListener
.onFailure(TransportClientNodesS

rvice.java:249) [elasticsearch-0.19.4.jar:na]

    at

org.elasticsearch.action.TransportActionNodeProxy$1.handleException(Transpor
tActionNodeProxy.java:84) [elast

csearch-0.19.4.jar:na]

    at

org.elasticsearch.transport.TransportService$Adapter$2$1.run(TransportServic
e.java:310) [elasticsearch-0.19.

.jar:na]

    at

java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.ja
va:886) [na:1.6.0_25]

    at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:9
08) [na:1.6.0_25]

    at java.lang.Thread.run(Thread.java:662) [na:1.6.0_25]

2012-05-22 20:16:52,252 [elasticsearch[generic]-pool-3-thread-28] ERROR
r.b.elasticsearch.ElasticGenerator - Failed to

And after this kind of errors I need to restart my load-client.

There is no error logs on server side.

Can anybody explain this?

Thanks!

Best regards,

Alexey Krylov


(Shay Banon) #2

Are you using the TransportClient? It means that it sent a ping to the node
its connected to, and did not get a response. When you execute an index
operation, do you wait for it? (using actionGet)? If not, you can easily
overload the node with concurrent indexing requests.

On Tue, May 22, 2012 at 10:00 PM, Алексей Крылов
aleksey.krylov@gmail.comwrote:

Hi,****


Last week I found great article about elasticsearch as a database and this
week I’m trying to test this case.****


Currently I have only one server for Elastic, but this is doesn’t matter
for the first-time look.****

My test case is an elastic Java-client which is continuously loads the
data. Data is a simple entity, which is serialized to JSON with Gson
Java-library, something like this:****


class MsgLog{****

private UUID id;****

private String text;****

private int templateID;****

private int templateID****

}****


The client just inserts the data in infinite loop (no bulks, just usual
client.index(IndexRequest)).****

After some irregular periods of this inserts-load I have this exception:**
**


2012-05-22 20:16:51,940 [Timer-0] INFO r.b.elasticsearch.ElasticGenerator

  • inserts: [68] ops/sec, avg: [1364] ops/sec****

logs count: [21298659], free: [7]****

2012-05-22 20:16:52,242 [elasticsearch[generic]-pool-3-thread-1] INFO
org.elasticsearch.client.transport - [Sprite] fa****

led to get node info for [#transport#-1][inet[/xxxxxxx:9300]],
disconnecting...****

org.elasticsearch.transport.ReceiveTimeoutTransportException:
[][inet[/xxxxxxxxx:9300]][cluster/nodes/info] request****

id [21299220] timed out after [5001ms]****

    at

org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:347)
~[elasticsearch-0****

19.4.jar:na]****

    at

java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
[na:1.6.0_25]****

    at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
[na:1.6.0_25]****

    at java.lang.Thread.run(Thread.java:662) [na:1.6.0_25]****

2012-05-22 20:16:52,250 [elasticsearch[generic]-pool-3-thread-1] ERROR
r.b.elasticsearch.ElasticGenerator - Failed to e****

ecute index****

org.elasticsearch.client.transport.NoNodeAvailableException: No node
available****

    at

org.elasticsearch.client.transport.TransportClientNodesService$RetryListener.onFailure(TransportClientNodesS


rvice.java:249) [elasticsearch-0.19.4.jar:na]****

    at

org.elasticsearch.action.TransportActionNodeProxy$1.handleException(TransportActionNodeProxy.java:84)
[elast****

csearch-0.19.4.jar:na]****

    at

org.elasticsearch.transport.TransportService$Adapter$2$1.run(TransportService.java:310)
[elasticsearch-0.19.****

.jar:na]****

    at

java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
[na:1.6.0_25]****

    at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
[na:1.6.0_25]****

    at java.lang.Thread.run(Thread.java:662) [na:1.6.0_25]****

2012-05-22 20:16:52,252 [elasticsearch[generic]-pool-3-thread-28] ERROR
r.b.elasticsearch.ElasticGenerator - Failed to ****


And after this kind of errors I need to restart my load-client.****

There is no error logs on server side.****


Can anybody explain this?****

Thanks!****


Best regards,****

Alexey Krylov****



(system) #3