ES Scalability Issues

Hi,

I'm running tests for ES scalability and getting some strange results. I've
a two node cluster running on separate machines, with 8GB assigned to each
node and each writing to there respective local disk. Replication is set to
0 and number of shards to 4.

When I ran with only a one node cluster, I got a throughput of around 875
records per second. Then I deleted the index and added another node. This
time the throughput dropped to 784 records per second.

I'm using BulkRequestBuilder java api to persist two documents at a time
with the second being the child of the first.

Thanks,
Anand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hello Anand,

On Tue, Jul 2, 2013 at 12:39 PM, anand nalya anand.nalya@gmail.com wrote:

Hi,

I'm running tests for ES scalability and getting some strange results.
I've a two node cluster running on separate machines, with 8GB assigned to
each node and each writing to there respective local disk. Replication is
set to 0 and number of shards to 4.

When I ran with only a one node cluster, I got a throughput of around 875
records per second. Then I deleted the index and added another node. This
time the throughput dropped to 784 records per second.

I'm using BulkRequestBuilder java api to persist two documents at a time
with the second being the child of the first.

Does this mean you're only sending two documents in each bulk? If the
answer is yes, then maybe the speed drop is just because of the network
overhead (which can get a bit higher when you add a second node because
they have to talk to each other).

If you get the same problem with bigger bulks, then it's strange. I would
monitor the cluster with something like
SPMhttp://sematext.com/spm/elasticsearch-performance-monitoring/to
see where the bottleneck is.

Best regards,
Radu

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.