Insertion Spikes

Hi all,

As part of our performance exercize, we have been trying characterize
the Insertion performance of ElasticSearch (0.90.7). Here is our setup:

Nodes: 3 AWS m1.xlarge (16G)
Memory: 8G Heap on each node.

Indices: 5 aliases, 3 indexes per alias, 2 shards per index. (30 shards),
1 replica.
*Total: *60 shards across 3 machines in the cluster.

Client Nodes: 3 AWS m1.large (8G)
No. of threads per client: 50
Memory: 1 G (heap)
Overall: 1.4 M documents
Avg Doc size: 4K (Most messages were 500 - 1500 bytes, there were a
bunch of message >8K)

The graph below shows insertion spikes at various points. We tried to
correlate them with the merge times, but that did not seem to hold true.
There also seems to be no direct correlation between message size and
insertion times.

https://lh3.googleusercontent.com/-CDJ90Ty1w1A/Ux4F5gfWT2I/AAAAAAAACOE/9U866TNtP4I/s1600/insertion_95th_percentile.png

Insertion times have been plotted at 95th percentile.
Below is a plot for the size of messages:

https://lh5.googleusercontent.com/-YGntu5KejBg/Ux4GJg1UohI/AAAAAAAACOM/T4ciaVajD04/s1600/insertion_data_size.png
My question is: Is there something we are doing wrong in here or is there a
way to explain why the spikes occur for insertion times?

Any help will be appreciated.

Regards
Parag

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/665fe952-964b-4cf4-8ef8-fab95a529cc1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.