Load testing on elasticsearch cluster

Elastic Stack Elasticsearch
1

Hi,

Could anyone tell me how can i do a load testing on my elasticsearch
cluster.?
I want to test how many concurrent user requests it can handle.

Regards
Bharvi Dixit

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b8e3b24f-caa6-4772-9914-675fd2bc2f72%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Hi,
may be jMeter?
Regards,
Lukáš
Dne 1.12.2013 9:16 "Bharvi Dixit" bharvidixit@gmail.com napsal(a):

Hi,

Could anyone tell me how can i do a load testing on my elasticsearch
cluster.?
I want to test how many concurrent user requests it can handle.

Regards
Bharvi Dixit

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b8e3b24f-caa6-4772-9914-675fd2bc2f72%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAO9cvUZ_Koi9c_ZMLzVe0AnQm2S%2B%3DMh9UqRVAoZ2pRC2mysBEA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

I just completed some ES performance testing. Here's what I did:

  1. Get some test queries. I looked at some production traffic, extracted
    some common query types, and inserted pseudo-random data in place of
    variables (e.g. user search terms).

  2. Instrumented the various components of my infrastructure (NewRelic and
    SPM http://sematext.com/spm/index.html for Elasticsearch, NewRelic for my
    application and the load generators).

  3. Used Siege to hammer the cluster with requests. I experimented with
    various parameters, including concurrency, ramp-up, etc., in order to get
    the right load profile.

  4. Evaluate the results from the instrumentation. Identify the components
    which are the bottleneck.

Partway through the test, I switched from Siege to JMeter, which has much
better reporting.

The most important part of this was probably getting a good set of
representative data. I didn't have actual production ES queries to replay,
so I had to convert other queries to the ES format.

My cluster of ES nodes got to about 1700 QPS, and ES never broke a sweat --
the bottleneck was in my application. I'm confident that ES could handle
2-3 times more traffic than that before it started to work hard.

Ross

On Sunday, 1 December 2013 19:16:15 UTC+11, Bharvi Dixit wrote:

Hi,

Could anyone tell me how can i do a load testing on my elasticsearch
cluster.?
I want to test how many concurrent user requests it can handle.

Regards
Bharvi Dixit

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/43cb5e62-3472-4ef2-812c-f3dba9fa4531%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Thanks for the reply and sharing your experience, I will try the same.

Regards,
Bhavi Dixit

On Monday, 2 December 2013 08:13:16 UTC+5:30, Ross Simpson wrote:

I just completed some ES performance testing. Here's what I did:

  1. Get some test queries. I looked at some production traffic, extracted
    some common query types, and inserted pseudo-random data in place of
    variables (e.g. user search terms).

  2. Instrumented the various components of my infrastructure (NewRelic and
    SPM http://sematext.com/spm/index.html for Elasticsearch, NewRelic for
    my application and the load generators).

  3. Used Siege to hammer the cluster with requests. I experimented with
    various parameters, including concurrency, ramp-up, etc., in order to get
    the right load profile.

  4. Evaluate the results from the instrumentation. Identify the components
    which are the bottleneck.

Partway through the test, I switched from Siege to JMeter, which has much
better reporting.

The most important part of this was probably getting a good set of
representative data. I didn't have actual production ES queries to replay,
so I had to convert other queries to the ES format.

My cluster of ES nodes got to about 1700 QPS, and ES never broke a sweat
-- the bottleneck was in my application. I'm confident that ES could
handle 2-3 times more traffic than that before it started to work hard.

Ross

On Sunday, 1 December 2013 19:16:15 UTC+11, Bharvi Dixit wrote:

Hi,

Could anyone tell me how can i do a load testing on my elasticsearch
cluster.?
I want to test how many concurrent user requests it can handle.

Regards
Bharvi Dixit

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/aa1660c1-d321-44ac-a8af-c58eb8385593%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

Thanks for the reply and sharing your experience, I will try the same.

Regards,
Bharvi Dixit

On Monday, 2 December 2013 08:13:16 UTC+5:30, Ross Simpson wrote:

I just completed some ES performance testing. Here's what I did:

  1. Get some test queries. I looked at some production traffic, extracted
    some common query types, and inserted pseudo-random data in place of
    variables (e.g. user search terms).

  2. Instrumented the various components of my infrastructure (NewRelic and
    SPM http://sematext.com/spm/index.html for Elasticsearch, NewRelic for
    my application and the load generators).

  3. Used Siege to hammer the cluster with requests. I experimented with
    various parameters, including concurrency, ramp-up, etc., in order to get
    the right load profile.

  4. Evaluate the results from the instrumentation. Identify the components
    which are the bottleneck.

Partway through the test, I switched from Siege to JMeter, which has much
better reporting.

The most important part of this was probably getting a good set of
representative data. I didn't have actual production ES queries to replay,
so I had to convert other queries to the ES format.

My cluster of ES nodes got to about 1700 QPS, and ES never broke a sweat
-- the bottleneck was in my application. I'm confident that ES could
handle 2-3 times more traffic than that before it started to work hard.

Ross

On Sunday, 1 December 2013 19:16:15 UTC+11, Bharvi Dixit wrote:

Hi,

Could anyone tell me how can i do a load testing on my elasticsearch
cluster.?
I want to test how many concurrent user requests it can handle.

Regards
Bharvi Dixit

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5761c9d9-b5cb-4401-bd65-b76f173a5ad6%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.