[Stress Test] JavaAPI Search Cost too much time with small index

Elastic Stack Elasticsearch
1

Hi,

Currently I am load test my search server, Bellow is detail:

  1. I have 3 server running ES, each of them have 30Gb Ram & 24 Core. I set
    20Gb for ES.
  2. I have 3 webserver (Jetty) to receive request from client, then search
    in ES using java api, and then return json value.
  3. I have a index which have small data, (about 50000 record, 345.7mb)

To load test, I make a request to jetty server. (eg: /RandomSearch)

In this doGet function, I just random one character value from a-z to
search and then return value to client.

My problem is, it's running too low:

Here is stats:

Start

Current

Num Indexed

Total Time(ms)

Rate (ms)

Num Request(s)

Num Searched

Total Time(ms)

Rate (ms)

Num Request(s)

18.10.12 20:59:48 538

18.10.12 21:20:27 227

0

0

0.0

0

959626

526643575

548.801

826

But when I using head plugin ( myserver:9200/_plugin/head/) to search, I
notice that the result is really fast

(eg: Searched 5 of 5 shards. 7588 hits. 0.046 seconds)

I wonder whether I am wrong at java api request or java api is slower than
http request ?

The other problem is the json result is really slow. It take about 6-10
(second) to return to client. How could I improve that ?

Thanks & Best Regards All.

--

2

Can you share your load test scripts/programs ? Could be a nice way for
other users to test their hardware or settings.

David.

De : elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com]
De la part de kidkid
Envoyé : jeudi 18 octobre 2012 16:30
À : elasticsearch@googlegroups.com
Objet : [Stress Test] JavaAPI Search Cost too much time with small index.

Hi,

Currently I am load test my search server, Bellow is detail:

  1. I have 3 server running ES, each of them have 30Gb Ram & 24 Core. I set
    20Gb for ES.

  2. I have 3 webserver (Jetty) to receive request from client, then search in
    ES using java api, and then return json value.

  3. I have a index which have small data, (about 50000 record, 345.7mb)

To load test, I make a request to jetty server. (eg: /RandomSearch)

In this doGet function, I just random one character value from a-z to search
and then return value to client.

My problem is, it's running too low:

Here is stats:

Start

Current

Num Indexed

Total Time(ms)

Rate (ms)

Num Request(s)

Num Searched

Total Time(ms)

Rate (ms)

Num Request(s)

18.10.12 20:59:48 538

18.10.12 21:20:27 227

0

0

0.0

0

959626

526643575

548.801

826

But when I using head plugin ( myserver:9200/_plugin/head/) to search, I
notice that the result is really fast

(eg: Searched 5 of 5 shards. 7588 hits. 0.046 seconds)

I wonder whether I am wrong at java api request or java api is slower than
http request ?

The other problem is the json result is really slow. It take about 6-10
(second) to return to client. How could I improve that ?

Thanks & Best Regards All.

--

--

3

Hi,

Current, we have a site with is running with lot of user.
Then we make a script on this site so that every user will call to search
for testing.

In case you don't have a similar site, you could use Apache JMeter or
CometD to load test your site.

Currently, I'm looking a way to bring all data to memory. I still research
LRU of ES to improve searching performance.
Wait for your help all.

Thanks.

On Friday, October 19, 2012 12:41:59 AM UTC+7, David Pilato wrote:

Can you share your load test scripts/programs ? Could be a nice way for
other users to test their hardware or settings.

David.

De : elasti...@googlegroups.com <javascript:> [mailto:
elasti...@googlegroups.com <javascript:>] De la part de kidkid
Envoyé : jeudi 18 octobre 2012 16:30
À : elasti...@googlegroups.com <javascript:>
Objet : [Stress Test] JavaAPI Search Cost too much time with small
index.

Hi,

Currently I am load test my search server, Bellow is detail:

  1. I have 3 server running ES, each of them have 30Gb Ram & 24 Core. I set
    20Gb for ES.

  2. I have 3 webserver (Jetty) to receive request from client, then search
    in ES using java api, and then return json value.

  3. I have a index which have small data, (about 50000 record, 345.7mb)

To load test, I make a request to jetty server. (eg: /RandomSearch)

In this doGet function, I just random one character value from a-z to
search and then return value to client.

My problem is, it's running too low:

Here is stats:

Start

Current

Num Indexed

Total Time(ms)

Rate (ms)

Num Request(s)

Num Searched

Total Time(ms)

Rate (ms)

Num Request(s)

18.10.12 20:59:48 538

18.10.12 21:20:27 227

0

0

0.0

0

959626

526643575

548.801

826

But when I using head plugin ( myserver:9200/_plugin/head/) to search, I
notice that the result is really fast

(eg: Searched 5 of 5 shards. 7588 hits. 0.046 seconds)

I wonder whether I am wrong at java api request or java api is slower than
http request ?

The other problem is the json result is really slow. It take about 6-10
(second) to return to client. How could I improve that ?

Thanks & Best Regards All.

--

--

4

I would guess that the problem is somewhere in your app, then in elasticsearch. Its easy to test, use the total time returned from a search result and see if the problem is in the search executed by elasticsearch, or somewhere "above" it.

On Oct 19, 2012, at 11:21 AM, kidkid zkidkid@gmail.com wrote:

Hi,

Current, we have a site with is running with lot of user.
Then we make a script on this site so that every user will call to search for testing.

In case you don't have a similar site, you could use Apache JMeter or CometD to load test your site.

Currently, I'm looking a way to bring all data to memory. I still research LRU of ES to improve searching performance.
Wait for your help all.

Thanks.

On Friday, October 19, 2012 12:41:59 AM UTC+7, David Pilato wrote:
Can you share your load test scripts/programs ? Could be a nice way for other users to test their hardware or settings.

David.

De : elasti...@googlegroups.com [mailto:elasti...@googlegroups.com] De la part de kidkid
Envoyé : jeudi 18 octobre 2012 16:30
À : elasti...@googlegroups.com
Objet : [Stress Test] JavaAPI Search Cost too much time with small index.

Hi,

Currently I am load test my search server, Bellow is detail:

  1. I have 3 server running ES, each of them have 30Gb Ram & 24 Core. I set 20Gb for ES.

  2. I have 3 webserver (Jetty) to receive request from client, then search in ES using java api, and then return json value.

  3. I have a index which have small data, (about 50000 record, 345.7mb)

To load test, I make a request to jetty server. (eg: /RandomSearch)

In this doGet function, I just random one character value from a-z to search and then return value to client.

My problem is, it's running too low:

Here is stats:

Start
Current
Num Indexed
Total Time(ms)
Rate (ms)
Num Request(s)
Num Searched
Total Time(ms)
Rate (ms)
Num Request(s)
18.10.12 20:59:48 538
18.10.12 21:20:27 227
0
0
0.0
0
959626
526643575
548.801
826

But when I using head plugin ( myserver:9200/_plugin/head/) to search, I notice that the result is really fast

(eg: Searched 5 of 5 shards. 7588 hits. 0.046 seconds)

I wonder whether I am wrong at java api request or java api is slower than http request ?

The other problem is the json result is really slow. It take about 6-10 (second) to return to client. How could I improve that ?

Thanks & Best Regards All.

--

--

--