Performance difference between REST and Java API

Elastic Stack Elasticsearch
1

I'm benchmarking results for an ES cluster, using both the REST api and
native Java client. We're getting very different response times, between
each of these (the REST api is doing approximately 50% better in the 95th
percentile). I was wondering what the cause of this is, since the search
requests look identical to me (unless I missed something).

Here are the queries:

Our setup: 20 node cluster, 20 shards/2 replicas, and benchmarking from a
separate machine with JMeter (an HTTP sampler for each of the 20 nodes, and
a separate test plan with a Java sampler initializing the ES java client,
and sending queries).

Any help appreciated.

-M

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a5915f2f-ed31-4abb-af68-77c67a586aa9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

How about this?

Jörg

On Thu, Dec 18, 2014 at 7:48 PM, Marie Jacob jacob.marie@gmail.com wrote:

I'm benchmarking results for an ES cluster, using both the REST api and
native Java client. We're getting very different response times, between
each of these (the REST api is doing approximately 50% better in the 95th
percentile). I was wondering what the cause of this is, since the search
requests look identical to me (unless I missed something).

Here are the queries:

gist:509b3db873a30d8961ed · GitHub

Our setup: 20 node cluster, 20 shards/2 replicas, and benchmarking from a
separate machine with JMeter (an HTTP sampler for each of the 20 nodes, and
a separate test plan with a Java sampler initializing the ES java client,
and sending queries).

Any help appreciated.

-M

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a5915f2f-ed31-4abb-af68-77c67a586aa9%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/a5915f2f-ed31-4abb-af68-77c67a586aa9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFm1Eo-h6y%3D2NjvZtY7yGa3YRBcLZG%3DjAxAS9oxCKZeyw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Thanks Jörg.

I tried that, and it seems to make no difference. All I could gather are
these graphs (attached) from the bigdesk plugin that are showing
performance for the query and fetch phase (the first is during ES java
client use, the second is using the REST calls) . It's really weird that
the FETCH phase is taking any time at all, if I set _source = false.

On Thursday, December 18, 2014 5:33:05 PM UTC-5, Jörg Prante wrote:

How about this?

gist:509b3db873a30d8961ed · GitHub

Jörg

On Thu, Dec 18, 2014 at 7:48 PM, Marie Jacob <jacob...@gmail.com
<javascript:>> wrote:

I'm benchmarking results for an ES cluster, using both the REST api and
native Java client. We're getting very different response times, between
each of these (the REST api is doing approximately 50% better in the 95th
percentile). I was wondering what the cause of this is, since the search
requests look identical to me (unless I missed something).

Here are the queries:

gist:509b3db873a30d8961ed · GitHub

Our setup: 20 node cluster, 20 shards/2 replicas, and benchmarking from a
separate machine with JMeter (an HTTP sampler for each of the 20 nodes, and
a separate test plan with a Java sampler initializing the ES java client,
and sending queries).

Any help appreciated.

-M

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a5915f2f-ed31-4abb-af68-77c67a586aa9%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/a5915f2f-ed31-4abb-af68-77c67a586aa9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/060e3e34-1326-4d6d-a645-670924818d6c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

The idea is for tracing the issue to rebuild the query from simple to
complex and compare.

I would start with the query and add the filters step by step, to identify
the part which causes trouble.

On possible cause is that REST API is using some defaults which have to be
set in the Java API explicitly.

Jörg

On Fri, Dec 19, 2014 at 1:18 AM, Marie Jacob jacob.marie@gmail.com wrote:

Thanks Jörg.

I tried that, and it seems to make no difference. All I could gather are
these graphs (attached) from the bigdesk plugin that are showing
performance for the query and fetch phase (the first is during ES java
client use, the second is using the REST calls) . It's really weird that
the FETCH phase is taking any time at all, if I set _source = false.

On Thursday, December 18, 2014 5:33:05 PM UTC-5, Jörg Prante wrote:

How about this?

gist:509b3db873a30d8961ed · GitHub

Jörg

On Thu, Dec 18, 2014 at 7:48 PM, Marie Jacob jacob...@gmail.com wrote:

I'm benchmarking results for an ES cluster, using both the REST api and
native Java client. We're getting very different response times, between
each of these (the REST api is doing approximately 50% better in the 95th
percentile). I was wondering what the cause of this is, since the search
requests look identical to me (unless I missed something).

Here are the queries:

gist:509b3db873a30d8961ed · GitHub

Our setup: 20 node cluster, 20 shards/2 replicas, and benchmarking from
a separate machine with JMeter (an HTTP sampler for each of the 20 nodes,
and a separate test plan with a Java sampler initializing the ES java
client, and sending queries).

Any help appreciated.

-M

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/a5915f2f-ed31-4abb-af68-77c67a586aa9%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/a5915f2f-ed31-4abb-af68-77c67a586aa9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/060e3e34-1326-4d6d-a645-670924818d6c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/060e3e34-1326-4d6d-a645-670924818d6c%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHRJSL_wJoj-Q%2BypXC3Ej79fNPC-6j-BEJ-LMpPXU8yDQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.