Elasticsearch 1.1.0 Java API Slower Than Curl for Certain Queries


#1

I'm joining an existing cluster as a node via the Java API. I generate a
relatively complicated multi-index "indices" query that combines the
results of two different queries on two modestly sized indexes (150M
records combined). Execution of actionGet() on this query takes
approximately 1-2 seconds. On the other hand, if I dump the same query to
JSON via the XContent jsonBuilder() and execute it with curl, it takes only
14 ms. Results are identical in both cases.

Has anyone else had similar problems like this? Any thoughts on what might
be the source of the problem? My next thought was to try connecting to the
cluster via a TransportClient. Not sure if it will help or not...

Thanks!

  • Elliott

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fa4fb57e-3beb-4e80-8672-1981bcb8c35c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(vineeth mohan-2) #2

Hello Elliot ,

Is the sniffing enabled while creating the client.
This means that , it will determine which all machines are enabled and does
some load balancing.
Which means that it needs additional time while creating the client.

Thanks
Vineeth

On Wed, Aug 20, 2014 at 9:39 PM, Elliott Bradshaw ebradshaw1@gmail.com
wrote:

I'm joining an existing cluster as a node via the Java API. I generate a
relatively complicated multi-index "indices" query that combines the
results of two different queries on two modestly sized indexes (150M
records combined). Execution of actionGet() on this query takes
approximately 1-2 seconds. On the other hand, if I dump the same query to
JSON via the XContent jsonBuilder() and execute it with curl, it takes only
14 ms. Results are identical in both cases.

Has anyone else had similar problems like this? Any thoughts on what
might be the source of the problem? My next thought was to try connecting
to the cluster via a TransportClient. Not sure if it will help or not...

Thanks!

  • Elliott

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/fa4fb57e-3beb-4e80-8672-1981bcb8c35c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/fa4fb57e-3beb-4e80-8672-1981bcb8c35c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGdPd5m-8Tn_VEGwcpUhr29r7RyL36%3DimKCEsVhvF5U3GOg5mQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


#3

Vineeth,

Thanks, but according to the documentation, it looks like sniffing is only
supported when using the TransportClient. I'm using the standard
Node/Client configuration.

On Wednesday, August 20, 2014 12:24:45 PM UTC-4, vineeth mohan wrote:

Hello Elliot ,

Is the sniffing enabled while creating the client.
This means that , it will determine which all machines are enabled and
does some load balancing.
Which means that it needs additional time while creating the client.

Thanks
Vineeth

On Wed, Aug 20, 2014 at 9:39 PM, Elliott Bradshaw <ebrad...@gmail.com
<javascript:>> wrote:

I'm joining an existing cluster as a node via the Java API. I generate a
relatively complicated multi-index "indices" query that combines the
results of two different queries on two modestly sized indexes (150M
records combined). Execution of actionGet() on this query takes
approximately 1-2 seconds. On the other hand, if I dump the same query to
JSON via the XContent jsonBuilder() and execute it with curl, it takes only
14 ms. Results are identical in both cases.

Has anyone else had similar problems like this? Any thoughts on what
might be the source of the problem? My next thought was to try connecting
to the cluster via a TransportClient. Not sure if it will help or not...

Thanks!

  • Elliott

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/fa4fb57e-3beb-4e80-8672-1981bcb8c35c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/fa4fb57e-3beb-4e80-8672-1981bcb8c35c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/bfae9628-0212-4b03-8f76-4fb85aab23e9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(vineeth mohan-2) #4

Hello Eliott ,

I suspect if this has something to do with client creation.
Can you see the time taken for the query alone and not the client creation.

Other possible reasons can be refresh flag ON.
Please check that also.

Thanks
Vineeth

On Wed, Aug 20, 2014 at 10:11 PM, Elliott Bradshaw ebradshaw1@gmail.com
wrote:

Vineeth,

Thanks, but according to the documentation, it looks like sniffing is only
supported when using the TransportClient. I'm using the standard
Node/Client configuration.

On Wednesday, August 20, 2014 12:24:45 PM UTC-4, vineeth mohan wrote:

Hello Elliot ,

Is the sniffing enabled while creating the client.
This means that , it will determine which all machines are enabled and
does some load balancing.
Which means that it needs additional time while creating the client.

Thanks
Vineeth

On Wed, Aug 20, 2014 at 9:39 PM, Elliott Bradshaw ebrad...@gmail.com
wrote:

I'm joining an existing cluster as a node via the Java API. I generate
a relatively complicated multi-index "indices" query that combines the
results of two different queries on two modestly sized indexes (150M
records combined). Execution of actionGet() on this query takes
approximately 1-2 seconds. On the other hand, if I dump the same query to
JSON via the XContent jsonBuilder() and execute it with curl, it takes only
14 ms. Results are identical in both cases.

Has anyone else had similar problems like this? Any thoughts on what
might be the source of the problem? My next thought was to try connecting
to the cluster via a TransportClient. Not sure if it will help or not...

Thanks!

  • Elliott

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/fa4fb57e-3beb-4e80-8672-1981bcb8c35c%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/fa4fb57e-3beb-4e80-8672-1981bcb8c35c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/bfae9628-0212-4b03-8f76-4fb85aab23e9%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/bfae9628-0212-4b03-8f76-4fb85aab23e9%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGdPd5m7B_iaHJwudFcFEbwcVab3hjEo5KWWg_3hLwmaDhuKDA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


#5

Hi Vineeth,

The system is currently deployed on a Tomcat server and the client is
created a single time on application start up. The time I gave of 1-2
seconds is for the query alone, as far as I know.

Thanks,

Elliott

On Wednesday, August 20, 2014 12:46:34 PM UTC-4, vineeth mohan wrote:

Hello Eliott ,

I suspect if this has something to do with client creation.
Can you see the time taken for the query alone and not the client creation.

Other possible reasons can be refresh flag ON.
Please check that also.

Thanks
Vineeth

On Wed, Aug 20, 2014 at 10:11 PM, Elliott Bradshaw <ebrad...@gmail.com
<javascript:>> wrote:

Vineeth,

Thanks, but according to the documentation, it looks like sniffing is
only supported when using the TransportClient. I'm using the standard
Node/Client configuration.

On Wednesday, August 20, 2014 12:24:45 PM UTC-4, vineeth mohan wrote:

Hello Elliot ,

Is the sniffing enabled while creating the client.
This means that , it will determine which all machines are enabled and
does some load balancing.
Which means that it needs additional time while creating the client.

Thanks
Vineeth

On Wed, Aug 20, 2014 at 9:39 PM, Elliott Bradshaw ebrad...@gmail.com
wrote:

I'm joining an existing cluster as a node via the Java API. I generate
a relatively complicated multi-index "indices" query that combines the
results of two different queries on two modestly sized indexes (150M
records combined). Execution of actionGet() on this query takes
approximately 1-2 seconds. On the other hand, if I dump the same query to
JSON via the XContent jsonBuilder() and execute it with curl, it takes only
14 ms. Results are identical in both cases.

Has anyone else had similar problems like this? Any thoughts on what
might be the source of the problem? My next thought was to try connecting
to the cluster via a TransportClient. Not sure if it will help or not...

Thanks!

  • Elliott

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/fa4fb57e-3beb-4e80-8672-1981bcb8c35c%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/fa4fb57e-3beb-4e80-8672-1981bcb8c35c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/bfae9628-0212-4b03-8f76-4fb85aab23e9%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/bfae9628-0212-4b03-8f76-4fb85aab23e9%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/16c49c37-4c0c-4b37-85a5-2969aa6c5ee4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #6

Since curl requests use the Java API, it is impossible to execute this
faster than Java API.

Can you post the Java code and curl request you use? Have you compared the
queries sent? What does the query look like? How does the result look like?

Without this information it is very hard to discuss any issue, it will be
just guessing.

Jörg

On Wed, Aug 20, 2014 at 8:02 PM, Elliott Bradshaw ebradshaw1@gmail.com
wrote:

Hi Vineeth,

The system is currently deployed on a Tomcat server and the client is
created a single time on application start up. The time I gave of 1-2
seconds is for the query alone, as far as I know.

Thanks,

Elliott

On Wednesday, August 20, 2014 12:46:34 PM UTC-4, vineeth mohan wrote:

Hello Eliott ,

I suspect if this has something to do with client creation.
Can you see the time taken for the query alone and not the client
creation.

Other possible reasons can be refresh flag ON.
Please check that also.

Thanks
Vineeth

On Wed, Aug 20, 2014 at 10:11 PM, Elliott Bradshaw ebrad...@gmail.com
wrote:

Vineeth,

Thanks, but according to the documentation, it looks like sniffing is
only supported when using the TransportClient. I'm using the standard
Node/Client configuration.

On Wednesday, August 20, 2014 12:24:45 PM UTC-4, vineeth mohan wrote:

Hello Elliot ,

Is the sniffing enabled while creating the client.
This means that , it will determine which all machines are enabled and
does some load balancing.
Which means that it needs additional time while creating the client.

Thanks
Vineeth

On Wed, Aug 20, 2014 at 9:39 PM, Elliott Bradshaw ebrad...@gmail.com
wrote:

I'm joining an existing cluster as a node via the Java API. I
generate a relatively complicated multi-index "indices" query that combines
the results of two different queries on two modestly sized indexes (150M
records combined). Execution of actionGet() on this query takes
approximately 1-2 seconds. On the other hand, if I dump the same query to
JSON via the XContent jsonBuilder() and execute it with curl, it takes only
14 ms. Results are identical in both cases.

Has anyone else had similar problems like this? Any thoughts on what
might be the source of the problem? My next thought was to try connecting
to the cluster via a TransportClient. Not sure if it will help or not...

Thanks!

  • Elliott

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/fa4fb57e-3beb-4e80-8672-1981bcb8c35c%40goo
glegroups.com
https://groups.google.com/d/msgid/elasticsearch/fa4fb57e-3beb-4e80-8672-1981bcb8c35c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/bfae9628-0212-4b03-8f76-4fb85aab23e9%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/bfae9628-0212-4b03-8f76-4fb85aab23e9%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/16c49c37-4c0c-4b37-85a5-2969aa6c5ee4%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/16c49c37-4c0c-4b37-85a5-2969aa6c5ee4%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHuXWpN2RU1SdDJzQsAy9xxf39cdSSmRD1nPs4_i8PMGQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


#7

Jorg,

Thanks for the response. I understand that curl hits the Java API, that's
why the issue is so strange.

I will try to make some more tweaks tomorrow, and I can send the query as
well. I can't post the java code as it's extensive and spans several
classes (the resultant query is relatively complex). However, I log the
JSON dump of the final QueryBuilder object and use that in my curl request.

  • Elliott

On Wednesday, August 20, 2014 2:55:50 PM UTC-4, Jörg Prante wrote:

Since curl requests use the Java API, it is impossible to execute this
faster than Java API.

Can you post the Java code and curl request you use? Have you compared the
queries sent? What does the query look like? How does the result look like?

Without this information it is very hard to discuss any issue, it will be
just guessing.

Jörg

On Wed, Aug 20, 2014 at 8:02 PM, Elliott Bradshaw <ebrad...@gmail.com
<javascript:>> wrote:

Hi Vineeth,

The system is currently deployed on a Tomcat server and the client is
created a single time on application start up. The time I gave of 1-2
seconds is for the query alone, as far as I know.

Thanks,

Elliott

On Wednesday, August 20, 2014 12:46:34 PM UTC-4, vineeth mohan wrote:

Hello Eliott ,

I suspect if this has something to do with client creation.
Can you see the time taken for the query alone and not the client
creation.

Other possible reasons can be refresh flag ON.
Please check that also.

Thanks
Vineeth

On Wed, Aug 20, 2014 at 10:11 PM, Elliott Bradshaw ebrad...@gmail.com
wrote:

Vineeth,

Thanks, but according to the documentation, it looks like sniffing is
only supported when using the TransportClient. I'm using the standard
Node/Client configuration.

On Wednesday, August 20, 2014 12:24:45 PM UTC-4, vineeth mohan wrote:

Hello Elliot ,

Is the sniffing enabled while creating the client.
This means that , it will determine which all machines are enabled and
does some load balancing.
Which means that it needs additional time while creating the client.

Thanks
Vineeth

On Wed, Aug 20, 2014 at 9:39 PM, Elliott Bradshaw ebrad...@gmail.com
wrote:

I'm joining an existing cluster as a node via the Java API. I
generate a relatively complicated multi-index "indices" query that combines
the results of two different queries on two modestly sized indexes (150M
records combined). Execution of actionGet() on this query takes
approximately 1-2 seconds. On the other hand, if I dump the same query to
JSON via the XContent jsonBuilder() and execute it with curl, it takes only
14 ms. Results are identical in both cases.

Has anyone else had similar problems like this? Any thoughts on what
might be the source of the problem? My next thought was to try connecting
to the cluster via a TransportClient. Not sure if it will help or not...

Thanks!

  • Elliott

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/fa4fb57e-3beb-4e80-8672-1981bcb8c35c%40goo
glegroups.com
https://groups.google.com/d/msgid/elasticsearch/fa4fb57e-3beb-4e80-8672-1981bcb8c35c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/bfae9628-0212-4b03-8f76-4fb85aab23e9%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/bfae9628-0212-4b03-8f76-4fb85aab23e9%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/16c49c37-4c0c-4b37-85a5-2969aa6c5ee4%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/16c49c37-4c0c-4b37-85a5-2969aa6c5ee4%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/cb19a3a8-c8dc-48c8-aa44-ebd718667be3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(vineeth mohan-2) #8

Hello Elliott ,

There could be many other variables here.
One is that the first time when you run the query , it would be slow and
the second time when you run it , then it is fast.
You might have always executed the java code first and then tried the
normal curl.

Please reproduce the issue in a java code with just the query and paste
here.
That might help us better debug the issue.

Thanks
Vineeth

On Thu, Aug 21, 2014 at 7:46 AM, Elliott Bradshaw ebradshaw1@gmail.com
wrote:

Jorg,

Thanks for the response. I understand that curl hits the Java API, that's
why the issue is so strange.

I will try to make some more tweaks tomorrow, and I can send the query as
well. I can't post the java code as it's extensive and spans several
classes (the resultant query is relatively complex). However, I log the
JSON dump of the final QueryBuilder object and use that in my curl request.

  • Elliott

On Wednesday, August 20, 2014 2:55:50 PM UTC-4, Jörg Prante wrote:

Since curl requests use the Java API, it is impossible to execute this
faster than Java API.

Can you post the Java code and curl request you use? Have you compared
the queries sent? What does the query look like? How does the result look
like?

Without this information it is very hard to discuss any issue, it will be
just guessing.

Jörg

On Wed, Aug 20, 2014 at 8:02 PM, Elliott Bradshaw ebrad...@gmail.com
wrote:

Hi Vineeth,

The system is currently deployed on a Tomcat server and the client is
created a single time on application start up. The time I gave of 1-2
seconds is for the query alone, as far as I know.

Thanks,

Elliott

On Wednesday, August 20, 2014 12:46:34 PM UTC-4, vineeth mohan wrote:

Hello Eliott ,

I suspect if this has something to do with client creation.
Can you see the time taken for the query alone and not the client
creation.

Other possible reasons can be refresh flag ON.
Please check that also.

Thanks
Vineeth

On Wed, Aug 20, 2014 at 10:11 PM, Elliott Bradshaw ebrad...@gmail.com
wrote:

Vineeth,

Thanks, but according to the documentation, it looks like sniffing is
only supported when using the TransportClient. I'm using the standard
Node/Client configuration.

On Wednesday, August 20, 2014 12:24:45 PM UTC-4, vineeth mohan wrote:

Hello Elliot ,

Is the sniffing enabled while creating the client.
This means that , it will determine which all machines are enabled
and does some load balancing.
Which means that it needs additional time while creating the client.

Thanks
Vineeth

On Wed, Aug 20, 2014 at 9:39 PM, Elliott Bradshaw <ebrad...@gmail.com

wrote:

I'm joining an existing cluster as a node via the Java API. I
generate a relatively complicated multi-index "indices" query that combines
the results of two different queries on two modestly sized indexes (150M
records combined). Execution of actionGet() on this query takes
approximately 1-2 seconds. On the other hand, if I dump the same query to
JSON via the XContent jsonBuilder() and execute it with curl, it takes only
14 ms. Results are identical in both cases.

Has anyone else had similar problems like this? Any thoughts on
what might be the source of the problem? My next thought was to try
connecting to the cluster via a TransportClient. Not sure if it will help
or not...

Thanks!

  • Elliott

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/fa4fb57e-3be
b-4e80-8672-1981bcb8c35c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/fa4fb57e-3beb-4e80-8672-1981bcb8c35c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/bfae9628-0212-4b03-8f76-4fb85aab23e9%40goo
glegroups.com
https://groups.google.com/d/msgid/elasticsearch/bfae9628-0212-4b03-8f76-4fb85aab23e9%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/16c49c37-4c0c-4b37-85a5-2969aa6c5ee4%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/16c49c37-4c0c-4b37-85a5-2969aa6c5ee4%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/cb19a3a8-c8dc-48c8-aa44-ebd718667be3%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/cb19a3a8-c8dc-48c8-aa44-ebd718667be3%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGdPd5mFpcEj7hURgUqqaA6y0ZrVSwTnb-J335QhSdnru0eFug%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


#9

Hi guys,

Thanks for getting back to me.

Here's the JSON for the query:

{ "bool" : {
"must" : [ {
"indices" : {
"indices" : "index1",
"query" : {
"filtered" : {
"query" : {
"function_score" : {
"query" : {
"multi_match" : {
"query" : "this is my query string",
"fields" : [ "field1^1.2",
"field2", "field3", "field4", "field5" ],
"type" : "cross_fields",
"operator" : "AND",
"boost" : 1.2,
"fuzziness" : "AUTO", //does not
work unfortunately on cross_fields :frowning: ...
"minimum_should_match" : "75%",
"tie_breaker" : 0.1
}
},
"gauss" : {
"fields6" : {
"origin" : 1,
"scale" : 25
}
}
}
},
"filter" : {
"and" : {
"filters" : []
}
}
}
}
}
},
{
"indices" : {
"indices" : "index2",
"query" : {
"filtered" : {
"query" : {
"multi_match" : {
"query" : "this is my query string",
"fields" : [ "field7^1.2", "field6",
"field9", "field10", "field11" ],
"type" : "cross_fields",
"operator" : "AND",
"boost" : 1.2,
"fuzziness" : "AUTO", //does not work
unfortunately on cross_fields :frowning: ...
"minimum_should_match" : "75%",
"tie_breaker" : 0.1
}
},
"filter" : {
"and" : {
"filters" : []
}
}
}
}
}
}
]
}

This JSON is output to a log by the following Java code:

QueryBuilder query = getQuery();

String json = query.toXContent(jsonBuilder(), null).string();
logger.debug(json); //Outputs JSON that I copy into my curl body
(displayed above)

I then execute the query:

logger.debug("Search request beginning");

SearchResponse response = client
.prepareSearch()
.setQuery(query)
.setExplain(false)
.setSize(10)
.execute().actionGet();

logger.debug("Search request complete");

The time between the two debug statements is always between 1-2 seconds.
This is after running many queries via a webservice. The curl request
always runs in 10-30 ms.

Any other thoughts?

On Wednesday, August 20, 2014 10:23:49 PM UTC-4, vineeth mohan wrote:

Hello Elliott ,

There could be many other variables here.
One is that the first time when you run the query , it would be slow and
the second time when you run it , then it is fast.
You might have always executed the java code first and then tried the
normal curl.

Please reproduce the issue in a java code with just the query and paste
here.
That might help us better debug the issue.

Thanks
Vineeth

On Thu, Aug 21, 2014 at 7:46 AM, Elliott Bradshaw <ebrad...@gmail.com
<javascript:>> wrote:

Jorg,

Thanks for the response. I understand that curl hits the Java API,
that's why the issue is so strange.

I will try to make some more tweaks tomorrow, and I can send the query as
well. I can't post the java code as it's extensive and spans several
classes (the resultant query is relatively complex). However, I log the
JSON dump of the final QueryBuilder object and use that in my curl request.

  • Elliott

On Wednesday, August 20, 2014 2:55:50 PM UTC-4, Jörg Prante wrote:

Since curl requests use the Java API, it is impossible to execute this
faster than Java API.

Can you post the Java code and curl request you use? Have you compared
the queries sent? What does the query look like? How does the result look
like?

Without this information it is very hard to discuss any issue, it will
be just guessing.

Jörg

On Wed, Aug 20, 2014 at 8:02 PM, Elliott Bradshaw ebrad...@gmail.com
wrote:

Hi Vineeth,

The system is currently deployed on a Tomcat server and the client is
created a single time on application start up. The time I gave of 1-2
seconds is for the query alone, as far as I know.

Thanks,

Elliott

On Wednesday, August 20, 2014 12:46:34 PM UTC-4, vineeth mohan wrote:

Hello Eliott ,

I suspect if this has something to do with client creation.
Can you see the time taken for the query alone and not the client
creation.

Other possible reasons can be refresh flag ON.
Please check that also.

Thanks
Vineeth

On Wed, Aug 20, 2014 at 10:11 PM, Elliott Bradshaw <ebrad...@gmail.com

wrote:

Vineeth,

Thanks, but according to the documentation, it looks like sniffing is
only supported when using the TransportClient. I'm using the standard
Node/Client configuration.

On Wednesday, August 20, 2014 12:24:45 PM UTC-4, vineeth mohan wrote:

Hello Elliot ,

Is the sniffing enabled while creating the client.
This means that , it will determine which all machines are enabled
and does some load balancing.
Which means that it needs additional time while creating the client.

Thanks
Vineeth

On Wed, Aug 20, 2014 at 9:39 PM, Elliott Bradshaw <
ebrad...@gmail.com> wrote:

I'm joining an existing cluster as a node via the Java API. I
generate a relatively complicated multi-index "indices" query that combines
the results of two different queries on two modestly sized indexes (150M
records combined). Execution of actionGet() on this query takes
approximately 1-2 seconds. On the other hand, if I dump the same query to
JSON via the XContent jsonBuilder() and execute it with curl, it takes only
14 ms. Results are identical in both cases.

Has anyone else had similar problems like this? Any thoughts on
what might be the source of the problem? My next thought was to try
connecting to the cluster via a TransportClient. Not sure if it will help
or not...

Thanks!

  • Elliott

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/fa4fb57e-3be
b-4e80-8672-1981bcb8c35c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/fa4fb57e-3beb-4e80-8672-1981bcb8c35c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/bfae9628-0212-4b03-8f76-4fb85aab23e9%40goo
glegroups.com
https://groups.google.com/d/msgid/elasticsearch/bfae9628-0212-4b03-8f76-4fb85aab23e9%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/16c49c37-4c0c-4b37-85a5-2969aa6c5ee4%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/16c49c37-4c0c-4b37-85a5-2969aa6c5ee4%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/cb19a3a8-c8dc-48c8-aa44-ebd718667be3%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/cb19a3a8-c8dc-48c8-aa44-ebd718667be3%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/81f86a13-eea0-4b86-8e56-981eb2045d5e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


#10

Hey I think I got it!

So in my curl request, I was specifying the two indexes in the rest
endpoint, but I did not specify them in my java search request. My only
other index is a suggest index, and I would think that it would be ignored
when I ran the query, but I guess not.

Thanks a lot for your time. This was definitely one of those bang your
head against the wall issues.

On Thursday, August 21, 2014 11:06:32 AM UTC-4, Elliott Bradshaw wrote:

Hi guys,

Thanks for getting back to me.

Here's the JSON for the query:

{ "bool" : {
"must" : [ {
"indices" : {
"indices" : "index1",
"query" : {
"filtered" : {
"query" : {
"function_score" : {
"query" : {
"multi_match" : {
"query" : "this is my query
string",
"fields" : [ "field1^1.2",
"field2", "field3", "field4", "field5" ],
"type" : "cross_fields",
"operator" : "AND",
"boost" : 1.2,
"fuzziness" : "AUTO", //does not
work unfortunately on cross_fields :frowning: ...
"minimum_should_match" : "75%",
"tie_breaker" : 0.1
}
},
"gauss" : {
"fields6" : {
"origin" : 1,
"scale" : 25
}
}
}
},
"filter" : {
"and" : {
"filters" : []
}
}
}
}
}
},
{
"indices" : {
"indices" : "index2",
"query" : {
"filtered" : {
"query" : {
"multi_match" : {
"query" : "this is my query string",
"fields" : [ "field7^1.2", "field6",
"field9", "field10", "field11" ],
"type" : "cross_fields",
"operator" : "AND",
"boost" : 1.2,
"fuzziness" : "AUTO", //does not work
unfortunately on cross_fields :frowning: ...
"minimum_should_match" : "75%",
"tie_breaker" : 0.1
}
},
"filter" : {
"and" : {
"filters" : []
}
}
}
}
}
}
]
}

This JSON is output to a log by the following Java code:

QueryBuilder query = getQuery();

String json = query.toXContent(jsonBuilder(), null).string();
logger.debug(json); //Outputs JSON that I copy into my curl body
(displayed above)

I then execute the query:

logger.debug("Search request beginning");

SearchResponse response = client
.prepareSearch()
.setQuery(query)
.setExplain(false)
.setSize(10)
.execute().actionGet();

logger.debug("Search request complete");

The time between the two debug statements is always between 1-2 seconds.
This is after running many queries via a webservice. The curl request
always runs in 10-30 ms.

Any other thoughts?

On Wednesday, August 20, 2014 10:23:49 PM UTC-4, vineeth mohan wrote:

Hello Elliott ,

There could be many other variables here.
One is that the first time when you run the query , it would be slow and
the second time when you run it , then it is fast.
You might have always executed the java code first and then tried the
normal curl.

Please reproduce the issue in a java code with just the query and paste
here.
That might help us better debug the issue.

Thanks
Vineeth

On Thu, Aug 21, 2014 at 7:46 AM, Elliott Bradshaw ebrad...@gmail.com
wrote:

Jorg,

Thanks for the response. I understand that curl hits the Java API,
that's why the issue is so strange.

I will try to make some more tweaks tomorrow, and I can send the query
as well. I can't post the java code as it's extensive and spans several
classes (the resultant query is relatively complex). However, I log the
JSON dump of the final QueryBuilder object and use that in my curl request.

  • Elliott

On Wednesday, August 20, 2014 2:55:50 PM UTC-4, Jörg Prante wrote:

Since curl requests use the Java API, it is impossible to execute this
faster than Java API.

Can you post the Java code and curl request you use? Have you compared
the queries sent? What does the query look like? How does the result look
like?

Without this information it is very hard to discuss any issue, it will
be just guessing.

Jörg

On Wed, Aug 20, 2014 at 8:02 PM, Elliott Bradshaw ebrad...@gmail.com
wrote:

Hi Vineeth,

The system is currently deployed on a Tomcat server and the client is
created a single time on application start up. The time I gave of 1-2
seconds is for the query alone, as far as I know.

Thanks,

Elliott

On Wednesday, August 20, 2014 12:46:34 PM UTC-4, vineeth mohan wrote:

Hello Eliott ,

I suspect if this has something to do with client creation.
Can you see the time taken for the query alone and not the client
creation.

Other possible reasons can be refresh flag ON.
Please check that also.

Thanks
Vineeth

On Wed, Aug 20, 2014 at 10:11 PM, Elliott Bradshaw <
ebrad...@gmail.com> wrote:

Vineeth,

Thanks, but according to the documentation, it looks like sniffing
is only supported when using the TransportClient. I'm using the standard
Node/Client configuration.

On Wednesday, August 20, 2014 12:24:45 PM UTC-4, vineeth mohan wrote:

Hello Elliot ,

Is the sniffing enabled while creating the client.
This means that , it will determine which all machines are enabled
and does some load balancing.
Which means that it needs additional time while creating the client.

Thanks
Vineeth

On Wed, Aug 20, 2014 at 9:39 PM, Elliott Bradshaw <
ebrad...@gmail.com> wrote:

I'm joining an existing cluster as a node via the Java API. I
generate a relatively complicated multi-index "indices" query that combines
the results of two different queries on two modestly sized indexes (150M
records combined). Execution of actionGet() on this query takes
approximately 1-2 seconds. On the other hand, if I dump the same query to
JSON via the XContent jsonBuilder() and execute it with curl, it takes only
14 ms. Results are identical in both cases.

Has anyone else had similar problems like this? Any thoughts on
what might be the source of the problem? My next thought was to try
connecting to the cluster via a TransportClient. Not sure if it will help
or not...

Thanks!

  • Elliott

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/fa4fb57e-3be
b-4e80-8672-1981bcb8c35c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/fa4fb57e-3beb-4e80-8672-1981bcb8c35c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/bfae9628-021
2-4b03-8f76-4fb85aab23e9%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/bfae9628-0212-4b03-8f76-4fb85aab23e9%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/16c49c37-4c0c-4b37-85a5-2969aa6c5ee4%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/16c49c37-4c0c-4b37-85a5-2969aa6c5ee4%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/cb19a3a8-c8dc-48c8-aa44-ebd718667be3%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/cb19a3a8-c8dc-48c8-aa44-ebd718667be3%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b7b073db-eef5-4680-9d4e-05a261ffd0b3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #11