Running the same ES query through the Java API and the ES console gives different number of results

Elastic Stack Elasticsearch
1

Hi,

When i run the below query through the Java API i get a certain number of
results but when i run the same query through the elastic search console i
get fewer records. The Elastic search version is 0.19.2 and same is the
version of the Jar being used. Has anyone faced this situation before?

{
"from" : 0,
"size" : 2,
"query" : {
"filtered" : {
"query" : {"match_all" : {}}
,"filter" : {
"and" : {
"filters" :
[ {
"exists" : {"field" : "id"}}
, {"range" : {"startDate" : {"from" : null,"to" :
"2013-02-20","include_lower" : true,"include_upper" : true}}}
, {"term" : {"completed" : "true"}}, {"term" : {"active" :
"true"}}, {"term" : {"traded" : "false"}}, {"term" : {"working" :
"false"}}, {"term" : {"secondhand" : "false"}
} ]
}
}
}
},
"fields" : [ "name", "price", "marketprice", "id" ],
"sort" : [ {"id" : {"order" : "asc"}} ]
}

Thanks,
John

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

I think i should make my post more precise.
When i run this query from the Elastic Search head/console, it returns the
same number of records as requested but running it through the Java API
returns more, almost double.

On Wednesday, 20 February 2013 16:39:29 UTC-6, john wrote:

Hi,

When i run the below query through the Java API i get a certain number of
results but when i run the same query through the Elasticsearch console i
get fewer records. The Elastic search version is 0.19.2 and same is the
version of the Jar being used. Has anyone faced this situation before?

{
"from" : 0,
"size" : 2,
"query" : {
"filtered" : {
"query" : {"match_all" : {}}
,"filter" : {
"and" : {
"filters" :
[ {
"exists" : {"field" : "id"}}
, {"range" : {"startDate" : {"from" : null,"to" :
"2013-02-20","include_lower" : true,"include_upper" : true}}}
, {"term" : {"completed" : "true"}}, {"term" : {"active" :
"true"}}, {"term" : {"traded" : "false"}}, {"term" : {"working" :
"false"}}, {"term" : {"secondhand" : "false"}
} ]
}
}
}
},
"fields" : [ "name", "price", "marketprice", "id" ],
"sort" : [ {"id" : {"order" : "asc"}} ]
}

Thanks,
John

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Can you gist your Java code somewhere (or post it)?

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

This is essentially how i'm writing that code to run the Java API Query.

Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name",
"XXXXXXXX").put("client.transport.sniff", true).put("node.client", true)
.put("client.transport.ping_timeout","10s").build();
Client client = new TransportClient(settings).addTransportAddress(new
InetSocketTransportAddress("10.16.150.100", 9300));
QueryBuilder queryBuilderObj = boolQuery().must(termQuery("XXXX", "YYYY"));
SearchRequestBuilder srb = client.prepareSearch("XXXXX")
.setSearchType(DFS_QUERY_THEN_FETCH).setQuery(queryBuilderObj)
.addField("price").addField("id").setFrom(0).setSize(3).setExplain(true);
SearchResponse response = srb.execute().actionGet();

On Wednesday, 20 February 2013 17:16:36 UTC-6, Jörg Prante wrote:

Can you gist your Java code somewhere (or post it)?

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

This is not the same query at all.
What could be nice is to sysout your query using srb.string() or toString() and run exactly this query with curl.

You should have same results.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 21 févr. 2013 à 17:14, john john2roll2@gmail.com a écrit :

This is essentially how i'm writing that code to run the Java API Query.

Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name", "XXXXXXXX").put("client.transport.sniff", true).put("node.client", true).put("client.transport.ping_timeout","10s").build();
Client client = new TransportClient(settings).addTransportAddress(new InetSocketTransportAddress("10.16.150.100", 9300));
QueryBuilder queryBuilderObj = boolQuery().must(termQuery("XXXX", "YYYY"));
SearchRequestBuilder srb = client.prepareSearch("XXXXX").setSearchType(DFS_QUERY_THEN_FETCH).setQuery(queryBuilderObj) .addField("price").addField("id").setFrom(0).setSize(3).setExplain(true);
SearchResponse response = srb.execute().actionGet();

On Wednesday, 20 February 2013 17:16:36 UTC-6, Jörg Prante wrote:
Can you gist your Java code somewhere (or post it)?

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

6

David,

i did not provide the exact query building code as i thought probably Jörg
wanted to know how i'm running the query. The query that initally posted
was taken after a toString() on the searchRequestBuilder.
Will try to run the same query using curl.

On Thursday, 21 February 2013 10:20:32 UTC-6, David Pilato wrote:

This is not the same query at all.
What could be nice is to sysout your query using srb.string() or
toString() and run exactly this query with curl.

You should have same results.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr
| @scrutmydocs https://twitter.com/scrutmydocs

Le 21 févr. 2013 à 17:14, john <john2...@gmail.com <javascript:>> a écrit
:

This is essentially how i'm writing that code to run the Java API Query.

Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name",
"XXXXXXXX").put("client.transport.sniff", true).put("node.client", true)
.put("client.transport.ping_timeout","10s").build();
Client client = new TransportClient(settings).addTransportAddress(new
InetSocketTransportAddress("10.16.150.100", 9300));
QueryBuilder queryBuilderObj = boolQuery().must(termQuery("XXXX", "YYYY"));
SearchRequestBuilder srb = client.prepareSearch("XXXXX")
.setSearchType(DFS_QUERY_THEN_FETCH).setQuery(queryBuilderObj)
.addField("price").addField("id").setFrom(0).setSize(3).setExplain(true);
SearchResponse response = srb.execute().actionGet();

On Wednesday, 20 February 2013 17:16:36 UTC-6, Jörg Prante wrote:

Can you gist your Java code somewhere (or post it)?

Jörg

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

7

I tried running the same query using curl (on the elastic Search) and got
the same number of records as requested.

On Thursday, 21 February 2013 10:34:43 UTC-6, john wrote:

David,

i did not provide the exact query building code as i thought probably Jörg
wanted to know how i'm running the query. The query that initally posted
was taken after a toString() on the searchRequestBuilder.
Will try to run the same query using curl.

On Thursday, 21 February 2013 10:20:32 UTC-6, David Pilato wrote:

This is not the same query at all.
What could be nice is to sysout your query using srb.string() or
toString() and run exactly this query with curl.

You should have same results.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr
| @scrutmydocs https://twitter.com/scrutmydocs

Le 21 févr. 2013 à 17:14, john john2...@gmail.com a écrit :

This is essentially how i'm writing that code to run the Java API Query.

Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name",
"XXXXXXXX").put("client.transport.sniff", true).put("node.client", true)
.put("client.transport.ping_timeout","10s").build();
Client client = new TransportClient(settings).addTransportAddress(new
InetSocketTransportAddress("10.16.150.100", 9300));
QueryBuilder queryBuilderObj = boolQuery().must(termQuery("XXXX",
"YYYY"));
SearchRequestBuilder srb = client.prepareSearch("XXXXX")
.setSearchType(DFS_QUERY_THEN_FETCH).setQuery(queryBuilderObj)
.addField("price").addField("id").setFrom(0).setSize(3).setExplain(true);
SearchResponse response = srb.execute().actionGet();

On Wednesday, 20 February 2013 17:16:36 UTC-6, Jörg Prante wrote:

Can you gist your Java code somewhere (or post it)?

Jörg

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

8

So both the Elasticsearch console and curl are returning me the same
number of records as requested, but the java API returns me almost double
the requested number

On Thursday, 21 February 2013 13:42:59 UTC-6, john wrote:

I tried running the same query using curl (on the elastic Search) and got
the same number of records as requested.

On Thursday, 21 February 2013 10:34:43 UTC-6, john wrote:

David,

i did not provide the exact query building code as i thought
probably Jörg wanted to know how i'm running the query. The query that
initally posted was taken after a toString() on the searchRequestBuilder.
Will try to run the same query using curl.

On Thursday, 21 February 2013 10:20:32 UTC-6, David Pilato wrote:

This is not the same query at all.
What could be nice is to sysout your query using srb.string() or
toString() and run exactly this query with curl.

You should have same results.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr
| @scrutmydocs https://twitter.com/scrutmydocs

Le 21 févr. 2013 à 17:14, john john2...@gmail.com a écrit :

This is essentially how i'm writing that code to run the Java API Query.

Settings settings = ImmutableSettings.settingsBuilder().put("
cluster.name", "XXXXXXXX").put("client.transport.sniff", true).put("node.client",
true).put("client.transport.ping_timeout","10s").build();
Client client = new TransportClient(settings).addTransportAddress(new
InetSocketTransportAddress("10.16.150.100", 9300));
QueryBuilder queryBuilderObj = boolQuery().must(termQuery("XXXX",
"YYYY"));
SearchRequestBuilder srb = client.prepareSearch("XXXXX")
.setSearchType(DFS_QUERY_THEN_FETCH).setQuery(queryBuilderObj)
.addField("price").addField("id")
.setFrom(0).setSize(3).setExplain(true);
SearchResponse response = srb.execute().actionGet();

On Wednesday, 20 February 2013 17:16:36 UTC-6, Jörg Prante wrote:

Can you gist your Java code somewhere (or post it)?

Jörg

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

9

So i figured out whats causing those extra number of records. The query is
run on all the shards and the users get to see the records returned from
each of those shards hence those extra set of records.
Now i'm trying to figure out whats the cause for that.

On Friday, 22 February 2013 12:54:36 UTC-6, john wrote:

So both the Elasticsearch console and curl are returning me the same
number of records as requested, but the java API returns me almost double
the requested number

On Thursday, 21 February 2013 13:42:59 UTC-6, john wrote:

I tried running the same query using curl (on the elastic Search) and got
the same number of records as requested.

On Thursday, 21 February 2013 10:34:43 UTC-6, john wrote:

David,

i did not provide the exact query building code as i thought
probably Jörg wanted to know how i'm running the query. The query that
initally posted was taken after a toString() on the searchRequestBuilder.
Will try to run the same query using curl.

On Thursday, 21 February 2013 10:20:32 UTC-6, David Pilato wrote:

This is not the same query at all.
What could be nice is to sysout your query using srb.string() or
toString() and run exactly this query with curl.

You should have same results.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr
| @scrutmydocs https://twitter.com/scrutmydocs

Le 21 févr. 2013 à 17:14, john john2...@gmail.com a écrit :

This is essentially how i'm writing that code to run the Java API Query.

Settings settings = ImmutableSettings.settingsBuilder().put("
cluster.name", "XXXXXXXX").put("client.transport.sniff", true).put("node.client",
true).put("client.transport.ping_timeout","10s").build();
Client client = new TransportClient(settings).addTransportAddress(new
InetSocketTransportAddress("10.16.150.100", 9300));
QueryBuilder queryBuilderObj = boolQuery().must(termQuery("XXXX",
"YYYY"));
SearchRequestBuilder srb = client.prepareSearch("XXXXX")
.setSearchType(DFS_QUERY_THEN_FETCH).setQuery(queryBuilderObj)
.addField("price").addField("id")
.setFrom(0).setSize(3).setExplain(true);
SearchResponse response = srb.execute().actionGet();

On Wednesday, 20 February 2013 17:16:36 UTC-6, Jörg Prante wrote:

Can you gist your Java code somewhere (or post it)?

Jörg

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

10

This issue was resolved in this below post

https://groups.google.com/d/topic/elasticsearch/rZdPLltx-Sg/discussion

thanks,
john

On Friday, 22 February 2013 13:25:04 UTC-6, john wrote:

So i figured out whats causing those extra number of records. The query is
run on all the shards and the users get to see the records returned from
each of those shards hence those extra set of records.
Now i'm trying to figure out whats the cause for that.

On Friday, 22 February 2013 12:54:36 UTC-6, john wrote:

So both the Elasticsearch console and curl are returning me the same
number of records as requested, but the java API returns me almost double
the requested number

On Thursday, 21 February 2013 13:42:59 UTC-6, john wrote:

I tried running the same query using curl (on the elastic Search) and
got the same number of records as requested.

On Thursday, 21 February 2013 10:34:43 UTC-6, john wrote:

David,

i did not provide the exact query building code as i thought
probably Jörg wanted to know how i'm running the query. The query that
initally posted was taken after a toString() on the searchRequestBuilder.
Will try to run the same query using curl.

On Thursday, 21 February 2013 10:20:32 UTC-6, David Pilato wrote:

This is not the same query at all.
What could be nice is to sysout your query using srb.string() or
toString() and run exactly this query with curl.

You should have same results.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr
| @scrutmydocs https://twitter.com/scrutmydocs

Le 21 févr. 2013 à 17:14, john john2...@gmail.com a écrit :

This is essentially how i'm writing that code to run the Java API
Query.

Settings settings = ImmutableSettings.settingsBuilder().put("
cluster.name", "XXXXXXXX").put("client.transport.sniff", true).put("node.client",
true).put("client.transport.ping_timeout","10s").build();
Client client = new TransportClient(settings).addTransportAddress(new
InetSocketTransportAddress("10.16.150.100", 9300));
QueryBuilder queryBuilderObj = boolQuery().must(termQuery("XXXX",
"YYYY"));
SearchRequestBuilder srb = client.prepareSearch("XXXXX")
.setSearchType(DFS_QUERY_THEN_FETCH).setQuery(queryBuilderObj)
.addField("price").addField("id")
.setFrom(0).setSize(3).setExplain(true);
SearchResponse response = srb.execute().actionGet();

On Wednesday, 20 February 2013 17:16:36 UTC-6, Jörg Prante wrote:

Can you gist your Java code somewhere (or post it)?

Jörg

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.