Weird difference in total results when executing directly against REST or through Java API

Elastic Stack Elasticsearch
1

Hi,

I just noticed I have a weird difference in the number of results depending
on whether I execute my query as a direct REST (with curl) or via JAVA API.

So for instance, if I use the REST call:

{
"from": 0,
"size": 30,
"query": {
"filtered": {
"query": {
"query_string": {
"query": "iphone",
"default_operator": "and",
"analyze_wildcard": true
}
},
"filter": {
"and": {
"filters": [
{
"range": {
"cantidad": {
"from": 6.35,
"to": 527.12,
"include_lower": true,
"include_upper": true
}
}
},
{
"term": {
"categorias1": "Electrónica"
}
}
]
}
}
}
},
"explain": false,
"sort": [
{
"_score": {}
}
]
}

then I get these results:

{

  • took: 552
  • timed_out: false
  • _shards: {
    • total: 3
    • successful: 3
    • failed: 0
      }
  • hits: {
    • total: 578
    • max_score: 1
    • hits: [
      • {
        • _index: cuestamenos
        • _type: producto
        • _id: XqvBlh3JSxy9WBgXRfbNLQ
        • _score: 1
        • _source: {
          • nombre: Altec Lansing T612 Altavoz para iPhone/iPod

.....
.....
.....
.....

As you can see, 578 results.

But when I use the JAVA API (and I make sure it's exactly the same query
because I print the generated JSON out in the logs and it matches exactly):

*def* response = search.execute().actionGet()

def result = [:]

result.query = querystring

result.hits = response?.hits()

result.total = result?.hits?.totalHits()

LOG.debug "Search returned ${result.total ?: 0} result(s)."

then it says:

2011-03-18 13:12:18,228 [http-8080-4] DEBUG util.ElasticSearchUtil - Search
returned 576 result(s).

And doing some more tests, it turns out that the JAVA API always displays
exactly 2 less than the REST approach.

Any clue why the JAVA API call returns always exactly 2 results less than
the REST call?

I'm working with latest from master as of today --> 0.16.0.SNAPSHOT

Thanks.

2

Just thinking loud, but could it be that ES is not picking up the inclusive
range in the JAVA API, hence discarding the first and last match in the
price range?

          "range": {
            "cantidad": {
              "from": 6.35,
              "to": 527.12,
              "include_lower": true,
              "include_upper": true
            }
          }

On Fri, Mar 18, 2011 at 1:16 PM, Enrique Medina Montenegro <
e.medina.m@gmail.com> wrote:

Hi,

I just noticed I have a weird difference in the number of results depending
on whether I execute my query as a direct REST (with curl) or via JAVA API.

So for instance, if I use the REST call:

{
"from": 0,
"size": 30,
"query": {
"filtered": {
"query": {
"query_string": {
"query": "iphone",
"default_operator": "and",
"analyze_wildcard": true
}
},
"filter": {
"and": {
"filters": [
{
"range": {
"cantidad": {
"from": 6.35,
"to": 527.12,
"include_lower": true,
"include_upper": true
}
}
},
{
"term": {
"categorias1": "Electrónica"
}
}
]
}
}
}
},
"explain": false,
"sort": [
{
"_score": {}
}
]
}

then I get these results:

{

  • took: 552
  • timed_out: false
  • _shards: {
    • total: 3
    • successful: 3
    • failed: 0
      }
  • hits: {
    • total: 578
    • max_score: 1
    • hits: [
      • {
        • _index: cuestamenos
        • _type: producto
        • _id: XqvBlh3JSxy9WBgXRfbNLQ
        • _score: 1
        • _source: {
          • nombre: Altec Lansing T612 Altavoz para iPhone/iPod

.....
.....
.....
.....

As you can see, 578 results.

But when I use the JAVA API (and I make sure it's exactly the same query
because I print the generated JSON out in the logs and it matches exactly):

def response = search.execute().actionGet()

def result = [:]

result.query = querystring

result.hits = response?.hits()

result.total = result?.hits?.totalHits()

LOG.debug "Search returned ${result.total ?: 0} result(s)."

then it says:

2011-03-18 13:12:18,228 [http-8080-4] DEBUG util.ElasticSearchUtil -
Search returned 576 result(s).

And doing some more tests, it turns out that the JAVA API always displays
exactly 2 less than the REST approach.

Any clue why the JAVA API call returns always exactly 2 results less than
the REST call?

I'm working with latest from master as of today --> 0.16.0.SNAPSHOT

Thanks.

3

Can you post how you build the query using the Java API?
On Friday, March 18, 2011 at 2:25 PM, Enrique Medina Montenegro wrote:

Just thinking loud, but could it be that ES is not picking up the inclusive range in the JAVA API, hence discarding the first and last match in the price range?

"range": {
"cantidad": {
"from": 6.35,
"to": 527.12,
"include_lower": true,
"include_upper": true
}
}

On Fri, Mar 18, 2011 at 1:16 PM, Enrique Medina Montenegro e.medina.m@gmail.com wrote:

Hi,

I just noticed I have a weird difference in the number of results depending on whether I execute my query as a direct REST (with curl) or via JAVA API.

So for instance, if I use the REST call:

{
"from": 0,
"size": 30,
"query": {
"filtered": {
"query": {
"query_string": {
"query": "iphone",
"default_operator": "and",
"analyze_wildcard": true
}
},
"filter": {
"and": {
"filters": [
{
"range": {
"cantidad": {
"from": 6.35,
"to": 527.12,
"include_lower": true,
"include_upper": true
}
}
},
{
"term": {
"categorias1": "Electrónica"
}
}
]
}
}
}
},
"explain": false,
"sort": [
{
"_score": {}
}
]
}

then I get these results:

{
took: 552
timed_out: false
_shards: {
total: 3
successful: 3
failed: 0

}
hits: {
total: 578
max_score: 1
hits: [
{
_index: cuestamenos
_type: producto
_id: XqvBlh3JSxy9WBgXRfbNLQ
_score: 1
_source: {
nombre: Altec Lansing T612 Altavoz para iPhone/iPod

.....
.....
.....
.....

As you can see, 578 results.

But when I use the JAVA API (and I make sure it's exactly the same query because I print the generated JSON out in the logs and it matches exactly):

def response = search.execute().actionGet()

def result = [:]
result.query = querystring
result.hits = response?.hits()
result.total = result?.hits?.totalHits()

LOG.debug "Search returned ${result.total ?: 0} result(s)."

then it says:

2011-03-18 13:12:18,228 [http-8080-4] DEBUG util.ElasticSearchUtil - Search returned 576 result(s).

And doing some more tests, it turns out that the JAVA API always displays exactly 2 less than the REST approach.

Any clue why the JAVA API call returns always exactly 2 results less than the REST call?

I'm working with latest from master as of today --> 0.16.0.SNAPSHOT

Thanks.

4

Shay,

I was doing some more research, and I noticed that I was using a
rangeFilter(), and then I changed it to numericRangeFilter(), but with the
same result :frowning:

numericRangeFilter("cantidad").from(searchParams.cantidad.selected)

.to(searchParams.cantidad.to)

.includeLower(true).includeUpper(true)

Regards.

On Fri, Mar 18, 2011 at 1:34 PM, Shay Banon shay.banon@elasticsearch.comwrote:

Can you post how you build the query using the Java API?

On Friday, March 18, 2011 at 2:25 PM, Enrique Medina Montenegro wrote:

Just thinking loud, but could it be that ES is not picking up the inclusive
range in the JAVA API, hence discarding the first and last match in the
price range?

          "range": {
            "cantidad": {
              "from": 6.35,
              "to": 527.12,
              "include_lower": true,
              "include_upper": true
            }
          }

On Fri, Mar 18, 2011 at 1:16 PM, Enrique Medina Montenegro <
e.medina.m@gmail.com> wrote:

Hi,

I just noticed I have a weird difference in the number of results depending
on whether I execute my query as a direct REST (with curl) or via JAVA API.

So for instance, if I use the REST call:

{
"from": 0,
"size": 30,
"query": {
"filtered": {
"query": {
"query_string": {
"query": "iphone",
"default_operator": "and",
"analyze_wildcard": true
}
},
"filter": {
"and": {
"filters": [
{
"range": {
"cantidad": {
"from": 6.35,
"to": 527.12,
"include_lower": true,
"include_upper": true
}
}
},
{
"term": {
"categorias1": "Electrónica"
}
}
]
}
}
}
},
"explain": false,
"sort": [
{
"_score": {}
}
]
}

then I get these results:

{

  • took: 552
  • timed_out: false
  • _shards: {
    • total: 3
    • successful: 3
    • failed: 0
      }
  • hits: {
    • total: 578
    • max_score: 1
    • hits: [
      • {
        • _index: cuestamenos
        • _type: producto
        • _id: XqvBlh3JSxy9WBgXRfbNLQ
        • _score: 1
        • _source: {
          • nombre: Altec Lansing T612 Altavoz para iPhone/iPod

.....
.....
.....
.....

As you can see, 578 results.

But when I use the JAVA API (and I make sure it's exactly the same query
because I print the generated JSON out in the logs and it matches exactly):

def response = search.execute().actionGet()

def result = [:]

result.query = querystring

result.hits = response?.hits()

result.total = result?.hits?.totalHits()

LOG.debug "Search returned ${result.total ?: 0} result(s)."

then it says:

2011-03-18 13:12:18,228 [http-8080-4] DEBUG util.ElasticSearchUtil -
Search returned 576 result(s).

And doing some more tests, it turns out that the JAVA API always displays
exactly 2 less than the REST approach.

Any clue why the JAVA API call returns always exactly 2 results less than
the REST call?

I'm working with latest from master as of today --> 0.16.0.SNAPSHOT

Thanks.

5

Forgot to mention that both values are JAVA Float values...

On Fri, Mar 18, 2011 at 1:41 PM, Enrique Medina Montenegro <
e.medina.m@gmail.com> wrote:

Shay,

I was doing some more research, and I noticed that I was using a
rangeFilter(), and then I changed it to numericRangeFilter(), but with the
same result :frowning:

numericRangeFilter("cantidad").from(searchParams.cantidad.selected)

.to(searchParams.cantidad.to)

.includeLower(true).includeUpper(true)

Regards.

On Fri, Mar 18, 2011 at 1:34 PM, Shay Banon shay.banon@elasticsearch.comwrote:

Can you post how you build the query using the Java API?

On Friday, March 18, 2011 at 2:25 PM, Enrique Medina Montenegro wrote:

Just thinking loud, but could it be that ES is not picking up the
inclusive range in the JAVA API, hence discarding the first and last match
in the price range?

          "range": {
            "cantidad": {
              "from": 6.35,
              "to": 527.12,
              "include_lower": true,
              "include_upper": true
            }
          }

On Fri, Mar 18, 2011 at 1:16 PM, Enrique Medina Montenegro <
e.medina.m@gmail.com> wrote:

Hi,

I just noticed I have a weird difference in the number of results
depending on whether I execute my query as a direct REST (with curl) or via
JAVA API.

So for instance, if I use the REST call:

{
"from": 0,
"size": 30,
"query": {
"filtered": {
"query": {
"query_string": {
"query": "iphone",
"default_operator": "and",
"analyze_wildcard": true
}
},
"filter": {
"and": {
"filters": [
{
"range": {
"cantidad": {
"from": 6.35,
"to": 527.12,
"include_lower": true,
"include_upper": true
}
}
},
{
"term": {
"categorias1": "Electrónica"
}
}
]
}
}
}
},
"explain": false,
"sort": [
{
"_score": {}
}
]
}

then I get these results:

{

  • took: 552
  • timed_out: false
  • _shards: {
    • total: 3
    • successful: 3
    • failed: 0
      }
  • hits: {
    • total: 578
    • max_score: 1
    • hits: [
      • {
        • _index: cuestamenos
        • _type: producto
        • _id: XqvBlh3JSxy9WBgXRfbNLQ
        • _score: 1
        • _source: {
          • nombre: Altec Lansing T612 Altavoz para iPhone/iPod

.....
.....
.....
.....

As you can see, 578 results.

But when I use the JAVA API (and I make sure it's exactly the same query
because I print the generated JSON out in the logs and it matches exactly):

def response = search.execute().actionGet()

def result = [:]

result.query = querystring

result.hits = response?.hits()

result.total = result?.hits?.totalHits()

LOG.debug "Search returned ${result.total ?: 0} result(s)."

then it says:

2011-03-18 13:12:18,228 [http-8080-4] DEBUG util.ElasticSearchUtil -
Search returned 576 result(s).

And doing some more tests, it turns out that the JAVA API always displays
exactly 2 less than the REST approach.

Any clue why the JAVA API call returns always exactly 2 results less than
the REST call?

I'm working with latest from master as of today --> 0.16.0.SNAPSHOT

Thanks.

6

Some more feedback...

If I substract/add 0.001 to each boundary in the range, then it works fine.
Could it be that using the JAVA API something is being rounded up/down
somehow?

Example:

[6.35, 527.12] --> Doesn't work
[6.351, 527.121] -> Works

On Fri, Mar 18, 2011 at 1:42 PM, Enrique Medina Montenegro <
e.medina.m@gmail.com> wrote:

Forgot to mention that both values are JAVA Float values...

On Fri, Mar 18, 2011 at 1:41 PM, Enrique Medina Montenegro <
e.medina.m@gmail.com> wrote:

Shay,

I was doing some more research, and I noticed that I was using a
rangeFilter(), and then I changed it to numericRangeFilter(), but with the
same result :frowning:

numericRangeFilter("cantidad").from(searchParams.cantidad.selected)

.to(searchParams.cantidad.to)

.includeLower(true).includeUpper(true)

Regards.

On Fri, Mar 18, 2011 at 1:34 PM, Shay Banon <shay.banon@elasticsearch.com

wrote:

Can you post how you build the query using the Java API?

On Friday, March 18, 2011 at 2:25 PM, Enrique Medina Montenegro wrote:

Just thinking loud, but could it be that ES is not picking up the
inclusive range in the JAVA API, hence discarding the first and last match
in the price range?

          "range": {
            "cantidad": {
              "from": 6.35,
              "to": 527.12,
              "include_lower": true,
              "include_upper": true
            }
          }

On Fri, Mar 18, 2011 at 1:16 PM, Enrique Medina Montenegro <
e.medina.m@gmail.com> wrote:

Hi,

I just noticed I have a weird difference in the number of results
depending on whether I execute my query as a direct REST (with curl) or via
JAVA API.

So for instance, if I use the REST call:

{
"from": 0,
"size": 30,
"query": {
"filtered": {
"query": {
"query_string": {
"query": "iphone",
"default_operator": "and",
"analyze_wildcard": true
}
},
"filter": {
"and": {
"filters": [
{
"range": {
"cantidad": {
"from": 6.35,
"to": 527.12,
"include_lower": true,
"include_upper": true
}
}
},
{
"term": {
"categorias1": "Electrónica"
}
}
]
}
}
}
},
"explain": false,
"sort": [
{
"_score": {}
}
]
}

then I get these results:

{

  • took: 552
  • timed_out: false
  • _shards: {
    • total: 3
    • successful: 3
    • failed: 0
      }
  • hits: {
    • total: 578
    • max_score: 1
    • hits: [
      • {
        • _index: cuestamenos
        • _type: producto
        • _id: XqvBlh3JSxy9WBgXRfbNLQ
        • _score: 1
        • _source: {
          • nombre: Altec Lansing T612 Altavoz para iPhone/iPod

.....
.....
.....
.....

As you can see, 578 results.

But when I use the JAVA API (and I make sure it's exactly the same query
because I print the generated JSON out in the logs and it matches exactly):

def response = search.execute().actionGet()

def result = [:]

result.query = querystring

result.hits = response?.hits()

result.total = result?.hits?.totalHits()

LOG.debug "Search returned ${result.total ?: 0} result(s)."

then it says:

2011-03-18 13:12:18,228 [http-8080-4] DEBUG util.ElasticSearchUtil -
Search returned 576 result(s).

And doing some more tests, it turns out that the JAVA API always displays
exactly 2 less than the REST approach.

Any clue why the JAVA API call returns always exactly 2 results less than
the REST call?

I'm working with latest from master as of today --> 0.16.0.SNAPSHOT

Thanks.