Paging on 0.90

Simon_Gaeremynck · May 20, 2013, 5:19pm

Hello,

I'm seeing the following paging issue on 0.90 that we weren't seeing on
0.20.6

I have a query with size set to 2 and from set to 0 that returns the
documents: [ docA, docB ]
When I change the size to 1 the result set is [ docB ]
When I query with {size: 1, from: 1} I get [ docB ] again.
I would expect that {size: 1, from: 0} would return [ docA ] ?

Both docA and docB have a _score of 1. The query orders descending on
_score first and then sorts ascending on a document field (string). The
sort field of document A should come before the sort field of document B.

Some details:

Number of shards: 1
Number of hits for the query: 2 (the 2 documents are the only hits)
There is no indexing going on in the background
Total size of the index: ~2000 documents

The query:
{
"query": {
"filtered": {
"query": {
"query_string": {
"fields": [ "q_high^2.0", "q_low^0.75" ],
"query": "*"
}
},
"filter": {
"and": [
{ "term": { "_type": "resource" } },
{ "term": { "resourceType": "group" } },
{ "ids": { "values": [ "g:camtest:groupChild-PTmmhkau0kf",
"g:camtest:groupParent-TP9mWsIp1k5" ] } }
]
}
}
},
"from": 0,
"size": 2,
"sort": [
{ "_score": { "order": "desc" } },
{ "sortField": { "order": "asc" } }
],
"min_score": 0.2,
"explain": true
}

Kind regards,

Simon

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

simonw_2 · May 20, 2013, 8:19pm

hey,

I assume this is related
to BytesRefOrdValComparator ignores highest value in a segment during binarySearch · Issue #2991 · elastic/elasticsearch · GitHub
which is essentially a sorting bug. Would you have the spare cycles to try
if the latest master fixes this problem (or maybe try branch 0.90)

simon

On Monday, May 20, 2013 7:19:09 PM UTC+2, Simon Gaeremynck wrote:

Hello,

I'm seeing the following paging issue on 0.90 that we weren't seeing on
0.20.6

I have a query with size set to 2 and from set to 0 that returns the
documents: [ docA, docB ]
When I change the size to 1 the result set is [ docB ]
When I query with {size: 1, from: 1} I get [ docB ] again.
I would expect that {size: 1, from: 0} would return [ docA ] ?

Both docA and docB have a _score of 1. The query orders descending on
_score first and then sorts ascending on a document field (string). The
sort field of document A should come before the sort field of document B.

Some details:

Number of shards: 1

Number of hits for the query: 2 (the 2 documents are the only hits)

There is no indexing going on in the background

Total size of the index: ~2000 documents

The query:
{
"query": {
"filtered": {
"query": {
"query_string": {
"fields": [ "q_high^2.0", "q_low^0.75" ],
"query": "*"
}
},
"filter": {
"and": [
{ "term": { "_type": "resource" } },
{ "term": { "resourceType": "group" } },
{ "ids": { "values": [ "g:camtest:groupChild-PTmmhkau0kf",
"g:camtest:groupParent-TP9mWsIp1k5" ] } }
]
}
}
},
"from": 0,
"size": 2,
"sort": [
{ "_score": { "order": "desc" } },
{ "sortField": { "order": "asc" } }
],
"min_score": 0.2,
"explain": true
}

Kind regards,

Simon

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Karel_Minarik_2 · May 21, 2013, 7:08am

Just a note that Tire, the Ruby client, has quite extensive integration
test suite for
pagination: https://github.com/karmi/tire/blob/master/test/integration/active_record_searchable_test.rb#L191,
which is passing even on 0.90, so this might be highly specific to some
conditions.

Karel

On Monday, May 20, 2013 10:19:39 PM UTC+2, simonw wrote:

hey,

I assume this is related to
BytesRefOrdValComparator ignores highest value in a segment during binarySearch · Issue #2991 · elastic/elasticsearch · GitHub
which is essentially a sorting bug. Would you have the spare cycles to try
if the latest master fixes this problem (or maybe try branch 0.90)

simon

On Monday, May 20, 2013 7:19:09 PM UTC+2, Simon Gaeremynck wrote:

Hello,

I'm seeing the following paging issue on 0.90 that we weren't seeing on
0.20.6

I have a query with size set to 2 and from set to 0 that returns the
documents: [ docA, docB ]
When I change the size to 1 the result set is [ docB ]
When I query with {size: 1, from: 1} I get [ docB ] again.
I would expect that {size: 1, from: 0} would return [ docA ] ?

Both docA and docB have a _score of 1. The query orders descending on
_score first and then sorts ascending on a document field (string). The
sort field of document A should come before the sort field of document B.

Some details:

Number of shards: 1

Number of hits for the query: 2 (the 2 documents are the only hits)

There is no indexing going on in the background

Total size of the index: ~2000 documents

The query:
{
"query": {
"filtered": {
"query": {
"query_string": {
"fields": [ "q_high^2.0", "q_low^0.75" ],
"query": "*"
}
},
"filter": {
"and": [
{ "term": { "_type": "resource" } },
{ "term": { "resourceType": "group" } },
{ "ids": { "values": [ "g:camtest:groupChild-PTmmhkau0kf",
"g:camtest:groupParent-TP9mWsIp1k5" ] } }
]
}
}
},
"from": 0,
"size": 2,
"sort": [
{ "_score": { "order": "desc" } },
{ "sortField": { "order": "asc" } }
],
"min_score": 0.2,
"explain": true
}

Kind regards,

Simon

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Simon_Gaeremynck · May 21, 2013, 7:39am

Hi Simon,

Just verified that master (e0825686f323bcb9bb0348ce61068e4af1763316) fixes the issue we're seeing.

Good stuff

Simon

On 20 May 2013, at 21:19, simonw simon.willnauer@elasticsearch.com wrote:

hey,

I assume this is related to BytesRefOrdValComparator ignores highest value in a segment during binarySearch · Issue #2991 · elastic/elasticsearch · GitHub
which is essentially a sorting bug. Would you have the spare cycles to try if the latest master fixes this problem (or maybe try branch 0.90)

simon

On Monday, May 20, 2013 7:19:09 PM UTC+2, Simon Gaeremynck wrote:
Hello,

I'm seeing the following paging issue on 0.90 that we weren't seeing on 0.20.6

I have a query with size set to 2 and from set to 0 that returns the documents: [ docA, docB ]
When I change the size to 1 the result set is [ docB ]
When I query with {size: 1, from: 1} I get [ docB ] again.
I would expect that {size: 1, from: 0} would return [ docA ] ?

Both docA and docB have a _score of 1. The query orders descending on _score first and then sorts ascending on a document field (string). The sort field of document A should come before the sort field of document B.

Some details:

Number of shards: 1

Number of hits for the query: 2 (the 2 documents are the only hits)

There is no indexing going on in the background

Total size of the index: ~2000 documents

The query:
{
"query": {
"filtered": {
"query": {
"query_string": {
"fields": [ "q_high^2.0", "q_low^0.75" ],
"query": "*"
}
},
"filter": {
"and": [
{ "term": { "_type": "resource" } },
{ "term": { "resourceType": "group" } },
{ "ids": { "values": [ "g:camtest:groupChild-PTmmhkau0kf", "g:camtest:groupParent-TP9mWsIp1k5" ] } }
]
}
}
},
"from": 0,
"size": 2,
"sort": [
{ "_score": { "order": "desc" } },
{ "sortField": { "order": "asc" } }
],
"min_score": 0.2,
"explain": true
}

Kind regards,

Simon

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/4jkFmvucijY/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

simonw_2 · May 21, 2013, 7:52am

perfect! thanks for verifying!

simon

On Tuesday, May 21, 2013 9:39:20 AM UTC+2, Simon Gaeremynck wrote:

Hi Simon,

Just verified that master (e0825686f323bcb9bb0348ce61068e4af1763316) fixes
the issue we're seeing.

Good stuff

Simon

On 20 May 2013, at 21:19, simonw <simon.w...@elasticsearch.com<javascript:>>
wrote:

hey,

I assume this is related to
BytesRefOrdValComparator ignores highest value in a segment during binarySearch · Issue #2991 · elastic/elasticsearch · GitHub
which is essentially a sorting bug. Would you have the spare cycles to
try if the latest master fixes this problem (or maybe try branch 0.90)

simon

On Monday, May 20, 2013 7:19:09 PM UTC+2, Simon Gaeremynck wrote:
Hello,

I'm seeing the following paging issue on 0.90 that we weren't seeing on
0.20.6

I have a query with size set to 2 and from set to 0 that returns the
documents: [ docA, docB ]
When I change the size to 1 the result set is [ docB ]
When I query with {size: 1, from: 1} I get [ docB ] again.
I would expect that {size: 1, from: 0} would return [ docA ] ?

Both docA and docB have a _score of 1. The query orders descending on
_score first and then sorts ascending on a document field (string). The
sort field of document A should come before the sort field of document B.

Some details:

Number of shards: 1

Number of hits for the query: 2 (the 2 documents are the only hits)

There is no indexing going on in the background

Total size of the index: ~2000 documents

The query:
{
"query": {
"filtered": {
"query": {
"query_string": {
"fields": [ "q_high^2.0", "q_low^0.75" ],
"query": "*"
}
},
"filter": {
"and": [
{ "term": { "_type": "resource" } },
{ "term": { "resourceType": "group" } },
{ "ids": { "values": [ "g:camtest:groupChild-PTmmhkau0kf",
"g:camtest:groupParent-TP9mWsIp1k5" ] } }
]
}
}
},
"from": 0,
"size": 2,
"sort": [
{ "_score": { "order": "desc" } },
{ "sortField": { "order": "asc" } }
],
"min_score": 0.2,
"explain": true
}

Kind regards,

Simon

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/4jkFmvucijY/unsubscribe?hl=en-US.

To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Topic		Replies	Views
Getting inconsistent records while using From + Size pagination approach Elasticsearch	1	666	August 23, 2018
Inconsistent paging Elasticsearch	4	1774	July 6, 2017
Pagination Elastic Search Elasticsearch	8	633	March 12, 2019
Inconsistent paging Elasticsearch	2	347	July 6, 2017
Changing size parameter changes ordering Elasticsearch	4	365	July 6, 2017

Paging on 0.90

Related topics