Changing size parameter changes ordering

eric_2 · May 22, 2013, 10:33pm

I'm using the following search:

gist.github.com

https://gist.github.com/emarthinsen/5631387

gistfile1.sh

curl -X GET 'http://elasticsearch.local/listings/_search?pretty' -d '
{
  "from": 0,
  "size": 20000,

  "query": {
    "bool": {
      "must": [
        {"term": {"state":{"term":"active"}}},
        {"term": {"account_type":{"term":"free"}}},

This file has been truncated. show original

There are around 20,000 documents in the database - fairly small. This
specific query returns around 300 results.

I'm using a sort function that generates a hash off of a random string
combined with the document's id. It's entirely deterministic and should
only generate a different order if the salt changes.

When 'from" is set to zero and "size" is set to 10,000 (a number larger
than the total size of the index), then the first 10 documents show up in a
certain order. If I change "size" to 10, then I'd expect the documents to
show up in the same order as the first 10 documents from the query when
"size" was set to 10,000. As you can guess, it doesn't. The first 10
documents show up in completely different orders across the two searches.

Can anyone explain what's going on and how to fix it?

Regards,
Eric

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Aldo_Sarmiento · May 22, 2013, 11:26pm

Interestingly enough, I think we may be experiencing similar issues:

https://groups.google.com/forum/#!topic/elasticsearch/kOL2uPgaPm8

On Wednesday, May 22, 2013 3:33:40 PM UTC-7, er...@sortfolio.com wrote:

I'm using the following search:

gist:5631387 · GitHub

There are around 20,000 documents in the database - fairly small. This
specific query returns around 300 results.

I'm using a sort function that generates a hash off of a random string
combined with the document's id. It's entirely deterministic and should
only generate a different order if the salt changes.

When 'from" is set to zero and "size" is set to 10,000 (a number larger
than the total size of the index), then the first 10 documents show up in a
certain order. If I change "size" to 10, then I'd expect the documents to
show up in the same order as the first 10 documents from the query when
"size" was set to 10,000. As you can guess, it doesn't. The first 10
documents show up in completely different orders across the two searches.

Can anyone explain what's going on and how to fix it?

Regards,
Eric

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

eric_2 · May 23, 2013, 2:26am

Let's hope!

On Wednesday, May 22, 2013 7:26:13 PM UTC-4, Aldo Sarmiento wrote:

Interestingly enough, I think we may be experiencing similar issues:

Redirecting to Google Groups

On Wednesday, May 22, 2013 3:33:40 PM UTC-7, er...@sortfolio.com wrote:

I'm using the following search:

gist:5631387 · GitHub

There are around 20,000 documents in the database - fairly small. This
specific query returns around 300 results.

I'm using a sort function that generates a hash off of a random string
combined with the document's id. It's entirely deterministic and should
only generate a different order if the salt changes.

When 'from" is set to zero and "size" is set to 10,000 (a number larger
than the total size of the index), then the first 10 documents show up in a
certain order. If I change "size" to 10, then I'd expect the documents to
show up in the same order as the first 10 documents from the query when
"size" was set to 10,000. As you can guess, it doesn't. The first 10
documents show up in completely different orders across the two searches.

Can anyone explain what's going on and how to fix it?

Regards,
Eric

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

simonw_2 · May 23, 2013, 7:27am

On Thursday, May 23, 2013 12:33:40 AM UTC+2, er...@sortfolio.com wrote:

I'm using the following search:

gist:5631387 · GitHub

There are around 20,000 documents in the database - fairly small. This
specific query returns around 300 results.

I'm using a sort function that generates a hash off of a random string
combined with the document's id. It's entirely deterministic and should
only generate a different order if the salt changes.

When 'from" is set to zero and "size" is set to 10,000 (a number larger
than the total size of the index), then the first 10 documents show up in a
certain order. If I change "size" to 10, then I'd expect the documents to
show up in the same order as the first 10 documents from the query when
"size" was set to 10,000. As you can guess, it doesn't. The first 10
documents show up in completely different orders across the two searches.

Can anyone explain what's going on and how to fix it?

can you build a simple testcase / gist that reproduces the problem somehow?

simon

Regards,
Eric

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.