# Scores and order differ after reindex
**URL:** https://discuss.elastic.co/t/scores-and-order-differ-after-reindex/14541
**Category:** Elasticsearch
**Created:** [November 22, 2013, 10:27am UTC](https://discuss.elastic.co/t/scores-and-order-differ-after-reindex/14541 "2013-11-22T10:27:47Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@Fluckx](https://discuss.elastic.co/u/Fluckx)
#### Post date: [November 22, 2013, 10:27am UTC](https://discuss.elastic.co/t/scores-and-order-differ-after-reindex/14541/1 "2013-11-22T10:27:47Z")
Hello,
I am currently writing a unittest to verify the response of the
elasticsearch. The reason for this is so we can run these same tests on
higher versions of elasticsearch to see if it's safe to upgrade.
The flow of the unittest:
1. Create a new index
2. Put the mapping
3. Insert data in bulk
4. flush index
5. optimize index to 1 segment
6. refresh index
7. perform queries and assertions
8. remove index
The problem is that every time i run these unittests they're unreliable
because the results i get return in different orders.
For example
First run of the unittest i would get the result of a query in this order:
Document A
Document B
Document C
Document D
Document E
Document F
The second run of the unittest ( immediately after ), the results are
something like this
Document A
Document B
Document D
Document C
Document E
Document F
The third run is something similar again.
If I look at the resultset i noticed the scores are different with each run
( keep in mind that every run it creates a new index ). The issue I have is
that when I recreate the same index 10 times and run my queries that
suddenly some items score higher than others. While the elasticsearch
version is the same and the data is the exact same ( it's a file that
contains all the bulk data ).
Anybody that can explain why this is or how i can get around this issue?
I'd assume that running a query on an index that is built the exact same
way 5 times should return the same results every time? Especially since i
flush - optimize and refresh. I assume all the documents are indexed.
The index isn't that big ( around 8000 documents ).
Extra information:
Version: 0.90.5
OS: linux
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [November 22, 2013, 1:23pm UTC](https://discuss.elastic.co/t/scores-and-order-differ-after-reindex/14541/2 "2013-11-22T13:23:47Z")
When you use the bulk, are you providing id for each doc?
Or are they auto generated?
I suppose that you have more than 1 shard for your index, right?
On a side note, you probably don't need to optimize your index.
--
David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com)
@dadoonet | @elasticsearchfr
Le 22 novembre 2013 at 11:27:50, Fluckx ([filip.van.houtryve@gmail.com](mailto:filip.van.houtryve@gmail.com)) a écrit:
Hello,
I am currently writing a unittest to verify the response of the elasticsearch. The reason for this is so we can run these same tests on higher versions of elasticsearch to see if it's safe to upgrade.
The flow of the unittest:
1. Create a new index
2. Put the mapping
3. Insert data in bulk
4. flush index
5. optimize index to 1 segment
6. refresh index
7. perform queries and assertions
8. remove index
The problem is that every time i run these unittests they're unreliable because the results i get return in different orders.
For example
First run of the unittest i would get the result of a query in this order:
Document A
Document B
Document C
Document D
Document E
Document F
The second run of the unittest ( immediately after ), the results are something like this
Document A
Document B
Document D
Document C
Document E
Document F
The third run is something similar again.
If I look at the resultset i noticed the scores are different with each run ( keep in mind that every run it creates a new index ). The issue I have is that when I recreate the same index 10 times and run my queries that suddenly some items score higher than others. While the elasticsearch version is the same and the data is the exact same ( it's a file that contains all the bulk data ).
Anybody that can explain why this is or how i can get around this issue? I'd assume that running a query on an index that is built the exact same way 5 times should return the same results every time? Especially since i flush - optimize and refresh. I assume all the documents are indexed.
The index isn't that big ( around 8000 documents ).
Extra information:
## Version: 0.90.5 OS: linux
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Fluckx](https://discuss.elastic.co/u/Fluckx)
#### Post date: [November 22, 2013, 2:54pm UTC](https://discuss.elastic.co/t/scores-and-order-differ-after-reindex/14541/3 "2013-11-22T14:54:25Z")
Hi David!
Thanks for the reply.
All the documents are indexed with their own id's ( not auto generated ).
This unittest runs it's queries on a single node with 1 shard ( The
production cluster has replication and multiple shards of course, but this
unittest just creates an index - inserts data - tries the queries - and
removes the index again ).
I have also discovered what the problem was. It's really stupid, but the
reason that some documents kept switching order is because they had the
exact same score.
So I decided to add a sort to the query so the return order is more
consistent.
On Friday, 22 November 2013 14:23:47 UTC+1, David Pilato wrote:
> When you use the bulk, are you providing id for each doc?
> Or are they auto generated?
>
> I suppose that you have more than 1 shard for your index, right?
>
> On a side note, you probably don't need to optimize your index.
>
> --
> _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_
> @dadoonet[https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Fdadoonet&sa=D&sntz=1&usg=AFQjCNE-DMC3YEu3X\_lhRIhUzuSZGsaSqA](https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Fdadoonet&sa=D&sntz=1&usg=AFQjCNE-DMC3YEu3X_lhRIhUzuSZGsaSqA)
> | @elasticsearchfr[https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Felasticsearchfr&sa=D&sntz=1&usg=AFQjCNGfXdQ98RWFMJXdiqpKnZb5GMg0zA](https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Felasticsearchfr&sa=D&sntz=1&usg=AFQjCNGfXdQ98RWFMJXdiqpKnZb5GMg0zA)
>
> Le 22 novembre 2013 at 11:27:50, Fluckx ([filip.van...@gmail.com](mailto:filip.van...@gmail.com)\)
> a écrit:
>
> Hello,
>
> I am currently writing a unittest to verify the response of the
> elasticsearch. The reason for this is so we can run these same tests on
> higher versions of elasticsearch to see if it's safe to upgrade.
>
> The flow of the unittest:
>
> 1. Create a new index
> 2. Put the mapping
> 3. Insert data in bulk
> 4. flush index
> 5. optimize index to 1 segment
> 6. refresh index
> 7. perform queries and assertions
> 8. remove index
>
> The problem is that every time i run these unittests they're unreliable
> because the results i get return in different orders.
> For example
>
> First run of the unittest i would get the result of a query in this order:
>
> Document A
> Document B
> Document C
> Document D
> Document E
> Document F
>
> The second run of the unittest ( immediately after ), the results are
> something like this
>
> Document A
> Document B
> Document D
> Document C
> Document E
> Document F
>
> The third run is something similar again.
>
> If I look at the resultset i noticed the scores are different with each
> run ( keep in mind that every run it creates a new index ). The issue I
> have is that when I recreate the same index 10 times and run my queries
> that suddenly some items score higher than others. While the elasticsearch
> version is the same and the data is the exact same ( it's a file that
> contains all the bulk data ).
>
> Anybody that can explain why this is or how i can get around this issue?
> I'd assume that running a query on an index that is built the exact same
> way 5 times should return the same results every time? Especially since i
> flush - optimize and refresh. I assume all the documents are indexed.
>
> The index isn't that big ( around 8000 documents ).
>
> Extra information:
>
> ## Version: 0.90.5 OS: linux
>
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [November 22, 2013, 3:01pm UTC](https://discuss.elastic.co/t/scores-and-order-differ-after-reindex/14541/4 "2013-11-22T15:01:11Z")
Ha! Thanks for the update.
--
David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com)
@dadoonet | @elasticsearchfr
Le 22 novembre 2013 at 15:54:32, Fluckx ([filip.van.houtryve@gmail.com](mailto:filip.van.houtryve@gmail.com)) a écrit:
Hi David!
Thanks for the reply.
All the documents are indexed with their own id's ( not auto generated ).
This unittest runs it's queries on a single node with 1 shard ( The production cluster has replication and multiple shards of course, but this unittest just creates an index - inserts data - tries the queries - and removes the index again ).
I have also discovered what the problem was. It's really stupid, but the reason that some documents kept switching order is because they had the exact same score.
So I decided to add a sort to the query so the return order is more consistent.
On Friday, 22 November 2013 14:23:47 UTC+1, David Pilato wrote:
When you use the bulk, are you providing id for each doc?
Or are they auto generated?
I suppose that you have more than 1 shard for your index, right?
On a side note, you probably don't need to optimize your index.
--
David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com)
@dadoonet | @elasticsearchfr
Le 22 novembre 2013 at 11:27:50, Fluckx ([filip.van...@gmail.com](mailto:filip.van...@gmail.com)) a écrit:
Hello,
I am currently writing a unittest to verify the response of the elasticsearch. The reason for this is so we can run these same tests on higher versions of elasticsearch to see if it's safe to upgrade.
The flow of the unittest:
1. Create a new index
2. Put the mapping
3. Insert data in bulk
4. flush index
5. optimize index to 1 segment
6. refresh index
7. perform queries and assertions
8. remove index
The problem is that every time i run these unittests they're unreliable because the results i get return in different orders.
For example
First run of the unittest i would get the result of a query in this order:
Document A
Document B
Document C
Document D
Document E
Document F
The second run of the unittest ( immediately after ), the results are something like this
Document A
Document B
Document D
Document C
Document E
Document F
The third run is something similar again.
If I look at the resultset i noticed the scores are different with each run ( keep in mind that every run it creates a new index ). The issue I have is that when I recreate the same index 10 times and run my queries that suddenly some items score higher than others. While the elasticsearch version is the same and the data is the exact same ( it's a file that contains all the bulk data ).
Anybody that can explain why this is or how i can get around this issue? I'd assume that running a query on an index that is built the exact same way 5 times should return the same results every time? Especially since i flush - optimize and refresh. I assume all the documents are indexed.
The index isn't that big ( around 8000 documents ).
Extra information:
## Version: 0.90.5 OS: linux
## You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Fluckx](https://discuss.elastic.co/u/Fluckx)
#### Post date: [November 29, 2013, 1:43pm UTC](https://discuss.elastic.co/t/scores-and-order-differ-after-reindex/14541/5 "2013-11-29T13:43:33Z")
Hello again.
I seem to be running into the same issue again. Unfortunately it's not as
simple as the sorting order this time.
It works most of the time, but occasionally the last two items switch order.
The \_score of both items also differ, but they're never very fart apart( up
to maximum 0.03 difference ). Occasionally they switch order because the
last item scores minimally higher than the item before it.
For clarity:
if i run my query multiple times on the same index - the scores don't
change. But since the index is recreated every time the unittest is run -
the scores do change ( which is a little weird i suppose ).
Elasticsearch version is still 0.90.5.
On Friday, 22 November 2013 16:01:11 UTC+1, David Pilato wrote:
> Ha! Thanks for the update.
>
> --
> _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_
> @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr)
>
> Le 22 novembre 2013 at 15:54:32, Fluckx ([filip.van...@gmail.com](mailto:filip.van...@gmail.com)\)
> a écrit:
>
> Hi David!
>
> Thanks for the reply.
>
> All the documents are indexed with their own id's ( not auto generated ).
>
> This unittest runs it's queries on a single node with 1 shard ( The
> production cluster has replication and multiple shards of course, but this
> unittest just creates an index - inserts data - tries the queries - and
> removes the index again ).
>
> I have also discovered what the problem was. It's really stupid, but the
> reason that some documents kept switching order is because they had the
> exact same score.
> So I decided to add a sort to the query so the return order is more
> consistent.
>
> On Friday, 22 November 2013 14:23:47 UTC+1, David Pilato wrote:
>
> > When you use the bulk, are you providing id for each doc?
> > Or are they auto generated?
> >
> > I suppose that you have more than 1 shard for your index, right?
> >
> > On a side note, you probably don't need to optimize your index.
> >
> > ```
> > --
> >
> > ```
> >
> > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_
> > @dadoonet[https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Fdadoonet&sa=D&sntz=1&usg=AFQjCNE-DMC3YEu3X\_lhRIhUzuSZGsaSqA](https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Fdadoonet&sa=D&sntz=1&usg=AFQjCNE-DMC3YEu3X_lhRIhUzuSZGsaSqA)
> > | @elasticsearchfr[https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Felasticsearchfr&sa=D&sntz=1&usg=AFQjCNGfXdQ98RWFMJXdiqpKnZb5GMg0zA](https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Felasticsearchfr&sa=D&sntz=1&usg=AFQjCNGfXdQ98RWFMJXdiqpKnZb5GMg0zA)
> >
> > Le 22 novembre 2013 at 11:27:50, Fluckx ([filip.van...@gmail.com](mailto:filip.van...@gmail.com)) a écrit:
> >
> > Hello,
> >
> > I am currently writing a unittest to verify the response of the
> > elasticsearch. The reason for this is so we can run these same tests on
> > higher versions of elasticsearch to see if it's safe to upgrade.
> >
> > The flow of the unittest:
> >
> > 1. Create a new index
> > 2. Put the mapping
> > 3. Insert data in bulk
> > 4. flush index
> > 5. optimize index to 1 segment
> > 6. refresh index
> > 7. perform queries and assertions
> > 8. remove index
> >
> > The problem is that every time i run these unittests they're unreliable
> > because the results i get return in different orders.
> > For example
> >
> > First run of the unittest i would get the result of a query in this order:
> >
> > Document A
> > Document B
> > Document C
> > Document D
> > Document E
> > Document F
> >
> > The second run of the unittest ( immediately after ), the results are
> > something like this
> >
> > Document A
> > Document B
> > Document D
> > Document C
> > Document E
> > Document F
> >
> > The third run is something similar again.
> >
> > If I look at the resultset i noticed the scores are different with each
> > run ( keep in mind that every run it creates a new index ). The issue I
> > have is that when I recreate the same index 10 times and run my queries
> > that suddenly some items score higher than others. While the elasticsearch
> > version is the same and the data is the exact same ( it's a file that
> > contains all the bulk data ).
> >
> > Anybody that can explain why this is or how i can get around this issue?
> > I'd assume that running a query on an index that is built the exact same
> > way 5 times should return the same results every time? Especially since i
> > flush - optimize and refresh. I assume all the documents are indexed.
> >
> > The index isn't that big ( around 8000 documents ).
> >
> > Extra information:
> >
> > ## Version: 0.90.5 OS: linux
> >
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/17100e17-6a7e-471a-a9da-37ecb01d1a47%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/17100e17-6a7e-471a-a9da-37ecb01d1a47%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [November 29, 2013, 1:53pm UTC](https://discuss.elastic.co/t/scores-and-order-differ-after-reindex/14541/6 "2013-11-29T13:53:25Z")
Are you indexing new documents in the meantime?
--
David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com)
@dadoonet | @elasticsearchfr
Le 29 novembre 2013 at 14:43:36, Fluckx ([filip.van.houtryve@gmail.com](mailto:filip.van.houtryve@gmail.com)) a écrit:
Hello again.
I seem to be running into the same issue again. Unfortunately it's not as simple as the sorting order this time.
It works most of the time, but occasionally the last two items switch order.
The \_score of both items also differ, but they're never very fart apart( up to maximum 0.03 difference ). Occasionally they switch order because the last item scores minimally higher than the item before it.
For clarity:
if i run my query multiple times on the same index - the scores don't change. But since the index is recreated every time the unittest is run - the scores do change ( which is a little weird i suppose ).
Elasticsearch version is still 0.90.5.
On Friday, 22 November 2013 16:01:11 UTC+1, David Pilato wrote:
Ha! Thanks for the update.
--
David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com)
@dadoonet | @elasticsearchfr
Le 22 novembre 2013 at 15:54:32, Fluckx ([filip.van...@gmail.com](mailto:filip.van...@gmail.com)) a écrit:
Hi David!
Thanks for the reply.
All the documents are indexed with their own id's ( not auto generated ).
This unittest runs it's queries on a single node with 1 shard ( The production cluster has replication and multiple shards of course, but this unittest just creates an index - inserts data - tries the queries - and removes the index again ).
I have also discovered what the problem was. It's really stupid, but the reason that some documents kept switching order is because they had the exact same score.
So I decided to add a sort to the query so the return order is more consistent.
On Friday, 22 November 2013 14:23:47 UTC+1, David Pilato wrote:
When you use the bulk, are you providing id for each doc?
Or are they auto generated?
I suppose that you have more than 1 shard for your index, right?
On a side note, you probably don't need to optimize your index.
--
David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com)
@dadoonet | @elasticsearchfr
Le 22 novembre 2013 at 11:27:50, Fluckx ([filip.van...@gmail.com](mailto:filip.van...@gmail.com)) a écrit:
Hello,
I am currently writing a unittest to verify the response of the elasticsearch. The reason for this is so we can run these same tests on higher versions of elasticsearch to see if it's safe to upgrade.
The flow of the unittest:
1. Create a new index
2. Put the mapping
3. Insert data in bulk
4. flush index
5. optimize index to 1 segment
6. refresh index
7. perform queries and assertions
8. remove index
The problem is that every time i run these unittests they're unreliable because the results i get return in different orders.
For example
First run of the unittest i would get the result of a query in this order:
Document A
Document B
Document C
Document D
Document E
Document F
The second run of the unittest ( immediately after ), the results are something like this
Document A
Document B
Document D
Document C
Document E
Document F
The third run is something similar again.
If I look at the resultset i noticed the scores are different with each run ( keep in mind that every run it creates a new index ). The issue I have is that when I recreate the same index 10 times and run my queries that suddenly some items score higher than others. While the elasticsearch version is the same and the data is the exact same ( it's a file that contains all the bulk data ).
Anybody that can explain why this is or how i can get around this issue? I'd assume that running a query on an index that is built the exact same way 5 times should return the same results every time? Especially since i flush - optimize and refresh. I assume all the documents are indexed.
The index isn't that big ( around 8000 documents ).
Extra information:
## Version: 0.90.5 OS: linux
## You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
## You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/17100e17-6a7e-471a-a9da-37ecb01d1a47%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/17100e17-6a7e-471a-a9da-37ecb01d1a47%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/etPan.52989c55.3f2dba31.a94c%40MacBook-Air-de-David.local](https://groups.google.com/d/msgid/elasticsearch/etPan.52989c55.3f2dba31.a94c%40MacBook-Air-de-David.local).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Fluckx](https://discuss.elastic.co/u/Fluckx)
#### Post date: [November 29, 2013, 2:35pm UTC](https://discuss.elastic.co/t/scores-and-order-differ-after-reindex/14541/7 "2013-11-29T14:35:52Z")
Hi David!
Thanks for the swift reply.
There are no documents being indexed in the meantime.
I generate a unique name for the index.
I am currently
1. Creating the index
2. Putting the mapping
3. Inserting the data from the bulkfile
4. flushing and refreshing the index ( $client-\>indices()-\>flush ( array(
'index' =\> $indexname,
'refresh' =\> true,
) );
this is all done in the setup of the unittest ( before any test is run )
and the index is removed after all tests are run.
On Friday, 29 November 2013 14:53:25 UTC+1, David Pilato wrote:
> Are you indexing new documents in the meantime?
>
> --
> _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_
> @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr)
>
> Le 29 novembre 2013 at 14:43:36, Fluckx ([filip.van...@gmail.com](mailto:filip.van...@gmail.com)\)
> a écrit:
>
> Hello again.
>
> I seem to be running into the same issue again. Unfortunately it's not as
> simple as the sorting order this time.
> It works most of the time, but occasionally the last two items switch
> order.
>
> The \_score of both items also differ, but they're never very fart apart(
> up to maximum 0.03 difference ). Occasionally they switch order because the
> last item scores minimally higher than the item before it.
>
> For clarity:
>
> if i run my query multiple times on the same index - the scores don't
> change. But since the index is recreated every time the unittest is run -
> the scores do change ( which is a little weird i suppose ).
>
> Elasticsearch version is still 0.90.5.
>
> On Friday, 22 November 2013 16:01:11 UTC+1, David Pilato wrote:
>
> > Ha! Thanks for the update.
> >
> > ```
> > --
> >
> > ```
> >
> > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_
> > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr)
> >
> > Le 22 novembre 2013 at 15:54:32, Fluckx ([filip.van...@gmail.com](mailto:filip.van...@gmail.com)) a écrit:
> >
> > Hi David!
> >
> > Thanks for the reply.
> >
> > All the documents are indexed with their own id's ( not auto generated ).
> >
> > This unittest runs it's queries on a single node with 1 shard ( The
> > production cluster has replication and multiple shards of course, but this
> > unittest just creates an index - inserts data - tries the queries - and
> > removes the index again ).
> >
> > I have also discovered what the problem was. It's really stupid, but the
> > reason that some documents kept switching order is because they had the
> > exact same score.
> > So I decided to add a sort to the query so the return order is more
> > consistent.
> >
> > On Friday, 22 November 2013 14:23:47 UTC+1, David Pilato wrote:
> >
> > > When you use the bulk, are you providing id for each doc?
> > > Or are they auto generated?
> > >
> > > I suppose that you have more than 1 shard for your index, right?
> > >
> > > On a side note, you probably don't need to optimize your index.
> > >
> > > ```
> > > --
> > >
> > > ```
> > >
> > > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_
> > > @dadoonet[https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Fdadoonet&sa=D&sntz=1&usg=AFQjCNE-DMC3YEu3X\_lhRIhUzuSZGsaSqA](https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Fdadoonet&sa=D&sntz=1&usg=AFQjCNE-DMC3YEu3X_lhRIhUzuSZGsaSqA)
> > > | @elasticsearchfr[https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Felasticsearchfr&sa=D&sntz=1&usg=AFQjCNGfXdQ98RWFMJXdiqpKnZb5GMg0zA](https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Felasticsearchfr&sa=D&sntz=1&usg=AFQjCNGfXdQ98RWFMJXdiqpKnZb5GMg0zA)
> > >
> > > Le 22 novembre 2013 at 11:27:50, Fluckx ([filip.van...@gmail.com](mailto:filip.van...@gmail.com)) a
> > > écrit:
> > >
> > > Hello,
> > >
> > > I am currently writing a unittest to verify the response of the
> > > elasticsearch. The reason for this is so we can run these same tests on
> > > higher versions of elasticsearch to see if it's safe to upgrade.
> > >
> > > The flow of the unittest:
> > >
> > > 1. Create a new index
> > > 2. Put the mapping
> > > 3. Insert data in bulk
> > > 4. flush index
> > > 5. optimize index to 1 segment
> > > 6. refresh index
> > > 7. perform queries and assertions
> > > 8. remove index
> > >
> > > The problem is that every time i run these unittests they're unreliable
> > > because the results i get return in different orders.
> > > For example
> > >
> > > First run of the unittest i would get the result of a query in this
> > > order:
> > >
> > > Document A
> > > Document B
> > > Document C
> > > Document D
> > > Document E
> > > Document F
> > >
> > > The second run of the unittest ( immediately after ), the results are
> > > something like this
> > >
> > > Document A
> > > Document B
> > > Document D
> > > Document C
> > > Document E
> > > Document F
> > >
> > > The third run is something similar again.
> > >
> > > If I look at the resultset i noticed the scores are different with each
> > > run ( keep in mind that every run it creates a new index ). The issue I
> > > have is that when I recreate the same index 10 times and run my queries
> > > that suddenly some items score higher than others. While the elasticsearch
> > > version is the same and the data is the exact same ( it's a file that
> > > contains all the bulk data ).
> > >
> > > Anybody that can explain why this is or how i can get around this issue?
> > > I'd assume that running a query on an index that is built the exact same
> > > way 5 times should return the same results every time? Especially since i
> > > flush - optimize and refresh. I assume all the documents are indexed.
> > >
> > > The index isn't that big ( around 8000 documents ).
> > >
> > > Extra information:
> > >
> > > ## Version: 0.90.5 OS: linux
> > >
> > > You received this message because you are subscribed to the Google
> > > Groups "elasticsearch" group.
> > > To unsubscribe from this group and stop receiving emails from it, send
> > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > >
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "elasticsearch" group.
> > > To unsubscribe from this group and stop receiving emails from it, send an
> > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > To view this discussion on the web visit
> > [https://groups.google.com/d/msgid/elasticsearch/17100e17-6a7e-471a-a9da-37ecb01d1a47%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/17100e17-6a7e-471a-a9da-37ecb01d1a47%40googlegroups.com)
> > .
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/ded45227-b50e-4fb1-9abc-488054ecad7e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/ded45227-b50e-4fb1-9abc-488054ecad7e%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Henrik\_Nordvik](https://discuss.elastic.co/u/Henrik_Nordvik)
#### Post date: [December 2, 2013, 9:23am UTC](https://discuss.elastic.co/t/scores-and-order-differ-after-reindex/14541/8 "2013-12-02T09:23:21Z")
Hi,
Are you using dfs\_query\_then\_fetch? It should help getting the same score.
> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.
If that doesn't help then it could he because lucene uses internal doc-ids
as tie-breaker when the scores are the same. Not sure what the best fix for
this is though.
>
>
> not really sure if this is a bug or not.
> the first part is:
> - is it normal t…hat shards have different number of max\_docs? what could cause that? a fast insertion + delete(that my guess is wont be replicated to the other shards). and of course, i guess that if they have different number of max docs, they most likely will also have different term freq and whatnot.
>
> the second, based on if the previous is true:
> - is it then possible to have a consistent sorting(based on score) with this scenario?
>
> i currently have for a simple match query, completely different result lists based on the shards that the query hits.
[http://web.archiveorange.com/archive/v/AAfXfnLZCbyQTykIeQWm](http://web.archiveorange.com/archive/v/AAfXfnLZCbyQTykIeQWm)
-
Henrik Nordvik
On Friday, November 29, 2013 3:35:52 PM UTC+1, Fluckx wrote:
> Hi David!
>
> Thanks for the swift reply.
> There are no documents being indexed in the meantime.
>
> I generate a unique name for the index.
>
> I am currently
>
> 1. Creating the index
> 2. Putting the mapping
> 3. Inserting the data from the bulkfile
> 4. flushing and refreshing the index ( $client-\>indices()-\>flush ( array(
> 'index' =\> $indexname,
> 'refresh' =\> true,
> ) );
>
> this is all done in the setup of the unittest ( before any test is run )
> and the index is removed after all tests are run.
>
> On Friday, 29 November 2013 14:53:25 UTC+1, David Pilato wrote:
>
> > Are you indexing new documents in the meantime?
> >
> > --
> > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_
> > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr)
> >
> > Le 29 novembre 2013 at 14:43:36, Fluckx ([filip.van...@gmail.com](mailto:filip.van...@gmail.com)) a écrit:
> >
> > Hello again.
> >
> > I seem to be running into the same issue again. Unfortunately it's not as
> > simple as the sorting order this time.
> > It works most of the time, but occasionally the last two items switch
> > order.
> >
> > The \_score of both items also differ, but they're never very fart apart(
> > up to maximum 0.03 difference ). Occasionally they switch order because the
> > last item scores minimally higher than the item before it.
> >
> > For clarity:
> >
> > if i run my query multiple times on the same index - the scores don't
> > change. But since the index is recreated every time the unittest is run -
> > the scores do change ( which is a little weird i suppose ).
> >
> > Elasticsearch version is still 0.90.5.
> >
> > On Friday, 22 November 2013 16:01:11 UTC+1, David Pilato wrote:
> >
> > > Ha! Thanks for the update.
> > >
> > > ```
> > > --
> > >
> > > ```
> > >
> > > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_
> > > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr)
> > >
> > > Le 22 novembre 2013 at 15:54:32, Fluckx ([filip.van...@gmail.com](mailto:filip.van...@gmail.com)) a
> > > écrit:
> > >
> > > Hi David!
> > >
> > > Thanks for the reply.
> > >
> > > All the documents are indexed with their own id's ( not auto generated ).
> > >
> > > This unittest runs it's queries on a single node with 1 shard ( The
> > > production cluster has replication and multiple shards of course, but this
> > > unittest just creates an index - inserts data - tries the queries - and
> > > removes the index again ).
> > >
> > > I have also discovered what the problem was. It's really stupid, but the
> > > reason that some documents kept switching order is because they had the
> > > exact same score.
> > > So I decided to add a sort to the query so the return order is more
> > > consistent.
> > >
> > > On Friday, 22 November 2013 14:23:47 UTC+1, David Pilato wrote:
> > >
> > > > When you use the bulk, are you providing id for each doc?
> > > > Or are they auto generated?
> > > >
> > > > I suppose that you have more than 1 shard for your index, right?
> > > >
> > > > On a side note, you probably don't need to optimize your index.
> > > >
> > > > ```
> > > > --
> > > >
> > > > ```
> > > >
> > > > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_
> > > > @dadoonet[https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Fdadoonet&sa=D&sntz=1&usg=AFQjCNE-DMC3YEu3X\_lhRIhUzuSZGsaSqA](https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Fdadoonet&sa=D&sntz=1&usg=AFQjCNE-DMC3YEu3X_lhRIhUzuSZGsaSqA)
> > > > | @elasticsearchfr[https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Felasticsearchfr&sa=D&sntz=1&usg=AFQjCNGfXdQ98RWFMJXdiqpKnZb5GMg0zA](https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Felasticsearchfr&sa=D&sntz=1&usg=AFQjCNGfXdQ98RWFMJXdiqpKnZb5GMg0zA)
> > > >
> > > > Le 22 novembre 2013 at 11:27:50, Fluckx ([filip.van...@gmail.com](mailto:filip.van...@gmail.com)) a
> > > > écrit:
> > > >
> > > > Hello,
> > > >
> > > > I am currently writing a unittest to verify the response of the
> > > > elasticsearch. The reason for this is so we can run these same tests on
> > > > higher versions of elasticsearch to see if it's safe to upgrade.
> > > >
> > > > The flow of the unittest:
> > > >
> > > > 1. Create a new index
> > > > 2. Put the mapping
> > > > 3. Insert data in bulk
> > > > 4. flush index
> > > > 5. optimize index to 1 segment
> > > > 6. refresh index
> > > > 7. perform queries and assertions
> > > > 8. remove index
> > > >
> > > > The problem is that every time i run these unittests they're unreliable
> > > > because the results i get return in different orders.
> > > > For example
> > > >
> > > > First run of the unittest i would get the result of a query in this
> > > > order:
> > > >
> > > > Document A
> > > > Document B
> > > > Document C
> > > > Document D
> > > > Document E
> > > > Document F
> > > >
> > > > The second run of the unittest ( immediately after ), the results are
> > > > something like this
> > > >
> > > > Document A
> > > > Document B
> > > > Document D
> > > > Document C
> > > > Document E
> > > > Document F
> > > >
> > > > The third run is something similar again.
> > > >
> > > > If I look at the resultset i noticed the scores are different with each
> > > > run ( keep in mind that every run it creates a new index ). The issue I
> > > > have is that when I recreate the same index 10 times and run my queries
> > > > that suddenly some items score higher than others. While the elasticsearch
> > > > version is the same and the data is the exact same ( it's a file that
> > > > contains all the bulk data ).
> > > >
> > > > Anybody that can explain why this is or how i can get around this
> > > > issue? I'd assume that running a query on an index that is built the exact
> > > > same way 5 times should return the same results every time? Especially
> > > > since i flush - optimize and refresh. I assume all the documents are
> > > > indexed.
> > > >
> > > > The index isn't that big ( around 8000 documents ).
> > > >
> > > > Extra information:
> > > >
> > > > ## Version: 0.90.5 OS: linux
> > > >
> > > > You received this message because you are subscribed to the Google
> > > > Groups "elasticsearch" group.
> > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > >
> > > > --
> > > > You received this message because you are subscribed to the Google
> > > > Groups "elasticsearch" group.
> > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > >
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "elasticsearch" group.
> > > To unsubscribe from this group and stop receiving emails from it, send an
> > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > To view this discussion on the web visit
> > > [https://groups.google.com/d/msgid/elasticsearch/17100e17-6a7e-471a-a9da-37ecb01d1a47%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/17100e17-6a7e-471a-a9da-37ecb01d1a47%40googlegroups.com)
> > > .
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/4bbd4cbd-0117-4b15-99ee-720a0cf45980%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4bbd4cbd-0117-4b15-99ee-720a0cf45980%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Fluckx](https://discuss.elastic.co/u/Fluckx)
#### Post date: [December 2, 2013, 10:12am UTC](https://discuss.elastic.co/t/scores-and-order-differ-after-reindex/14541/9 "2013-12-02T10:12:48Z")
Hey,
I doubt that will fix the problem. my queries are run on a node which only
has 1 shard and 0 replicas, so I don't think the query\_then\_fetch will
solve this. I will try it though. If it doesn't work I will try to post a
sample to further demonstrate my problem.
The weirdest thing for me is that the scores differ. If I create a file
with bulk inserts and I create 2 indexes and insert this bulk data into
both.
If I then run the same query against both indexes, shouldn't they return
the exact same result with the exact same scores? The data is identical and
both indexes are optimized and refreshed after the insert ( to make sure
there is no data falling behind ). The "scoring" algorithm has the same
information ( besides being created at a different time ).
On Monday, 2 December 2013 10:23:21 UTC+1, Henrik Nordvik wrote:
> Hi,
> Are you using dfs\_query\_then\_fetch? It should help getting the same score.
>
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/blog/understanding-query-then-fetch-vs-dfs-query-then-fetch/)
>
> If that doesn't help then it could he because lucene uses internal doc-ids
> as tie-breaker when the scores are the same. Not sure what the best fix for
> this is though.
> [inconsistent sorting by score due to differences between primary and replicas · Issue #3578 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/3578)
> [http://web.archiveorange.com/archive/v/AAfXfnLZCbyQTykIeQWm](http://web.archiveorange.com/archive/v/AAfXfnLZCbyQTykIeQWm)
>
> -
>
> Henrik Nordvik
>
> On Friday, November 29, 2013 3:35:52 PM UTC+1, Fluckx wrote:
>
> > Hi David!
> >
> > Thanks for the swift reply.
> > There are no documents being indexed in the meantime.
> >
> > I generate a unique name for the index.
> >
> > I am currently
> >
> > 1. Creating the index
> > 2. Putting the mapping
> > 3. Inserting the data from the bulkfile
> > 4. flushing and refreshing the index ( $client-\>indices()-\>flush ( array(
> > 'index' =\> $indexname,
> > 'refresh' =\> true,
> > ) );
> >
> > this is all done in the setup of the unittest ( before any test is run )
> > and the index is removed after all tests are run.
> >
> > On Friday, 29 November 2013 14:53:25 UTC+1, David Pilato wrote:
> >
> > > Are you indexing new documents in the meantime?
> > >
> > > --
> > > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_
> > > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr)
> > >
> > > Le 29 novembre 2013 at 14:43:36, Fluckx ([filip.van...@gmail.com](mailto:filip.van...@gmail.com)) a
> > > écrit:
> > >
> > > Hello again.
> > >
> > > I seem to be running into the same issue again. Unfortunately it's not
> > > as simple as the sorting order this time.
> > > It works most of the time, but occasionally the last two items switch
> > > order.
> > >
> > > The \_score of both items also differ, but they're never very fart apart(
> > > up to maximum 0.03 difference ). Occasionally they switch order because the
> > > last item scores minimally higher than the item before it.
> > >
> > > For clarity:
> > >
> > > if i run my query multiple times on the same index - the scores don't
> > > change. But since the index is recreated every time the unittest is run -
> > > the scores do change ( which is a little weird i suppose ).
> > >
> > > Elasticsearch version is still 0.90.5.
> > >
> > > On Friday, 22 November 2013 16:01:11 UTC+1, David Pilato wrote:
> > >
> > > > Ha! Thanks for the update.
> > > >
> > > > ```
> > > > --
> > > >
> > > > ```
> > > >
> > > > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_
> > > > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr)
> > > >
> > > > Le 22 novembre 2013 at 15:54:32, Fluckx ([filip.van...@gmail.com](mailto:filip.van...@gmail.com)) a
> > > > écrit:
> > > >
> > > > Hi David!
> > > >
> > > > Thanks for the reply.
> > > >
> > > > All the documents are indexed with their own id's ( not auto generated
> > > > ).
> > > >
> > > > This unittest runs it's queries on a single node with 1 shard ( The
> > > > production cluster has replication and multiple shards of course, but this
> > > > unittest just creates an index - inserts data - tries the queries - and
> > > > removes the index again ).
> > > >
> > > > I have also discovered what the problem was. It's really stupid, but
> > > > the reason that some documents kept switching order is because they had the
> > > > exact same score.
> > > > So I decided to add a sort to the query so the return order is more
> > > > consistent.
> > > >
> > > > On Friday, 22 November 2013 14:23:47 UTC+1, David Pilato wrote:
> > > >
> > > > > When you use the bulk, are you providing id for each doc?
> > > > > Or are they auto generated?
> > > > >
> > > > > I suppose that you have more than 1 shard for your index, right?
> > > > >
> > > > > On a side note, you probably don't need to optimize your index.
> > > > >
> > > > > ```
> > > > > --
> > > > >
> > > > > ```
> > > > >
> > > > > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_
> > > > > @dadoonet[https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Fdadoonet&sa=D&sntz=1&usg=AFQjCNE-DMC3YEu3X\_lhRIhUzuSZGsaSqA](https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Fdadoonet&sa=D&sntz=1&usg=AFQjCNE-DMC3YEu3X_lhRIhUzuSZGsaSqA)
> > > > > | @elasticsearchfr[https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Felasticsearchfr&sa=D&sntz=1&usg=AFQjCNGfXdQ98RWFMJXdiqpKnZb5GMg0zA](https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Felasticsearchfr&sa=D&sntz=1&usg=AFQjCNGfXdQ98RWFMJXdiqpKnZb5GMg0zA)
> > > > >
> > > > > Le 22 novembre 2013 at 11:27:50, Fluckx ([filip.van...@gmail.com](mailto:filip.van...@gmail.com)) a
> > > > > écrit:
> > > > >
> > > > > Hello,
> > > > >
> > > > > I am currently writing a unittest to verify the response of the
> > > > > elasticsearch. The reason for this is so we can run these same tests on
> > > > > higher versions of elasticsearch to see if it's safe to upgrade.
> > > > >
> > > > > The flow of the unittest:
> > > > >
> > > > > 1. Create a new index
> > > > > 2. Put the mapping
> > > > > 3. Insert data in bulk
> > > > > 4. flush index
> > > > > 5. optimize index to 1 segment
> > > > > 6. refresh index
> > > > > 7. perform queries and assertions
> > > > > 8. remove index
> > > > >
> > > > > The problem is that every time i run these unittests they're
> > > > > unreliable because the results i get return in different orders.
> > > > > For example
> > > > >
> > > > > First run of the unittest i would get the result of a query in this
> > > > > order:
> > > > >
> > > > > Document A
> > > > > Document B
> > > > > Document C
> > > > > Document D
> > > > > Document E
> > > > > Document F
> > > > >
> > > > > The second run of the unittest ( immediately after ), the results are
> > > > > something like this
> > > > >
> > > > > Document A
> > > > > Document B
> > > > > Document D
> > > > > Document C
> > > > > Document E
> > > > > Document F
> > > > >
> > > > > The third run is something similar again.
> > > > >
> > > > > If I look at the resultset i noticed the scores are different with
> > > > > each run ( keep in mind that every run it creates a new index ). The issue
> > > > > I have is that when I recreate the same index 10 times and run my queries
> > > > > that suddenly some items score higher than others. While the elasticsearch
> > > > > version is the same and the data is the exact same ( it's a file that
> > > > > contains all the bulk data ).
> > > > >
> > > > > Anybody that can explain why this is or how i can get around this
> > > > > issue? I'd assume that running a query on an index that is built the exact
> > > > > same way 5 times should return the same results every time? Especially
> > > > > since i flush - optimize and refresh. I assume all the documents are
> > > > > indexed.
> > > > >
> > > > > The index isn't that big ( around 8000 documents ).
> > > > >
> > > > > Extra information:
> > > > >
> > > > > ## Version: 0.90.5 OS: linux
> > > > >
> > > > > You received this message because you are subscribed to the Google
> > > > > Groups "elasticsearch" group.
> > > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > >
> > > > > --
> > > > > You received this message because you are subscribed to the Google
> > > > > Groups "elasticsearch" group.
> > > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > >
> > > > --
> > > > You received this message because you are subscribed to the Google
> > > > Groups "elasticsearch" group.
> > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > To view this discussion on the web visit
> > > > [https://groups.google.com/d/msgid/elasticsearch/17100e17-6a7e-471a-a9da-37ecb01d1a47%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/17100e17-6a7e-471a-a9da-37ecb01d1a47%40googlegroups.com)
> > > > .
> > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c748cf41-5b82-41bc-b2e4-7034eb513fb3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c748cf41-5b82-41bc-b2e4-7034eb513fb3%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Fluckx](https://discuss.elastic.co/u/Fluckx)
#### Post date: [December 2, 2013, 10:43am UTC](https://discuss.elastic.co/t/scores-and-order-differ-after-reindex/14541/10 "2013-12-02T10:43:45Z")
Hello again,
I noticed that my max number of documents and deleted documents differed. I
don't even know how this is possible ( concidering the same file should
yield the same results every time? ).
I optimized my index with `max_num_segments` set to 1 after inserting the
data.
Ever since the scoring has been "stable" across all the indexes.
If I have some spare time I should try and see why 1 bulkfile has 2
different amounts of deleted documents across indexes. It _does_ explain
why the scoring is different every time i queried and why it's now the same.
On Monday, 2 December 2013 11:12:48 UTC+1, Fluckx wrote:
> Hey,
>
> I doubt that will fix the problem. my queries are run on a node which only
> has 1 shard and 0 replicas, so I don't think the query\_then\_fetch will
> solve this. I will try it though. If it doesn't work I will try to post a
> sample to further demonstrate my problem.
>
> The weirdest thing for me is that the scores differ. If I create a file
> with bulk inserts and I create 2 indexes and insert this bulk data into
> both.
> If I then run the same query against both indexes, shouldn't they return
> the exact same result with the exact same scores? The data is identical and
> both indexes are optimized and refreshed after the insert ( to make sure
> there is no data falling behind ). The "scoring" algorithm has the same
> information ( besides being created at a different time ).
>
> On Monday, 2 December 2013 10:23:21 UTC+1, Henrik Nordvik wrote:
>
> > Hi,
> > Are you using dfs\_query\_then\_fetch? It should help getting the same score.
> >
> > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/blog/understanding-query-then-fetch-vs-dfs-query-then-fetch/)
> >
> > If that doesn't help then it could he because lucene uses internal
> > doc-ids as tie-breaker when the scores are the same. Not sure what the best
> > fix for this is though.
> > [inconsistent sorting by score due to differences between primary and replicas · Issue #3578 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/3578)
> > [http://web.archiveorange.com/archive/v/AAfXfnLZCbyQTykIeQWm](http://web.archiveorange.com/archive/v/AAfXfnLZCbyQTykIeQWm)
> >
> > -
> >
> > Henrik Nordvik
> >
> > On Friday, November 29, 2013 3:35:52 PM UTC+1, Fluckx wrote:
> >
> > > Hi David!
> > >
> > > Thanks for the swift reply.
> > > There are no documents being indexed in the meantime.
> > >
> > > I generate a unique name for the index.
> > >
> > > I am currently
> > >
> > > 1. Creating the index
> > > 2. Putting the mapping
> > > 3. Inserting the data from the bulkfile
> > > 4. flushing and refreshing the index ( $client-\>indices()-\>flush ( array(
> > > 'index' =\> $indexname,
> > > 'refresh' =\> true,
> > > ) );
> > >
> > > this is all done in the setup of the unittest ( before any test is run )
> > > and the index is removed after all tests are run.
> > >
> > > On Friday, 29 November 2013 14:53:25 UTC+1, David Pilato wrote:
> > >
> > > > Are you indexing new documents in the meantime?
> > > >
> > > > --
> > > > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_
> > > > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr)
> > > >
> > > > Le 29 novembre 2013 at 14:43:36, Fluckx ([filip.van...@gmail.com](mailto:filip.van...@gmail.com)) a
> > > > écrit:
> > > >
> > > > Hello again.
> > > >
> > > > I seem to be running into the same issue again. Unfortunately it's not
> > > > as simple as the sorting order this time.
> > > > It works most of the time, but occasionally the last two items switch
> > > > order.
> > > >
> > > > The \_score of both items also differ, but they're never very fart
> > > > apart( up to maximum 0.03 difference ). Occasionally they switch order
> > > > because the last item scores minimally higher than the item before it.
> > > >
> > > > For clarity:
> > > >
> > > > if i run my query multiple times on the same index - the scores don't
> > > > change. But since the index is recreated every time the unittest is run -
> > > > the scores do change ( which is a little weird i suppose ).
> > > >
> > > > Elasticsearch version is still 0.90.5.
> > > >
> > > > On Friday, 22 November 2013 16:01:11 UTC+1, David Pilato wrote:
> > > >
> > > > > Ha! Thanks for the update.
> > > > >
> > > > > ```
> > > > > --
> > > > >
> > > > > ```
> > > > >
> > > > > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_
> > > > > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr)
> > > > >
> > > > > Le 22 novembre 2013 at 15:54:32, Fluckx ([filip.van...@gmail.com](mailto:filip.van...@gmail.com)) a
> > > > > écrit:
> > > > >
> > > > > Hi David!
> > > > >
> > > > > Thanks for the reply.
> > > > >
> > > > > All the documents are indexed with their own id's ( not auto generated
> > > > > ).
> > > > >
> > > > > This unittest runs it's queries on a single node with 1 shard ( The
> > > > > production cluster has replication and multiple shards of course, but this
> > > > > unittest just creates an index - inserts data - tries the queries - and
> > > > > removes the index again ).
> > > > >
> > > > > I have also discovered what the problem was. It's really stupid, but
> > > > > the reason that some documents kept switching order is because they had the
> > > > > exact same score.
> > > > > So I decided to add a sort to the query so the return order is more
> > > > > consistent.
> > > > >
> > > > > On Friday, 22 November 2013 14:23:47 UTC+1, David Pilato wrote:
> > > > >
> > > > > > When you use the bulk, are you providing id for each doc?
> > > > > > Or are they auto generated?
> > > > > >
> > > > > > I suppose that you have more than 1 shard for your index, right?
> > > > > >
> > > > > > On a side note, you probably don't need to optimize your index.
> > > > > >
> > > > > > ```
> > > > > > --
> > > > > >
> > > > > > ```
> > > > > >
> > > > > > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_
> > > > > > @dadoonet[https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Fdadoonet&sa=D&sntz=1&usg=AFQjCNE-DMC3YEu3X\_lhRIhUzuSZGsaSqA](https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Fdadoonet&sa=D&sntz=1&usg=AFQjCNE-DMC3YEu3X_lhRIhUzuSZGsaSqA)
> > > > > > | @elasticsearchfr[https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Felasticsearchfr&sa=D&sntz=1&usg=AFQjCNGfXdQ98RWFMJXdiqpKnZb5GMg0zA](https://www.google.com/url?q=https%3A%2F%2Ftwitter.com%2Felasticsearchfr&sa=D&sntz=1&usg=AFQjCNGfXdQ98RWFMJXdiqpKnZb5GMg0zA)
> > > > > >
> > > > > > Le 22 novembre 2013 at 11:27:50, Fluckx ([filip.van...@gmail.com](mailto:filip.van...@gmail.com)) a
> > > > > > écrit:
> > > > > >
> > > > > > Hello,
> > > > > >
> > > > > > I am currently writing a unittest to verify the response of the
> > > > > > elasticsearch. The reason for this is so we can run these same tests on
> > > > > > higher versions of elasticsearch to see if it's safe to upgrade.
> > > > > >
> > > > > > The flow of the unittest:
> > > > > >
> > > > > > 1. Create a new index
> > > > > > 2. Put the mapping
> > > > > > 3. Insert data in bulk
> > > > > > 4. flush index
> > > > > > 5. optimize index to 1 segment
> > > > > > 6. refresh index
> > > > > > 7. perform queries and assertions
> > > > > > 8. remove index
> > > > > >
> > > > > > The problem is that every time i run these unittests they're
> > > > > > unreliable because the results i get return in different orders.
> > > > > > For example
> > > > > >
> > > > > > First run of the unittest i would get the result of a query in this
> > > > > > order:
> > > > > >
> > > > > > Document A
> > > > > > Document B
> > > > > > Document C
> > > > > > Document D
> > > > > > Document E
> > > > > > Document F
> > > > > >
> > > > > > The second run of the unittest ( immediately after ), the results are
> > > > > > something like this
> > > > > >
> > > > > > Document A
> > > > > > Document B
> > > > > > Document D
> > > > > > Document C
> > > > > > Document E
> > > > > > Document F
> > > > > >
> > > > > > The third run is something similar again.
> > > > > >
> > > > > > If I look at the resultset i noticed the scores are different with
> > > > > > each run ( keep in mind that every run it creates a new index ). The issue
> > > > > > I have is that when I recreate the same index 10 times and run my queries
> > > > > > that suddenly some items score higher than others. While the elasticsearch
> > > > > > version is the same and the data is the exact same ( it's a file that
> > > > > > contains all the bulk data ).
> > > > > >
> > > > > > Anybody that can explain why this is or how i can get around this
> > > > > > issue? I'd assume that running a query on an index that is built the exact
> > > > > > same way 5 times should return the same results every time? Especially
> > > > > > since i flush - optimize and refresh. I assume all the documents are
> > > > > > indexed.
> > > > > >
> > > > > > The index isn't that big ( around 8000 documents ).
> > > > > >
> > > > > > Extra information:
> > > > > >
> > > > > > ## Version: 0.90.5 OS: linux
> > > > > >
> > > > > > You received this message because you are subscribed to the Google
> > > > > > Groups "elasticsearch" group.
> > > > > > To unsubscribe from this group and stop receiving emails from it,
> > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > > >
> > > > > > --
> > > > > > You received this message because you are subscribed to the Google
> > > > > > Groups "elasticsearch" group.
> > > > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > >
> > > > > --
> > > > > You received this message because you are subscribed to the Google
> > > > > Groups "elasticsearch" group.
> > > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > To view this discussion on the web visit
> > > > > [https://groups.google.com/d/msgid/elasticsearch/17100e17-6a7e-471a-a9da-37ecb01d1a47%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/17100e17-6a7e-471a-a9da-37ecb01d1a47%40googlegroups.com)
> > > > > .
> > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e7a76307-6cf3-4a1f-8f1c-93229e766fe3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e7a76307-6cf3-4a1f-8f1c-93229e766fe3%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 2:03am UTC](https://discuss.elastic.co/t/scores-and-order-differ-after-reindex/14541/11 "2017-07-06T02:03:55Z")