Hit result change using same query?

[lufeng1102 (Lufeng)]

Hi I meet an issue that get different results when query the indexing using same query sentence. The query sentence like this

ES and Kibana version is 5.4.1

---

and the index schema like this

---

part the first query result

{
"took": 5,
"timed_out": false,
"_shards": {
"total": 10,
"successful": 10,
"failed": 0
},
"hits": {
"total": 69,
"max_score": 31.639772,
"hits": [
{
"_index": "dui_resource",
"_type": "skill",
"_id": "100001049",
"_score": 3.5021076,
"_source": {
"skill_name": "panzz",
"skill_id": "100001049",
"create_user_id": 1000000024,
"create_time": 1498980589,
"create_user_name": "潘珍珍1",
"skill_desc": "",
"company_id": "2",
"team_id": "2",
"last_modify_time": 1498980589,
"skill_type": "1",
"version": "",
"skill_class": "1",
"attach_products": "",
"is_deleted": "false",
"email": "335998774@qq.com"
},
"highlight": {
"create_user_name": [
"潘珍珍1"
]
}
},
{
"_index": "dui_resource",
"_type": "skill",
"_id": "100001201",
"_score": 3.5021076,
"_source": {
"skill_name": "pzz音乐717",
"skill_id": "100001201",
"create_user_id": 1000000024,
"create_time": 1500257113,
"create_user_name": "潘珍珍1",
"skill_desc": "",
"company_id": "2",
"team_id": "2",
"last_modify_time": 1500257113,
"skill_type": "1",
"version": "",
"skill_class": "1",
"attach_products": "",
"is_deleted": "false",
"email": "335998774@qq.com"
},
"highlight": {
"create_user_name": [
"潘珍珍1"
]
}
},
{
"_index": "dui_resource",
"_type": "skill",
"_id": "100001365",
"_score": 3.5021076,
"_source": {
"skill_name": "testfromBuiltin",
"skill_id": "100001365",
"create_user_id": 1000000024,
"create_time": 1502099891,
"create_user_name": "潘珍珍1",
"skill_desc": "",
"company_id": "2",
"team_id": "2",
"last_modify_time": 1502100440,
"skill_type": "1",
"version": "[{state:2,publishTime:1502099917,version:1,result:1},{state:2,publishTime:1502099962,version:2,result:1},{state:2,publishTime:1502100440,version:3,result:1}]",
"skill_class": "1",
"attach_products": "",
"is_deleted": "false",
"email": "335998774@qq.com"
},
"highlight": {
"create_user_name": [
"潘珍珍1"
]
}
},

---

and the second query result is

{
"took": 51,
"timed_out": false,
"_shards": {
"total": 10,
"successful": 10,
"failed": 0
},
"hits": {
"total": 69,
"max_score": 31.914755,
"hits": [
{
"_index": "dui_resource",
"_type": "skill",
"_id": "100001201",
"_score": 3.4787264,
"_source": {
"skill_name": "pzz音乐717",
"skill_id": "100001201",
"create_user_id": 1000000024,
"create_time": 1500257113,
"create_user_name": "潘珍珍1",
"skill_desc": "",
"company_id": "2",
"team_id": "2",
"last_modify_time": 1500257113,
"skill_type": "1",
"version": "",
"skill_class": "1",
"attach_products": "",
"is_deleted": "false",
"email": "335998774@qq.com"
},
"highlight": {
"create_user_name": [
"潘珍珍1"
]
}
},
{
"_index": "dui_resource",
"_type": "skill",
"_id": "100001049",
"_score": 3.4787264,
"_source": {
"skill_name": "panzz",
"skill_id": "100001049",
"create_user_id": 1000000024,
"create_time": 1498980589,
"create_user_name": "潘珍珍1",
"skill_desc": "",
"company_id": "2",
"team_id": "2",
"last_modify_time": 1498980589,
"skill_type": "1",
"version": "",
"skill_class": "1",
"attach_products": "",
"is_deleted": "false",
"email": "335998774@qq.com"
},
"highlight": {
"create_user_name": [
"潘珍珍1"
]
}
},
{
"_index": "dui_resource",
"_type": "skill",
"_id": "100001365",
"_score": 3.4787264,
"_source": {
"skill_name": "testfromBuiltin",
"skill_id": "100001365",
"create_user_id": 1000000024,
"create_time": 1502099891,
"create_user_name": "潘珍珍1",
"skill_desc": "",
"company_id": "2",
"team_id": "2",
"last_modify_time": 1502100440,
"skill_type": "1",
"version": "[{state:2,publishTime:1502099917,version:1,result:1},{state:2,publishTime:1502099962,version:2,result:1},{state:2,publishTime:1502100440,version:3,result:1}]",
"skill_class": "1",
"attach_products": "",
"is_deleted": "false",
"email": "335998774@qq.com"
},
"highlight": {
"create_user_name": [
"潘珍珍1"
]
}
},

so these two queries hit different documents.

How can I get idempotent result using same query several times?

Thanks.

[abdon (Abdon Pijpelink)]

If you look closely at the two responses, you are hitting the same documents. The total number of hits is 69 in both cases; it's just the ordering of the results that's different.

The ordering is different because all documents have the same score. For documents with the same score, the order in which Elasticsearch returns them is basically undefined.

If you want to get a consistent ordering, you need to add a secondary sort order to your request. For example, you could use the documents' unique ID (_uid) as a secondary sort order. If you do that, documents that have the same score will be sorted on their ID.

To do so, add this clause to your search request:

"sort": [
    {
      "_score" : {
        "order" : "desc"
      }
    },{
      "_uid": {
        "order": "asc"
      }
    }
  ]

[lufeng1102 (Lufeng)]

Hi abdon.

The secondary sort works. Thanks for you quick reply and detailed analysis.