Duplicate documents with exactly same index/type/id in ES5.4.2

My ES cluster has only one node. The "model" index has 5 primary shards and 1 replica shard.

After indexing several documents, I run the below query:

GET https://9.111.255.174:9200/model/cubes/_search

The response is as below.

{
  "took": 21,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 1,
    "hits": [
      {
        "_index": "model",
        "_type": "cubes",
        "_id": "slot_allocation_cube_1",
        "_score": 1,
        "_source": {
         ......
        }
      },
      {
        "_index": "model",
        "_type": "cubes",
        "_id": "slot_allocation_cube_1",
        "_score": 1,
        "_source": {
        ......
        }
      }
    ]
  }
}

We can see that there are two duplicated documents with the same "_id" as "slot_allocation_cube_1". I'm sure I didn't use customer routing during data indexing.

After investigation, I find the data is correct on shard 0. However, there is something wrong with shard 3.

I run the below query

GET https://9.111.255.174:9200/model/cubes/_search?preference=_shards: 3

The response is as below.

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 1,
    "hits": [
      {
        "_index": "model",
        "_type": "cubes",
        "_id": "slot_allocation_cube_1",
        "_score": 1,
        "_source": {
         ......
        }
      }
    ]
  }
}

However, when I specify the id and run the below query

GET https://9.111.255.174:9200/model/cubes/slot_allocation_cube_1?preference=_shards:3

It failed!!!

 {
      "error": {
        "root_cause": [
          {
            "type": "remote_transport_exception",
            "reason": "[client_jfp4][9.111.255.174:9300][indices:data/read/get[s]]"
          }
        ],
        "type": "null_pointer_exception",
        "reason": null
      },
      "status": 500
    }

It is really strange. Can someone help me on this? Thanks a million!!!

Could anyone help on this issue?

:sob::sob::sob::sob::sob::sob::sob::sob::sob::sob::sob:

What do the logs show when you get that 500 error?

What does the search show if you add ?explain=true to the end? It should show which shard it is hitting and more.

Can you show your index settings as well?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.