Elasticsearch returns two documents with same ID but different versions

ES version: 5.3.1

Some documents returns twice, updated and previous version

rq /purchases/payments/_search?q=_id:AYKCmqWVUjd1dFdxujwy&version=true

Why it happens? How to fix it?

That is a very old version that has been EOL for a long time. I would recommend you upgrade as soon as possible.

I have never seen this happen. How many nodes do you have in the cluster? How are these configured? Are you by any chance using routing?

If you at some point used routing and got the value wrong, you could end up writing the same document to multiple shards. To see if this is the case you could run the query you used to get these results with a few different routing values and see if you at some point only get one of the results.

1 Like

Welcome to our community! :smiley:

Please don't post pictures of text, logs or code. They are difficult to read, impossible to search and replicate (if it's code), and some people may not be even able to see them :slight_smile:

Thanks for the answer

Nodes: 3
Indices: 47
Memory: 46GB / 78GB
Total Shards: 326
Unassigned Shards: 0
Documents: 536,778,749
Data: 962GB
Version: 5.3.1

26 GB on each node

What does your elasticsearch.yml file look like?

One of node, others is the same

node.name: node1
cluster.name: cluster
network.host: ip1
discovery.zen.ping.unicast.hosts: ["ip2", "ip3"]
discovery.zen.minimum_master_nodes: 2
path.repo: ["/opt/shares/es"]
http.cors.enabled: true
http.cors.allow-origin: "*"
indices.query.bool.max_clause_count: 10000
xpack.security.enabled: false

Additional: search results is not stable, some request returns two documents, some returns only one document with the latest version

Another thing you can try is to run the test with different preference settings, potentially combined with different routing values. This could show if there is a difference between primaries and replicas.

1 Like

Thank you, it works

preference=_primary

Only one actual document, stable

preference=_replica

Only two documents like first my message, stable

This could show if there is a difference between primaries and replicas

As I see, I have difference... Have I any chance to fix it?

If you reduce the number of replicas for the index to 0 and then set it back to the original value the replica will be created as a copy of the primary. Note that a lot of improvements have been made to resiliency in newer version to avoid issues like this.

1 Like

Yes, you definitely need to upgrade to a version of Elasticsearch that is supported.

1 Like

Thank you, I will try this. Upgrade to new version in the future planned. Tell me please, how to reduce the chance of this problem? Add more nodes?

You need to upgrade to improve resiliency for issues like this.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.