DuplicateKeyError

Elastic Search
1

Hi

I'm getting an error while adding a document by POST to documents:
Exception: Unable to save document: Swiftype::Documents::DuplicateKeyError:

How's that even possible? If the ID already exists, wouldn't it just update the fields?

Greetings Pascal

2

So weird. It only happens on a few documents: 22 out of 2.2 millions. But why?

{
    "id": "taxirothorn.ch",
    "domain": "taxirothorn.ch",
    "timestamp": "2021-01-25T16:45:31.798315+01:00"
}

For example on this. Have no idea with what magic AppSearch creates the actual ID stored in ElasticSearch or how it keeps track in .ent-search-document-backend

3

Hi @pesc! Good question. Are you able to reproduce the error when trying to index the individual document? I've seen DuplicateKeyError issues when the same ID is used in multiple documents in the same batch before.

4

Yes, happens on single document indexing too

curl --location --request POST 'https://api.searchzone.ch/api/as/v1/engines/domains/documents' \
--header 'Authorization: Bearer private-?????' \
--header 'Content-Type: application/json' \
--data-raw '{
    "id": "taxirothorn.ch",
    "domain": "taxirothorn.ch",
    "timestamp": "2021-01-25T16:45:31.798315+01:00"
}'
[{"id":"taxirothorn.ch","errors":["Unable to save document"]}]
5

Fixed it by manually deleting the entries out of ElasticSearch directly:

.ent-search-document-backend-[index]-document_type_id-external_id-unique-constraint

Don't ask me how it's possible that this happens...

6

Hm, glad you got it worked out. Was the entry in that index recreated when you indexed the document successfully?

7

Yes, everything worked properly :+1:

1 Like