"Unable to save document" while indexing document

Elastic Search
1

HI,
Using app search and Elasticsearch version 7.10
A few ids have got corrupted while indexing appsearch, now when i am trying to index them even v through Postman i get

[
    {
        "id": "123456a2-e619-43f9-98c2-a28282123456",
        "errors": [
            "Unable to save document"
        ]
    }
]

Below are the appsearch logs while saving:

{"ecs":{"version":"1.5.0"},"@timestamp":"2021-11-22T06:28:00.221+00:00","labels":{"index_date":"2021.11.22"},"message":"Exception: Unable to save document: Swiftype::Documents::DuplicateKeyError: Swiftype::Documents::DuplicateKeyError","error.stack_trace":"0: Swiftype::Documents::DuplicateKeyError (Swiftype::Documents::DuplicateKeyError)\n","error.type":"Swiftype::Documents::DuplicateKeyError","error.message":"Swiftype::Documents::DuplicateKeyError","tags":["7e89dca624189a8f6bb046b22f44a120"],"log.level":"ERROR","log.logger":"app-server","service.type":"enterprise-search","service.version":"7.10.0","process.name":"app-server","process.pid":37,"process.thread.id":48494}

I am not able to delete/patch/get by id for these ids , is there any way to fix it so i can freshly insert them again without deleting the engine.

Thank you,
Subhasis Dash
``

2

Hi @Subhasis_Dash !

I'm afraid that you have ran into a known issue for App Search versions before 7.12. The backing indexes for the documents can become inconsistent under specific circumstances in case documents are updated / deleted at nearly the same time from different processes.

In order to fix this issue, you can:

  1. Reindex documents into a fresh engine from the source.

  2. In case reindexing is not an option, you can remove the conflicting documents via the App Search API.

  3. If you're not able to delete them using App Search API, so you might need to delete them from the underlying Enterprise Search indices in Elasticsearch. There are 3 indices of interest:

  • .ent-search-engine-<ENGINE_ID> (engine-index)
  • .ent-search-document-backend-<ENGINE_ID> (document-backend)
  • .ent-search-document-backend-<ENGINE_ID>-document_type_id-external_id-unique-constraint (constraint-index)

To get the <ENGINE_ID> of a particular engine, query the .ent-search-actastic-engines_v<N> index and look for the record that contains the engine name to locate its id (pick the index with the highest out of the .ent-search-actastic-engines_v* indices).

Please take a snapshot of all the dot indices in the cluster before proceeding to ensure no data is lost in the process.

Version 7.12 of App Search fixes this issue, so you might want to upgrade to 7.12 to avoid any related issue in the future.

I hope this helps, let me know about the result!