I'm updating the index with new/updated documents nightly. After each update, I perform a cleanup: getting all documents currently in the index and checking if they still should be present, based on the path field of each document.
During update, I calculate the document ID based on the document title - which means that the ID changes in case the title changes, creating a duplicate document. However, given the path has not changed, the duplicated document will never be cleaned up.
The first solution that pops into mind is to check if there are documents pointing to the same path, then delete the older one - but I need a timestamp for that. I was expecting elastic to keep a document creation timestamp but this doesn't seem to be present in the metadata through the app search API.
Is there anything I can do apart from adding a timestamp manually?
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.