Reindexing docs that don’t yet exist in new index only (by doc id)

I need to reindex some indices so that it will change some object mapping to nested. I plan on setting the doc IDs to be the same in the source index and destination index. The challenge is that the source indices will continue to be written to, so I am trying to see if it’s possible to only index documents that don’t exist in the destination.

One way that I thought of doing is adding a flag on the source docs once the reindex is done, and then use this flag in subsequent reindex as a query to filter the docs that had already been reindexed previously.

I would like to know if there is a simpler way to handle this that I am unaware of, including potentially a simple script that check for existence of a doc id in the destination index to skip the doc.

If you have a time field that flag the create time in your docment, you can filter the incremental document by this time field.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.