How to delete huge number of documents from Index? What can be better option?

Elastic Stack Elasticsearch
1

For example, we are having 2 billion documents in an index in ES and want to delete 1 billion documents based on some data from Oracle database?

So i have to copy data from Oracle 1 billion records and based on Primary Key value of Oracle - same present as Column in Index on ES, want to delete those records?

Please suggest any better option.

I can think of below options but not sure

Option1 - Delete 1000 documents by passing in Loops - but this will take huge time

Option2 - add all these documents 1 billion from Oracle to ES index with some new column as DELETE = YES. Based on this column DELETE = YES , take PK value and search the same in ES index - which in turn gets 2 billion records for Deletion. Delete the same.. This is just idea - but not sure how to achieve this?

Thanks in advance.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.