For example, we are having 2 billion documents in an index in ES and want to delete 1 billion documents based on some data from Oracle database?
So i have to copy data from Oracle 1 billion records and based on Primary Key value of Oracle - same present as Column in Index on ES, want to delete those records?
Please suggest any better option.
I can think of below options but not sure
Option1 - Delete 1000 documents by passing in Loops - but this will take huge time
Option2 - add all these documents 1 billion from Oracle to ES index with some new column as DELETE = YES. Based on this column DELETE = YES , take PK value and search the same in ES index - which in turn gets 2 billion records for Deletion. Delete the same.. This is just idea - but not sure how to achieve this?
Thanks in advance.