Deleting data deleted from the source in Elasticsearch

Elastic Stack Elasticsearch
1

Hi guys,
I transfer data between MSSQL and Elasticsearch using LogStash. I want the data deleted on the MSSQL side to be deleted in Elasticsearch. Is it possible?

Please say if more details are required.

2

The thing with DELETEs is that you would need to create a temporary table of things to DELETE in elasticsearch, scan that table on a regular basis and for each id, delete the document in elasticsearch and remove that id from the deletion table in MySQL.

So basically, I'd recommend modifying the application layer if possible and send/update/delete data to elasticsearch in the same "transaction" as you are sending your data to the database.

I shared most of my thoughts there: http://david.pilato.fr/blog/2015/05/09/advanced-search-for-your-legacy-application/

Have also a look at this "live coding" recording.

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.