Delete lines in Elasticsearch previous deleted in SQL DB

In my logstash file, I input my data from a mysql database and then do some transformations and insert in elasticsearch. I used the action update to always have the most updated information. However, if I delete some lines of mysql database, how I can delete those lines from elasticsearch?

Thank you,

You might be able to do it using an AFTER DELETE trigger in MySQL.

Thank you. However, from the side of logstash what can I do?

Well, it is possible. You could use an elasticsearch input to read all the documents, then use a jdbc_streaming to look them up in MySQL. If you find it in the DB then drop {} the event, otherwise send it to an elasticsearch output with the action option set to delete.

You would need to be careful around error handling. A database error will result in you deleting records from elasticsearch. It sounds really fragile to me.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.