Delete lines in Elasticsearch previous deleted in SQL DB

[Francisca_Lima]

Hello,
In my logstash file, I input my data from a mysql database and then do some transformations and insert in elasticsearch. I used the action update to always have the most updated information. However, if I delete some lines of mysql database, how I can delete those lines from elasticsearch?

Thank you,
Francisca

[Badger]

@Francisca_Lima

However, if I delete some lines of mysql database, how I can delete those lines from elasticsearch?

You might be able to do it using an AFTER DELETE trigger in MySQL.

[Francisca_Lima]

Thank you. However, from the side of logstash what can I do?

[Badger]

Well, it is possible. You could use an elasticsearch input to read all the documents, then use a jdbc_streaming to look them up in MySQL. If you find it in the DB then drop {} the event, otherwise send it to an elasticsearch output with the action option set to delete.

You would need to be careful around error handling. A database error will result in you deleting records from elasticsearch. It sounds really fragile to me.