Clear index before update

Elastic Stack Logstash
1

Hello,

I use Logstash to feed my ES from a MysQL database. Everything seems to work so far but the update. When a row is deleted on the MySQL database, the record is still present on the ES side. Is there any solution like a parameter in my Logstash conf file to clean the index before recreating it ?

Many thanks in advance !!!

Anne

1 Like
2

No, there's no such option. And even if it did, the index would be incomplete for the duration of the reindexing. What you need to do is create a new index and delete the old one. Index aliases can help here.

1 Like
3

Thanks for your clear and fast reply !

That's what I did in my crontab :
# Run this command every day at 01:50 AM
# Delete the whole index of elasticsearch / INDEX
50 1 * * * root curl -X DELETE http://localhost:9200/INDEX/
# Run this command every day at 01:55 AM
# Recreate an index called 'INDEX' in elasticsearch and set the mapping
55 1 * * * root curl -X PUT http://localhost:9200/INDEX -d @/path/to/my/mapping/file/mapping.json
# Run this command every day at 02:00 AM
# Execute the logstash command to get data from my MySQL database as put in the conf file
0 2 * * * root /path/to/my/logstash -f /path/to/my/logstash/conf/file/get-data.conf

Many thanks !

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.