I'm relatively new to Logstash, and even though I've browsed the web for a few days now, I can not seem to find out how my problem should be handled.
I have a database with roughly 35 million records. Every minute, around 500 updates happen on that database, and I want to keep those changes in ElasticSearch as soon as possible.
Right now (before I met LogStash) I did the following every 10 seconds:
- On update insert a row into a trigger table
- A process reading from that trigger table and update the values in ElasticSearch.
But I think using LogStash, it could go much faster and easier, although I have not found out how.
Could you guys/girls help me out?