I need to get data from a Firebase database into Elasticsearch. I've considered using Firebase cloud functions, Flashlight and Google Pub/Sub. The big issue that I see with cloud functions and flashlight is reliability. If the write to elasticsearch fails due to a network outage, the service is down, etc. there isn't a way to retry the write and neither solution is persistent. Google Pub/Sub is persistent and can retry the write, but I'm concerned about message ordering. I can probably implement timestamp checking as a solution, but I feel like I'm reinventing the wheel.
Any chance any of you have tackled something like this before and can give me some suggestions on the best way to proceed?
Thanks!
Troy