Hi everyone, I'm new to Elasticsearch.
Currently I'm trying to replicate my db from postgresql and transform it to Elasticsearch. I've tried using google datastream (since my db is deployed in cloudSql) to cloud storage, then using dataflow to ingest the output from datastream to Elasticsearch but I need to somehow transform the data from datastream output to csv or pubsub message since the template is only supported in csv / ingest from pubsub. I think it's possible by using cloud function but I feel like there will be too much process. I'm also looking at some psql extension like zombodb, but it's not fit with my need where I want to transform the data first before storing it to Elasticsearch.
Is there any suggestion or recommendation that I can look for ?
Thank you in advance