Hi all,
I'm trying to solve the following problem: I have data in RDSs (Aurora MySQL), a subset of which I'm going to map to Elasticsearch using the AWS Database Migration Service. Unfortunately, we have full deployments in 4 AWS regions, but need to consolidate the output onto one Elasticsearch instance. The source databases aren't publicly accessible and AWS DMS doesn't support Elasticsearch targets in different regions, so I need to forward all the ES documents to a primary ES instance in one region. Here's a simplified diagram:
The forwarding ES doesn't need to keep anything; all searching will be done on the primary ES. What's the best way to consolidate documents from multiple Elasticsearch instances onto this single primary instance?
Thanks!