Using mongo connector to sync collections between mongo and ES 7

[gidgid]

Hi,
We're been using Mongo 3.4 and ES 2.4 and syncing them using Mongo Connector. We're finally upgrading our ES to 7.5. Reading online it seems that Mongo Connector doesnt work with versions that are higher than 5.x and so my questions are:

1. Can we still use Mongo Connector with the latest ES version (currently 7.5.2)?
2. If not, what are the recommended alternatives?

Thanks

[ropc (Romain Chanu)]

@gidgid, let me try to answer your questions.

1. Mongo Connector is not developed by Elastic - this is a third-party MongoDB Connector. As such, Elastic cannot provide any support. There is an existing issue requesting for support for Elasticsearch 7.x. You may want to collaborate there and subscribe to this issue for further updates.

2. You might come across the logtash-input-mongodb plugin which is also available in RubyGems. This is a community plugin and it is not supported by Elastic. From the GitHub repository, there is no activity since 2017 so I am not sure whether this would work with Logstash / Elasticsearch 7.x. You can refer to our support matrix for the list of plugins supported by Elastic.

3. Some other alternatives:

• Use MongoDB Connector for Apache Kafka and use Logstash with Kafka input plugin. This may be worth taking a look.
• I also came accross this Discuss post where a community user was using Logstash with the JDBC input plugin. However, this could come with its own set of challenges as described in the post.

I hope that helps.