Elastic Search Connect to Mongo Database Server

[Gehan_Fernando]

basically, I know the difference between MongoDB and the ES. In my server I have installed MongoDB instance separately will name it as (MDB1) and I installed ES instance separately will name it as (ESS). Currently, I am working on document management system, daily storing millions different kind of document information in Mongo DB. I have browsed many websites but I didn't get my answer as expected.

My question is, I want to connect ESS to MDB1, coz I using ES to search and analyze the data. Is there have any method to connect ES to existing MongoDB instance. (I am using C#.net)

[mayya (Mayya Sharipova)]

There is no official connector from the Elasticsearch side. There are third-party connectors you can use. You can also do http bulk update queries to load data to ES.

[Gehan_Fernando]

Configure MongoDB on Elasticsearch

1. Install MongoDB
2. Configure MongoDB

net:
    bindIp: 127.0.0.1
    port: 27017
replication:
    oplogSizeMB: 1024
    replSetName: res0
storage:
    dbPath: C:\data\db\rs0
systemLog:
    timeStampFormat: iso8601-utc
    destination: file
    path: C:\data\log\rs0\mongod.log

• Create two file using above configuration save it as primary.cfg and secondary.cfg.

• Install as Service:

mongod --config "C:\Program Files\MongoDB\Server\3.6\primary.cfg" --install --serviceName "MongoDB_Primary" --serviceDisplayName "MongoDB Primary"
mongod --config "C:\Program Files\MongoDB\Server\3.6\secondary.cfg" --install --serviceName "MongoDB_Secondary" --serviceDisplayName "MongoDB Secondary"

• Start services (Primary and Secondary instances):

3. Connect to the primary database
4. Use rs.initiate() command
5. Use rs.add("127.0.0.1:27018") command
6. Use rs.config() command
7. Use rs.status() command
8. Install Java latest version and set java home path in variable environment
9. Install Elasticsearch
10. Go to Elasticsearch folder and open config folder
11. Set following configuration in elasticsearch.yml file:

cluster.name: home_server
node.name: gehan-pc
path.data: C:\Elasticsearch\data
path.logs: C:\Elasticsearch\logs
network.host: 127.0.0.1
network.bind_host: 0.0.0.0
http.port: 9200
http.cors.enabled : true
http.cors.allow-origin : "*"
http.cors.allow-methods : OPTIONS, HEAD, GET, POST, PUT, DELETE
http.cors.allow-headers : X-Requested-With,X-Auth-Token,Content-Type, Content-Length

12. Install following files:

pip install mongo-connector[elastic5]
pip install elastic2-doc-manager[elastic5]

13. Start Elasticsearch instance
14. Run this command:

mongo-connector --auto-commit-interval=0 -d elastic_doc_manager -t localhost:9200

15. Open http://localhost:9200/_cat/indices?v to check is mongo collections sync with Elasticsearch server

[dadoonet (David Pilato)]

Thank you @Gehan_Fernando for sharing your recipe. I updated the format a bit as this can be useful for other members. I hope I did not change anything important (specifically, I updated a bit the mongo config).