Integrating mongodb with elasticSearch

coys · May 2, 2012, 5:25am

Hi, I'm trying to integrate mongodb with elasticsearch and for some
reason I cant seem to get this working. I think there's something
messed up with the version number combination I'm using - mongodb-
linux 64 2.0.4 , elasticsearch 0.19.0 with the elasticsearch-river-
mongdb jar version being 1.1 and the mongo-java-driver version being
2.7.2. It seems to create the indices when I bring elasticsearch up
but I cant insert a record , following the example given by
richardwilly98, it just hangs and there's no response when i run
db.person.save(p) on the mongotest db. Could someone please tell me
where I'm going wrong and if you've got this working could you please
tell me the version numbers that you're using? Thanks!

lsx · May 2, 2012, 5:34am

Hello.
Please see the bad post of issue(which posted by me.)

github.com/richardwilly98/elasticsearch-river-mongodb

NoSuchElementException and no search result.

opened 10:48AM - 06 Apr 12 UTC

closed 02:30PM - 06 Apr 12 UTC

xunitc

Hi, I forgot something or did something wrong? there is no search result. I goog…le by "elasticsearch IndexMissingException" but can not solve it. These(1-5) are what I do: 1, bin/mongod --directoryperdb --dbpath=/var/data/db --logpath=/var/data/log/mongodb.log --fork 2, bin/elasticsearch 3, curl -XPUT 'http://localhost:9200/_river/mongodb/_meta' -d '{ "type": "mongodb", "mongodb": { "db": "testmongo", "collection": "person" }, "index": { "name": "mongoindex", "type": "person" } }' 4, bin/mongo use testmongo db.person.save({firstName: "John", lastName: "Doe"}) 5, curl -XGET "localhost:9200/testmongo/person/_search?q=firstName:John&pretty=true" And I got: { "error" : "IndexMissingException[[testmongo] missing]", "status" : 404 } When I tail the elasticsearch.log, there are some exceptions: java.util.NoSuchElementException at java.util.LinkedList$ListItr.next(LinkedList.java:715) at com.mongodb.DBCursor._next(DBCursor.java:453) at com.mongodb.DBCursor.next(DBCursor.java:533) at org.elasticsearch.river.mongodb.MongoDBRiver$Slurper.processFullCollection(MongoDBRiver.java:378) at org.elasticsearch.river.mongodb.MongoDBRiver$Slurper.run(MongoDBRiver.java:353) at java.lang.Thread.run(Thread.java:636) [2012-04-06 18:19:30,636][INFO ][river.mongodb ] [Rama-Tut] [mongodb][mongodb] No known previous slurping time for this collection My environment: debian 6 64bit openjdk 6 64bit mongodb 2.0.4 Linux 64-bit elasticsearch 0.19.2 and plugin -install elasticsearch/elasticsearch-mapper-attachments/1.2.0 plugin -install richardwilly98/elasticsearch-river-mongodb/1.1.0 So, what's my wrong please?

You will do 2 things:

open the oplog of mongodb.
indexname is 'mongoindex', not the example's 'testmongo'.

Good Luck.

2012/5/2 coys gautam719@gmail.com:

Hi, I'm trying to integrate mongodb with elasticsearch and for some
reason I cant seem to get this working. I think there's something
messed up with the version number combination I'm using - mongodb-
linux 64 2.0.4 , elasticsearch 0.19.0 with the elasticsearch-river-
mongdb jar version being 1.1 and the mongo-java-driver version being
2.7.2. It seems to create the indices when I bring elasticsearch up
but I cant insert a record , following the example given by
richardwilly98, it just hangs and there's no response when i run
db.person.save(p) on the mongotest db. Could someone please tell me
where I'm going wrong and if you've got this working could you please
tell me the version numbers that you're using? Thanks!

coys · May 2, 2012, 7:26am

Thanks for the reply! Sorry, but I'm still stuck:( I'm now using
elasticsearch 0-19.2 and mongodb 2.0.4. Thanks for the querying on
mongoindex tip! When I query on the mongoindex using curl it tells me that
5 shards were successfully created but I have 0 hits for firstname John
even though I inserted John Doe into mongotest/person. I'm running mongo in
--replSet mode and I think it's most probably a wrong file version
somewhere. The elasticsearch.log contains the following error
org.elasticsearch.common.collect.ComputationException: java.lang.NoClassDefFoundError:
org/elasticsearch/client/action/bulk/BulkRequestBuilder and says that
even though it could load the mongodb river it failed to create river
mongodb. Also, my plugins/river-mongodb directory has the following 2 files
which I manually added mongo-java-driver-2.7.2.jar and
elastic-river-mongodb-1.1.0-SNAPSHOT.jar. Thanks!

coys · May 2, 2012, 11:26pm

Just an update, I got this working now, I was using an older version of
mongo. 2.0.4 works for me, thanks

Jalpesh1 · May 31, 2017, 6:42am

Hi Coys,

Please help in integration of mongodb with elastic search.

Thanks & Regards,
Jalpesh

dadoonet · May 31, 2017, 6:54am

It's preferable to open a new question and if needed link to existing posts.