# ElasticSearch with MongoDB doesn't indexing big objects **URL:** \ **Category:** Elasticsearch\ **Created:** [March 20, 2013, 9:58am UTC](https://discuss.elastic.co/t/elasticsearch-with-mongodb-doesnt-indexing-big-objects/11218 "2013-03-20T09:58:41Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Martin\_Pernica\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/martin_pernica_2/32/2428_2.png) [@Martin\_Pernica\_2](https://discuss.elastic.co/u/Martin_Pernica_2)\ **Post date:** [March 20, 2013, 9:58am UTC](https://discuss.elastic.co/t/elasticsearch-with-mongodb-doesnt-indexing-big-objects/11218/1 "2013-03-20T09:58:41Z")
Hello, I trying to insert big object like: { "object": { "text": "Let's do it again!", "boolTest": false }, "type": "coolType", "tags": [ "" ], "subObject1": { "count": 0, "last3": [], "array": [] }, "subObject2": { "count": 0, "last3": [], "array": [] }, "subObject3": { "count": 0, "last3": [], "array": [] }, "usrID": "5141a5a4d8f3a79c09000001", "created": Date(1363527664000), "lastUpdate": Date(1363527664000) } But I getting error: org.elasticsearch.index.mapper.MapperParsingException: object mapping for [stream] tried to parse as object, but got EOF, has a concrete value been provided to it? at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:457) at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:486) at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:430) at org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:318) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:157) at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:533) at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:431) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) [2013-03-20 10:35:05,697][WARN ][org.elasticsearch.river.mongodb.MongoDBRiver$Indexer] failed to executefailure in bulk execution: [0]: index [stream], type [stream], id [514982c9b7f3bfbdb488ca81], message [MapperParsingException[object mapping for [stream] tried to parse as object, but got EOF, has a concrete value been provided to it?]] [2013-03-20 10:35:05,698][INFO ][org.elasticsearch.river.mongodb.MongoDBRiver$Indexer] Indexed 1 documents, 1 insertions 0, updates, 0 deletions, 0 documents per second Where can be problem please? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [March 20, 2013, 10:07am UTC](https://discuss.elastic.co/t/elasticsearch-with-mongodb-doesnt-indexing-big-objects/11218/2 "2013-03-20T10:07:58Z")
Hey, the "Date" type you are trying to index with created and lastUpdate timestamp fields does not exist for JSON (it is mongo db specific and has to be converted before indexing). If this is not your problem (or you run into more problems), can you provide a complete entity you are trying to index? I cannot reproduce your exception. On Wed, Mar 20, 2013 at 10:58 AM, Martin Pernica [pernica@pria.cz](mailto:pernica@pria.cz) wrote: > Hello, > > I trying to insert big object like: > > { > "object": { > "text": "Let's do it again!", > "boolTest": false > }, > "type": "coolType", > "tags": [ > "" > ], > "subObject1": { > "count": 0, > "last3": , > "array": > }, > "subObject2": { > "count": 0, > "last3": , > "array": > }, > "subObject3": { > "count": 0, > "last3": , > "array": > }, > "usrID": "5141a5a4d8f3a79c09000001", > "created": Date(1363527664000), > "lastUpdate": Date(1363527664000) > } > > But I getting error: > > org.elasticsearch.index.mapper.MapperParsingException: object mapping for > [stream] tried to parse as object, but got EOF, has a concrete value been > provided to it? at > org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:457) > at > org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:486) > at > org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:430) > at > org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:318) > at > org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:157) > at > org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:533) > at > org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:431) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:722) [2013-03-20 10:35:05,697][WARN > ][org.elasticsearch.river.mongodb.MongoDBRiver$Indexer] failed to > executefailure in bulk execution: [0]: index [stream], type [stream], id > [514982c9b7f3bfbdb488ca81], message [MapperParsingException[object mapping > for [stream] tried to parse as object, but got EOF, has a concrete value > been provided to it?]] [2013-03-20 10:35:05,698][INFO > ][org.elasticsearch.river.mongodb.MongoDBRiver$Indexer] Indexed 1 > documents, 1 insertions 0, updates, 0 deletions, 0 documents per second > > Where can be problem please? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Martin\_Pernica\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/martin_pernica_2/32/2428_2.png) [@Martin\_Pernica\_2](https://discuss.elastic.co/u/Martin_Pernica_2)\ **Post date:** [March 20, 2013, 10:19am UTC](https://discuss.elastic.co/t/elasticsearch-with-mongodb-doesnt-indexing-big-objects/11218/3 "2013-03-20T10:19:10Z")
Is there anyway how to convert Date into compatible type for ES, without changing application logic? Or what is the best practice to store date (with timezone and etc.) in document-based databases? Yeah, I trying to insert this entity now (some fields have changed name, I cannot provide exactly same entity): { "object": { "text": "A test", "type": "", "boolValue": true, "fileId": "", "link": "[http://static.php.net/www.php.net/images/php.gif](http://static.php.net/www.php.net/images/php.gif)", "author": "", "local": true }, "type": "someType", "tags": [ "" ], "subObject": { "count": 0, "last3": , "array": }, "subObject2": { "count": 0, "last3": , "array": }, "subObject3": { "count": 0, "last3": , "array": }, "usrID": "5141a5a4d8f3a79c09000001", "created": Date(1363556619000), "lastUpdate": Date(1363556619000) } Dne středa, 20. března 2013 11:07:58 UTC+1 Alexander Reelsen napsal(a): > Hey, > > the "Date" type you are trying to index with created and lastUpdate > timestamp fields does not exist for JSON (it is mongo db specific and has > to be converted before indexing). > If this is not your problem (or you run into more problems), can you > provide a complete entity you are trying to index? I cannot reproduce your > exception. > > On Wed, Mar 20, 2013 at 10:58 AM, Martin Pernica \ > > > wrote: > > > Hello, > > > > I trying to insert big object like: > > > > { > > "object": { > > "text": "Let's do it again!", > > "boolTest": false > > }, > > "type": "coolType", > > "tags": [ > > "" > > ], > > "subObject1": { > > "count": 0, > > "last3": , > > "array": > > }, > > "subObject2": { > > "count": 0, > > "last3": , > > "array": > > }, > > "subObject3": { > > "count": 0, > > "last3": , > > "array": > > }, > > "usrID": "5141a5a4d8f3a79c09000001", > > "created": Date(1363527664000), > > "lastUpdate": Date(1363527664000) > > } > > > > But I getting error: > > > > org.elasticsearch.index.mapper.MapperParsingException: object mapping for > > [stream] tried to parse as object, but got EOF, has a concrete value been > > provided to it? at > > org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:457) > > at > > org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:486) > > at > > org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:430) > > at > > org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:318) > > at > > org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:157) > > at > > org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:533) > > at > > org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:431) > > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > at java.lang.Thread.run(Thread.java:722) [2013-03-20 10:35:05,697][WARN > > ][org.elasticsearch.river.mongodb.MongoDBRiver$Indexer] failed to > > executefailure in bulk execution: [0]: index [stream], type [stream], id > > [514982c9b7f3bfbdb488ca81], message [MapperParsingException[object mapping > > for [stream] tried to parse as object, but got EOF, has a concrete value > > been provided to it?]] [2013-03-20 10:35:05,698][INFO > > ][org.elasticsearch.river.mongodb.MongoDBRiver$Indexer] Indexed 1 > > documents, 1 insertions 0, updates, 0 deletions, 0 documents per second > > > > Where can be problem please? > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [March 20, 2013, 11:39am UTC](https://discuss.elastic.co/t/elasticsearch-with-mongodb-doesnt-indexing-big-objects/11218/4 "2013-03-20T11:39:21Z")
Hey I have never tried it, but I guess the mongodb river might help you See [GitHub - richardwilly98/elasticsearch-river-mongodb: MongoDB River Plugin for ElasticSearch](https://github.com/richardwilly98/elasticsearch-river-mongodb/) On Wed, Mar 20, 2013 at 11:19 AM, Martin Pernica [pernica@pria.cz](mailto:pernica@pria.cz) wrote: > Is there anyway how to convert Date into compatible type for ES, without > changing application logic? Or what is the best practice to store date > (with timezone and etc.) in document-based databases? > Yeah, I trying to insert this entity now (some fields have changed name, I > cannot provide exactly same entity): > > { > "object": { > "text": "A test", > "type": "", > "boolValue": true, > "fileId": "", > "link": "[http://static.php.net/www.php.net/images/php.gif](http://static.php.net/www.php.net/images/php.gif)", > "author": "", > "local": true > }, > "type": "someType", > "tags": [ > "" > ], > "subObject": { > "count": 0, > "last3": , > "array": > }, > "subObject2": { > "count": 0, > "last3": , > "array": > }, > "subObject3": { > "count": 0, > "last3": , > "array": > }, > "usrID": "5141a5a4d8f3a79c09000001", > "created": Date(1363556619000), > "lastUpdate": Date(1363556619000) > } > > Dne středa, 20. března 2013 11:07:58 UTC+1 Alexander Reelsen napsal(a): > > > Hey, > > > > the "Date" type you are trying to index with created and lastUpdate > > timestamp fields does not exist for JSON (it is mongo db specific and has > > to be converted before indexing). > > If this is not your problem (or you run into more problems), can you > > provide a complete entity you are trying to index? I cannot reproduce your > > exception. > > > > On Wed, Mar 20, 2013 at 10:58 AM, Martin Pernica [per...@pria.cz](mailto:per...@pria.cz) wrote: > > > > > Hello, > > > > > > I trying to insert big object like: > > > > > > { > > > "object": { > > > "text": "Let's do it again!", > > > "boolTest": false > > > }, > > > "type": "coolType", > > > "tags": [ > > > "" > > > ], > > > "subObject1": { > > > "count": 0, > > > "last3": , > > > "array": > > > }, > > > "subObject2": { > > > "count": 0, > > > "last3": , > > > "array": > > > }, > > > "subObject3": { > > > "count": 0, > > > "last3": , > > > "array": > > > }, > > > "usrID": "5141a5a4d8f3a79c09000001", > > > "created": Date(1363527664000), > > > "lastUpdate": Date(1363527664000) > > > } > > > > > > But I getting error: > > > > > > org.elasticsearch.index.**mapper.MapperParsingException: object mapping > > > for [stream] tried to parse as object, but got EOF, has a concrete value > > > been provided to it? at org.elasticsearch.index.** > > > mapper.object.ObjectMapper.\*\*parse(ObjectMapper.java:457) at > > > org.elasticsearch.index.\*\*mapper.DocumentMapper.parse(\*\*DocumentMapper.java:486) > > > at org.elasticsearch.index.\*\*mapper.DocumentMapper.parse(\*\*DocumentMapper.java:430) > > > at org.elasticsearch.index.shard. **service.InternalIndexShard.** > > > prepareIndex(\*_InternalIndexShard.java:318) at > > > org.elasticsearch.action.bulk. **TransportShardBulkAction.** > > > shardOperationOnPrimary(TransportShardBulkAction.java:**157) at > > > org.elasticsearch.action.support.replication. > > > TransportShardReplicationOpera**tionAction$AsyncShardOperationAction. > > > \*performOnPrimary( **TransportShardReplicationOpera** tionAction.java:533) > > > at org.elasticsearch.action. **support.replication.** > > > TransportShardReplicationOperationAction$_ > > > AsyncShardOperationAction$1.\*\*run( **TransportShardReplicationOpera** tionAction.java:431) > > > at java.util.concurrent.\*\*ThreadPoolExecutor.runWorker(\*\*ThreadPoolExecutor.java:1145) > > > at java.util.concurrent.\*\*ThreadPoolExecutor$Worker.run(\*\*ThreadPoolExecutor.java:615) > > > at java.lang.Thread.run(Thread.\*\*java:722) [2013-03-20 > > > 10:35:05,697][WARN][org.elasticsearch.river.\*\*mongodb.MongoDBRiver$Indexer] > > > failed to executefailure in bulk execution: [0]: index [stream], type > > > [stream], id [514982c9b7f3bfbdb488ca81], message > > > [MapperParsingException[object mapping for [stream] tried to parse as > > > object, but got EOF, has a concrete value been provided to it?]] > > > [2013-03-20 10:35:05,698][INFO][org.elasticsearch.river.\*\*mongodb.MongoDBRiver$Indexer] > > > Indexed 1 documents, 1 insertions 0, updates, 0 deletions, 0 documents per > > > second > > > > > > Where can be problem please? > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to elasticsearc...@\*\*[googlegroups.com](http://googlegroups.com). > > > > > > For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out) > > > . > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:45am UTC](https://discuss.elastic.co/t/elasticsearch-with-mongodb-doesnt-indexing-big-objects/11218/5 "2017-07-06T02:45:38Z")