ElasticSearch with MongoDB doesn't indexing big objects

Hello,

I trying to insert big object like:

{
"object": {
"text": "Let's do it again!",
"boolTest": false
},
"type": "coolType",
"tags": [
""
],
"subObject1": {
"count": 0,
"last3": [],
"array": []
},
"subObject2": {
"count": 0,
"last3": [],
"array": []
},
"subObject3": {
"count": 0,
"last3": [],
"array": []
},
"usrID": "5141a5a4d8f3a79c09000001",
"created": Date(1363527664000),
"lastUpdate": Date(1363527664000)
}

But I getting error:

org.elasticsearch.index.mapper.MapperParsingException: object mapping for
[stream] tried to parse as object, but got EOF, has a concrete value been
provided to it? at
org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:457)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:486)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:430)
at
org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:318)
at
org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:157)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:533)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:431)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722) [2013-03-20 10:35:05,697][WARN
][org.elasticsearch.river.mongodb.MongoDBRiver$Indexer] failed to
executefailure in bulk execution: [0]: index [stream], type [stream], id
[514982c9b7f3bfbdb488ca81], message [MapperParsingException[object mapping
for [stream] tried to parse as object, but got EOF, has a concrete value
been provided to it?]] [2013-03-20 10:35:05,698][INFO
][org.elasticsearch.river.mongodb.MongoDBRiver$Indexer] Indexed 1
documents, 1 insertions 0, updates, 0 deletions, 0 documents per second

Where can be problem please?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hey,

the "Date" type you are trying to index with created and lastUpdate
timestamp fields does not exist for JSON (it is mongo db specific and has
to be converted before indexing).
If this is not your problem (or you run into more problems), can you
provide a complete entity you are trying to index? I cannot reproduce your
exception.

On Wed, Mar 20, 2013 at 10:58 AM, Martin Pernica pernica@pria.cz wrote:

Hello,

I trying to insert big object like:

{
"object": {
"text": "Let's do it again!",
"boolTest": false
},
"type": "coolType",
"tags": [
""
],
"subObject1": {
"count": 0,
"last3": [],
"array": []
},
"subObject2": {
"count": 0,
"last3": [],
"array": []
},
"subObject3": {
"count": 0,
"last3": [],
"array": []
},
"usrID": "5141a5a4d8f3a79c09000001",
"created": Date(1363527664000),
"lastUpdate": Date(1363527664000)
}

But I getting error:

org.elasticsearch.index.mapper.MapperParsingException: object mapping for
[stream] tried to parse as object, but got EOF, has a concrete value been
provided to it? at
org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:457)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:486)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:430)
at
org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:318)
at
org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:157)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:533)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:431)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722) [2013-03-20 10:35:05,697][WARN
][org.elasticsearch.river.mongodb.MongoDBRiver$Indexer] failed to
executefailure in bulk execution: [0]: index [stream], type [stream], id
[514982c9b7f3bfbdb488ca81], message [MapperParsingException[object mapping
for [stream] tried to parse as object, but got EOF, has a concrete value
been provided to it?]] [2013-03-20 10:35:05,698][INFO
][org.elasticsearch.river.mongodb.MongoDBRiver$Indexer] Indexed 1
documents, 1 insertions 0, updates, 0 deletions, 0 documents per second

Where can be problem please?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Is there anyway how to convert Date into compatible type for ES, without
changing application logic? Or what is the best practice to store date
(with timezone and etc.) in document-based databases?
Yeah, I trying to insert this entity now (some fields have changed name, I
cannot provide exactly same entity):

{
"object": {
"text": "A test",
"type": "",
"boolValue": true,
"fileId": "",
"link": "http://static.php.net/www.php.net/images/php.gif",
"author": "",
"local": true
},
"type": "someType",
"tags": [
""
],
"subObject": {
"count": 0,
"last3": [],
"array": []
},
"subObject2": {
"count": 0,
"last3": [],
"array": []
},
"subObject3": {
"count": 0,
"last3": [],
"array": []
},
"usrID": "5141a5a4d8f3a79c09000001",
"created": Date(1363556619000),
"lastUpdate": Date(1363556619000)
}

Dne středa, 20. března 2013 11:07:58 UTC+1 Alexander Reelsen napsal(a):

Hey,

the "Date" type you are trying to index with created and lastUpdate
timestamp fields does not exist for JSON (it is mongo db specific and has
to be converted before indexing).
If this is not your problem (or you run into more problems), can you
provide a complete entity you are trying to index? I cannot reproduce your
exception.

On Wed, Mar 20, 2013 at 10:58 AM, Martin Pernica <per...@pria.cz<javascript:>

wrote:

Hello,

I trying to insert big object like:

{
"object": {
"text": "Let's do it again!",
"boolTest": false
},
"type": "coolType",
"tags": [
""
],
"subObject1": {
"count": 0,
"last3": [],
"array": []
},
"subObject2": {
"count": 0,
"last3": [],
"array": []
},
"subObject3": {
"count": 0,
"last3": [],
"array": []
},
"usrID": "5141a5a4d8f3a79c09000001",
"created": Date(1363527664000),
"lastUpdate": Date(1363527664000)
}

But I getting error:

org.elasticsearch.index.mapper.MapperParsingException: object mapping for
[stream] tried to parse as object, but got EOF, has a concrete value been
provided to it? at
org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:457)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:486)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:430)
at
org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:318)
at
org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:157)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:533)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:431)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722) [2013-03-20 10:35:05,697][WARN
][org.elasticsearch.river.mongodb.MongoDBRiver$Indexer] failed to
executefailure in bulk execution: [0]: index [stream], type [stream], id
[514982c9b7f3bfbdb488ca81], message [MapperParsingException[object mapping
for [stream] tried to parse as object, but got EOF, has a concrete value
been provided to it?]] [2013-03-20 10:35:05,698][INFO
][org.elasticsearch.river.mongodb.MongoDBRiver$Indexer] Indexed 1
documents, 1 insertions 0, updates, 0 deletions, 0 documents per second

Where can be problem please?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hey

I have never tried it, but I guess the mongodb river might help you

See https://github.com/richardwilly98/elasticsearch-river-mongodb/

On Wed, Mar 20, 2013 at 11:19 AM, Martin Pernica pernica@pria.cz wrote:

Is there anyway how to convert Date into compatible type for ES, without
changing application logic? Or what is the best practice to store date
(with timezone and etc.) in document-based databases?
Yeah, I trying to insert this entity now (some fields have changed name, I
cannot provide exactly same entity):

{
"object": {
"text": "A test",
"type": "",
"boolValue": true,
"fileId": "",
"link": "http://static.php.net/www.php.net/images/php.gif",
"author": "",
"local": true
},
"type": "someType",
"tags": [
""
],
"subObject": {
"count": 0,
"last3": [],
"array": []
},
"subObject2": {
"count": 0,
"last3": [],
"array": []
},
"subObject3": {
"count": 0,
"last3": [],
"array": []
},
"usrID": "5141a5a4d8f3a79c09000001",
"created": Date(1363556619000),
"lastUpdate": Date(1363556619000)
}

Dne středa, 20. března 2013 11:07:58 UTC+1 Alexander Reelsen napsal(a):

Hey,

the "Date" type you are trying to index with created and lastUpdate
timestamp fields does not exist for JSON (it is mongo db specific and has
to be converted before indexing).
If this is not your problem (or you run into more problems), can you
provide a complete entity you are trying to index? I cannot reproduce your
exception.

On Wed, Mar 20, 2013 at 10:58 AM, Martin Pernica per...@pria.cz wrote:

Hello,

I trying to insert big object like:

{
"object": {
"text": "Let's do it again!",
"boolTest": false
},
"type": "coolType",
"tags": [
""
],
"subObject1": {
"count": 0,
"last3": [],
"array": []
},
"subObject2": {
"count": 0,
"last3": [],
"array": []
},
"subObject3": {
"count": 0,
"last3": [],
"array": []
},
"usrID": "5141a5a4d8f3a79c09000001",
"created": Date(1363527664000),
"lastUpdate": Date(1363527664000)
}

But I getting error:

org.elasticsearch.index.mapper.MapperParsingException: object mapping
for [stream] tried to parse as object, but got EOF, has a concrete value
been provided to it? at org.elasticsearch.index.

mapper.object.ObjectMapper.**parse(ObjectMapper.java:457) at
org.elasticsearch.index.**mapper.DocumentMapper.parse(**DocumentMapper.java:486)
at org.elasticsearch.index.**mapper.DocumentMapper.parse(**DocumentMapper.java:430)
at org.elasticsearch.index.shard.service.InternalIndexShard.
prepareIndex(*InternalIndexShard.java:318) at
org.elasticsearch.action.bulk.TransportShardBulkAction.
shardOperationOnPrimary(TransportShardBulkAction.java:157) at
org.elasticsearch.action.support.replication.
TransportShardReplicationOpera
tionAction$AsyncShardOperationAction.
*performOnPrimary(TransportShardReplicationOperationAction.java:533)
at org.elasticsearch.action.support.replication.
TransportShardReplicationOpera
tionAction$

AsyncShardOperationAction$1.**run(TransportShardReplicationOperationAction.java:431)
at java.util.concurrent.**ThreadPoolExecutor.runWorker(**ThreadPoolExecutor.java:1145)
at java.util.concurrent.**ThreadPoolExecutor$Worker.run(**ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.**java:722) [2013-03-20
10:35:05,697][WARN ][org.elasticsearch.river.**mongodb.MongoDBRiver$Indexer]
failed to executefailure in bulk execution: [0]: index [stream], type
[stream], id [514982c9b7f3bfbdb488ca81], message
[MapperParsingException[object mapping for [stream] tried to parse as
object, but got EOF, has a concrete value been provided to it?]]
[2013-03-20 10:35:05,698][INFO ][org.elasticsearch.river.**mongodb.MongoDBRiver$Indexer]
Indexed 1 documents, 1 insertions 0, updates, 0 deletions, 0 documents per
second

Where can be problem please?

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@**googlegroups.com.

For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.