Mapping for objects with an arbitrary amount of properties

Elastic Stack Elasticsearch
1

i have a given document:

{
"foo": {}
}

where foo can have an arbitrary amount of properties. lets assume i will
import millions of documents into my index, in which each of foo's
properties do have other values.

that means my mapping which will be built dynamically will grow enormous.
is there any kind of way where i can tell elasticsearch something like

take everything you have in foo and just accept it as it is (or
stringify foo) without having a resulting million-lines-mapping???

or do i have to care by myself, before indexing documents?

if so, there a 2 solutions i think

  1. JSON-Stringify

JSON.stringify(obj.foo)

  1. map every property in foo into key/value pairs, and create an array of
    objects:

// object
{
"foo": [
{"key": "bar1", "value": "bar1's value"},
{"key": "bar2", "value": "bar2's value"}
]
}

// resulting mapping
{
"type": {
"properties": {
"foo": {
"properties": {
"key": {
"type": "string"
},
"value": {
"type": "string"
}
}
}
}
}
}

would you prefer then solution 1 or 2, and why?

appreciate your help!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1207c3c0-3fba-4b6e-9cfc-b501afca1614%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Not sure I understood completely the use case. Having real data could help.

That said, if you do solution 2 I think you will need to use nested documents to index sub documents in foo.
Nested docs are actual Lucene docs. So indexing
{
"foo": [
{"key": "bar1", "value": "bar1's value"},
{"key": "bar2", "value": "bar2's value"}
]
}

will result in 3 docs (the main one and one for each key/value pair).

Nested documents are fine but complicate then a little queries.

For example 1, could you illustrate how your document will look like?

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 28 novembre 2013 at 12:57:04, hereandnow (hereandnow78@googlemail.com) a écrit:

i have a given document:

{
"foo": {}
}

where foo can have an arbitrary amount of properties. lets assume i will import millions of documents into my index, in which each of foo's properties do have other values.

that means my mapping which will be built dynamically will grow enormous. is there any kind of way where i can tell elasticsearch something like

take everything you have in foo and just accept it as it is (or stringify foo) without having a resulting million-lines-mapping???

or do i have to care by myself, before indexing documents?

if so, there a 2 solutions i think

  1. JSON-Stringify

JSON.stringify(obj.foo)

  1. map every property in foo into key/value pairs, and create an array of objects:

// object
{
"foo": [
{"key": "bar1", "value": "bar1's value"},
{"key": "bar2", "value": "bar2's value"}
]
}

// resulting mapping
{
"type": {
"properties": {
"foo": {
"properties": {
"key": {
"type": "string"
},
"value": {
"type": "string"
}
}
}
}
}
}

would you prefer then solution 1 or 2, and why?

appreciate your help!

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1207c3c0-3fba-4b6e-9cfc-b501afca1614%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.5297b623.26f324ba.3e14%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.

3

thanks for your answer. i found a solution for this. i dont need to search
on 'foo' so i can just disable the field, to prevent bloating of my mapping

"foo": {
"type": "object",
"enabled": false
}

Am Donnerstag, 28. November 2013 12:57:01 UTC+1 schrieb hereandnow:

i have a given document:

{
"foo": {}
}

where foo can have an arbitrary amount of properties. lets assume i will
import millions of documents into my index, in which each of foo's
properties do have other values.

that means my mapping which will be built dynamically will grow enormous.
is there any kind of way where i can tell elasticsearch something like

take everything you have in foo and just accept it as it is (or
stringify foo) without having a resulting million-lines-mapping???

or do i have to care by myself, before indexing documents?

if so, there a 2 solutions i think

  1. JSON-Stringify

JSON.stringify(obj.foo)

  1. map every property in foo into key/value pairs, and create an array
    of objects:

// object
{
"foo": [
{"key": "bar1", "value": "bar1's value"},
{"key": "bar2", "value": "bar2's value"}
]
}

// resulting mapping
{
"type": {
"properties": {
"foo": {
"properties": {
"key": {
"type": "string"
},
"value": {
"type": "string"
}
}
}
}
}
}

would you prefer then solution 1 or 2, and why?

appreciate your help!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3df422a7-37d2-43d5-bb96-75865fe67740%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

thanks for your answer. i found a solution for this. i dont need to search
on 'foo' so i can just disable the field, to prevent bloating of my mapping

"foo": {
"type": "object",
"enabled": false
}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a008f1a0-90a2-4531-a1e9-bd7ee000eaf1%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.