Dynamic template for nested object at arbitrary locations in document?

Elastic Stack Elasticsearch
1

Hi,

We have user defined data coming in that may have interspersed within its
data a "packet" of well structured data that we'd like to index as a nested
document.

so for example:

form1 = {
"name": "dan",
"location": "zambia",
"CASEBLOCK": {
"item_id": "asfsaf",
"create": {
etc, etc...
}
}

form2 = {
"name": "logistics",
"role": "foo",
"manifest": {
"customer1": {
"CASEBLOCK": ....
},
"customer2": {
"CASEBLOCK": ...
}
}
}

form3 = {
"foo": { "CASEBLOCK": ...},
"bar": { "hello": {"goodbye": {"CASEBLOCK": ... } } }
}

etc. etc.

1: Is there a dynamic_template matching definition that allows me to
capture all instances of CASEBOCK? I want to be able to do a nested query
on the caseblock to float up the parent doc.

I have little control in how customers might set up their document
structure, but I do know that that tag and the structure within that tag
are well defined. In the example for form1, I can capture that with a
dynamic template. However, form2 and form3 examples, I have little
knowledge ahead of time to know how to structure an arbitrary match rule.

I've experimented with making multiple dynamic templates with path_matches
like:

path_match: ".CASEBLOCK" => this is indexed
path_match: "..CASEBLOCK
path_match: "...CASEBLOCK
...

but the deeper objects don't get detected despite the overall mapping of
the form documents being set to dynamic:true

2: With the arbitrary depth and path, is it possible to have a query based
upon properties in the nested object be done without knowledge of its
depth? The documentation seems to suggest that knowing the full depth of a
nested object is required.

So instead of needing to do form1.CASEBLOCK AND
form2.manifest.customer1.CASEBLOCK etc etc. I just want to interrogate the
index and query CASEBLOCK.foo and see which parent docs have it. I won't
necessarily now the path's depth ahead of time...but i will know the
structured object's information.

I'm running 0.90.5

Thanks,

Dan

Thanks,

Dan

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Hi Dan,
if you switch to the ordinary "match" in the dynamic template, instead of
the "path_match", that matches on the field name regardless of its path.
Did you give it a try already?

Regarding your query, I would play around with "path":"just_name", which
allows you to collapse a field to the top-level of the json object, and
index it using just its name, instead of the whole path. Have a look here:

. That said, I'm not sure how this would work with nested documents, as I
haven't tried it myself. Maybe you can give it a shot and let us know how
it went?

On Saturday, October 19, 2013 7:06:48 AM UTC+2, dmyung wrote:

Hi,

We have user defined data coming in that may have interspersed within its
data a "packet" of well structured data that we'd like to index as a nested
document.

so for example:

form1 = {
"name": "dan",
"location": "zambia",
"CASEBLOCK": {
"item_id": "asfsaf",
"create": {
etc, etc...
}
}

form2 = {
"name": "logistics",
"role": "foo",
"manifest": {
"customer1": {
"CASEBLOCK": ....
},
"customer2": {
"CASEBLOCK": ...
}
}
}

form3 = {
"foo": { "CASEBLOCK": ...},
"bar": { "hello": {"goodbye": {"CASEBLOCK": ... } } }
}

etc. etc.

1: Is there a dynamic_template matching definition that allows me to
capture all instances of CASEBOCK? I want to be able to do a nested query
on the caseblock to float up the parent doc.

I have little control in how customers might set up their document
structure, but I do know that that tag and the structure within that tag
are well defined. In the example for form1, I can capture that with a
dynamic template. However, form2 and form3 examples, I have little
knowledge ahead of time to know how to structure an arbitrary match rule.

I've experimented with making multiple dynamic templates with path_matches
like:

path_match: ".CASEBLOCK" => this is indexed
path_match: "..CASEBLOCK
path_match: "...CASEBLOCK
...

but the deeper objects don't get detected despite the overall mapping of
the form documents being set to dynamic:true

2: With the arbitrary depth and path, is it possible to have a query based
upon properties in the nested object be done without knowledge of its
depth? The documentation seems to suggest that knowing the full depth of a
nested object is required.

So instead of needing to do form1.CASEBLOCK AND
form2.manifest.customer1.CASEBLOCK etc etc. I just want to interrogate the
index and query CASEBLOCK.foo and see which parent docs have it. I won't
necessarily now the path's depth ahead of time...but i will know the
structured object's information.

I'm running 0.90.5

Thanks,

Dan

Thanks,

Dan

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.