# Cannot search in sub-attribute **URL:** \ **Category:** Elasticsearch\ **Created:** [June 1, 2013, 2:18pm UTC](https://discuss.elastic.co/t/cannot-search-in-sub-attribute/12217 "2013-06-01T14:18:03Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Chris\_Kinsalb](https://avatars.discourse-cdn.com/v4/letter/c/e99b99/32.png) [@Chris\_Kinsalb](https://discuss.elastic.co/u/Chris_Kinsalb)\ **Post date:** [June 1, 2013, 2:18pm UTC](https://discuss.elastic.co/t/cannot-search-in-sub-attribute/12217/1 "2013-06-01T14:18:03Z")
Hello, I created a doc type which has several attributes including an attribute @fields configured like this: "@fields": { "type": "object", "index": "not\_analyzed", "dynamic": true, "path": "full" } Here is a sample input: { "@fields": { "hostname": "walter", "ip": "192.168.10.237", "user": { "firstname": "Donny", "lastname": "Kerabatsos" }, "message": "Shut the f\* up, Donny" } } Now I can search for a doc by referencing the hostname like this: { "query": { "filtered": { "filter": { "and": [ { "term": { "@fields.hostname": "walter" } } ] } } } } But this search yields\* no result:\* { "query": { "filtered": { "filter": { "and": [ { "term": { "@fields.user.firstname": "Donny" } } ] } } } } I don't understand why, but perhaps I misunderstood something about the object type. My impression was, that it is possible to insert an arbitrary json and perform searches on any of it's attributes (also "nested" attributes), due to the dynamic mapping. 1.) What do I have to do in order to make this work? 2.) I _DO not_ want to create a default schema for this, since the json data which is inserted into @fields changes (different hierarchies, attribute names, etc.). Thanks a lot & cheers, - Chris -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [June 1, 2013, 3:19pm UTC](https://discuss.elastic.co/t/cannot-search-in-sub-attribute/12217/2 "2013-06-01T15:19:40Z")
you should use the match query instead of the term query, since the internal fields ends up being analyzed. I see you try to set the index field on the object mapping to mark it as not\_analyzed, this will not work and is not supported (I think you assume that al internal mapping will inherit it, which they do not). if you want to achieve this, check into dynamic templates in the root object mappings. On Sat, Jun 1, 2013 at 4:18 PM, Chris Kinsalb [hc.kinsalb@gmail.com](mailto:hc.kinsalb@gmail.com) wrote: > Hello, > I created a doc type which has several attributes including an attribute > @fields configured like this: > "@fields": { > "type": "object", > "index": "not\_analyzed", > "dynamic": true, > "path": "full" > } > Here is a sample input: > { > "@fields": { > "hostname": "walter", > "ip": "192.168.10.237", > "user": { > "firstname": "Donny", > "lastname": "Kerabatsos" > }, > "message": "Shut the f\* up, Donny" > } > } > Now I can search for a doc by referencing the hostname like this: > { > "query": { > "filtered": { > "filter": { > "and": [ > { > "term": { > "@fields.hostname": "walter" > } > } > ] > } > } > } > } > But this search yields\* no result:\* > { > "query": { > "filtered": { > "filter": { > "and": [ > { > "term": { > "@fields.user.firstname": "Donny" > } > } > ] > } > } > } > } > I don't understand why, but perhaps I misunderstood something about the > object type. My impression was, that it is possible to insert an arbitrary > json and perform searches on any of it's attributes (also "nested" > attributes), due to the dynamic mapping. > 1.) What do I have to do in order to make this work? > 2.) I _DO not_ want to create a default schema for this, since the json > data which is inserted into @fields changes (different hierarchies, > attribute names, etc.). > Thanks a lot & cheers, > > - Chris > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Chris\_Kinsalb](https://avatars.discourse-cdn.com/v4/letter/c/e99b99/32.png) [@Chris\_Kinsalb](https://discuss.elastic.co/u/Chris_Kinsalb)\ **Post date:** [June 3, 2013, 6:18am UTC](https://discuss.elastic.co/t/cannot-search-in-sub-attribute/12217/3 "2013-06-03T06:18:54Z")
Hello Shay, thanks for the explanation. Yes, that is indeed what I thought. However, the problem is that I wanted my input to be totally schema-less, which I cannot seem to achieve using dynamic templates since I will not know what the input objects will look like. However, I do have an idea on how to achieve this: Using dynamic templates could I do something along these lines? ``` "dynamic_templates": [ { "fields_template": { "mapping": { "index": "not_analyzed" }, "match": "fields.*" } } ] ``` Now, whenever I get a json input I would transform it to a flat hierarchy, e.g.: input: { foo: 'bar', user: { firstname: 'bla', lastname: 'narf' } } Transform output: { fields.foo: 'bar', fields.user.firstname: 'bla', fields.user.lastname: 'narf' } This way: - each field will automatically have the correct type (e.g. string or integer) - I can run facet queries (statistics) on the fields Does this sound like a viable solution to you? (will try it out now) Cheers & regards, - Chris On Saturday, June 1, 2013 5:19:40 PM UTC+2, kimchy wrote: > you should use the match query instead of the term query, since the > internal fields ends up being analyzed. > > I see you try to set the index field on the object mapping to mark it as > not\_analyzed, this will not work and is not supported (I think you assume > that al internal mapping will inherit it, which they do not). if you want > to achieve this, check into dynamic templates in the root object mappings. > > On Sat, Jun 1, 2013 at 4:18 PM, Chris Kinsalb \<[hc.ki...@gmail.com](mailto:hc.ki...@gmail.com)\ > > > wrote: > > > Hello, > > > > I created a doc type which has several attributes including an attribute > > @fields configured like this: > > > > "@fields": { > > "type": "object", > > "index": "not\_analyzed", > > "dynamic": true, > > "path": "full" > > } > > > > Here is a sample input: > > > > { > > "@fields": { > > "hostname": "walter", > > "ip": "192.168.10.237", > > "user": { > > "firstname": "Donny", > > "lastname": "Kerabatsos" > > }, > > "message": "Shut the f\* up, Donny" > > } > > } > > > > Now I can search for a doc by referencing the hostname like this: > > > > { > > "query": { > > "filtered": { > > "filter": { > > "and": [ > > { > > "term": { > > "@fields.hostname": "walter" > > } > > } > > ] > > } > > } > > } > > } > > > > But this search yields\* no result:\* > > > > { > > "query": { > > "filtered": { > > "filter": { > > "and": [ > > { > > "term": { > > "@fields.user.firstname": "Donny" > > } > > } > > ] > > } > > } > > } > > } > > > > I don't understand why, but perhaps I misunderstood something about the > > object type. My impression was, that it is possible to insert an arbitrary > > json and perform searches on any of it's attributes (also "nested" > > attributes), due to the dynamic mapping. > > > > 1.) What do I have to do in order to make this work? > > 2.) I _DO not_ want to create a default schema for this, since the json > > data which is inserted into @fields changes (different hierarchies, > > attribute names, etc.). > > > > Thanks a lot & cheers, > > > > - Chris > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Chris\_Kinsalb](https://avatars.discourse-cdn.com/v4/letter/c/e99b99/32.png) [@Chris\_Kinsalb](https://discuss.elastic.co/u/Chris_Kinsalb)\ **Post date:** [June 3, 2013, 11:46am UTC](https://discuss.elastic.co/t/cannot-search-in-sub-attribute/12217/4 "2013-06-03T11:46:18Z")
fyi: My own suggested solution _works as intended_. I can now add arbitrarily nested objects, with each attribute flattened down to its full path. The query to server:9200/index/{doctype}/ \_mapping shows that the field types are created correctly was well. - chris On Monday, June 3, 2013 8:18:54 AM UTC+2, Chris Kinsalb wrote: > Hello Shay, > > thanks for the explanation. > Yes, that is indeed what I thought. However, the problem is that I wanted > my input to be totally schema-less, which I cannot seem to achieve using > dynamic templates since I will not know what the input objects will look > like. > > However, I do have an idea on how to achieve this: > > Using dynamic templates could I do something along these lines? > > ``` > "dynamic_templates": [ > { > "fields_template": { > "mapping": { > "index": "not_analyzed" > }, > "match": "fields.*" > } > } > ] > > ``` > > Now, whenever I get a json input I would transform it to a flat hierarchy, > e.g.: > > input: > > { > foo: 'bar', > user: { > firstname: 'bla', > lastname: 'narf' > } > } > > Transform output: > > { > fields.foo: 'bar', > fields.user.firstname: 'bla', > fields.user.lastname: 'narf' > } > > This way: > > - each field will automatically have the correct type (e.g. string or > integer) > - I can run facet queries (statistics) on the fields > > Does this sound like a viable solution to you? > (will try it out now) > > Cheers & regards, > > - Chris > > On Saturday, June 1, 2013 5:19:40 PM UTC+2, kimchy wrote: > > > you should use the match query instead of the term query, since the > > internal fields ends up being analyzed. > > > > I see you try to set the index field on the object mapping to mark it as > > not\_analyzed, this will not work and is not supported (I think you assume > > that al internal mapping will inherit it, which they do not). if you want > > to achieve this, check into dynamic templates in the root object mappings. > > > > On Sat, Jun 1, 2013 at 4:18 PM, Chris Kinsalb [hc.ki...@gmail.com](mailto:hc.ki...@gmail.com) wrote: > > > > > Hello, > > > > > > I created a doc type which has several attributes including an > > > attribute @fields configured like this: > > > > > > "@fields": { > > > "type": "object", > > > "index": "not\_analyzed", > > > "dynamic": true, > > > "path": "full" > > > } > > > > > > Here is a sample input: > > > > > > { > > > "@fields": { > > > "hostname": "walter", > > > "ip": "192.168.10.237", > > > "user": { > > > "firstname": "Donny", > > > "lastname": "Kerabatsos" > > > }, > > > "message": "Shut the f\* up, Donny" > > > } > > > } > > > > > > Now I can search for a doc by referencing the hostname like this: > > > > > > { > > > "query": { > > > "filtered": { > > > "filter": { > > > "and": [ > > > { > > > "term": { > > > "@fields.hostname": "walter" > > > } > > > } > > > ] > > > } > > > } > > > } > > > } > > > > > > But this search yields\* no result:\* > > > > > > { > > > "query": { > > > "filtered": { > > > "filter": { > > > "and": [ > > > { > > > "term": { > > > "@fields.user.firstname": "Donny" > > > } > > > } > > > ] > > > } > > > } > > > } > > > } > > > > > > I don't understand why, but perhaps I misunderstood something about the > > > object type. My impression was, that it is possible to insert an arbitrary > > > json and perform searches on any of it's attributes (also "nested" > > > attributes), due to the dynamic mapping. > > > > > > 1.) What do I have to do in order to make this work? > > > 2.) I _DO not_ want to create a default schema for this, since the json > > > data which is inserted into @fields changes (different hierarchies, > > > attribute names, etc.). > > > > > > Thanks a lot & cheers, > > > > > > - Chris > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:33am UTC](https://discuss.elastic.co/t/cannot-search-in-sub-attribute/12217/5 "2017-07-06T02:33:20Z")