# JSON structure & querying object hierarchies **URL:** \ **Category:** Elasticsearch\ **Created:** [June 14, 2010, 7:45pm UTC](https://discuss.elastic.co/t/json-structure-querying-object-hierarchies/3019 "2010-06-14T19:45:26Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![MJ\_Suhonos](https://avatars.discourse-cdn.com/v4/letter/m/a3d4f5/32.png) [@MJ\_Suhonos](https://discuss.elastic.co/u/MJ_Suhonos)\ **Post date:** [June 14, 2010, 7:45pm UTC](https://discuss.elastic.co/t/json-structure-querying-object-hierarchies/3019/1 "2010-06-14T19:45:26Z")
Hi all, Apologies if this is a silly question, but I'm still learning the commonalities of JSON. I'm able to make a JSON structure which is being indexed, but not queryable using object-dot-notation. eg. a document with: "id" : ["12345", { "special" : "09876" } If I do an open query for either value (eg. ?q=12345 or q?=09876), I get the document. But, I cannot query for "id.special" or simply "id" (eg. ?q=id:12345 or ?q=id.special:09876), neither match. What I'm looking for is to be able to query a term in a "hierarchy" and have it match values within/below, whether in objects, arrays, etc (eg. ?q=id:09876 would match). Am I searching incorrectly or is there a proper way to express this in JSON that ES could handle? Is something like this even possible? Thanks in advance, MJ ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [June 14, 2010, 7:57pm UTC](https://discuss.elastic.co/t/json-structure-querying-object-hierarchies/3019/2 "2010-06-14T19:57:09Z")
Can you oust the json you index, the one you specified is not proper JSON.... On Mon, Jun 14, 2010 at 10:45 PM, MJ Suhonos [suhonos@gmail.com](mailto:suhonos@gmail.com) wrote: > Hi all, > > Apologies if this is a silly question, but I'm still learning the > commonalities of JSON. I'm able to make a JSON structure which is > being indexed, but not queryable using object-dot-notation. eg. a > document with: > > "id" : ["12345", > { > "special" : "09876" > } > > If I do an open query for either value (eg. ?q=12345 or q?=09876), I > get the document. But, I cannot query for "id.special" or simply > "id" (eg. ?q=id:12345 or ?q=id.special:09876), neither match. > > What I'm looking for is to be able to query a term in a "hierarchy" > and have it match values within/below, whether in objects, arrays, etc > (eg. ?q=id:09876 would match). Am I searching incorrectly or is there > a proper way to express this in JSON that ES could handle? Is > something like this even possible? > > Thanks in advance, > MJ ---
**Author:** ![MJ\_Suhonos](https://avatars.discourse-cdn.com/v4/letter/m/a3d4f5/32.png) [@MJ\_Suhonos](https://discuss.elastic.co/u/MJ_Suhonos)\ **Post date:** [June 14, 2010, 8:49pm UTC](https://discuss.elastic.co/t/json-structure-querying-object-hierarchies/3019/3 "2010-06-14T20:49:55Z")
Apologies -- this got truncated. The document is actually quite large, but the relevant portion (which does seem to get parsed fine by ES and is valid JSON) is: { "id" : ["81881", { "special" : "051110" }] } Thanks, MJ On Jun 14, 3:57 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > Can you oust the json you index, the one you specified is not proper > JSON.... > > On Mon, Jun 14, 2010 at 10:45 PM, MJ Suhonos [suho...@gmail.com](mailto:suho...@gmail.com) wrote: > > > Hi all, > > > Apologies if this is a silly question, but I'm still learning the > > commonalities of JSON. I'm able to make a JSON structure which is > > being indexed, but not queryable using object-dot-notation. eg. a > > document with: > > > "id" : ["12345", > > { > > "special" : "09876" > > } > > > If I do an open query for either value (eg. ?q=12345 or q?=09876), I > > get the document. But, I cannot query for "id.special" or simply > > "id" (eg. ?q=id:12345 or ?q=id.special:09876), neither match. > > > What I'm looking for is to be able to query a term in a "hierarchy" > > and have it match values within/below, whether in objects, arrays, etc > > (eg. ?q=id:09876 would match). Am I searching incorrectly or is there > > a proper way to express this in JSON that ES could handle? Is > > something like this even possible? > > > Thanks in advance, > > MJ ---
**Author:** ![Diptamay](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/diptamay/32/2805_2.png) [@Diptamay](https://discuss.elastic.co/u/Diptamay)\ **Post date:** [June 15, 2010, 2:23am UTC](https://discuss.elastic.co/t/json-structure-querying-object-hierarchies/3019/4 "2010-06-15T02:23:39Z")
Instead of { "id" : ["81881", { "special" : "051110" }] } Represent your json as { "id" : [81881, 051110] } Then if you query using q=id:81881 or q=id:051110, it will work. Ideally you would want to keep your array elements homogenous. On Jun 14, 4:49 pm, MJ Suhonos [suho...@gmail.com](mailto:suho...@gmail.com) wrote: > Apologies -- this got truncated. The document is actually quite > large, but the relevant portion (which does seem to get parsed fine by > ES and is valid JSON) is: > > { > "id" : ["81881", > { > "special" : "051110" > }] > > } > > Thanks, > MJ > > On Jun 14, 3:57 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > > > Can you oust the json you index, the one you specified is not proper > > JSON.... > > > On Mon, Jun 14, 2010 at 10:45 PM, MJ Suhonos [suho...@gmail.com](mailto:suho...@gmail.com) wrote: > > > > > Hi all, > > > > Apologies if this is a silly question, but I'm still learning the > > > commonalities of JSON. I'm able to make a JSON structure which is > > > being indexed, but not queryable using object-dot-notation. eg. a > > > document with: > > > > "id" : ["12345", > > > { > > > "special" : "09876" > > > } > > > > If I do an open query for either value (eg. ?q=12345 or q?=09876), I > > > get the document. But, I cannot query for "id.special" or simply > > > "id" (eg. ?q=id:12345 or ?q=id.special:09876), neither match. > > > > What I'm looking for is to be able to query a term in a "hierarchy" > > > and have it match values within/below, whether in objects, arrays, etc > > > (eg. ?q=id:09876 would match). Am I searching incorrectly or is there > > > a proper way to express this in JSON that ES could handle? Is > > > something like this even possible? > > > > Thanks in advance, > > > MJ ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [June 15, 2010, 7:37am UTC](https://discuss.elastic.co/t/json-structure-querying-object-hierarchies/3019/5 "2010-06-15T07:37:01Z")
Yes, this is probably the problem here... -shay.banon On Tue, Jun 15, 2010 at 5:23 AM, diptamay [diptamay@gmail.com](mailto:diptamay@gmail.com) wrote: > Instead of > { > "id" : ["81881", > { > "special" : "051110" > }] > } > > Represent your json as > { > "id" : [81881, 051110] > } > > Then if you query using q=id:81881 or q=id:051110, it will work. > > Ideally you would want to keep your array elements homogenous. > > On Jun 14, 4:49 pm, MJ Suhonos [suho...@gmail.com](mailto:suho...@gmail.com) wrote: > > > Apologies -- this got truncated. The document is actually quite > > large, but the relevant portion (which does seem to get parsed fine by > > ES and is valid JSON) is: > > > > { > > "id" : ["81881", > > { > > "special" : "051110" > > }] > > > > } > > > > Thanks, > > MJ > > > > On Jun 14, 3:57 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > > > > > Can you oust the json you index, the one you specified is not proper > > > JSON.... > > > > > On Mon, Jun 14, 2010 at 10:45 PM, MJ Suhonos [suho...@gmail.com](mailto:suho...@gmail.com) > > > wrote: > > > > > > > Hi all, > > > > > > Apologies if this is a silly question, but I'm still learning the > > > > commonalities of JSON. I'm able to make a JSON structure which is > > > > being indexed, but not queryable using object-dot-notation. eg. a > > > > document with: > > > > > > "id" : ["12345", > > > > { > > > > "special" : "09876" > > > > } > > > > > > If I do an open query for either value (eg. ?q=12345 or q?=09876), I > > > > get the document. But, I cannot query for "id.special" or simply > > > > "id" (eg. ?q=id:12345 or ?q=id.special:09876), neither match. > > > > > > What I'm looking for is to be able to query a term in a "hierarchy" > > > > and have it match values within/below, whether in objects, arrays, > > > > etc > > > > (eg. ?q=id:09876 would match). Am I searching incorrectly or is > > > > there > > > > a proper way to express this in JSON that ES could handle? Is > > > > something like this even possible? > > > > > > Thanks in advance, > > > > MJ ---
**Author:** ![MJ\_Suhonos](https://avatars.discourse-cdn.com/v4/letter/m/a3d4f5/32.png) [@MJ\_Suhonos](https://discuss.elastic.co/u/MJ_Suhonos)\ **Post date:** [June 15, 2010, 4:52pm UTC](https://discuss.elastic.co/t/json-structure-querying-object-hierarchies/3019/6 "2010-06-15T16:52:14Z")
Thanks diptamay, I was expecting the response was that Elastic Search isn't intended to be used for searching in this way. Let me clarify my use a case a bit to explain my approach. A clearer example is this: { "title" : ["Robin Hood", { "subtitle" : "Men in tights" }] } Ideally, I'd like to be able to do a search on q?title:tights and have this match, since the user may not distinguish the subtitle (they may simply think it is "part of the title"). Semantically, subtitle _is_ a sub-object of title, and q?title.subtitle:tights should _also_ match, if I were searching at that level of specificity. I shouldn't have to separate them into homogeneous units, since q? title:tights still wouldn't match, eg: { "title" : "Robin Hood", "subtitle" : "Men in tights" } And I lose the semantic label of "subtitle" if I do: { "title" : ["Robin Hood", "Men in tights"] } I've been following Elastic Search for a little while now, and for my problem domain (library science), having an index which is data-model neutral but still provides the advanced structure, faceting, etc. features is an ideal solution that we have been waiting for a very long time. Unfortunately, if I have to limit my JSON structure to ordered arrays and key-value objects in order to be able to search them, then it's not a viable option. Shay (and thanks in advance for taking the time to look at my question, I really appreciate it a lot), is there an architectural reason why ES can't handle this kind of JSON construct in the way I describe above? I can imagine from a performance perspective that indexing a hierarchy would be potentially expensive, but I'd be willing to make that trade-off for this kind of functionality. MJ > On Tue, Jun 15, 2010 at 5:23 AM, diptamay [dipta...@gmail.com](mailto:dipta...@gmail.com) wrote: > > > Instead of > > { > > "id" : ["81881", > > { > > "special" : "051110" > > }] > > } > > > Represent your json as > > { > > "id" : [81881, 051110] > > } > > > Then if you query using q=id:81881 or q=id:051110, it will work. > > > Ideally you would want to keep your array elements homogenous. > > > On Jun 14, 4:49 pm, MJ Suhonos [suho...@gmail.com](mailto:suho...@gmail.com) wrote: > > > > > Apologies -- this got truncated. The document is actually quite > > > large, but the relevant portion (which does seem to get parsed fine by > > > ES and is valid JSON) is: > > > > { > > > "id" : ["81881", > > > { > > > "special" : "051110" > > > }] > > > > } > > > > Thanks, > > > MJ > > > > On Jun 14, 3:57 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > > > > > Can you oust the json you index, the one you specified is not proper > > > > JSON.... > > > > > On Mon, Jun 14, 2010 at 10:45 PM, MJ Suhonos [suho...@gmail.com](mailto:suho...@gmail.com) > > > > wrote: > > > > > > > > > Hi all, > > > > > > Apologies if this is a silly question, but I'm still learning the > > > > > commonalities of JSON. I'm able to make a JSON structure which is > > > > > being indexed, but not queryable using object-dot-notation. eg. a > > > > > document with: > > > > > > "id" : ["12345", > > > > > { > > > > > "special" : "09876" > > > > > } > > > > > > If I do an open query for either value (eg. ?q=12345 or q?=09876), I > > > > > get the document. But, I cannot query for "id.special" or simply > > > > > "id" (eg. ?q=id:12345 or ?q=id.special:09876), neither match. > > > > > > What I'm looking for is to be able to query a term in a "hierarchy" > > > > > and have it match values within/below, whether in objects, arrays, > > > > > etc > > > > > (eg. ?q=id:09876 would match). Am I searching incorrectly or is > > > > > there > > > > > a proper way to express this in JSON that ES could handle? Is > > > > > something like this even possible? > > > > > > Thanks in advance, > > > > > MJ ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [June 15, 2010, 5:33pm UTC](https://discuss.elastic.co/t/json-structure-querying-object-hierarchies/3019/7 "2010-06-15T17:33:15Z")
On Tue, 2010-06-15 at 09:52 -0700, MJ Suhonos wrote: > Thanks diptamay, I was expecting the response was that Elastic Search > isn't intended to be used for searching in this way. Let me clarify > my use a case a bit to explain my approach. A clearer example is > this: > > { > "title" : ["Robin Hood", > { > "subtitle" : "Men in tights" > }] > > } Why not just: { "title": "Robin Hood", "subtitle: "Men in tights" } Then search for ?q=title:tights%20or%20subtitle:tights I don't think you lose anything semantically by expressing it as above. clint ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [June 15, 2010, 5:50pm UTC](https://discuss.elastic.co/t/json-structure-querying-object-hierarchies/3019/8 "2010-06-15T17:50:58Z")
The way elasticsearch works is that array elements should keep the same "type" of json, either string/int/... or object or another array, but not mix them. The limitation is because of how that json is mapped to a Lucene document (at the end, it gets deconstructed into a Lucene Document). I am not sure it is such a constraint, and I would say it make more sense when it comes to modeling your data (make more sense in statically typed langs, but not just). -shay.banon On Tue, Jun 15, 2010 at 8:33 PM, Clinton Gormley [clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk)wrote: > On Tue, 2010-06-15 at 09:52 -0700, MJ Suhonos wrote: > > > Thanks diptamay, I was expecting the response was that Elastic Search > > isn't intended to be used for searching in this way. Let me clarify > > my use a case a bit to explain my approach. A clearer example is > > this: > > > > { > > "title" : ["Robin Hood", > > { > > "subtitle" : "Men in tights" > > }] > > > > } > > Why not just: > { > "title": "Robin Hood", > "subtitle: "Men in tights" > } > > Then search for ?q=title:tights%20or%20subtitle:tights > > I don't think you lose anything semantically by expressing it as above. > > clint ---
**Author:** ![MJ\_Suhonos](https://avatars.discourse-cdn.com/v4/letter/m/a3d4f5/32.png) [@MJ\_Suhonos](https://discuss.elastic.co/u/MJ_Suhonos)\ **Post date:** [June 15, 2010, 6:07pm UTC](https://discuss.elastic.co/t/json-structure-querying-object-hierarchies/3019/9 "2010-06-15T18:07:06Z")
Clint, thanks -- you're absolutely right, the better way is to construct my search queries more intelligently. It'll take some more mapping in the application layer, but I'll give that a go. Shay -- thanks for the explanation, and understood; I certainly agree with the "purity" of requiring homogeneity among JSON elements, and of course, if it forces me to think harder about my data model, so much the better. (This is one of the reasons I'm so keen on Elastic Search, it frees you to focus on your data model design instead of worrying about limitations) Thanks all for your comments, and wonderful work once again on ES! I'm very excited to play with faceting in 0.9. 🙂 MJ On Jun 15, 1:50 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > The way elasticsearch works is that array elements should keep the same > "type" of json, either string/int/... or object or another array, but not > mix them. The limitation is because of how that json is mapped to a Lucene > document (at the end, it gets deconstructed into a Lucene Document). I am > not sure it is such a constraint, and I would say it make more sense when it > comes to modeling your data (make more sense in statically typed langs, but > not just). > > -shay.banon > > On Tue, Jun 15, 2010 at 8:33 PM, Clinton Gormley [clin...@iannounce.co.uk](mailto:clin...@iannounce.co.uk)wrote: > > > On Tue, 2010-06-15 at 09:52 -0700, MJ Suhonos wrote: > > > > > Thanks diptamay, I was expecting the response was that Elastic Search > > > isn't intended to be used for searching in this way. Let me clarify > > > my use a case a bit to explain my approach. A clearer example is > > > this: > > > > { > > > "title" : ["Robin Hood", > > > { > > > "subtitle" : "Men in tights" > > > }] > > > > } > > > Why not just: > > { > > "title": "Robin Hood", > > "subtitle: "Men in tights" > > } > > > Then search for ?q=title:tights%20or%20subtitle:tights > > > I don't think you lose anything semantically by expressing it as above. > > > clint ---
**Author:** ![mooky](https://avatars.discourse-cdn.com/v4/letter/m/43a26b/32.png) [@mooky](https://discuss.elastic.co/u/mooky)\ **Post date:** [June 15, 2010, 8:31pm UTC](https://discuss.elastic.co/t/json-structure-querying-object-hierarchies/3019/10 "2010-06-15T20:31:30Z")
Or, depending on your requirement / goal, get a more google-style search experience by use of the "all" field (what is it called in ES?) -N On 15 Jun 2010, at 18:33, Clinton Gormley [clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk) wrote: > On Tue, 2010-06-15 at 09:52 -0700, MJ Suhonos wrote: > > > Thanks diptamay, I was expecting the response was that Elastic Search > > isn't intended to be used for searching in this way. Let me clarify > > my use a case a bit to explain my approach. A clearer example is > > this: > > > > { > > "title" : ["Robin Hood", > > { > > "subtitle" : "Men in tights" > > }] > > > > } > > Why not just: > { > "title": "Robin Hood", > "subtitle: "Men in tights" > } > > Then search for ?q=title:tights%20or%20subtitle:tights > > I don't think you lose anything semantically by expressing it as > above. > > clint ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [June 15, 2010, 8:54pm UTC](https://discuss.elastic.co/t/json-structure-querying-object-hierarchies/3019/11 "2010-06-15T20:54:31Z")
Its still called "all", the name of the field is "\_all" (as is the convention for internal fields). It is automatically used when not "prefix field" is specified in the query. -shay.banon On Tue, Jun 15, 2010 at 11:31 PM, Nick Minutello [nick.minutello@gmail.com](mailto:nick.minutello@gmail.com)wrote: > Or, depending on your requirement / goal, get a more google-style search > experience by use of the "all" field (what is it called in ES?) > > -N > > On 15 Jun 2010, at 18:33, Clinton Gormley [clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk) wrote: > > On Tue, 2010-06-15 at 09:52 -0700, MJ Suhonos wrote: > > > > Thanks diptamay, I was expecting the response was that Elastic Search > > > isn't intended to be used for searching in this way. Let me clarify > > > my use a case a bit to explain my approach. A clearer example is > > > this: > > > > > > { > > > "title" : ["Robin Hood", > > > { > > > "subtitle" : "Men in tights" > > > }] > > > > > > } > > > > Why not just: > > { > > "title": "Robin Hood", > > "subtitle: "Men in tights" > > } > > > > Then search for ?q=title:tights%20or%20subtitle:tights > > > > I don't think you lose anything semantically by expressing it as above. > > > > clint ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:23am UTC](https://discuss.elastic.co/t/json-structure-querying-object-hierarchies/3019/12 "2017-07-06T04:23:17Z")