Facet filters using nested objects

Marcin_Dojwa · May 23, 2012, 8:52am

Hi,

I have the following problem, please check out

gist.github.com

https://gist.github.com/anonymous/d3234db747e6e845d1e5

curl_queries.sh

curl -XPUT 'http://localhost:9200/test'
curl -XPUT 'http://localhost:9200/test/type1/_mapping' -d '{"type1":{"properties":{"testobj":{"properties":{"idtest":{"type":"long","include_in_all":false},"obj1":{"type":"nested","dynamic":"true","properties":{"date":{"type":"date","format":"dateOptionalTime","include_in_all":false},"id":{"type":"long","include_in_all":false}}}}}}}}'
curl -XPOST 'http://localhost:9200/test/type1/1' -d '{"testobj":{"idtest":1,"obj1":[{"date":"2012-01-01T00:02:00Z","id":4},{"date":"2012-01-01T02:11:00Z","id":6}]}}'
curl -XPOST 'http://localhost:9200/test/type1/_search?pretty=true' -d '{"facets":{"facet1":{"date_histogram":{"field":"date","interval":"hour"},"nested":"testobj.obj1","facet_filter":{"and":[{"range":{"testobj.obj1.date":{"from":"2012-01-01T00:00:00Z","to":"2012-02-01T00:00:00Z"}}}]}}}}'
curl -XPOST 'http://localhost:9200/test/type1/_search?pretty=true' -d '{"facets":{"facet1":{"date_histogram":{"field":"date","interval":"hour"},"nested":"testobj.obj1","facet_filter":{"and":[{"exists":{"field":"testobj.idtest"}},{"range":{"testobj.obj1.date":{"from":"2012-01-01T00:00:00Z","to":"2012-02-01T00:00:00Z"}}}]}}}}'

date.js

{
   "testobj":{
      "idtest":1,
      "obj1":[
         {
            "date":"2012-01-01T00:02:00Z",
            "id":4
         },
         {
            "date":"2012-01-01T02:11:00Z",

This file has been truncated. show original

mapping.js

{
   "type1":{
      "properties":{
         "testobj":{
            "properties":{
               "idtest":{
                  "type":"long",
                  "include_in_all":false
               },
               "obj1":{

This file has been truncated. show original

There are more than three files. show original

query_ok.js - executes OK and this is OK
query_failed.js - this returns empty set because there is exists filter
in facet_filter that checks testobj.idtest which is not in nested object
(testobj.obj1).

So the question is: how to filter documents on any document fields (here:
testobj.idtest) using facets on field from nested object (here:
testobj.obj1.date)?

Thanks.
Best regards
Marcin.

Marcin_Dojwa · May 25, 2012, 2:25pm

Anyone ?

2012/5/23 Marcin Dojwa m.dojwa@livechatinc.com

Hi,

I have the following problem, please check out
Facet filters using nested objects · GitHub

query_ok.js - executes OK and this is OK

query_failed.js - this returns empty set because there is exists filter
in facet_filter that checks testobj.idtest which is not in nested object
(testobj.obj1).

So the question is: how to filter documents on any document fields (here:
testobj.idtest) using facets on field from nested object (here:
testobj.obj1.date)?

Thanks.
Best regards
Marcin.

Clinton_Gormley · May 25, 2012, 2:31pm

Hi Marcin

I don't see anything obviously wrong, but it is better to gist actual
curl statements that allow people to copy-and-paste in order to test
locally, otherwise, speaking for myself, I don't have the time

clint

2012/5/23 Marcin Dojwa m.dojwa@livechatinc.com
Hi,

    I have the following problem, please check
    out https://gist.github.com/d3234db747e6e845d1e5
    1. query_ok.js - executes OK and this is OK
    2. query_failed.js - this returns empty set because there is
    exists filter in facet_filter that checks testobj.idtest which
    is not in nested object (testobj.obj1).
    
    
    So the question is: how to filter documents on any document
    fields (here: testobj.idtest) using facets on field from
    nested object (here: testobj.obj1.date)?
    
    
    Thanks.
    Best regards
    Marcin.

benjamin_leviant · May 25, 2012, 3:23pm

Hello,

why not this :

{
"facets": {
"facet1": {
"date_histogram": {
"field": "date",
"interval": "hour"
},
"facet_filter": {
"and": [
{
"exists": {
"field": "testobj.idtest"
}
},
{
"nested": {
"path": "testobj.obj1",
"query": {
"range": {
"testobj.obj1.date": {
"from": "2012-01-01T00:00:00Z",
"to": "2012-02-01T00:00:00Z"
}
}
}
}
}
]
}
}
}
}

I hope it can help.

Regards

Benjamin

On Fri, May 25, 2012 at 4:31 PM, Clinton Gormley clint@traveljury.comwrote:

Hi Marcin

I don't see anything obviously wrong, but it is better to gist actual
curl statements that allow people to copy-and-paste in order to test
locally, otherwise, speaking for myself, I don't have the time

clint
2012/5/23 Marcin Dojwa m.dojwa@livechatinc.com
Hi,
    I have the following problem, please check
    out https://gist.github.com/d3234db747e6e845d1e5
    1. query_ok.js - executes OK and this is OK
    2. query_failed.js - this returns empty set because there is
    exists filter in facet_filter that checks testobj.idtest which
    is not in nested object (testobj.obj1).


    So the question is: how to filter documents on any document
    fields (here: testobj.idtest) using facets on field from
    nested object (here: testobj.obj1.date)?


    Thanks.
    Best regards
    Marcin.

Marcin_Dojwa · May 25, 2012, 3:30pm

Hi,

You are right, I added curl_queries.sh in gist

gist.github.com

https://gist.github.com/anonymous/d3234db747e6e845d1e5

curl_queries.sh

curl -XPUT 'http://localhost:9200/test'
curl -XPUT 'http://localhost:9200/test/type1/_mapping' -d '{"type1":{"properties":{"testobj":{"properties":{"idtest":{"type":"long","include_in_all":false},"obj1":{"type":"nested","dynamic":"true","properties":{"date":{"type":"date","format":"dateOptionalTime","include_in_all":false},"id":{"type":"long","include_in_all":false}}}}}}}}'
curl -XPOST 'http://localhost:9200/test/type1/1' -d '{"testobj":{"idtest":1,"obj1":[{"date":"2012-01-01T00:02:00Z","id":4},{"date":"2012-01-01T02:11:00Z","id":6}]}}'
curl -XPOST 'http://localhost:9200/test/type1/_search?pretty=true' -d '{"facets":{"facet1":{"date_histogram":{"field":"date","interval":"hour"},"nested":"testobj.obj1","facet_filter":{"and":[{"range":{"testobj.obj1.date":{"from":"2012-01-01T00:00:00Z","to":"2012-02-01T00:00:00Z"}}}]}}}}'
curl -XPOST 'http://localhost:9200/test/type1/_search?pretty=true' -d '{"facets":{"facet1":{"date_histogram":{"field":"date","interval":"hour"},"nested":"testobj.obj1","facet_filter":{"and":[{"exists":{"field":"testobj.idtest"}},{"range":{"testobj.obj1.date":{"from":"2012-01-01T00:00:00Z","to":"2012-02-01T00:00:00Z"}}}]}}}}'

date.js

{
   "testobj":{
      "idtest":1,
      "obj1":[
         {
            "date":"2012-01-01T00:02:00Z",
            "id":4
         },
         {
            "date":"2012-01-01T02:11:00Z",

This file has been truncated. show original

mapping.js

{
   "type1":{
      "properties":{
         "testobj":{
            "properties":{
               "idtest":{
                  "type":"long",
                  "include_in_all":false
               },
               "obj1":{

This file has been truncated. show original

There are more than three files. show original

Best regards.

2012/5/25 Clinton Gormley clint@traveljury.com

Hi Marcin

I don't see anything obviously wrong, but it is better to gist actual
curl statements that allow people to copy-and-paste in order to test
locally, otherwise, speaking for myself, I don't have the time

clint
2012/5/23 Marcin Dojwa m.dojwa@livechatinc.com
Hi,
    I have the following problem, please check
    out https://gist.github.com/d3234db747e6e845d1e5
    1. query_ok.js - executes OK and this is OK
    2. query_failed.js - this returns empty set because there is
    exists filter in facet_filter that checks testobj.idtest which
    is not in nested object (testobj.obj1).


    So the question is: how to filter documents on any document
    fields (here: testobj.idtest) using facets on field from
    nested object (here: testobj.obj1.date)?


    Thanks.
    Best regards
    Marcin.

Clinton_Gormley · May 25, 2012, 4:37pm

You are right, I added curl_queries.sh in gist
Facet filters using nested objects · GitHub

Got it.

The problem is that your facet is running an a nested object:

     "nested":"testobj.obj1",

which means that it can't see field testobj.idtest

To be clear, a nested object is a SEPARATE doc internally. So either,
you need to use the 'exists' filter as part of the query, or you can
configure your main object to also store a copy of the nested object
data:

{
type: "nested",
include_in_root: true,
properties: {.... }
}

Then, instead of running your facet on the nested object, run it on the
main object (just by leaving out this line: "nested":"testobj.obj1")

Depending on your data, this may or may not suit your purposes

clint

Marcin_Dojwa · May 25, 2012, 5:22pm

Hi Benjamin,

Thank you for your help.
The one you gave does not work because testobj.obj1 is a nested object and
the date_histogram works with testobj.obj1.date field. It would work if the
facet didn't work with field of nested object.
So, any other ideas ? The question is if this is possible in ES

Best regards.

2012/5/25 benjamin leviant benjamin.leviant@gmail.com

Hello,

why not this :

{
"facets": {
"facet1": {
"date_histogram": {
"field": "date",
"interval": "hour"
},
"facet_filter": {
"and": [
{
"exists": {
"field": "testobj.idtest"
}
},
{
"nested": {
"path": "testobj.obj1",
"query": {
"range": {
"testobj.obj1.date": {
"from": "2012-01-01T00:00:00Z",
"to": "2012-02-01T00:00:00Z"
}
}
}
}
}
]
}
}
}
}

I hope it can help.

Regards

Benjamin

On Fri, May 25, 2012 at 4:31 PM, Clinton Gormley clint@traveljury.comwrote:
Hi Marcin

I don't see anything obviously wrong, but it is better to gist actual
curl statements that allow people to copy-and-paste in order to test
locally, otherwise, speaking for myself, I don't have the time

clint
2012/5/23 Marcin Dojwa m.dojwa@livechatinc.com
Hi,
    I have the following problem, please check
    out https://gist.github.com/d3234db747e6e845d1e5
    1. query_ok.js - executes OK and this is OK
    2. query_failed.js - this returns empty set because there is
    exists filter in facet_filter that checks testobj.idtest which
    is not in nested object (testobj.obj1).


    So the question is: how to filter documents on any document
    fields (here: testobj.idtest) using facets on field from
    nested object (here: testobj.obj1.date)?


    Thanks.
    Best regards
    Marcin.

Marcin_Dojwa · May 25, 2012, 5:32pm

Hi Clinton,

Thank you,

I was afraid that these were the only one solutions. I did it the first way

I added exists filter to query but I had to separate two different facets
and create two different queries instead of one query with two facets
filtered with facet filters But this works as well.
In my case the copy of testobj.obj1 object will not work because I need to
count the number of testobj.obj1 elements not the number of root documents
(am I right this would count the number of documents not the number of
testobj.obj1 elements?).
I just wanted to know if it is possible to access root document's fields
from the nested query. If not I will leave the solution like I said before.

Thank you all for your help.

2012/5/25 Clinton Gormley clint@traveljury.com

You are right, I added curl_queries.sh in gist
Facet filters using nested objects · GitHub

Got it.

The problem is that your facet is running an a nested object:
    "nested":"testobj.obj1",
which means that it can't see field testobj.idtest

To be clear, a nested object is a SEPARATE doc internally. So either,
you need to use the 'exists' filter as part of the query, or you can
configure your main object to also store a copy of the nested object
data:

{
type: "nested",
include_in_root: true,
properties: {.... }
}

Then, instead of running your facet on the nested object, run it on the
main object (just by leaving out this line: "nested":"testobj.obj1")

Depending on your data, this may or may not suit your purposes

clint

Clinton_Gormley · May 25, 2012, 6:31pm

Hiya

I was afraid that these were the only one solutions. I did it the
first way - I added exists filter to query but I had to separate two
different facets and create two different queries instead of one query
with two facets filtered with facet filters But this works as well.
In my case the copy of testobj.obj1 object will not work because I
need to count the number of testobj.obj1 elements not the number of
root documents (am I right this would count the number of documents
not the number of testobj.obj1 elements?).
I just wanted to know if it is possible to access root document's
fields from the nested query. If not I will leave the solution like I
said before.

It's not possible. But what about including a copy of that field in
your nested docs? You would have to do it manually, and it'd take more
space, but you'd achieve what you're after

clint

Thank you all for your help.

2012/5/25 Clinton Gormley clint@traveljury.com

    >
    > You are right, I added curl_queries.sh in gist
    > https://gist.github.com/d3234db747e6e845d1e5
    
    
    Got it.
    
    The problem is that your facet is running an a nested object:
    
            "nested":"testobj.obj1",
    
    which means that it can't see field testobj.idtest
    
    To be clear, a nested object is a SEPARATE doc internally.  So
    either,
    you need to use the 'exists' filter as part of the query, or
    you can
    configure your main object to also store a copy of the nested
    object
    data:
    
     {
       type: "nested",
       include_in_root: true,
       properties: {.... }
     }
    
    Then, instead of running your facet on the nested object, run
    it on the
    main object (just by leaving out this line:
    "nested":"testobj.obj1")
    
    Depending on your data, this may or may not suit your purposes
    
    clint

Marcin_Dojwa · May 25, 2012, 6:44pm

You are right this solves the problem Thank you very much.

Best regards.

2012/5/25 Clinton Gormley clint@traveljury.com

Hiya

I was afraid that these were the only one solutions. I did it the
first way - I added exists filter to query but I had to separate two
different facets and create two different queries instead of one query
with two facets filtered with facet filters But this works as well.
In my case the copy of testobj.obj1 object will not work because I
need to count the number of testobj.obj1 elements not the number of
root documents (am I right this would count the number of documents
not the number of testobj.obj1 elements?).
I just wanted to know if it is possible to access root document's
fields from the nested query. If not I will leave the solution like I
said before.

It's not possible. But what about including a copy of that field in
your nested docs? You would have to do it manually, and it'd take more
space, but you'd achieve what you're after

clint
Thank you all for your help.

2012/5/25 Clinton Gormley clint@traveljury.com
    >
    > You are right, I added curl_queries.sh in gist
    > https://gist.github.com/d3234db747e6e845d1e5


    Got it.

    The problem is that your facet is running an a nested object:

            "nested":"testobj.obj1",

    which means that it can't see field testobj.idtest

    To be clear, a nested object is a SEPARATE doc internally.  So
    either,
    you need to use the 'exists' filter as part of the query, or
    you can
    configure your main object to also store a copy of the nested
    object
    data:

     {
       type: "nested",
       include_in_root: true,
       properties: {.... }
     }

    Then, instead of running your facet on the nested object, run
    it on the
    main object (just by leaving out this line:
    "nested":"testobj.obj1")

    Depending on your data, this may or may not suit your purposes

    clint

Topic		Replies	Views
Nested type, facet, and filters Elasticsearch	6	368	July 6, 2017
Facets and filtering on nested documents Elasticsearch	5	431	July 6, 2017
Facet on nested object Elasticsearch	2	420	July 6, 2017
Facets on nested objects, plus facet_filter Elasticsearch	4	510	July 6, 2017
Facets on Object Types Elasticsearch	2	276	July 6, 2017

Facet filters using nested objects

Related Topics