Facetted Search Misbehaving

I have an index for a Store with a type for Items, i.e. /store/items.
among other properties, Items have a Title (analyzed text), a
Description (analyzed text), and Tags (not-analyzed text).

I want to be able to show the facets over Tags with counts, so if a
facet of the Tag "Yellow" has a count of 12, for example, then when the
user adds that Tag to the filter she will see only 12 items.

I am using a Filtered Query with Aggs, as shown below, on Elasticsearch
1.1.0 on a single node:

GET _search {
"query": {
"filtered": {
"query": {
"multi_match": {
"query": "Large Widgets"
,"fields": [ "title^3", "description" ]
}}
,"filter": {
"terms": {
"tags": [ "Colorful" ]
,"execution": "and"
}}}}
,"aggs": {"available_tags": {"terms": {"field": "tags"}}, "size": 20}
}

I have two problems:

  1. no matter what value I pass for the aggs/size I get 10 aggregations.
    I want to get more than 10.

  2. the hits count that comes back when adding the new tag to the filter
    doesn't match the doc_count that came with the aggregations, for
    example, the aggregations might show a doc_count of 12 for the tag
    "Yellow", but if I add "Yellow" to the filter terms so that it reads
    "tags": [ "Colorful", "Yellow" ]I get 17 hits instead of the expected 12.

am I doing something wrong? is there a bug somewhere?

TIA,

Igal

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org.
For more options, visit https://groups.google.com/d/optout.

Sent from my iPhone

On Apr 20, 2014, at 3:06 PM, "Igal @ getRailo.org" igal@getrailo.org wrote:

I have an index for a Store with a type for Items, i.e. /store/items. among other properties, Items have a Title (analyzed text), a Description (analyzed text), and Tags (not-analyzed text).

I want to be able to show the facets over Tags with counts, so if a facet of the Tag "Yellow" has a count of 12, for example, then when the user adds that Tag to the filter she will see only 12 items.

I am using a Filtered Query with Aggs, as shown below, on Elasticsearch 1.1.0 on a single node:

GET _search {
"query": {
"filtered": {
"query": {
"multi_match": {
"query": "Large Widgets"
,"fields": [ "title^3", "description" ]
}}
,"filter": {
"terms": {
"tags": [ "Colorful" ]
,"execution": "and"
}}}}
,"aggs": {"available_tags": {"terms": {"field": "tags"}}, "size": 20}
}

I have two problems:

  1. no matter what value I pass for the aggs/size I get 10 aggregations. I want to get more than 10.

  2. the hits count that comes back when adding the new tag to the filter doesn't match the doc_count that came with the aggregations, for example, the aggregations might show a doc_count of 12 for the tag "Yellow", but if I add "Yellow" to the filter terms so that it reads "tags": [ "Colorful", "Yellow" ]I get 17 hits instead of the expected 12.

If I recall the terms filter is an or filter. So you are getting colorful or yellow. It might have an and option. Not sure, on my phone now. Otherwise use a bool filter with two must subfilters to do an and.

Nik

am I doing something wrong? is there a bug somewhere?

TIA,

Igal

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/E9B277B9-31A2-4DC6-A4F9-A30E0C5265CF%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

hey Nik,

thanks for your reply.

you are correct about the default operator being "or", but as I've posted
in the original post I explicitly set it to "and" via "execution": "and"
and I see the difference in the results between "or" and "and" so I know
that it is working. for some reason the numbers still do not match some of
the time though.

Igal

On Sunday, April 20, 2014 12:32:31 PM UTC-7, Nikolas Everett wrote:

Sent from my iPhone

On Apr 20, 2014, at 3:06 PM, "Igal @ getRailo.org" <ig...@getrailo.org<javascript:>>
wrote:

I have an index for a Store with a type for Items, i.e. /store/items.
among other properties, Items have a Title (analyzed text), a Description
(analyzed text), and Tags (not-analyzed text).

I want to be able to show the facets over Tags with counts, so if a
facet of the Tag "Yellow" has a count of 12, for example, then when the
user adds that Tag to the filter she will see only 12 items.

I am using a Filtered Query with Aggs, as shown below, on Elasticsearch
1.1.0 on a single node:

GET _search {
"query": {
"filtered": {
"query": {
"multi_match": {
"query": "Large Widgets"
,"fields": [ "title^3", "description" ]
}}
,"filter": {
"terms": {
"tags": [ "Colorful" ]
,"execution": "and"
}}}}
,"aggs": {"available_tags": {"terms": {"field": "tags"}}, "size": 20}
}

I have two problems:

  1. no matter what value I pass for the aggs/size I get 10 aggregations.
    I want to get more than 10.

  2. the hits count that comes back when adding the new tag to the filter
    doesn't match the doc_count that came with the aggregations, for example,
    the aggregations might show a doc_count of 12 for the tag "Yellow", but if
    I add "Yellow" to the filter terms so that it reads "tags": [ "Colorful",
    "Yellow" ]I get 17 hits instead of the expected 12.

If I recall the terms filter is an or filter. So you are getting colorful
or yellow. It might have an and option. Not sure, on my phone now.
Otherwise use a bool filter with two must subfilters to do an and.

Nik

am I doing something wrong? is there a bug somewhere?

TIA,

Igal

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Sorry, wasn't reading too closely. If the execution:and is taking sometimes then that sounds like a bug.

Nik

Sent from my iPhone

On Apr 20, 2014, at 3:49 PM, Igal igal@getrailo.org wrote:

hey Nik,

thanks for your reply.

you are correct about the default operator being "or", but as I've posted in the original post I explicitly set it to "and" via "execution": "and" and I see the difference in the results between "or" and "and" so I know that it is working. for some reason the numbers still do not match some of the time though.

Igal

On Sunday, April 20, 2014 12:32:31 PM UTC-7, Nikolas Everett wrote:

Sent from my iPhone

On Apr 20, 2014, at 3:06 PM, "Igal @ getRailo.org" ig...@getrailo.org wrote:

I have an index for a Store with a type for Items, i.e. /store/items. among other properties, Items have a Title (analyzed text), a Description (analyzed text), and Tags (not-analyzed text).

I want to be able to show the facets over Tags with counts, so if a facet of the Tag "Yellow" has a count of 12, for example, then when the user adds that Tag to the filter she will see only 12 items.

I am using a Filtered Query with Aggs, as shown below, on Elasticsearch 1.1.0 on a single node:

GET _search {
"query": {
"filtered": {
"query": {
"multi_match": {
"query": "Large Widgets"
,"fields": [ "title^3", "description" ]
}}
,"filter": {
"terms": {
"tags": [ "Colorful" ]
,"execution": "and"
}}}}
,"aggs": {"available_tags": {"terms": {"field": "tags"}}, "size": 20}
}

I have two problems:

  1. no matter what value I pass for the aggs/size I get 10 aggregations. I want to get more than 10.

  2. the hits count that comes back when adding the new tag to the filter doesn't match the doc_count that came with the aggregations, for example, the aggregations might show a doc_count of 12 for the tag "Yellow", but if I add "Yellow" to the filter terms so that it reads "tags": [ "Colorful", "Yellow" ]I get 17 hits instead of the expected 12.

If I recall the terms filter is an or filter. So you are getting colorful or yellow. It might have an and option. Not sure, on my phone now. Otherwise use a bool filter with two must subfilters to do an and.

Nik

am I doing something wrong? is there a bug somewhere?

TIA,

Igal

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Thanks Nik -- that's what I was thinking too.

I was wondering if someone from the ES team could confirm that my code
is correct for what I'm trying to achieve.

Igal

On 4/20/2014 1:09 PM, Nik Everett wrote:

Sorry, wasn't reading too closely. If the execution:and is taking
sometimes then that sounds like a bug.

Nik

Sent from my iPhone

On Apr 20, 2014, at 3:49 PM, Igal <igal@getrailo.org
mailto:igal@getrailo.org> wrote:

hey Nik,

thanks for your reply.

you are correct about the default operator being "or", but as I've
posted in the original post I explicitly set it to "and" via
"execution": "and" and I see the difference in the results between
"or" and "and" so I know that it is working. for some reason the
numbers still do not match some of the time though.

Igal

On Sunday, April 20, 2014 12:32:31 PM UTC-7, Nikolas Everett wrote:

Sent from my iPhone

> On Apr 20, 2014, at 3:06 PM, "Igal @ getRailo.org
<http://getRailo.org>" <ig...@getrailo.org <javascript:>> wrote:
>
> I have an index for a Store with a type for Items, i.e.
/store/items.  among other properties, Items have a Title
(analyzed text), a Description (analyzed text), and Tags
(not-analyzed text).
>
> I want to be able to show the facets over Tags with counts, so
if a facet of the Tag "Yellow" has a count of 12, for example,
then when the user adds that Tag to the filter she will see only
12 items.
>
> I am using a Filtered Query with Aggs, as shown below, on
Elasticsearch 1.1.0 on a single node:
>
> GET _search {
>  "query": {
>    "filtered": {
>       "query": {
>        "multi_match": {
>           "query": "Large Widgets"
>          ,"fields": [ "title^3", "description" ]
>       }}
>      ,"filter": {
>        "terms": {
>           "tags": [ "Colorful" ]
>          ,"execution": "and"
>  }}}}
>  ,"aggs": {"available_tags": {"terms": {"field": "tags"}},
"size": 20}
> }
>
> I have two problems:
>
> 1) no matter what value I pass for the aggs/size I get 10
aggregations.  I want to get more than 10.
>
> 2) the hits count that comes back when adding the new tag to
the filter doesn't match the doc_count that came with the
aggregations, for example, the aggregations might show a
doc_count of 12 for the tag "Yellow", but if I add "Yellow" to
the filter terms so that it reads "tags": [ "Colorful", "Yellow"
]I get 17 hits instead of the expected 12.

If I recall the terms filter is an or filter. So you are getting
colorful or yellow. It might have an and option. Not sure, on my
phone now. Otherwise use a bool filter with two must subfilters
to do an and.

Nik
>
> am I doing something wrong?  is there a bug somewhere?
>
> TIA,
>
>
> Igal
>
>
>
>
> --
> You received this message because you are subscribed to the
Google Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from
it, send an email to elasticsearc...@googlegroups.com <javascript:>.
> To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org
<https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org>.

> For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com
mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/xYRaJa04wfc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com
mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.com
https://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Igal Sapir
Railo Core Developer

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/53556529.7010709%40getrailo.org.
For more options, visit https://groups.google.com/d/optout.

I noticed that the problem usually happens when the number of results in
the agg is rather small.

I have 5 shards on that index, and the aggs return 10 results (again, I'm
not sure why not more when I request specifically as mentioned in item 1 in
the OP.

it looks like when the agg value is less than 50 (5 shards * 10 aggs) I get
this problem, so the aggs shows a smaller value than the actual hits
returned.

is that a known issue? any ideas on how I can get it to work properly?

I also posted
to lucene - Faceted Search and Aggregations Do Not Work Properly - Stack Overflow
since there was no much response here.

thanks,

Igal

On Monday, April 21, 2014 11:36:25 AM UTC-7, Igal wrote:

Thanks Nik -- that's what I was thinking too.

I was wondering if someone from the ES team could confirm that my code is
correct for what I'm trying to achieve.

Igal

On 4/20/2014 1:09 PM, Nik Everett wrote:

Sorry, wasn't reading too closely. If the execution:and is taking
sometimes then that sounds like a bug.

Nik

Sent from my iPhone

On Apr 20, 2014, at 3:49 PM, Igal igal@getrailo.org wrote:

hey Nik,

thanks for your reply.

you are correct about the default operator being "or", but as I've
posted in the original post I explicitly set it to "and" via "execution":
"and" and I see the difference in the results between "or" and "and" so I
know that it is working. for some reason the numbers still do not match
some of the time though.

Igal

On Sunday, April 20, 2014 12:32:31 PM UTC-7, Nikolas Everett wrote:

Sent from my iPhone

On Apr 20, 2014, at 3:06 PM, "Igal @ getRailo.org" ig...@getrailo.org
wrote:

I have an index for a Store with a type for Items, i.e. /store/items.
among other properties, Items have a Title (analyzed text), a Description
(analyzed text), and Tags (not-analyzed text).

I want to be able to show the facets over Tags with counts, so if a
facet of the Tag "Yellow" has a count of 12, for example, then when the
user adds that Tag to the filter she will see only 12 items.

I am using a Filtered Query with Aggs, as shown below, on Elasticsearch
1.1.0 on a single node:

GET _search {
"query": {
"filtered": {
"query": {
"multi_match": {
"query": "Large Widgets"
,"fields": [ "title^3", "description" ]
}}
,"filter": {
"terms": {
"tags": [ "Colorful" ]
,"execution": "and"
}}}}
,"aggs": {"available_tags": {"terms": {"field": "tags"}}, "size": 20}
}

I have two problems:

  1. no matter what value I pass for the aggs/size I get 10 aggregations.
    I want to get more than 10.

  2. the hits count that comes back when adding the new tag to the filter
    doesn't match the doc_count that came with the aggregations, for example,
    the aggregations might show a doc_count of 12 for the tag "Yellow", but if
    I add "Yellow" to the filter terms so that it reads "tags": [ "Colorful",
    "Yellow" ]I get 17 hits instead of the expected 12.

If I recall the terms filter is an or filter. So you are getting colorful
or yellow. It might have an and option. Not sure, on my phone now.
Otherwise use a bool filter with two must subfilters to do an and.

Nik

am I doing something wrong? is there a bug somewhere?

TIA,

Igal

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/xYRaJa04wfc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.comhttps://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Igal Sapir
Railo Core Developerhttp://getRailo.org/

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b43fd972-ae9c-4368-9f4e-2428d71eb91f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

for whoever finds this thread in the future, see

On 4/24/2014 10:55 AM, Igal wrote:

I noticed that the problem usually happens when the number of results
in the agg is rather small.

I have 5 shards on that index, and the aggs return 10 results (again,
I'm not sure why not more when I request specifically as mentioned in
item 1 in the OP.

it looks like when the agg value is less than 50 (5 shards * 10 aggs)
I get this problem, so the aggs shows a smaller value than the actual
hits returned.

is that a known issue? any ideas on how I can get it to work properly?

I also posted
to lucene - Faceted Search and Aggregations Do Not Work Properly - Stack Overflow
since there was no much response here.

thanks,

Igal

On Monday, April 21, 2014 11:36:25 AM UTC-7, Igal wrote:

Thanks Nik -- that's what I was thinking too.

I was wondering if someone from the ES team could confirm that my
code is correct for what I'm trying to achieve.


Igal

On 4/20/2014 1:09 PM, Nik Everett wrote:
Sorry, wasn't reading too closely. If the execution:and is taking
sometimes then that sounds like a bug.

Nik

Sent from my iPhone

On Apr 20, 2014, at 3:49 PM, Igal <igal@getrailo.org
<mailto:igal@getrailo.org>> wrote:
hey Nik,

thanks for your reply.

you are correct about the default operator being "or", but as
I've posted in the original post I explicitly set it to "and"
via "execution": "and" and I see the difference in the results
between "or" and "and" so I know that it is working.  for some
reason the numbers still do not match some of the time though.


Igal

On Sunday, April 20, 2014 12:32:31 PM UTC-7, Nikolas Everett wrote:



    Sent from my iPhone

    > On Apr 20, 2014, at 3:06 PM, "Igal @ getRailo.org
    <http://getRailo.org>" <ig...@getrailo.org> wrote:
    >
    > I have an index for a Store with a type for Items, i.e.
    /store/items.  among other properties, Items have a Title
    (analyzed text), a Description (analyzed text), and Tags
    (not-analyzed text).
    >
    > I want to be able to show the facets over Tags with
    counts, so if a facet of the Tag "Yellow" has a count of 12,
    for example, then when the user adds that Tag to the filter
    she will see only 12 items.
    >
    > I am using a Filtered Query with Aggs, as shown below, on
    Elasticsearch 1.1.0 on a single node:
    >
    > GET _search {
    >  "query": {
    >    "filtered": {
    >       "query": {
    >        "multi_match": {
    >           "query": "Large Widgets"
    >          ,"fields": [ "title^3", "description" ]
    >       }}
    >      ,"filter": {
    >        "terms": {
    >           "tags": [ "Colorful" ]
    >          ,"execution": "and"
    >  }}}}
    >  ,"aggs": {"available_tags": {"terms": {"field": "tags"}},
    "size": 20}
    > }
    >
    > I have two problems:
    >
    > 1) no matter what value I pass for the aggs/size I get 10
    aggregations.  I want to get more than 10.
    >
    > 2) the hits count that comes back when adding the new tag
    to the filter doesn't match the doc_count that came with the
    aggregations, for example, the aggregations might show a
    doc_count of 12 for the tag "Yellow", but if I add "Yellow"
    to the filter terms so that it reads "tags": [ "Colorful",
    "Yellow" ]I get 17 hits instead of the expected 12.

    If I recall the terms filter is an or filter. So you are
    getting colorful or yellow. It might have an and option. Not
    sure, on my phone now. Otherwise use a bool filter with two
    must subfilters to do an and.

    Nik
    >
    > am I doing something wrong?  is there a bug somewhere?
    >
    > TIA,
    >
    >
    > Igal
    >
    >
    >
    >
    > --
    > You received this message because you are subscribed to
    the Google Groups "elasticsearch" group.
    > To unsubscribe from this group and stop receiving emails
    from it, send an email to elasticsearc...@googlegroups.com.
    > To view this discussion on the web visit
    https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org
    <https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org>.

    > For more options, visit https://groups.google.com/d/optout
    <https://groups.google.com/d/optout>.

-- 
You received this message because you are subscribed to the
Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to elasticsearch+unsubscribe@googlegroups.com
<mailto:elasticsearch+unsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
-- 
You received this message because you are subscribed to a topic
in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/xYRaJa04wfc/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/xYRaJa04wfc/unsubscribe>.
To unsubscribe from this group and all its topics, send an email
to elasticsearch+unsubscribe@googlegroups.com
<mailto:elasticsearch+unsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
-- 
Igal Sapir
Railo Core Developer
http://getRailo.org/

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/xYRaJa04wfc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com
mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b43fd972-ae9c-4368-9f4e-2428d71eb91f%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b43fd972-ae9c-4368-9f4e-2428d71eb91f%40googlegroups.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Igal Sapir
Railo Core Developer

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/53652882.4040406%40getrailo.org.
For more options, visit https://groups.google.com/d/optout.

looks like the "size" issue was my mistake. I've placed it in the
"available_tags" scope instead of the "terms" scope. so the line for
the aggs in the request should read instead:

 ,"aggs": {"available_tags": {"terms": {"field": "tags", "size": 20}}}

On 5/3/2014 10:33 AM, Igal @ getRailo.org wrote:

for whoever finds this thread in the future, see
terms facet gives wrong count with n_shards > 1 · Issue #1305 · elastic/elasticsearch · GitHub

On 4/24/2014 10:55 AM, Igal wrote:

I noticed that the problem usually happens when the number of results
in the agg is rather small.

I have 5 shards on that index, and the aggs return 10 results (again,
I'm not sure why not more when I request specifically as mentioned in
item 1 in the OP.

it looks like when the agg value is less than 50 (5 shards * 10 aggs)
I get this problem, so the aggs shows a smaller value than the actual
hits returned.

is that a known issue? any ideas on how I can get it to work properly?

I also posted to
lucene - Faceted Search and Aggregations Do Not Work Properly - Stack Overflow
since there was no much response here.

thanks,

Igal

On Monday, April 21, 2014 11:36:25 AM UTC-7, Igal wrote:

Thanks Nik -- that's what I was thinking too.

I was wondering if someone from the ES team could confirm that my
code is correct for what I'm trying to achieve.


Igal

On 4/20/2014 1:09 PM, Nik Everett wrote:
Sorry, wasn't reading too closely. If the execution:and is
taking sometimes then that sounds like a bug.

Nik

Sent from my iPhone

On Apr 20, 2014, at 3:49 PM, Igal <igal@getrailo.org
<mailto:igal@getrailo.org>> wrote:
hey Nik,

thanks for your reply.

you are correct about the default operator being "or", but as
I've posted in the original post I explicitly set it to "and"
via "execution": "and" and I see the difference in the results
between "or" and "and" so I know that it is working.  for some
reason the numbers still do not match some of the time though.


Igal

On Sunday, April 20, 2014 12:32:31 PM UTC-7, Nikolas Everett
wrote:



    Sent from my iPhone

    > On Apr 20, 2014, at 3:06 PM, "Igal @ getRailo.org
    <http://getRailo.org>" <ig...@getrailo.org> wrote:
    >
    > I have an index for a Store with a type for Items, i.e.
    /store/items.  among other properties, Items have a Title
    (analyzed text), a Description (analyzed text), and Tags
    (not-analyzed text).
    >
    > I want to be able to show the facets over Tags with
    counts, so if a facet of the Tag "Yellow" has a count of
    12, for example, then when the user adds that Tag to the
    filter she will see only 12 items.
    >
    > I am using a Filtered Query with Aggs, as shown below, on
    Elasticsearch 1.1.0 on a single node:
    >
    > GET _search {
    >  "query": {
    >    "filtered": {
    >       "query": {
    >        "multi_match": {
    >           "query": "Large Widgets"
    >          ,"fields": [ "title^3", "description" ]
    >       }}
    >      ,"filter": {
    >        "terms": {
    >           "tags": [ "Colorful" ]
    >          ,"execution": "and"
    >  }}}}
    >  ,"aggs": {"available_tags": {"terms": {"field":
    "tags"}}, "size": 20}
    > }
    >
    > I have two problems:
    >
    > 1) no matter what value I pass for the aggs/size I get 10
    aggregations.  I want to get more than 10.
    >
    > 2) the hits count that comes back when adding the new tag
    to the filter doesn't match the doc_count that came with
    the aggregations, for example, the aggregations might show
    a doc_count of 12 for the tag "Yellow", but if I add
    "Yellow" to the filter terms so that it reads "tags": [
    "Colorful", "Yellow" ]I get 17 hits instead of the expected
    12.

    If I recall the terms filter is an or filter. So you are
    getting colorful or yellow. It might have an and option.
    Not sure, on my phone now. Otherwise use a bool filter with
    two must subfilters to do an and.

    Nik
    >
    > am I doing something wrong?  is there a bug somewhere?
    >
    > TIA,
    >
    >
    > Igal
    >
    >
    >
    >
    > --
    > You received this message because you are subscribed to
    the Google Groups "elasticsearch" group.
    > To unsubscribe from this group and stop receiving emails
    from it, send an email to elasticsearc...@googlegroups.com.
    > To view this discussion on the web visit
    https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org
    <https://groups.google.com/d/msgid/elasticsearch/53541AB6.4000403%40getrailo.org>.

    > For more options, visit
    https://groups.google.com/d/optout
    <https://groups.google.com/d/optout>.

-- 
You received this message because you are subscribed to the
Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to elasticsearch+unsubscribe@googlegroups.com
<mailto:elasticsearch+unsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/f6ce5c0d-37ec-45b6-972b-03dbd426e7de%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
-- 
You received this message because you are subscribed to a topic
in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/xYRaJa04wfc/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/xYRaJa04wfc/unsubscribe>.
To unsubscribe from this group and all its topics, send an email
to elasticsearch+unsubscribe@googlegroups.com
<mailto:elasticsearch+unsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/B4C0631C-7665-4380-BDD6-E6C84292CAEB%40gmail.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
-- 
Igal Sapir
Railo Core Developer
http://getRailo.org/

--
You received this message because you are subscribed to a topic in
the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/xYRaJa04wfc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com
mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b43fd972-ae9c-4368-9f4e-2428d71eb91f%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b43fd972-ae9c-4368-9f4e-2428d71eb91f%40googlegroups.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Igal Sapir
Railo Core Developer
http://getRailo.org/

--
Igal Sapir
Railo Core Developer

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/53657072.8090807%40getrailo.org.
For more options, visit https://groups.google.com/d/optout.