# Facet with multiple counts
**URL:** https://discuss.elastic.co/t/facet-with-multiple-counts/11926
**Category:** Elasticsearch
**Created:** [May 12, 2013, 3:22pm UTC](https://discuss.elastic.co/t/facet-with-multiple-counts/11926 "2013-05-12T15:22:05Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@Alastair\_James](https://discuss.elastic.co/u/Alastair_James)
#### Post date: [May 12, 2013, 3:22pm UTC](https://discuss.elastic.co/t/facet-with-multiple-counts/11926/1 "2013-05-12T15:22:05Z")
Hi there.
I am trying to use ES to produce a table of several summed fields, grouped
by some other field.
Imagine I have a table of users. Each user has a country field. They also
have a boolean field 'likejazz' and 'likerock' (for ease I store these as 1
or 0 to make the following query faster).
Say I want to produce a table with rows for countries and columns for
number of users, number of users who likejazz and another column for those
who likerock, ordered by the first column decending In SQL I would write
this as:
SELECT country, COUNT(\*) AS nusers, SUM(likejazz) AS likejazz,
SUM(likerock) AS likerock
FROM users
GROUP BY country
ORDER BY nusers DESC
Is this possible in ES?
As far as I can see I can only calculate one column at a time (total,
likejazz or likerock) and would have to merge them in code. However, this
is a problem if the grouping is performed across a dimension with lots of
terms as, in order to merge accurately I need results for ALL terms.
As soon as I introduce a limit to any of the column facet calculations, I
may get an error. Imagine I try to retrieve the top 10 in each sub facet
search, the problem is that the ranking for the first column may not match
the following terms returned for the other columns.
Obviously in this case, I could get all the countries and merge, however
for collections with 1000s of terms this in inefficient.
Any pointers?
One approach may be to restrict the 2nd and 3rd facet search to just the
terms returned from the first, but not sure thats possible?
Cheers
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Boaz\_Leskes](https://discuss.elastic.co/u/Boaz_Leskes)
#### Post date: [May 13, 2013, 10:23am UTC](https://discuss.elastic.co/t/facet-with-multiple-counts/11926/2 "2013-05-13T10:23:53Z")
Hi James,
You can use the term statistics facet per field (likejazz, likerock) will
get you what you want for those fields:
[Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/search/facets/terms-stats-facet/) .
The user count is just a term facet on countries.
You'd need to request the facets to return the information for all
countries but I assume that shouldn't be a problem.
Does this help?
Boaz
On Sunday, May 12, 2013 5:22:05 PM UTC+2, Alastair James wrote:
> Hi there.
>
> I am trying to use ES to produce a table of several summed fields, grouped
> by some other field.
>
> Imagine I have a table of users. Each user has a country field. They also
> have a boolean field 'likejazz' and 'likerock' (for ease I store these as 1
> or 0 to make the following query faster).
>
> Say I want to produce a table with rows for countries and columns for
> number of users, number of users who likejazz and another column for those
> who likerock, ordered by the first column decending In SQL I would write
> this as:
>
> SELECT country, COUNT(\*) AS nusers, SUM(likejazz) AS likejazz,
> SUM(likerock) AS likerock
> FROM users
> GROUP BY country
> ORDER BY nusers DESC
>
> Is this possible in ES?
>
> As far as I can see I can only calculate one column at a time (total,
> likejazz or likerock) and would have to merge them in code. However, this
> is a problem if the grouping is performed across a dimension with lots of
> terms as, in order to merge accurately I need results for ALL terms.
>
> As soon as I introduce a limit to any of the column facet calculations, I
> may get an error. Imagine I try to retrieve the top 10 in each sub facet
> search, the problem is that the ranking for the first column may not match
> the following terms returned for the other columns.
>
> Obviously in this case, I could get all the countries and merge, however
> for collections with 1000s of terms this in inefficient.
>
> Any pointers?
>
> One approach may be to restrict the 2nd and 3rd facet search to just the
> terms returned from the first, but not sure thats possible?
>
> Cheers
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Alastair\_James](https://discuss.elastic.co/u/Alastair_James)
#### Post date: [May 13, 2013, 10:36am UTC](https://discuss.elastic.co/t/facet-with-multiple-counts/11926/3 "2013-05-13T10:36:54Z")
Hi there.
Thanks for the reply.
That would be fine for the above example, however the real problem is if I
am grouping over a high cardinality dimension, e.g. a facet with 1000s of
terms. In this case its impractical to return all the terms and merge
manually.
Any other ideas?
Regards
Al
On Monday, 13 May 2013 11:23:53 UTC+1, Boaz Leskes wrote:
> Hi James,
>
> You can use the term statistics facet per field (likejazz, likerock) will
> get you what you want for those fields:
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/search/facets/terms-stats-facet/) .
> The user count is just a term facet on countries.
>
> You'd need to request the facets to return the information for all
> countries but I assume that shouldn't be a problem.
>
> Does this help?
>
> Boaz
>
> On Sunday, May 12, 2013 5:22:05 PM UTC+2, Alastair James wrote:
>
> > Hi there.
> >
> > I am trying to use ES to produce a table of several summed fields,
> > grouped by some other field.
> >
> > Imagine I have a table of users. Each user has a country field. They also
> > have a boolean field 'likejazz' and 'likerock' (for ease I store these as 1
> > or 0 to make the following query faster).
> >
> > Say I want to produce a table with rows for countries and columns for
> > number of users, number of users who likejazz and another column for those
> > who likerock, ordered by the first column decending In SQL I would write
> > this as:
> >
> > SELECT country, COUNT(\*) AS nusers, SUM(likejazz) AS likejazz,
> > SUM(likerock) AS likerock
> > FROM users
> > GROUP BY country
> > ORDER BY nusers DESC
> >
> > Is this possible in ES?
> >
> > As far as I can see I can only calculate one column at a time (total,
> > likejazz or likerock) and would have to merge them in code. However, this
> > is a problem if the grouping is performed across a dimension with lots of
> > terms as, in order to merge accurately I need results for ALL terms.
> >
> > As soon as I introduce a limit to any of the column facet calculations, I
> > may get an error. Imagine I try to retrieve the top 10 in each sub facet
> > search, the problem is that the ranking for the first column may not match
> > the following terms returned for the other columns.
> >
> > Obviously in this case, I could get all the countries and merge, however
> > for collections with 1000s of terms this in inefficient.
> >
> > Any pointers?
> >
> > One approach may be to restrict the 2nd and 3rd facet search to just the
> > terms returned from the first, but not sure thats possible?
> >
> > Cheers
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Boaz\_Leskes](https://discuss.elastic.co/u/Boaz_Leskes)
#### Post date: [May 13, 2013, 10:52am UTC](https://discuss.elastic.co/t/facet-with-multiple-counts/11926/4 "2013-05-13T10:52:05Z")
For high cardinality fields your only "nice" option right now is to write a
plugin with a custom facet. Other than that here are the less clean options
I see:
1. Ask for top 500 terms, aggregate yourself, use the top 10 (or other
number of your choosing) and live with the potential in accuracies.
Elasticsearch does something similar with term facet anyway when
aggregating results from multiple nodes.
2. Make two round trips - one to get the top terms (countries in your
example), then one to get the counts (limited to terms you want)
3. Based on some assumptions on the total number of users per term, use
some number arithmetics to join both likejazz and likerock fields to a
single value and some that up. For example, if only have 100 users per term
you can sum 100\*likejazz+likerock and then extract the result from the
sum. In this approach the number of users is returned as total count per
term.
Does this make more sense?
Boaz
On Mon, May 13, 2013 at 12:36 PM, Alastair James [al.james@gmail.com](mailto:al.james@gmail.com) wrote:
> Hi there.
>
> Thanks for the reply.
>
> That would be fine for the above example, however the real problem is if I
> am grouping over a high cardinality dimension, e.g. a facet with 1000s of
> terms. In this case its impractical to return all the terms and merge
> manually.
>
> Any other ideas?
>
> Regards
>
> Al
>
> On Monday, 13 May 2013 11:23:53 UTC+1, Boaz Leskes wrote:
>
> > Hi James,
> >
> > You can use the term statistics facet per field (likejazz, likerock) will
> > get you what you want for those fields: [http://www.elasticsearch.org/](http://www.elasticsearch.org/)\*\*
> > guide/reference/api/search/\*\*facets/terms-stats-facet/[http://www.elasticsearch.org/guide/reference/api/search/facets/terms-stats-facet/](http://www.elasticsearch.org/guide/reference/api/search/facets/terms-stats-facet/) .
> > The user count is just a term facet on countries.
> >
> > You'd need to request the facets to return the information for all
> > countries but I assume that shouldn't be a problem.
> >
> > Does this help?
> >
> > Boaz
> >
> > On Sunday, May 12, 2013 5:22:05 PM UTC+2, Alastair James wrote:
> >
> > > Hi there.
> > >
> > > I am trying to use ES to produce a table of several summed fields,
> > > grouped by some other field.
> > >
> > > Imagine I have a table of users. Each user has a country field. They
> > > also have a boolean field 'likejazz' and 'likerock' (for ease I store these
> > > as 1 or 0 to make the following query faster).
> > >
> > > Say I want to produce a table with rows for countries and columns for
> > > number of users, number of users who likejazz and another column for those
> > > who likerock, ordered by the first column decending In SQL I would write
> > > this as:
> > >
> > > SELECT country, COUNT(\*) AS nusers, SUM(likejazz) AS likejazz,
> > > SUM(likerock) AS likerock
> > > FROM users
> > > GROUP BY country
> > > ORDER BY nusers DESC
> > >
> > > Is this possible in ES?
> > >
> > > As far as I can see I can only calculate one column at a time (total,
> > > likejazz or likerock) and would have to merge them in code. However, this
> > > is a problem if the grouping is performed across a dimension with lots of
> > > terms as, in order to merge accurately I need results for ALL terms.
> > >
> > > As soon as I introduce a limit to any of the column facet calculations,
> > > I may get an error. Imagine I try to retrieve the top 10 in each sub facet
> > > search, the problem is that the ranking for the first column may not match
> > > the following terms returned for the other columns.
> > >
> > > Obviously in this case, I could get all the countries and merge, however
> > > for collections with 1000s of terms this in inefficient.
> > >
> > > Any pointers?
> > >
> > > One approach may be to restrict the 2nd and 3rd facet search to just the
> > > terms returned from the first, but not sure thats possible?
> > >
> > > Cheers
> > >
> > > --
> > > You received this message because you are subscribed to a topic in the
> > > Google Groups "elasticsearch" group.
> > > To unsubscribe from this topic, visit
> > > [https://groups.google.com/d/topic/elasticsearch/OnBbz-nj194/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/OnBbz-nj194/unsubscribe?hl=en-US)
> > > .
> > > To unsubscribe from this group and all its topics, send an email to
> > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Alastair\_James](https://discuss.elastic.co/u/Alastair_James)
#### Post date: [May 13, 2013, 11:28am UTC](https://discuss.elastic.co/t/facet-with-multiple-counts/11926/5 "2013-05-13T11:28:32Z")
Hi. Thanks again.
Yes, it does make more sense.
If I create a custom facet can I return multiple values? Is this likely to
be fast? Are there any tutorials / pointers on this?
> 1. Ask for top 500 terms, aggregate yourself, use the top 10 (or other
> number of your choosing) and live with the potential in accuracies.
> Elasticsearch does something similar with term facet anyway when
> aggregating results from multiple nodes.
We cant really live with inaccuracies. So ES does this anyway? Is there any
details of this limitation and potential inaccuracy? Do you know exactly
how many terms does it merge from each node? I really feel this should be
written in big flashing bold letters on the facet documentation pages!
> 1. Make two round trips - one to get the top terms (countries in your
> example), then one to get the counts (limited to terms you want)
How could I limit to the terms I want? I know there is an 'exclude' field
for terms. Is there an 'include' one?
> 1. Based on some assumptions on the total number of users per term, use
> some number arithmetics to join both likejazz and likerock fields to a
> single value and some that up. For example, if only have 100 users per term
> you can sum 100\*likejazz+likerock and then extract the result from the
> sum. In this approach the number of users is returned as total count per
> term.
Interesting.... Will think this through. Not sure there is enough numeric
range on the return value to cope with the fields (there may be more than
two).
Cheers again.
On 13 May 2013 11:52, Boaz Leskes [b.leskes@gmail.com](mailto:b.leskes@gmail.com) wrote:
> For high cardinality fields your only "nice" option right now is to write
> a plugin with a custom facet. Other than that here are the less clean
> options I see:
>
> 1. Ask for top 500 terms, aggregate yourself, use the top 10 (or other
> number of your choosing) and live with the potential in accuracies.
> Elasticsearch does something similar with term facet anyway when
> aggregating results from multiple nodes.
> 2. Make two round trips - one to get the top terms (countries in your
> example), then one to get the counts (limited to terms you want)
> 3. Based on some assumptions on the total number of users per term, use
> some number arithmetics to join both likejazz and likerock fields to a
> single value and some that up. For example, if only have 100 users per term
> you can sum 100\*likejazz+likerock and then extract the result from the
> sum. In this approach the number of users is returned as total count per
> term.
>
> Does this make more sense?
>
> Boaz
>
> On Mon, May 13, 2013 at 12:36 PM, Alastair James [al.james@gmail.com](mailto:al.james@gmail.com)wrote:
>
> > Hi there.
> >
> > Thanks for the reply.
> >
> > That would be fine for the above example, however the real problem is if
> > I am grouping over a high cardinality dimension, e.g. a facet with 1000s of
> > terms. In this case its impractical to return all the terms and merge
> > manually.
> >
> > Any other ideas?
> >
> > Regards
> >
> > Al
> >
> > On Monday, 13 May 2013 11:23:53 UTC+1, Boaz Leskes wrote:
> >
> > > Hi James,
> > >
> > > You can use the term statistics facet per field (likejazz, likerock)
> > > will get you what you want for those fields:
> > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/**guide/reference/api/search/)\*\*
> > > facets/terms-stats-facet/[http://www.elasticsearch.org/guide/reference/api/search/facets/terms-stats-facet/](http://www.elasticsearch.org/guide/reference/api/search/facets/terms-stats-facet/) .
> > > The user count is just a term facet on countries.
> > >
> > > You'd need to request the facets to return the information for all
> > > countries but I assume that shouldn't be a problem.
> > >
> > > Does this help?
> > >
> > > Boaz
> > >
> > > On Sunday, May 12, 2013 5:22:05 PM UTC+2, Alastair James wrote:
> > >
> > > > Hi there.
> > > >
> > > > I am trying to use ES to produce a table of several summed fields,
> > > > grouped by some other field.
> > > >
> > > > Imagine I have a table of users. Each user has a country field. They
> > > > also have a boolean field 'likejazz' and 'likerock' (for ease I store these
> > > > as 1 or 0 to make the following query faster).
> > > >
> > > > Say I want to produce a table with rows for countries and columns for
> > > > number of users, number of users who likejazz and another column for those
> > > > who likerock, ordered by the first column decending In SQL I would write
> > > > this as:
> > > >
> > > > SELECT country, COUNT(\*) AS nusers, SUM(likejazz) AS likejazz,
> > > > SUM(likerock) AS likerock
> > > > FROM users
> > > > GROUP BY country
> > > > ORDER BY nusers DESC
> > > >
> > > > Is this possible in ES?
> > > >
> > > > As far as I can see I can only calculate one column at a time (total,
> > > > likejazz or likerock) and would have to merge them in code. However, this
> > > > is a problem if the grouping is performed across a dimension with lots of
> > > > terms as, in order to merge accurately I need results for ALL terms.
> > > >
> > > > As soon as I introduce a limit to any of the column facet calculations,
> > > > I may get an error. Imagine I try to retrieve the top 10 in each sub facet
> > > > search, the problem is that the ranking for the first column may not match
> > > > the following terms returned for the other columns.
> > > >
> > > > Obviously in this case, I could get all the countries and merge,
> > > > however for collections with 1000s of terms this in inefficient.
> > > >
> > > > Any pointers?
> > > >
> > > > One approach may be to restrict the 2nd and 3rd facet search to just
> > > > the terms returned from the first, but not sure thats possible?
> > > >
> > > > Cheers
> > > >
> > > > --
> > > > You received this message because you are subscribed to a topic in the
> > > > Google Groups "elasticsearch" group.
> > > > To unsubscribe from this topic, visit
> > > > [https://groups.google.com/d/topic/elasticsearch/OnBbz-nj194/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/OnBbz-nj194/unsubscribe?hl=en-US)
> > > > .
> > > > To unsubscribe from this group and all its topics, send an email to
> > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> [https://groups.google.com/d/topic/elasticsearch/OnBbz-nj194/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/OnBbz-nj194/unsubscribe?hl=en-US)
> .
> To unsubscribe from this group and all its topics, send an email to
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
Dr Alastair James
CTO [Ometria.com](http://Ometria.com)
Skype: al.james
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Boaz\_Leskes](https://discuss.elastic.co/u/Boaz_Leskes)
#### Post date: [May 13, 2013, 12:49pm UTC](https://discuss.elastic.co/t/facet-with-multiple-counts/11926/6 "2013-05-13T12:49:02Z")
Hi,
See my answers bellow.
Boaz
On Mon, May 13, 2013 at 1:28 PM, Alastair James [al.james@gmail.com](mailto:al.james@gmail.com) wrote:
> Hi. Thanks again.
>
> Yes, it does make more sense.
>
> If I create a custom facet can I return multiple values? Is this likely to
> be fast? Are there any tutorials / pointers on this?
It wil take quite a bit of coding but it's doable. If you write you own
facet you basically get to decide what to analyze and what json to return.
If done right it will be fast.
> > 1. Ask for top 500 terms, aggregate yourself, use the top 10 (or other
> > number of your choosing) and live with the potential in accuracies.
> > Elasticsearch does something similar with term facet anyway when
> > aggregating results from multiple nodes.
>
> We cant really live with inaccuracies. So ES does this anyway? Is there
> any details of this limitation and potential inaccuracy? Do you know
> exactly how many terms does it merge from each node? I really feel this
> should be written in big flashing bold letters on the
> facet documentation pages!
This discrepancies are inherit to the distributed nature of the problem.
It's the same one you run into as well. Think about the following - if you
have hundreds of gigabytes distributed on tens of machines bringing all the
data together will be really inefficient. Just moving a complete analysis
over the network will take a long time. Elasticsearch asks every node to do
the analysis locally and return the top terms back to the caller node. The
caller node then aggregates all the reports, adds things up and return it.
> > 1. Make two round trips - one to get the top terms (countries in your
> > example), then one to get the counts (limited to terms you want)
>
> How could I limit to the terms I want? I know there is an 'exclude' field
> for terms. Is there an 'include' one?
The term statistics facet doesn't support an exclude/include setup. I mean
changing the query to filter to the countries (in the example) you need.
This comes of course with a price.
>
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Alastair\_James](https://discuss.elastic.co/u/Alastair_James)
#### Post date: [May 13, 2013, 1:35pm UTC](https://discuss.elastic.co/t/facet-with-multiple-counts/11926/7 "2013-05-13T13:35:57Z")
Hi there. Thanks again.
Are there any examples for custom facets?
I quite understand about why only the top X terms are sent from each node
and combined, it would just be really nice to know what X is. At the moment
its basically saying "at some point, your facets are going to
become inaccurate, but you simply dont know when or if this ever occurs".
Would be nice to be able to get a feel for where this point is.
Cheers
On 13 May 2013 13:49, Boaz Leskes [b.leskes@gmail.com](mailto:b.leskes@gmail.com) wrote:
> Hi,
>
> See my answers bellow.
>
> Boaz
>
> On Mon, May 13, 2013 at 1:28 PM, Alastair James [al.james@gmail.com](mailto:al.james@gmail.com)wrote:
>
> > Hi. Thanks again.
> >
> > Yes, it does make more sense.
> >
> > If I create a custom facet can I return multiple values? Is this likely
> > to be fast? Are there any tutorials / pointers on this?
>
> It wil take quite a bit of coding but it's doable. If you write you own
> facet you basically get to decide what to analyze and what json to return.
> If done right it will be fast.
>
> > > 1. Ask for top 500 terms, aggregate yourself, use the top 10 (or other
> > > number of your choosing) and live with the potential in accuracies.
> > > Elasticsearch does something similar with term facet anyway when
> > > aggregating results from multiple nodes.
> >
> > We cant really live with inaccuracies. So ES does this anyway? Is there
> > any details of this limitation and potential inaccuracy? Do you know
> > exactly how many terms does it merge from each node? I really feel this
> > should be written in big flashing bold letters on the
> > facet documentation pages!
>
> This discrepancies are inherit to the distributed nature of the problem.
> It's the same one you run into as well. Think about the following - if you
> have hundreds of gigabytes distributed on tens of machines bringing all the
> data together will be really inefficient. Just moving a complete analysis
> over the network will take a long time. Elasticsearch asks every node to do
> the analysis locally and return the top terms back to the caller node. The
> caller node then aggregates all the reports, adds things up and return it.
>
> > > 1. Make two round trips - one to get the top terms (countries in your
> > > example), then one to get the counts (limited to terms you want)
> >
> > How could I limit to the terms I want? I know there is an 'exclude' field
> > for terms. Is there an 'include' one?
>
> The term statistics facet doesn't support an exclude/include setup. I mean
> changing the query to filter to the countries (in the example) you need.
> This comes of course with a price.
>
> > --
> > You received this message because you are subscribed to a topic in the
> > Google Groups "elasticsearch" group.
> > To unsubscribe from this topic, visit
> > [https://groups.google.com/d/topic/elasticsearch/OnBbz-nj194/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/OnBbz-nj194/unsubscribe?hl=en-US)
> > .
> > To unsubscribe from this group and all its topics, send an email to
> > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
Dr Alastair James
CTO [Ometria.com](http://Ometria.com)
Skype: al.james
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Boaz\_Leskes](https://discuss.elastic.co/u/Boaz_Leskes)
#### Post date: [May 14, 2013, 7:18am UTC](https://discuss.elastic.co/t/facet-with-multiple-counts/11926/8 "2013-05-14T07:18:03Z")
Hi,
There are a couple of plugins around that add a facet and, of course, there
is the code of Elasticsearch itself. A quick googling brings up the
following: [GitHub - imotov/elasticsearch-facet-script: Fully Scriptable Facets for ElasticSearch](https://github.com/imotov/elasticsearch-facet-script) ,
[GitHub - crate/elasticsearch-timefacets-plugin: Elasticsearch Timebased Facets](https://github.com/lovelysystems/elasticsearch-ls-plugins) ,
[GitHub - bleskes/elasticfacets: A set of facets and related tools for ElasticSearch](https://github.com/bleskes/elasticfacets) . Not all of them are up to date
with the latest version of Elasticsearch (mine for example is still at
0.20.x) which also shows you the down side of this approach. The faceting
api is still under heavy development and changes quite a bit (which is why
the classes all have internal in their name 🙂 ).
About the number of entries generated per shard - I've looked at the code
at it is now the size you actually asked for. So if you set your size to a
100 in the request it will retrieve 100 entries per shard and merge them to
a 100 entries output. I think it makes sense to make this configurable in
the future but at least for now that's not possible.
Cheers,
Boaz
On Mon, May 13, 2013 at 3:35 PM, Alastair James [al.james@gmail.com](mailto:al.james@gmail.com) wrote:
> Hi there. Thanks again.
>
> Are there any examples for custom facets?
>
> I quite understand about why only the top X terms are sent from each node
> and combined, it would just be really nice to know what X is. At the moment
> its basically saying "at some point, your facets are going to
> become inaccurate, but you simply dont know when or if this ever occurs".
> Would be nice to be able to get a feel for where this point is.
>
> Cheers
>
> On 13 May 2013 13:49, Boaz Leskes [b.leskes@gmail.com](mailto:b.leskes@gmail.com) wrote:
>
> > Hi,
> >
> > See my answers bellow.
> >
> > Boaz
> >
> > On Mon, May 13, 2013 at 1:28 PM, Alastair James [al.james@gmail.com](mailto:al.james@gmail.com)wrote:
> >
> > > Hi. Thanks again.
> > >
> > > Yes, it does make more sense.
> > >
> > > If I create a custom facet can I return multiple values? Is this likely
> > > to be fast? Are there any tutorials / pointers on this?
> >
> > It wil take quite a bit of coding but it's doable. If you write you own
> > facet you basically get to decide what to analyze and what json to return.
> > If done right it will be fast.
> >
> > > > 1. Ask for top 500 terms, aggregate yourself, use the top 10 (or other
> > > > number of your choosing) and live with the potential in accuracies.
> > > > Elasticsearch does something similar with term facet anyway when
> > > > aggregating results from multiple nodes.
> > >
> > > We cant really live with inaccuracies. So ES does this anyway? Is there
> > > any details of this limitation and potential inaccuracy? Do you know
> > > exactly how many terms does it merge from each node? I really feel this
> > > should be written in big flashing bold letters on the
> > > facet documentation pages!
> >
> > This discrepancies are inherit to the distributed nature of the problem.
> > It's the same one you run into as well. Think about the following - if you
> > have hundreds of gigabytes distributed on tens of machines bringing all the
> > data together will be really inefficient. Just moving a complete analysis
> > over the network will take a long time. Elasticsearch asks every node to do
> > the analysis locally and return the top terms back to the caller node. The
> > caller node then aggregates all the reports, adds things up and return it.
> >
> > > > 1. Make two round trips - one to get the top terms (countries in your
> > > > example), then one to get the counts (limited to terms you want)
> > >
> > > How could I limit to the terms I want? I know there is an 'exclude'
> > > field for terms. Is there an 'include' one?
> >
> > The term statistics facet doesn't support an exclude/include setup. I
> > mean changing the query to filter to the countries (in the example) you
> > need. This comes of course with a price.
> >
> > > --
> > > You received this message because you are subscribed to a topic in the
> > > Google Groups "elasticsearch" group.
> > > To unsubscribe from this topic, visit
> > > [https://groups.google.com/d/topic/elasticsearch/OnBbz-nj194/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/OnBbz-nj194/unsubscribe?hl=en-US)
> > > .
> > > To unsubscribe from this group and all its topics, send an email to
> > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
>
> --
> Dr Alastair James
> CTO [Ometria.com](http://Ometria.com)
> Skype: al.james
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> [https://groups.google.com/d/topic/elasticsearch/OnBbz-nj194/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/OnBbz-nj194/unsubscribe?hl=en-US)
> .
> To unsubscribe from this group and all its topics, send an email to
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Alastair\_James](https://discuss.elastic.co/u/Alastair_James)
#### Post date: [May 14, 2013, 7:56am UTC](https://discuss.elastic.co/t/facet-with-multiple-counts/11926/9 "2013-05-14T07:56:50Z")
Ok, thanks.
Good to know the number. I will see if requesting 10x the actual number
required helps.
Cheers
Al
On 14 May 2013 08:18, Boaz Leskes [b.leskes@gmail.com](mailto:b.leskes@gmail.com) wrote:
> Hi,
>
> There are a couple of plugins around that add a facet and, of course,
> there is the code of Elasticsearch itself. A quick googling brings up the
> following: [GitHub - imotov/elasticsearch-facet-script: Fully Scriptable Facets for ElasticSearch](https://github.com/imotov/elasticsearch-facet-script) ,
> [GitHub - crate/elasticsearch-timefacets-plugin: Elasticsearch Timebased Facets](https://github.com/lovelysystems/elasticsearch-ls-plugins) ,
> [GitHub - bleskes/elasticfacets: A set of facets and related tools for ElasticSearch](https://github.com/bleskes/elasticfacets) . Not all of them are up to date
> with the latest version of Elasticsearch (mine for example is still at
> 0.20.x) which also shows you the down side of this approach. The faceting
> api is still under heavy development and changes quite a bit (which is why
> the classes all have internal in their name 🙂 ).
>
> About the number of entries generated per shard - I've looked at the code
> at it is now the size you actually asked for. So if you set your size to a
> 100 in the request it will retrieve 100 entries per shard and merge them to
> a 100 entries output. I think it makes sense to make this configurable in
> the future but at least for now that's not possible.
>
> Cheers,
> Boaz
>
> On Mon, May 13, 2013 at 3:35 PM, Alastair James [al.james@gmail.com](mailto:al.james@gmail.com)wrote:
>
> > Hi there. Thanks again.
> >
> > Are there any examples for custom facets?
> >
> > I quite understand about why only the top X terms are sent from each node
> > and combined, it would just be really nice to know what X is. At the moment
> > its basically saying "at some point, your facets are going to
> > become inaccurate, but you simply dont know when or if this ever occurs".
> > Would be nice to be able to get a feel for where this point is.
> >
> > Cheers
> >
> > On 13 May 2013 13:49, Boaz Leskes [b.leskes@gmail.com](mailto:b.leskes@gmail.com) wrote:
> >
> > > Hi,
> > >
> > > See my answers bellow.
> > >
> > > Boaz
> > >
> > > On Mon, May 13, 2013 at 1:28 PM, Alastair James [al.james@gmail.com](mailto:al.james@gmail.com)wrote:
> > >
> > > > Hi. Thanks again.
> > > >
> > > > Yes, it does make more sense.
> > > >
> > > > If I create a custom facet can I return multiple values? Is this likely
> > > > to be fast? Are there any tutorials / pointers on this?
> > >
> > > It wil take quite a bit of coding but it's doable. If you write you own
> > > facet you basically get to decide what to analyze and what json to return.
> > > If done right it will be fast.
> > >
> > > > > 1. Ask for top 500 terms, aggregate yourself, use the top 10 (or
> > > > > other number of your choosing) and live with the potential in accuracies.
> > > > > Elasticsearch does something similar with term facet anyway when
> > > > > aggregating results from multiple nodes.
> > > >
> > > > We cant really live with inaccuracies. So ES does this anyway? Is there
> > > > any details of this limitation and potential inaccuracy? Do you know
> > > > exactly how many terms does it merge from each node? I really feel this
> > > > should be written in big flashing bold letters on the
> > > > facet documentation pages!
> > >
> > > This discrepancies are inherit to the distributed nature of the problem.
> > > It's the same one you run into as well. Think about the following - if you
> > > have hundreds of gigabytes distributed on tens of machines bringing all the
> > > data together will be really inefficient. Just moving a complete analysis
> > > over the network will take a long time. Elasticsearch asks every node to do
> > > the analysis locally and return the top terms back to the caller node. The
> > > caller node then aggregates all the reports, adds things up and return it.
> > >
> > > > > 1. Make two round trips - one to get the top terms (countries in your
> > > > > example), then one to get the counts (limited to terms you want)
> > > >
> > > > How could I limit to the terms I want? I know there is an 'exclude'
> > > > field for terms. Is there an 'include' one?
> > >
> > > The term statistics facet doesn't support an exclude/include setup. I
> > > mean changing the query to filter to the countries (in the example) you
> > > need. This comes of course with a price.
> > >
> > > > --
> > > > You received this message because you are subscribed to a topic in the
> > > > Google Groups "elasticsearch" group.
> > > > To unsubscribe from this topic, visit
> > > > [https://groups.google.com/d/topic/elasticsearch/OnBbz-nj194/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/OnBbz-nj194/unsubscribe?hl=en-US)
> > > > .
> > > > To unsubscribe from this group and all its topics, send an email to
> > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> >
> > --
> > Dr Alastair James
> > CTO [Ometria.com](http://Ometria.com)
> > Skype: al.james
> >
> > --
> > You received this message because you are subscribed to a topic in the
> > Google Groups "elasticsearch" group.
> > To unsubscribe from this topic, visit
> > [https://groups.google.com/d/topic/elasticsearch/OnBbz-nj194/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/OnBbz-nj194/unsubscribe?hl=en-US)
> > .
> > To unsubscribe from this group and all its topics, send an email to
> > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> [https://groups.google.com/d/topic/elasticsearch/OnBbz-nj194/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/OnBbz-nj194/unsubscribe?hl=en-US)
> .
> To unsubscribe from this group and all its topics, send an email to
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
Dr Alastair James
CTO [Ometria.com](http://Ometria.com)
Skype: al.james
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 2:36am UTC](https://discuss.elastic.co/t/facet-with-multiple-counts/11926/10 "2017-07-06T02:36:50Z")