Facets have got me down

It might be because I'm low on energy, but I'm having a real tough time
reconciling facet structure in my head right now. Does anyone know of a
good gist/blog example reference with some real world examples? I've been
good with the documentation except for facets, even though the samples work
well.

I'll give you an example below. Every example has the facet grouping
explicitly set:

"facets": {
    "colors": {  <- How is this grouping field dynamic to just match the field name?
        "terms": {
            "field": "colors",
            "all_terms": true
        }
    }

I'm also struggling with documents that can contain any variety of facets.
Some have none, some have many.

Thanks!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

I'm afraid I did not understand your question.
Here is some comment.
"facets": {
"colors": { <- this is a name you give to the facet
"terms": {
"field": "colors", <- the field you compute on
"all_terms": true
}
}
What is your concern here?

You just add this facet request with the search request and you get back hits and computed facet (with the same name you give).

Did you read this page? http://www.elasticsearch.org/guide/reference/api/search/facets/

HTH

David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 févr. 2013 à 22:00, jtreher@gmail.com a écrit :

It might be because I'm low on energy, but I'm having a real tough time reconciling facet structure in my head right now. Does anyone know of a good gist/blog example reference with some real world examples? I've been good with the documentation except for facets, even though the samples work well.

I'll give you an example below. Every example has the facet grouping explicitly set:

"facets": {
    "colors": {  <- How is this grouping field dynamic to just match the field name?
        "terms": {
            "field": "colors",
            "all_terms": true
        }
    }

I'm also struggling with documents that can contain any variety of facets. Some have none, some have many.

Thanks!

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

In the end, I'm trying to get the faceting count to work "automatically"
when I'm not filtering by facet, but just using an ad hoc query.

Every one of my documents has a child object called "facets" that has all
the key:value attributes that belong to it to describe it. One document
might have the colors key and another might have the genre key. With an ad
hoc query, I'd like ES to dive into the facets child object and group
together any matches just like it does with the standard syntax. However,
the facet "group" name will be one of the thousands of "keys."

input:
title: Mayhem
facets:
genre:horror
format: vhs
title: VHS player
facets:
remote control: yes
format: vhs

Search: Mayhem OR Player

output:
....
facets
genre
horror(1)
format
vhs(2)
remote control
yes(1)

On Wednesday, February 6, 2013 4:12:51 PM UTC-5, David Pilato wrote:

I'm afraid I did not understand your question.
Here is some comment.

"facets": {
    "colors": {  <- this is a name you give to the facet
        "terms": {
            "field": "colors", <- the field you compute on
            "all_terms": true
        }
    }

What is your concern here?

You just add this facet request with the search request and you get back
hits and computed facet (with the same name you give).

Did you read this page?
http://www.elasticsearch.org/guide/reference/api/search/facets/

HTH

David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 févr. 2013 à 22:00, jtr...@gmail.com <javascript:> a écrit :

It might be because I'm low on energy, but I'm having a real tough time
reconciling facet structure in my head right now. Does anyone know of a
good gist/blog example reference with some real world examples? I've been
good with the documentation except for facets, even though the samples work
well.

I'll give you an example below. Every example has the facet grouping
explicitly set:

"facets": {
    "colors": {  <- How is this grouping field dynamic to just match the field name?
        "terms": {
            "field": "colors",
            "all_terms": true
        }
    }

I'm also struggling with documents that can contain any variety of facets.
Some have none, some have many.

Thanks!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

So you don't know what fields you want to facet on? It sounds like
you want to facet on any field that is in the "facets" object, but you
don't really know what is there? If this is fact what you are trying
to do, then you could always pull the mapping for your index and parse
out any dynamic fields under the facets object then use those to
create the facets in your query.

On Wed, Feb 6, 2013 at 1:34 PM, jtreher@gmail.com wrote:

In the end, I'm trying to get the faceting count to work "automatically"
when I'm not filtering by facet, but just using an ad hoc query.

Every one of my documents has a child object called "facets" that has all
the key:value attributes that belong to it to describe it. One document
might have the colors key and another might have the genre key. With an ad
hoc query, I'd like ES to dive into the facets child object and group
together any matches just like it does with the standard syntax. However,
the facet "group" name will be one of the thousands of "keys."

input:
title: Mayhem
facets:
genre:horror
format: vhs
title: VHS player
facets:
remote control: yes
format: vhs

Search: Mayhem OR Player

output:
....
facets
genre
horror(1)
format
vhs(2)
remote control
yes(1)

On Wednesday, February 6, 2013 4:12:51 PM UTC-5, David Pilato wrote:

I'm afraid I did not understand your question.
Here is some comment.

"facets": {
    "colors": {  <- this is a name you give to the facet
        "terms": {
            "field": "colors", <- the field you compute on
            "all_terms": true
        }
    }

What is your concern here?

You just add this facet request with the search request and you get back
hits and computed facet (with the same name you give).

Did you read this page?
http://www.elasticsearch.org/guide/reference/api/search/facets/

HTH

David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 févr. 2013 à 22:00, jtr...@gmail.com a écrit :

It might be because I'm low on energy, but I'm having a real tough time
reconciling facet structure in my head right now. Does anyone know of a good
gist/blog example reference with some real world examples? I've been good
with the documentation except for facets, even though the samples work well.

I'll give you an example below. Every example has the facet grouping
explicitly set:

"facets": {
    "colors": {  <- How is this grouping field dynamic to just match

the field name?
"terms": {
"field": "colors",
"all_terms": true
}
}

I'm also struggling with documents that can contain any variety of facets.
Some have none, some have many.

Thanks!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

When I'm actually doing a facet filter, I will know which facet(s) to
filter on, but upon the initial ad hoc query, I will need to present the
facets to the user, grouped by count. Regardless, even when using a facet
filter or two, I'd still like to present the remaining, unused facets to
the user grouped by count.

On Wednesday, February 6, 2013 5:31:33 PM UTC-5, Matt Weber wrote:

So you don't know what fields you want to facet on? It sounds like
you want to facet on any field that is in the "facets" object, but you
don't really know what is there? If this is fact what you are trying
to do, then you could always pull the mapping for your index and parse
out any dynamic fields under the facets object then use those to
create the facets in your query.

On Wed, Feb 6, 2013 at 1:34 PM, <jtr...@gmail.com <javascript:>> wrote:

In the end, I'm trying to get the faceting count to work
"automatically"
when I'm not filtering by facet, but just using an ad hoc query.

Every one of my documents has a child object called "facets" that has
all
the key:value attributes that belong to it to describe it. One document
might have the colors key and another might have the genre key. With an
ad
hoc query, I'd like ES to dive into the facets child object and group
together any matches just like it does with the standard syntax.
However,
the facet "group" name will be one of the thousands of "keys."

input:
title: Mayhem
facets:
genre:horror
format: vhs
title: VHS player
facets:
remote control: yes
format: vhs

Search: Mayhem OR Player

output:
....
facets
genre
horror(1)
format
vhs(2)
remote control
yes(1)

On Wednesday, February 6, 2013 4:12:51 PM UTC-5, David Pilato wrote:

I'm afraid I did not understand your question.
Here is some comment.

"facets": { 
    "colors": {  <- this is a name you give to the facet 
        "terms": { 
            "field": "colors", <- the field you compute on 
            "all_terms": true 
        } 
    } 

What is your concern here?

You just add this facet request with the search request and you get
back

hits and computed facet (with the same name you give).

Did you read this page?
http://www.elasticsearch.org/guide/reference/api/search/facets/

HTH

David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 févr. 2013 à 22:00, jtr...@gmail.com a écrit :

It might be because I'm low on energy, but I'm having a real tough time
reconciling facet structure in my head right now. Does anyone know of a
good

gist/blog example reference with some real world examples? I've been
good

with the documentation except for facets, even though the samples work
well.

I'll give you an example below. Every example has the facet grouping
explicitly set:

"facets": { 
    "colors": {  <- How is this grouping field dynamic to just 

match

the field name?
"terms": {
"field": "colors",
"all_terms": true
}
}

I'm also struggling with documents that can contain any variety of
facets.

Some have none, some have many.

Thanks!

--
You received this message because you are subscribed to the Google
Groups

"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an

email to elasticsearc...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

@Matt Weber

Were you suggesting that I run a query to get ids and then run a subsequent
query using facets found in those results to get the appropriate count,
restricting the scope of the count to the id set?

Thanks!

On Thursday, February 7, 2013 9:39:17 AM UTC-5, jtr...@gmail.com wrote:

When I'm actually doing a facet filter, I will know which facet(s) to
filter on, but upon the initial ad hoc query, I will need to present the
facets to the user, grouped by count. Regardless, even when using a facet
filter or two, I'd still like to present the remaining, unused facets to
the user grouped by count.

On Wednesday, February 6, 2013 5:31:33 PM UTC-5, Matt Weber wrote:

So you don't know what fields you want to facet on? It sounds like
you want to facet on any field that is in the "facets" object, but you
don't really know what is there? If this is fact what you are trying
to do, then you could always pull the mapping for your index and parse
out any dynamic fields under the facets object then use those to
create the facets in your query.

On Wed, Feb 6, 2013 at 1:34 PM, jtr...@gmail.com wrote:

In the end, I'm trying to get the faceting count to work
"automatically"
when I'm not filtering by facet, but just using an ad hoc query.

Every one of my documents has a child object called "facets" that has
all
the key:value attributes that belong to it to describe it. One document
might have the colors key and another might have the genre key. With an
ad
hoc query, I'd like ES to dive into the facets child object and group
together any matches just like it does with the standard syntax.
However,
the facet "group" name will be one of the thousands of "keys."

input:
title: Mayhem
facets:
genre:horror
format: vhs
title: VHS player
facets:
remote control: yes
format: vhs

Search: Mayhem OR Player

output:
....
facets
genre
horror(1)
format
vhs(2)
remote control
yes(1)

On Wednesday, February 6, 2013 4:12:51 PM UTC-5, David Pilato wrote:

I'm afraid I did not understand your question.
Here is some comment.

"facets": { 
    "colors": {  <- this is a name you give to the facet 
        "terms": { 
            "field": "colors", <- the field you compute on 
            "all_terms": true 
        } 
    } 

What is your concern here?

You just add this facet request with the search request and you get
back

hits and computed facet (with the same name you give).

Did you read this page?
http://www.elasticsearch.org/guide/reference/api/search/facets/

HTH

David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 févr. 2013 à 22:00, jtr...@gmail.com a écrit :

It might be because I'm low on energy, but I'm having a real tough
time

reconciling facet structure in my head right now. Does anyone know of
a good

gist/blog example reference with some real world examples? I've been
good

with the documentation except for facets, even though the samples work
well.

I'll give you an example below. Every example has the facet grouping
explicitly set:

"facets": { 
    "colors": {  <- How is this grouping field dynamic to just 

match

the field name?
"terms": {
"field": "colors",
"all_terms": true
}
}

I'm also struggling with documents that can contain any variety of
facets.

Some have none, some have many.

Thanks!

--
You received this message because you are subscribed to the Google
Groups

"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an

email to elasticsearc...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

I did find this thread with a reply from the master himself. So, there is
no out of the box way to do what all of us what for an "ecommerce" style
mixed bag lot of filters for thousands of products.
https://groups.google.com/d/topic/elasticsearch/3_Oer8hT-9U/discussion

I agree with the OP about the ugly concatenation, but I also see this as a
suggested SOLR solution on StackOverflow. I think it would be nice to post
the "work around" within the documentation because it's really not clear
that it's not possible. Another to do for myself when I look at editing the
docs!

On Tuesday, February 12, 2013 1:02:26 PM UTC-5, jtr...@gmail.com wrote:

@Matt Weber

Were you suggesting that I run a query to get ids and then run a
subsequent query using facets found in those results to get the appropriate
count, restricting the scope of the count to the id set?

Thanks!

On Thursday, February 7, 2013 9:39:17 AM UTC-5, jtr...@gmail.com wrote:

When I'm actually doing a facet filter, I will know which facet(s) to
filter on, but upon the initial ad hoc query, I will need to present the
facets to the user, grouped by count. Regardless, even when using a facet
filter or two, I'd still like to present the remaining, unused facets to
the user grouped by count.

On Wednesday, February 6, 2013 5:31:33 PM UTC-5, Matt Weber wrote:

So you don't know what fields you want to facet on? It sounds like
you want to facet on any field that is in the "facets" object, but you
don't really know what is there? If this is fact what you are trying
to do, then you could always pull the mapping for your index and parse
out any dynamic fields under the facets object then use those to
create the facets in your query.

On Wed, Feb 6, 2013 at 1:34 PM, jtr...@gmail.com wrote:

In the end, I'm trying to get the faceting count to work
"automatically"
when I'm not filtering by facet, but just using an ad hoc query.

Every one of my documents has a child object called "facets" that has
all
the key:value attributes that belong to it to describe it. One
document
might have the colors key and another might have the genre key. With
an ad
hoc query, I'd like ES to dive into the facets child object and group
together any matches just like it does with the standard syntax.
However,
the facet "group" name will be one of the thousands of "keys."

input:
title: Mayhem
facets:
genre:horror
format: vhs
title: VHS player
facets:
remote control: yes
format: vhs

Search: Mayhem OR Player

output:
....
facets
genre
horror(1)
format
vhs(2)
remote control
yes(1)

On Wednesday, February 6, 2013 4:12:51 PM UTC-5, David Pilato wrote:

I'm afraid I did not understand your question.
Here is some comment.

"facets": { 
    "colors": {  <- this is a name you give to the facet 
        "terms": { 
            "field": "colors", <- the field you compute on 
            "all_terms": true 
        } 
    } 

What is your concern here?

You just add this facet request with the search request and you get
back

hits and computed facet (with the same name you give).

Did you read this page?
http://www.elasticsearch.org/guide/reference/api/search/facets/

HTH

David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 févr. 2013 à 22:00, jtr...@gmail.com a écrit :

It might be because I'm low on energy, but I'm having a real tough
time

reconciling facet structure in my head right now. Does anyone know of
a good

gist/blog example reference with some real world examples? I've been
good

with the documentation except for facets, even though the samples
work well.

I'll give you an example below. Every example has the facet grouping
explicitly set:

"facets": { 
    "colors": {  <- How is this grouping field dynamic to just 

match

the field name?
"terms": {
"field": "colors",
"all_terms": true
}
}

I'm also struggling with documents that can contain any variety of
facets.

Some have none, some have many.

Thanks!

--
You received this message because you are subscribed to the Google
Groups

"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an

email to elasticsearc...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

I still am not 100% clear on what you are trying to accomplish. It
sounded like you have a field called "facets" that is an object.
Within that there can be any number of fields (genre, format,
has_remote, etc), but you don't necessarily know these field names
because each product you index can have it's own set. Since ES
requires that you know what field you want to facet on at query time,
ie. you need to specify a field name, you are out of luck because you
don't know the field names. I was suggesting you pull the mapping of
the index you are searching across, then inspect the names of the
fields within the "facets" object. You can then use these names at
query time to setup your facets. Now I wouldn't pull the mapping for
every query, but you could do it on some pre-defined interval and
cache the results within your application.

On Tue, Feb 12, 2013 at 10:02 AM, jtreher@gmail.com wrote:

@Matt Weber

Were you suggesting that I run a query to get ids and then run a subsequent
query using facets found in those results to get the appropriate count,
restricting the scope of the count to the id set?

Thanks!

On Thursday, February 7, 2013 9:39:17 AM UTC-5, jtr...@gmail.com wrote:

When I'm actually doing a facet filter, I will know which facet(s) to
filter on, but upon the initial ad hoc query, I will need to present the
facets to the user, grouped by count. Regardless, even when using a facet
filter or two, I'd still like to present the remaining, unused facets to the
user grouped by count.

On Wednesday, February 6, 2013 5:31:33 PM UTC-5, Matt Weber wrote:

So you don't know what fields you want to facet on? It sounds like
you want to facet on any field that is in the "facets" object, but you
don't really know what is there? If this is fact what you are trying
to do, then you could always pull the mapping for your index and parse
out any dynamic fields under the facets object then use those to
create the facets in your query.

On Wed, Feb 6, 2013 at 1:34 PM, jtr...@gmail.com wrote:

In the end, I'm trying to get the faceting count to work
"automatically"
when I'm not filtering by facet, but just using an ad hoc query.

Every one of my documents has a child object called "facets" that has
all
the key:value attributes that belong to it to describe it. One document
might have the colors key and another might have the genre key. With an
ad
hoc query, I'd like ES to dive into the facets child object and group
together any matches just like it does with the standard syntax.
However,
the facet "group" name will be one of the thousands of "keys."

input:
title: Mayhem
facets:
genre:horror
format: vhs
title: VHS player
facets:
remote control: yes
format: vhs

Search: Mayhem OR Player

output:
....
facets
genre
horror(1)
format
vhs(2)
remote control
yes(1)

On Wednesday, February 6, 2013 4:12:51 PM UTC-5, David Pilato wrote:

I'm afraid I did not understand your question.
Here is some comment.

"facets": {
    "colors": {  <- this is a name you give to the facet
        "terms": {
            "field": "colors", <- the field you compute on
            "all_terms": true
        }
    }

What is your concern here?

You just add this facet request with the search request and you get
back
hits and computed facet (with the same name you give).

Did you read this page?
http://www.elasticsearch.org/guide/reference/api/search/facets/

HTH

David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 févr. 2013 à 22:00, jtr...@gmail.com a écrit :

It might be because I'm low on energy, but I'm having a real tough
time
reconciling facet structure in my head right now. Does anyone know of
a good
gist/blog example reference with some real world examples? I've been
good
with the documentation except for facets, even though the samples work
well.

I'll give you an example below. Every example has the facet grouping
explicitly set:

"facets": {
    "colors": {  <- How is this grouping field dynamic to just

match
the field name?
"terms": {
"field": "colors",
"all_terms": true
}
}

I'm also struggling with documents that can contain any variety of
facets.
Some have none, some have many.

Thanks!

--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to elasticsearc...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

@Matt,

I might have a grip on it now. I think the major disjunction that I and
others in my situation are having is the fundamental concept of faceted
navigation.

There are some sites, such as a real estate site or a travel site which use
the same standard list of facets for every search result. location, price,
to, from, # of bedrooms, "other options" etc, etc. The facet toolkit
out-of-the-box provided by ES works for this type of navigation. That's
what the documentation uses for an example.

However, for organizations that do ecommerce, your typical faceted
navigation is dependent on the category or products that you are browsing.
And, for an "ad hoc" query, you might pull in products from multiple
domains, therefore the facets are dynamic based on the results. For
instance, if you sell speakers, shirts, and ovens, you have facets for
those domains. There is no out of the box way to tell SOLR or ES as far as
I can tell, to use complex facets consisting of key/value pairs (except for
the mathematical ones). For instance, if I could give ES the ideal
structure below and have it consume a key/value field for facet name/type
it would work out. The workaround to give ES a concatenation of key+value
works, it's just that it feels wrong. So, I store both the ideal version,
when I know the facet and the workaround version for ad hoc queries to get
counts. The application can break apart the workaround easily enough.

Ideal:
{facets:[
{facet_name:name1,
facet_value:value1},
{facet_name:name2,
facet_value:value2}
]

Workaround to "flatten facets". In this case ### is just a place holder for
the app to break apart the strings.
facets:[name1###value1,name2###value2]

Finally, I know that sites like Amazon force you to pick a category before
presenting domain facets. This would give one the ability to find facets
for a category and setup a proper facet query for ES. However, our current
specs don't allow for this, but it's certainly something to consider.

On Tuesday, February 12, 2013 2:50:14 PM UTC-5, Matt Weber wrote:

I still am not 100% clear on what you are trying to accomplish. It
sounded like you have a field called "facets" that is an object.
Within that there can be any number of fields (genre, format,
has_remote, etc), but you don't necessarily know these field names
because each product you index can have it's own set. Since ES
requires that you know what field you want to facet on at query time,
ie. you need to specify a field name, you are out of luck because you
don't know the field names. I was suggesting you pull the mapping of
the index you are searching across, then inspect the names of the
fields within the "facets" object. You can then use these names at
query time to setup your facets. Now I wouldn't pull the mapping for
every query, but you could do it on some pre-defined interval and
cache the results within your application.

On Tue, Feb 12, 2013 at 10:02 AM, <jtr...@gmail.com <javascript:>>
wrote:

@Matt Weber

Were you suggesting that I run a query to get ids and then run a
subsequent
query using facets found in those results to get the appropriate count,
restricting the scope of the count to the id set?

Thanks!

On Thursday, February 7, 2013 9:39:17 AM UTC-5, jtr...@gmail.com wrote:

When I'm actually doing a facet filter, I will know which facet(s) to
filter on, but upon the initial ad hoc query, I will need to present
the

facets to the user, grouped by count. Regardless, even when using a
facet

filter or two, I'd still like to present the remaining, unused facets
to the

user grouped by count.

On Wednesday, February 6, 2013 5:31:33 PM UTC-5, Matt Weber wrote:

So you don't know what fields you want to facet on? It sounds like
you want to facet on any field that is in the "facets" object, but you
don't really know what is there? If this is fact what you are trying
to do, then you could always pull the mapping for your index and parse
out any dynamic fields under the facets object then use those to
create the facets in your query.

On Wed, Feb 6, 2013 at 1:34 PM, jtr...@gmail.com wrote:

In the end, I'm trying to get the faceting count to work
"automatically"
when I'm not filtering by facet, but just using an ad hoc query.

Every one of my documents has a child object called "facets" that
has

all
the key:value attributes that belong to it to describe it. One
document

might have the colors key and another might have the genre key. With
an

ad
hoc query, I'd like ES to dive into the facets child object and
group

together any matches just like it does with the standard syntax.
However,
the facet "group" name will be one of the thousands of "keys."

input:
title: Mayhem
facets:
genre:horror
format: vhs
title: VHS player
facets:
remote control: yes
format: vhs

Search: Mayhem OR Player

output:
....
facets
genre
horror(1)
format
vhs(2)
remote control
yes(1)

On Wednesday, February 6, 2013 4:12:51 PM UTC-5, David Pilato wrote:

I'm afraid I did not understand your question.
Here is some comment.

"facets": { 
    "colors": {  <- this is a name you give to the facet 
        "terms": { 
            "field": "colors", <- the field you compute on 
            "all_terms": true 
        } 
    } 

What is your concern here?

You just add this facet request with the search request and you get
back
hits and computed facet (with the same name you give).

Did you read this page?
http://www.elasticsearch.org/guide/reference/api/search/facets/

HTH

David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 févr. 2013 à 22:00, jtr...@gmail.com a écrit :

It might be because I'm low on energy, but I'm having a real tough
time
reconciling facet structure in my head right now. Does anyone know
of

a good
gist/blog example reference with some real world examples? I've
been

good
with the documentation except for facets, even though the samples
work

well.

I'll give you an example below. Every example has the facet
grouping

explicitly set:

"facets": { 
    "colors": {  <- How is this grouping field dynamic to just 

match
the field name?
"terms": {
"field": "colors",
"all_terms": true
}
}

I'm also struggling with documents that can contain any variety of
facets.
Some have none, some have many.

Thanks!

--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send

an
email to elasticsearc...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send

an
email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

We do something similar to your workaround for our ecommerce app. All our
dynamic product attributes are faceted against one field. Our app then
flattens things on the client side before returning the data to the user.
For efficiency, we do not use strings, but bit-shifted numerical values. A
client side mapping converts numeric ids into text values since our
taxonomy is known.

--
Ivan

On Wed, Feb 13, 2013 at 5:19 AM, jtreher@gmail.com wrote:

@Matt,

I might have a grip on it now. I think the major disjunction that I and
others in my situation are having is the fundamental concept of faceted
navigation.

There are some sites, such as a real estate site or a travel site which
use the same standard list of facets for every search result. location,
price, to, from, # of bedrooms, "other options" etc, etc. The facet toolkit
out-of-the-box provided by ES works for this type of navigation. That's
what the documentation uses for an example.

However, for organizations that do ecommerce, your typical faceted
navigation is dependent on the category or products that you are browsing.
And, for an "ad hoc" query, you might pull in products from multiple
domains, therefore the facets are dynamic based on the results. For
instance, if you sell speakers, shirts, and ovens, you have facets for
those domains. There is no out of the box way to tell SOLR or ES as far as
I can tell, to use complex facets consisting of key/value pairs (except for
the mathematical ones). For instance, if I could give ES the ideal
structure below and have it consume a key/value field for facet name/type
it would work out. The workaround to give ES a concatenation of key+value
works, it's just that it feels wrong. So, I store both the ideal version,
when I know the facet and the workaround version for ad hoc queries to get
counts. The application can break apart the workaround easily enough.

Ideal:
{facets:[
{facet_name:name1,
facet_value:value1},
{facet_name:name2,
facet_value:value2}
]

Workaround to "flatten facets". In this case ### is just a place holder
for the app to break apart the strings.
facets:[name1###value1,name2###value2]

Finally, I know that sites like Amazon force you to pick a category
before presenting domain facets. This would give one the ability to find
facets for a category and setup a proper facet query for ES. However, our
current specs don't allow for this, but it's certainly something to
consider.

On Tuesday, February 12, 2013 2:50:14 PM UTC-5, Matt Weber wrote:

I still am not 100% clear on what you are trying to accomplish. It
sounded like you have a field called "facets" that is an object.
Within that there can be any number of fields (genre, format,
has_remote, etc), but you don't necessarily know these field names
because each product you index can have it's own set. Since ES
requires that you know what field you want to facet on at query time,
ie. you need to specify a field name, you are out of luck because you
don't know the field names. I was suggesting you pull the mapping of
the index you are searching across, then inspect the names of the
fields within the "facets" object. You can then use these names at
query time to setup your facets. Now I wouldn't pull the mapping for
every query, but you could do it on some pre-defined interval and
cache the results within your application.

On Tue, Feb 12, 2013 at 10:02 AM, jtr...@gmail.com wrote:

@Matt Weber

Were you suggesting that I run a query to get ids and then run a
subsequent
query using facets found in those results to get the appropriate count,
restricting the scope of the count to the id set?

Thanks!

On Thursday, February 7, 2013 9:39:17 AM UTC-5, jtr...@gmail.comwrote:

When I'm actually doing a facet filter, I will know which facet(s) to
filter on, but upon the initial ad hoc query, I will need to present
the

facets to the user, grouped by count. Regardless, even when using a
facet

filter or two, I'd still like to present the remaining, unused facets
to the

user grouped by count.

On Wednesday, February 6, 2013 5:31:33 PM UTC-5, Matt Weber wrote:

So you don't know what fields you want to facet on? It sounds like
you want to facet on any field that is in the "facets" object, but
you

don't really know what is there? If this is fact what you are trying
to do, then you could always pull the mapping for your index and
parse

out any dynamic fields under the facets object then use those to
create the facets in your query.

On Wed, Feb 6, 2013 at 1:34 PM, jtr...@gmail.com wrote:

In the end, I'm trying to get the faceting count to work
"automatically"
when I'm not filtering by facet, but just using an ad hoc query.

Every one of my documents has a child object called "facets" that
has

all
the key:value attributes that belong to it to describe it. One
document

might have the colors key and another might have the genre key.
With an

ad
hoc query, I'd like ES to dive into the facets child object and
group

together any matches just like it does with the standard syntax.
However,
the facet "group" name will be one of the thousands of "keys."

input:
title: Mayhem
facets:
genre:horror
format: vhs
title: VHS player
facets:
remote control: yes
format: vhs

Search: Mayhem OR Player

output:
....
facets
genre
horror(1)
format
vhs(2)
remote control
yes(1)

On Wednesday, February 6, 2013 4:12:51 PM UTC-5, David Pilato
wrote:

I'm afraid I did not understand your question.
Here is some comment.

"facets": {
    "colors": {  <- this is a name you give to the facet
        "terms": {
            "field": "colors", <- the field you compute on
            "all_terms": true
        }
    }

What is your concern here?

You just add this facet request with the search request and you
get

back
hits and computed facet (with the same name you give).

Did you read this page?
http://www.elasticsearch.org/guide/reference/api/search/
facets/ http://www.elasticsearch.org/guide/reference/api/search/facets/

HTH

David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 févr. 2013 à 22:00, jtr...@gmail.com a écrit :

It might be because I'm low on energy, but I'm having a real tough
time
reconciling facet structure in my head right now. Does anyone know
of

a good
gist/blog example reference with some real world examples? I've
been

good
with the documentation except for facets, even though the samples
work

well.

I'll give you an example below. Every example has the facet
grouping

explicitly set:

"facets": {
    "colors": {  <- How is this grouping field dynamic to just

match
the field name?
"terms": {
"field": "colors",
"all_terms": true
}
}

I'm also struggling with documents that can contain any variety of
facets.
Some have none, some have many.

Thanks!

--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send

an
email to elasticsearc...@googlegroups.**com.

For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send

an
email to elasticsearc...@googlegroups.**com.
For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to elasticsearc...@**googlegroups.com.
For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

I apologize in advance if I should have started new thread with this question, but it does have some correlation to the above. I also apologize for the long winded post, I just want to make sure I get the message across

Just a quick background on our solution here. We have built a Spring web app that sits on top of an ElasticSearch cluster that will allow the Search Business team to define all the facets, synonyms, query boosts, recommended products, etc, for the Search solution at their respective ecommerce site. For those of you that are familiar with FAST search, our Spring app is similar to their Business Manager Tool. So for facets, using the ES mappings API, we present to the user a list of all indices in the ES cluster, then based on their selection, we then present all the attributes/properties that are included in the selected indices, and then they ultimately pick what attribute/property to use for that particular facet. They then can do all the related stuff (i.e. if range, designate ranges, if term, how many values to initially display, etc). So we don’t necessarily have the “dynamic facet” issue that has been discussed here, but the challenge we are facing that is somewhat relevant to this discussion is how can we manage what “context” the facet should appear in. What I mean by that is, for a given facet, I only want it to be included if the customer is in the “Electronics” category, or I may want to slot it globally and return it with all queries. The problem is that over time, the business will potentially create hundreds of facets, and we cannot have every request to ES tagged with hundreds of facet filters. Even if performance was not an issue (and in most of our cases, performance is the only issue ), this will not work given the following scenario (and there are others like it).

The search business team will set up multiple price range facets, each intended to only be evaluated and returned given the context the customer may be in. For example, if the customer is in the TVs category, the price range facet may look like:

$0 - $200
$200 - $400
$400 - $600…

But if they are in the cables category, it might look like:
$0 - $5
$5 - $10
$10 - $15…

So the business will create two different facets with the corresponding ranges and then add the appropriate “category” context to it, and therein lies our challenge, how to implement this?

As jtreher mentioned above, if we only needed to support this within the customer’s browse experience (like Amazon), the solution would be rather straight forward, since we would always know what category context the customer is in at any point in time, and we could then extract those facets accordingly from our cache and include in our ES Query. However, the business requirement is that we also support this model when the customer issues a global/ad-hoc query. In this scenario, we are in a bit of a chicken/egg situation, in that we won’t know what contexts are included until we evaluate the initial results, and only then can we apply the appropriate facet filtering.

Again, for those of you that have worked with FAST and its ecommerce layer, you know that they support all of this out-of-the-box. I don’t know all the nitty, gritty details of how they have implemented their contextual capability, but what I do know and don’t like, is that they require a full re-index when new facets are created or their context is changed. So obviously they are storing some metadata with each document that indicates what facet/context it belongs to, and this is used at query time to derive facets. Therefore, search query performance is optimized, but at a pretty heavy cost with the full re-indexing.

That all being said, I am struggling to find the best solution with ES. The different options I am looking at now are:

  1. Two-step query – Invoke global query with category facet to get all potential category contexts, then go to business managed facet cache and find all facets that match categories returned from initial global query and resubmit query again with all the appropriate facet filters. Pros – should work, Cons – very concerned with performance impact of this with both additional query load and latency.

  2. Do something like what FAST does, and that is when business user creates facet with context in business tool, add some metadata to the corresponding documents in index and then leverage that metadata at query time to create necessary facets. This might be similar to the facet flattening solution described above. Pros- query performance should be better than option1, Cons – not clear on what would need to be implemented and if this would work with ES, don’t want to get into situation where re-indexing is required for every facet transaction.

  3. Draw line in the sand and only support contextual faceting under “Browse” scenario.

  4. Other solutions???

Thanks for any feedback and a big thanks to all that have contributed to ElasticSearch, it totally rocks!

1 Like