Facet-Search property/property-value


(maik) #1

Hi,

I'm evaluating es for our ecommerce enivironment and ran in a problem:

I want to use es with its facet search for building the frontend filters.
For example: The user could browse some usb sticks and filter all sticks on
"capacity" with the value "4 GB".
I need the list of possible properties and its values to build the html.

A product could look like this:

product {
properties : {
capacity : "4 GB"
speed : "32 MB/s"
}
name : "very good usb stick"
}

The properties of course differ with each product.
A result I'm searching for could be:

results {
property-facet : {
...
...
facets : [
{ {property: "capacity", value: "4 GB"}, count: 455 },
{ {property: "capacity", value: "8 GB"}, count: 133 }

  ]  
}

}

Is there any way to do this with es?
Perhaps with another data schema?

Thx in advance
Greetings
maik


(Karussell) #2

The schema is good. Did you already tried the term facets?

http://www.elasticsearch.org/guide/reference/api/search/facets/terms-facet.html

Or what are you after?

Regards,
Peter.

On 8 Dez., 11:02, maik maik2...@googlemail.com wrote:

Hi,

I'm evaluating es for our ecommerce enivironment and ran in a problem:

I want to use es with its facet search for building the frontend filters.
For example: The user could browse some usb sticks and filter all sticks on
"capacity" with the value "4 GB".
I need the list of possible properties and its values to build the html.

A product could look like this:

product {
properties : {
capacity : "4 GB"
speed : "32 MB/s"
}
name : "very good usb stick"

}

The properties of course differ with each product.
A result I'm searching for could be:

results {
property-facet : {
...
...
facets : [
{ {property: "capacity", value: "4 GB"}, count: 455 },
{ {property: "capacity", value: "8 GB"}, count: 133 }

  ]
}

}

Is there any way to do this with es?
Perhaps with another data schema?

Thx in advance
Greetings
maik


(maik) #3

Hello Peter,

yes I tried the term facet.
Perhaps I'm missing some details about it, but I could not discover how to
configure a term facet to use the dynamic combination of a property and its
values.

If there was a field like this:
{
tags: ["a", "b", "c"]
}

the facet looks like this:
"facets" : {
"tag" : {
"terms" : {
"field" : "tag"
}
}
}

But how to configure it, if the tags are not a single string, but a
document like this: tags : [{property: "capacity", value: "4 GB"},
{property: "speed", value: "32 MB/s"}]?
In this case the key is the document and not a single string.

Its difficult for me to describe the Problem in english, but I hope you
understand it :slight_smile:

Greetings
maik


(Karussell) #4

did you try "field" : "tags.property" ?

Peter.


(maik) #5

Yes, but this only results in something like that:

facets : [
"capacity": 100,
"speed": 150
]

There is no connection to the value, and this is needed.

I helped me out in a way i don't like:
properties : ["capacity###4 GB", "speed###32 MB/s"]

A facet on the properties field results like this:
facets: [
"capacity###4 GB" : 30,
"capacity###8 GB" : 20,
"speed###16 MB/s": 10,
"speed###32 MB/s" : 40
]

I have to split the properties and values in the application with the
split-argument "###".
Thats not very fast.


(Karussell) #6

There is no connection to the value, and this is needed.

I see, hmmh but IMO this is a valid approach. Normally you could try
scripting but in this case not

http://www.elasticsearch.org/guide/reference/api/search/facets/terms-stats-facet.html

Thats not very fast.

Do not overestimate this or did you measure it :)?

Peter.

On 9 Dez., 12:13, maik maik2...@googlemail.com wrote:

Yes, but this only results in something like that:

facets : [
"capacity": 100,
"speed": 150
]

There is no connection to the value, and this is needed.

I helped me out in a way i don't like:
properties : ["capacity###4 GB", "speed###32 MB/s"]

A facet on the properties field results like this:
facets: [
"capacity###4 GB" : 30,
"capacity###8 GB" : 20,
"speed###16 MB/s": 10,
"speed###32 MB/s" : 40
]

I have to split the properties and values in the application with the
split-argument "###".
Thats not very fast.


(maik) #7

i didnt measure it. :slight_smile:
In my prototype its fast enough, but i dont like the way.

Es is a document orientated datastore and i hoped it could solve this issue
without this splitting thing.

As I can see, es today can only do faceted search on keys, which are are
single token (like the tag example).
It would be great if someday hole documents could be key, so my problem
will be natively solved without doing something strange.

I hoped I only missed the feature in the documentation.

Thx peter for your help.

Greetings
maik


(Clinton Gormley) #8

On Fri, 2011-12-09 at 03:13 -0800, maik wrote:

Yes, but this only results in something like that:

facets : [
"capacity": 100,
"speed": 150
]

Why not have two facets, one for tags.property.capacity and one for
tags.property.speed

that should give you what you want

clint


(maik) #9

Hi clint,

that would be the way, if I only want to count the properties.
But I need the property- / property-value combination.

Beside, with your approach I have to know every combination before building
the search request.

greets
maik


(Shay Banon) #10

You can't do generic property name / generic property value faceting, aside
from maybe using scripting (depends on exactly what you are after). But, I
think from your question that you know that you want to facet on something
like property name of capacity, no? If so, why not index the property name
as part of the json field name and facet on it? Or do you want to get all
the facets for all the properties defined for all the products?

On Fri, Dec 9, 2011 at 2:00 PM, maik maik2102@googlemail.com wrote:

Hi clint,

that would be the way, if I only want to count the properties.
But I need the property- / property-value combination.

Beside, with your approach I have to know every combination before
building the search request.

greets
maik


(maik) #11

Hi kimchy,

no I don't know what exactly properties I have to facet on.
I used some examples here to show you how the properties COULD look like in
my product-documents.
Alle products have different properties and values, some more, some less.

I only have a set of products (result of the search) and need all the
combinations of the properties/values to generate the frontend filters.

So, to make it visual:
A customer searched for "usb stick", gets the search result with a list of
products and additionally these filters on the side:

https://lh3.googleusercontent.com/-tGBltZbDsQw/TuWplIH9h3I/AAAAAAAAAww/rEdW5aqVj9E/s1600/Bildschirmfoto%25202011-12-12%2520um%252008.13.27.png

I thought es could bring the data for this filters with its facet search.

Greetings

maik


(Shay Banon) #12

Since we have the mapping information, we might be able to do this, i.e.
automatically generate facets for all the properly names assuming they are
used as field names. But, you can also do it yourself, you can ask for the
mapping, it will include all the fields names options, and build the facets
request based on that. (note, the properly names will need to be fields in
the document).

On Mon, Dec 12, 2011 at 9:15 AM, maik maik2102@googlemail.com wrote:

Hi kimchy,

no I don't know what exactly properties I have to facet on.
I used some examples here to show you how the properties COULD look like
in my product-documents.
Alle products have different properties and values, some more, some less.

I only have a set of products (result of the search) and need all the
combinations of the properties/values to generate the frontend filters.

So, to make it visual:
A customer searched for "usb stick", gets the search result with a list of
products and additionally these filters on the side:

https://lh3.googleusercontent.com/-tGBltZbDsQw/TuWplIH9h3I/AAAAAAAAAww/rEdW5aqVj9E/s1600/Bildschirmfoto%25202011-12-12%2520um%252008.13.27.png

I thought es could bring the data for this filters with its facet search.

Greetings

maik


(Rauan Maemirov) #13

Hi, Shay. I'm trying to implement the same scheme as topicstarter.
I didn't understand what do you mean by 'automatically generate facets for
all the properly names assuming they are used as field names'.
Should I directly define mapping for every property in properties list I
have? But what if there are dynamic? I.e. in first document the properties
are {color: 'black', size: 'XL'}, in the second are {capacity: '2Gb',
generation: '5'}, etc. (and there may be a lot of types of product,so
different document types is not a desirable solution)

понедельник, 12 декабря 2011 г., 19:08:15 UTC+6 пользователь kimchy написал:

Since we have the mapping information, we might be able to do this, i.e.
automatically generate facets for all the properly names assuming they are
used as field names. But, you can also do it yourself, you can ask for the
mapping, it will include all the fields names options, and build the facets
request based on that. (note, the properly names will need to be fields in
the document).


(maik) #14

To give an update on this:
I used the "workaround" with the concatenating of the property and its
values.
The application has to split the facet-results and group it manually by the
properties.

The speed is good and so we'll use most likely that way for facetting our
properties in our application.

On Friday, December 9, 2011 12:13:43 PM UTC+1, maik wrote:

Yes, but this only results in something like that:

facets : [
"capacity": 100,
"speed": 150
]

There is no connection to the value, and this is needed.

I helped me out in a way i don't like:
properties : ["capacity###4 GB", "speed###32 MB/s"]

A facet on the properties field results like this:
facets: [
"capacity###4 GB" : 30,
"capacity###8 GB" : 20,
"speed###16 MB/s": 10,
"speed###32 MB/s" : 40
]

I have to split the properties and values in the application with the
split-argument "###".
Thats not very fast.


(Shay Banon) #15

I meant using the get mapping API to get the list of relevant fields you
might care about when you facet...

On Fri, Jun 1, 2012 at 9:06 AM, Rauan Maemirov rauan1987@gmail.com wrote:

Hi, Shay. I'm trying to implement the same scheme as topicstarter.
I didn't understand what do you mean by 'automatically generate facets for
all the properly names assuming they are used as field names'.
Should I directly define mapping for every property in properties list I
have? But what if there are dynamic? I.e. in first document the properties
are {color: 'black', size: 'XL'}, in the second are {capacity: '2Gb',
generation: '5'}, etc. (and there may be a lot of types of product,so
different document types is not a desirable solution)

понедельник, 12 декабря 2011 г., 19:08:15 UTC+6 пользователь kimchy
написал:

Since we have the mapping information, we might be able to do this, i.e.
automatically generate facets for all the properly names assuming they are
used as field names. But, you can also do it yourself, you can ask for the
mapping, it will include all the fields names options, and build the facets
request based on that. (note, the properly names will need to be fields in
the document).


(Rauan Maemirov) #16

Hey, maik.

Do you then filter values by concatenating you facet again? I.e. filtering
properties by "capacity###4 Gb" or do you have different fields for proper
filtering?

пятница, 1 июня 2012 г., 13:24:05 UTC+6 пользователь maik написал:

To give an update on this:
I used the "workaround" with the concatenating of the property and its
values.
The application has to split the facet-results and group it manually by
the properties.

The speed is good and so we'll use most likely that way for facetting our
properties in our application.

On Friday, December 9, 2011 12:13:43 PM UTC+1, maik wrote:

Yes, but this only results in something like that:

facets : [
"capacity": 100,
"speed": 150
]

There is no connection to the value, and this is needed.

I helped me out in a way i don't like:
properties : ["capacity###4 GB", "speed###32 MB/s"]

A facet on the properties field results like this:
facets: [
"capacity###4 GB" : 30,
"capacity###8 GB" : 20,
"speed###16 MB/s": 10,
"speed###32 MB/s" : 40
]

I have to split the properties and values in the application with the
split-argument "###".
Thats not very fast.


(maik) #17

exactly.
I knew the property and the value at runtime, so I can build the query with
it.


(Rauan Maemirov) #18

Doesn't this solve exactly the same problem? (within conditions u've
mentioned in the topic description)
https://groups.google.com/group/elasticsearch/browse_thread/thread/0fb259db7552359a

вторник, 5 июня 2012 г., 15:45:58 UTC+6 пользователь maik написал:

exactly.
I knew the property and the value at runtime, so I can build the query
with it.


(Алексей Чернов) #19

Hi kimchy!

We currently have more than 2000 (and counting) properties in our db. Is
this ok to add all of them to mapping?

On Sunday, June 3, 2012 1:59:04 PM UTC+4, kimchy wrote:

I meant using the get mapping API to get the list of relevant fields you
might care about when you facet...

On Fri, Jun 1, 2012 at 9:06 AM, Rauan Maemirov <raua...@gmail.com<javascript:>

wrote:

Hi, Shay. I'm trying to implement the same scheme as topicstarter.
I didn't understand what do you mean by 'automatically generate facets
for all the properly names assuming they are used as field names'.
Should I directly define mapping for every property in properties list I
have? But what if there are dynamic? I.e. in first document the properties
are {color: 'black', size: 'XL'}, in the second are {capacity: '2Gb',
generation: '5'}, etc. (and there may be a lot of types of product,so
different document types is not a desirable solution)

понедельник, 12 декабря 2011 г., 19:08:15 UTC+6 пользователь kimchy
написал:

Since we have the mapping information, we might be able to do this, i.e.
automatically generate facets for all the properly names assuming they are
used as field names. But, you can also do it yourself, you can ask for the
mapping, it will include all the fields names options, and build the facets
request based on that. (note, the properly names will need to be fields in
the document).

--


(system) #20