Nested objects and facets

Elastic Stack Elasticsearch
1

Hello everybody

I'm working on a facet navigation and I've encountered an issue.

Let me explain what I'm trying to achieve first. Take a look at
http://www.pricerunner.co.uk/cl/2/TVs (not my site, just a basic example).
When you chose one filter, then it's applied to all facets but that
particular one (it's logical, because TVs cant be 30'' and 32'' at the same
time). That's simply achieved, but the problem arises when we want to add a
facet that is based on a field in a nested structure.

Let's say we've got a structure (mapping) like this:

curl -XPUT 'localhost:9200/items' -d '
{
"mappings" : {
"Item" : {
"dynamic" : false,
"properties" : {
"id" : { "type" : "string", "index" : "not_analyzed" },
"name" : { "type" : "string", "index" : "not_analyzed" },
"price" : { "type" : "double", "index" : "not_analyzed" },
"type" : { "type" : "string", "index" : "not_analyzed" },
"characteristics" : {
"type" : "nested",
"properties" : {
"name" : { "type" : "string", "index" : "not_analyzed" },
"value" : { "type" : "string", "index" : "not_analyzed" }
}
}
}
}
}
}
'

We want facets for the fields in the parent object - that's easy - and a
facet for characteristics (name -> value) - and thats troublesome.

So having the above mapping, my app would generate a query like this (let's
assume that some restrictions are already made):

curl -X POST 'localhost:9200/items/_search?pretty=true' -d '
{
"query" : {
"match_all" : {}
},
"facets" : {
"screen size" : {
"nested" : "characteristics",
"terms" : { "field" : "characteristics.value" },
"facet_filter" : {
"and" : [
{ "term" : { "characteristics.name" : "screen size" } }
]
}
},
"price" : {
"terms" : { "field" : "price" },
"facet_filter" : {
"and" : [
{ "term" : { "type" : "TV" } },
{ "term" : { "name" : "Shark largedisplay" } }
]
}
},
"type" : {
"terms" : { "field" : "type" },
"facet_filter" : {
"and" : [
{ "term" : { "name" : "Shark largedisplay" } }
]
}
},
"name" : {
"terms" : { "field" : "name" },
"facet_filter" : {
"and" : [
{ "term" : { "type" : "TV" } }
]
}
}
}
}
'

As you can see, I am able to provide some restrictions for regular facets.
Filter for the facet includes only conditions that are NOT related to the
field the facet is based on. But I'm not able to do this for the nested
facet, because I can't target fields of the parent objects from there.

I am curious about the 'proper' approach - this in my opinion should be a
common usage for the facets.
I would appriciate any given help.

Thanks in advance.
Paweł

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Hi,

I've tried to achieve something similar, without much luck, (and I don't
think we're the only ones).
What's the purpose of having the characteristics as a name/value object?
Perhaps you could try indexing these characteristics in the parent doc as
regular fields, (so "screen size" is the field name, not the value of the
"name" field) and just updating the mappings if a new one comes along, that
way you could perform the facets over them as you would normally.
If you need to work with mappings like this then I'm afraid it's not
possible yet (at least not according to:
https://groups.google.com/forum/?fromgroups=#!topic/elasticsearch/oDcyUNU1cW8
and a few others), but it seems to be a popular topic lately, so hopefully
there will be some development around this sometime soon,

Dustin

On Friday, March 15, 2013 12:14:05 AM UTC+11, Paweł Młynarczyk wrote:

Hello everybody

I'm working on a facet navigation and I've encountered an issue.

Let me explain what I'm trying to achieve first. Take a look at
TVs (1000+ products) compare today & see the best prices » (not my site, just a basic
example). When you chose one filter, then it's applied to all facets but
that particular one (it's logical, because TVs cant be 30'' and 32'' at the
same time). That's simply achieved, but the problem arises when we want to
add a facet that is based on a field in a nested structure.

Let's say we've got a structure (mapping) like this:

curl -XPUT 'localhost:9200/items' -d '
{
"mappings" : {
"Item" : {
"dynamic" : false,
"properties" : {
"id" : { "type" : "string", "index" : "not_analyzed" },
"name" : { "type" : "string", "index" : "not_analyzed" },
"price" : { "type" : "double", "index" : "not_analyzed" },
"type" : { "type" : "string", "index" : "not_analyzed" },
"characteristics" : {
"type" : "nested",
"properties" : {
"name" : { "type" : "string", "index" : "not_analyzed" },
"value" : { "type" : "string", "index" : "not_analyzed" }
}
}
}
}
}
}
'

We want facets for the fields in the parent object - that's easy - and a
facet for characteristics (name -> value) - and thats troublesome.

So having the above mapping, my app would generate a query like this
(let's assume that some restrictions are already made):

curl -X POST 'localhost:9200/items/_search?pretty=true' -d '
{
"query" : {
"match_all" : {}
},
"facets" : {
"screen size" : {
"nested" : "characteristics",
"terms" : { "field" : "characteristics.value" },
"facet_filter" : {
"and" : [
{ "term" : { "characteristics.name" : "screen size" } }
]
}
},
"price" : {
"terms" : { "field" : "price" },
"facet_filter" : {
"and" : [
{ "term" : { "type" : "TV" } },
{ "term" : { "name" : "Shark largedisplay" } }
]
}
},
"type" : {
"terms" : { "field" : "type" },
"facet_filter" : {
"and" : [
{ "term" : { "name" : "Shark largedisplay" } }
]
}
},
"name" : {
"terms" : { "field" : "name" },
"facet_filter" : {
"and" : [
{ "term" : { "type" : "TV" } }
]
}
}
}
}
'

As you can see, I am able to provide some restrictions for regular facets.
Filter for the facet includes only conditions that are NOT related to the
field the facet is based on. But I'm not able to do this for the nested
facet, because I can't target fields of the parent objects from there.

I am curious about the 'proper' approach - this in my opinion should be a
common usage for the facets.
I would appriciate any given help.

Thanks in advance.
Paweł

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Hi Dustin, thanks for your reply.

Indexing those characteristics as a regular fields would lead to a very
huge Item mapping, since it has to contain every characteristic. In
addition it would have to be a dynamic mapping since I don't have the
characteristics in advance.

W dniu piątek, 15 marca 2013 01:53:27 UTC+1 użytkownik Dustin Lashmar
napisał:

Hi,

I've tried to achieve something similar, without much luck, (and I don't
think we're the only ones).
What's the purpose of having the characteristics as a name/value object?
Perhaps you could try indexing these characteristics in the parent doc as
regular fields, (so "screen size" is the field name, not the value of the
"name" field) and just updating the mappings if a new one comes along, that
way you could perform the facets over them as you would normally.
If you need to work with mappings like this then I'm afraid it's not
possible yet (at least not according to:
Redirecting to Google Groups a few others), but it seems to be a popular topic lately, so hopefully
there will be some development around this sometime soon,

Dustin

On Friday, March 15, 2013 12:14:05 AM UTC+11, Paweł Młynarczyk wrote:

Hello everybody

I'm working on a facet navigation and I've encountered an issue.

Let me explain what I'm trying to achieve first. Take a look at
TVs (1000+ products) compare today & see the best prices » (not my site, just a basic
example). When you chose one filter, then it's applied to all facets but
that particular one (it's logical, because TVs cant be 30'' and 32'' at the
same time). That's simply achieved, but the problem arises when we want to
add a facet that is based on a field in a nested structure.

Let's say we've got a structure (mapping) like this:

curl -XPUT 'localhost:9200/items' -d '
{
"mappings" : {
"Item" : {
"dynamic" : false,
"properties" : {
"id" : { "type" : "string", "index" : "not_analyzed" },
"name" : { "type" : "string", "index" : "not_analyzed" },
"price" : { "type" : "double", "index" : "not_analyzed" },
"type" : { "type" : "string", "index" : "not_analyzed" },
"characteristics" : {
"type" : "nested",
"properties" : {
"name" : { "type" : "string", "index" : "not_analyzed" },
"value" : { "type" : "string", "index" : "not_analyzed" }
}
}
}
}
}
}
'

We want facets for the fields in the parent object - that's easy - and a
facet for characteristics (name -> value) - and thats troublesome.

So having the above mapping, my app would generate a query like this
(let's assume that some restrictions are already made):

curl -X POST 'localhost:9200/items/_search?pretty=true' -d '
{
"query" : {
"match_all" : {}
},
"facets" : {
"screen size" : {
"nested" : "characteristics",
"terms" : { "field" : "characteristics.value" },
"facet_filter" : {
"and" : [
{ "term" : { "characteristics.name" : "screen size" } }
]
}
},
"price" : {
"terms" : { "field" : "price" },
"facet_filter" : {
"and" : [
{ "term" : { "type" : "TV" } },
{ "term" : { "name" : "Shark largedisplay" } }
]
}
},
"type" : {
"terms" : { "field" : "type" },
"facet_filter" : {
"and" : [
{ "term" : { "name" : "Shark largedisplay" } }
]
}
},
"name" : {
"terms" : { "field" : "name" },
"facet_filter" : {
"and" : [
{ "term" : { "type" : "TV" } }
]
}
}
}
}
'

As you can see, I am able to provide some restrictions for regular
facets. Filter for the facet includes only conditions that are NOT related
to the field the facet is based on. But I'm not able to do this for the
nested facet, because I can't target fields of the parent objects from
there.

I am curious about the 'proper' approach - this in my opinion should be a
common usage for the facets.
I would appriciate any given help.

Thanks in advance.
Paweł

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Ok fair enough, sorry I couldn't help more. Maybe someone else will be able
to help.

On Friday, March 15, 2013 8:05:50 PM UTC+11, Paweł Młynarczyk wrote:

Hi Dustin, thanks for your reply.

Indexing those characteristics as a regular fields would lead to a very
huge Item mapping, since it has to contain every characteristic. In
addition it would have to be a dynamic mapping since I don't have the
characteristics in advance.

W dniu piątek, 15 marca 2013 01:53:27 UTC+1 użytkownik Dustin Lashmar
napisał:

Hi,

I've tried to achieve something similar, without much luck, (and I don't
think we're the only ones).
What's the purpose of having the characteristics as a name/value object?
Perhaps you could try indexing these characteristics in the parent doc as
regular fields, (so "screen size" is the field name, not the value of the
"name" field) and just updating the mappings if a new one comes along, that
way you could perform the facets over them as you would normally.
If you need to work with mappings like this then I'm afraid it's not
possible yet (at least not according to:
Redirecting to Google Groups a few others), but it seems to be a popular topic lately, so hopefully
there will be some development around this sometime soon,

Dustin

On Friday, March 15, 2013 12:14:05 AM UTC+11, Paweł Młynarczyk wrote:

Hello everybody

I'm working on a facet navigation and I've encountered an issue.

Let me explain what I'm trying to achieve first. Take a look at
TVs (1000+ products) compare today & see the best prices » (not my site, just a basic
example). When you chose one filter, then it's applied to all facets but
that particular one (it's logical, because TVs cant be 30'' and 32'' at the
same time). That's simply achieved, but the problem arises when we want to
add a facet that is based on a field in a nested structure.

Let's say we've got a structure (mapping) like this:

curl -XPUT 'localhost:9200/items' -d '
{
"mappings" : {
"Item" : {
"dynamic" : false,
"properties" : {
"id" : { "type" : "string", "index" : "not_analyzed" },
"name" : { "type" : "string", "index" : "not_analyzed" },
"price" : { "type" : "double", "index" : "not_analyzed" },
"type" : { "type" : "string", "index" : "not_analyzed" },
"characteristics" : {
"type" : "nested",
"properties" : {
"name" : { "type" : "string", "index" : "not_analyzed" },
"value" : { "type" : "string", "index" : "not_analyzed" }
}
}
}
}
}
}
'

We want facets for the fields in the parent object - that's easy - and a
facet for characteristics (name -> value) - and thats troublesome.

So having the above mapping, my app would generate a query like this
(let's assume that some restrictions are already made):

curl -X POST 'localhost:9200/items/_search?pretty=true' -d '
{
"query" : {
"match_all" : {}
},
"facets" : {
"screen size" : {
"nested" : "characteristics",
"terms" : { "field" : "characteristics.value" },
"facet_filter" : {
"and" : [
{ "term" : { "characteristics.name" : "screen size" } }
]
}
},
"price" : {
"terms" : { "field" : "price" },
"facet_filter" : {
"and" : [
{ "term" : { "type" : "TV" } },
{ "term" : { "name" : "Shark largedisplay" } }
]
}
},
"type" : {
"terms" : { "field" : "type" },
"facet_filter" : {
"and" : [
{ "term" : { "name" : "Shark largedisplay" } }
]
}
},
"name" : {
"terms" : { "field" : "name" },
"facet_filter" : {
"and" : [
{ "term" : { "type" : "TV" } }
]
}
}
}
}
'

As you can see, I am able to provide some restrictions for regular
facets. Filter for the facet includes only conditions that are NOT related
to the field the facet is based on. But I'm not able to do this for the
nested facet, because I can't target fields of the parent objects from
there.

I am curious about the 'proper' approach - this in my opinion should be
a common usage for the facets.
I would appriciate any given help.

Thanks in advance.
Paweł

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

Hi Paweł

Indexing those characteristics as a regular fields would lead to a
very huge Item mapping, since it has to contain every characteristic.
In addition it would have to be a dynamic mapping since I don't have
the characteristics in advance.

Yes, this is indeed a problem. Currently you can't use a filter facet
that is applied both to the parent doc and to the nested doc.

This is something that we hope to solve in a big refactoring of facets,
which will make them much more flexible.

however, in the meantime, I suggest you also index a field in the parent
whose value is something like: "key_value", eg "tvsize_32"

That field can be an array containing multiple values, one for each
characteristic.

That way, you can filter on that value, without having your mapping
explode

clint

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

6

Hi Clint, thanks for your reply.

however, in the meantime, I suggest you also index a field in the parent
whose value is something like: "key_value", eg "tvsize_32"

Do you mean something like this: Mappings for elasticsearch not nested stuff · GitHub ?

Paweł

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.