Facets across several types

fmpwizard · November 6, 2011, 4:15am

Hi,

This is what I'm trying to do, taking the example from
http://www.elasticsearch.org/guide/reference/api/search/facets/
Imagine that the Tags are on a separate type, not on the same article
type.
You know which tags go to which article because both article and the
corresponding tags have the same _id value.

Now I can search across both types (in my real app I have 6 types in
my search), by just omitting the type on the query url, but using
facets, I only get values matching the type that returns values.

What I mean is that if the query term is just a tag, the facet will
return just information from the tags type, not from the articles
type. Is there a way to get facet values from "all" types?

In other words, looking at the example from the site, you search for
"T*" and still get facet values for the tags, even though the tags are
not on the Article type.

I hope this was clear.

Thanks

Diego

kimchy · November 8, 2011, 3:52am

Not sure I understand... . Facets are only aggregated for actual hits.

On Sun, Nov 6, 2011 at 6:15 AM, fmpwizard fmpwizard@gmail.com wrote:

Hi,

This is what I'm trying to do, taking the example from
Elasticsearch Platform — Find real-time answers at scale | Elastic
Imagine that the Tags are on a separate type, not on the same article
type.
You know which tags go to which article because both article and the
corresponding tags have the same _id value.

Now I can search across both types (in my real app I have 6 types in
my search), by just omitting the type on the query url, but using
facets, I only get values matching the type that returns values.

What I mean is that if the query term is just a tag, the facet will
return just information from the tags type, not from the articles
type. Is there a way to get facet values from "all" types?

In other words, looking at the example from the site, you search for
"T*" and still get facet values for the tags, even though the tags are
not on the Article type.

I hope this was clear.

Thanks

Diego

fmpwizard · November 9, 2011, 2:52am

Let me try again.

See facet across multiple types · GitHub

The articles and tags have the same _id value because they are
related. Think of then as a one to one relationship from a database
point of view.

I then search for articles that start with T, how can I get a facet
with the tags for the two articles that match my query?

Thanks for taking the time
Diego

On Nov 7, 10:52 pm, Shay Banon kim...@gmail.com wrote:

Not sure I understand... . Facets are only aggregated for actual hits.

On Sun, Nov 6, 2011 at 6:15 AM, fmpwizard fmpwiz...@gmail.com wrote:

Hi,

This is what I'm trying to do, taking the example from
Elasticsearch Platform — Find real-time answers at scale | Elastic
Imagine that the Tags are on a separate type, not on the same article
type.
You know which tags go to which article because both article and the
corresponding tags have the same _id value.

Now I can search across both types (in my real app I have 6 types in
my search), by just omitting the type on the query url, but using
facets, I only get values matching the type that returns values.

What I mean is that if the query term is just a tag, the facet will
return just information from the tags type, not from the articles
type. Is there a way to get facet values from "all" types?

In other words, looking at the example from the site, you search for
"T*" and still get facet values for the tags, even though the tags are
not on the Article type.

I hope this was clear.

Thanks

Diego

fmpwizard · November 9, 2011, 5:29am

If I run the regular search like here:

gist.github.com

https://gist.github.com/fmpwizard/1350503

actual_result.sh

curl -X POST "http://localhost:9200/articles/_search?pretty=true" -d '
  {
    "query" : { "query_string" : {"query" : "T*"} },
    "facets" : {
      "tags" : { "terms" : {"field" : "tags"} }
    }
  }
'
{
  "took" : 5,

This file has been truncated. show original

I get this result:

gist.github.com

https://gist.github.com/fmpwizard/1350503

actual_result.sh

curl -X POST "http://localhost:9200/articles/_search?pretty=true" -d '
  {
    "query" : { "query_string" : {"query" : "T*"} },
    "facets" : {
      "tags" : { "terms" : {"field" : "tags"} }
    }
  }
'
{
  "took" : 5,

This file has been truncated. show original

(same gist )

The facet area is empty, I understand why it is empty, but I was
looking for a way around it, because in my use case, I know how the
articles and tags are related (using the id of each document type)

One way around it is by adding a __id field to each document on
articles and tags that has the same document id. I then do a regular
search not asking for the facets, just the articles with title of T*,
then I parse the _id fields of those results, and query the index
again, but this time looking for the _id values on the manually
updated __id fields (using the default OR operator) and this time
asking for a facet. This works, but I wonder if there is a cleaner
way.

Thanks

Diego

On Nov 8, 9:52 pm, fmpwizard fmpwiz...@gmail.com wrote:

Let me try again.

Seehttps://gist.github.com/1349360

The articles and tags have the same _id value because they are
related. Think of then as a one to one relationship from a database
point of view.

I then search for articles that start with T, how can I get a facet
with the tags for the two articles that match my query?

Thanks for taking the time
Diego

On Nov 7, 10:52 pm, Shay Banon kim...@gmail.com wrote:

Not sure I understand... . Facets are only aggregated for actual hits.

On Sun, Nov 6, 2011 at 6:15 AM,fmpwizardfmpwiz...@gmail.com wrote:

Hi,

This is what I'm trying to do, taking the example from
Elasticsearch Platform — Find real-time answers at scale | Elastic
Imagine that the Tags are on a separate type, not on the same article
type.
You know which tags go to which article because both article and the
corresponding tags have the same _id value.

Now I can search across both types (in my real app I have 6 types in
my search), by just omitting the type on the query url, but using
facets, I only get values matching the type that returns values.

What I mean is that if the query term is just a tag, the facet will
return just information from the tags type, not from the articles
type. Is there a way to get facet values from "all" types?

In other words, looking at the example from the site, you search for
"T*" and still get facet values for the tags, even though the tags are
not on the Article type.

I hope this was clear.

Thanks

Diego

Clinton_Gormley · November 9, 2011, 9:06am

Hiya

Now I can search across both types (in my real app I have 6 types in
my search), by just omitting the type on the query url, but using
facets, I only get values matching the type that returns values.

What I mean is that if the query term is just a tag, the facet will
return just information from the tags type, not from the articles
type. Is there a way to get facet values from "all" types?

The search API accepts a 'query' and a 'filter'.

The facets are calculated on the results of the 'query'. The top-level
'filter' parameter allows you to post-process the query results, after
the facets are calculated.

So if you run your query across both types (tags and articles), then you
set the filter to

{ "type": { "value": "articles" }}

... then your final result set should just include the articles, but
your facets should take the tags into account

clint

fmpwizard · November 9, 2011, 8:20pm

Thanks Clint, maybe I didn't fully understand your answer, but I
didn't get it to work.

These are the steps I'm doing:

Create data:

gist.github.com

https://gist.github.com/fmpwizard/1349360

gistfile1.sh

curl -X DELETE "http://localhost:9200/articles"
curl -X POST "http://localhost:9200/articles/article/1" -d '
{
"title" : "One"
}'

curl -X POST "http://localhost:9200/articles/article/2" -d '
{
"title" : "Two"
}'

This file has been truncated. show original

Search using filter:

gist.github.com

https://gist.github.com/fmpwizard/1352851

3rd.sh

curl -X POST "http://localhost:9200/articles/_search?pretty=true" -d '
  {
    "query" : { "query_string" : {"query" : "T*"} },
    "filter" : { "type": { "value": "article" }} ,
    "facets" : {
      "tags" : { "terms" : {"field" : "tags"} }
    }
  }
'

This file has been truncated. show original

Desired output:

gist.github.com

https://gist.github.com/fmpwizard/1352852

desired.json

"facets" : {
    "tags" : {
      "_type" : "terms",
      "missing" : 0,
      "total" : 2,
      "other" : 0,
      "terms" : [ {
        "term" : "foo",
        "count" : 2
      } ]

This file has been truncated. show original

Thanks

Diego

The search API accepts a 'query' and a 'filter'.

The facets are calculated on the results of the 'query'. The top-level
'filter' parameter allows you to post-process the query results, after
the facets are calculated.

So if you run your query across both types (tags and articles), then you
set the filter to
{ "type": { "value": "articles" }}
... then your final result set should just include the articles, but
your facets should take the tags into account

clint

Clinton_Gormley · November 10, 2011, 11:39am

Hi Diego

On Wed, 2011-11-09 at 12:20 -0800, fmpwizard wrote:

Thanks Clint, maybe I didn't fully understand your answer, but I
didn't get it to work.

Create data:
facet across multiple types · GitHub
Search using filter:
3rd.sh · GitHub
Desired output:
desired.json · GitHub

Right. That won't work. The query only matches docs with 't*' in the
_all field, which excludes the tag docs.

This, however, would:

gist.github.com

https://gist.github.com/clintongormley/1354663

gistfile1.txt

curl -XGET 'http://127.0.0.1:9200/articles/_search?pretty=1'  -d '
{
   "query" : {
      "bool" : {
         "should" : [
            {
               "filtered" : {
                  "filter" : {
                     "type" : {
                        "value" : "article"

This file has been truncated. show original

I use a bool clause to include 'articles-with-t*' and all 'tags', run
the facets on the tags, and filter the results to only include the
articles.

But frankly, you'd be better doing this in two separate queries

clint

fmpwizard · November 10, 2011, 3:17pm

Hi Clint,

This, however, would:

gist:1354663 · GitHub

I use a bool clause to include 'articles-with-t*' and all 'tags', run
the facets on the tags, and filter the results to only include the
articles.

But frankly, you'd be better doing this in two separate queries

Thanks for the example and I'll go ahead and use two separate queries.

Diego

clint