Is there a way to get n hits from each of my indices in one shoot?


(Mauricio Alarcon) #1

Hi Guys,

I know ES supports cross index searches and we're in love with it, but is
there a way to tell ES to return at most "n" hits per index in just one
shoot? or do I need to do a search per index?

Our problem is that some documents got higher score, eclipsing the hits
from its neighborhood which are also important for this use case.

Cheers


(Nick Dimiduk) #2

I believe the limit filter does what you require.
http://www.elasticsearch.org/guide/reference/query-dsl/limit-filter.html

-n

On Fri, Mar 9, 2012 at 6:19 AM, maverick mauricio.alarcon@gmail.com wrote:

Hi Guys,

I know ES supports cross index searches and we're in love with it, but is
there a way to tell ES to return at most "n" hits per index in just one
shoot? or do I need to do a search per index?

Our problem is that some documents got higher score, eclipsing the hits
from its neighborhood which are also important for this use case.

Cheers


(egaumer) #3

The limit filter works at a shard level. Shards are essentially index
partitions so a shard does not represent an entire index.

What you want is more classically known as "field collapsing" which is the
ability to collapse search results based on a unique key/field (a domain
name for instance).

There's been many requests for this feature and I think 0.19 actually
provides a solution called "Multi Search". It allows you submit a group of
queries as a single request and returns an array of "grouped" matches. With
this, you could construct a query against each index and limit the number
of results to N.

http://www.elasticsearch.org/guide/reference/api/multi-search.html

This should meet the requirement of field collapsing providing you have a
relatively small number of groups. I assume this is the case since each is
currently represented by its own index.

Of course you could do this manually at the cost of several round trip
requests and having to federate results. With multi-search, the engine
handles this for you.

-Eric

On Friday, March 9, 2012 1:08:23 PM UTC-5, Nick Dimiduk wrote:

I believe the limit filter does what you require.
http://www.elasticsearch.org/guide/reference/query-dsl/limit-filter.html

-n

On Fri, Mar 9, 2012 at 6:19 AM, maverick mauricio.alarcon@gmail.comwrote:

Hi Guys,

I know ES supports cross index searches and we're in love with it, but is
there a way to tell ES to return at most "n" hits per index in just one
shoot? or do I need to do a search per index?

Our problem is that some documents got higher score, eclipsing the hits
from its neighborhood which are also important for this use case.

Cheers


(Shay Banon) #4

Yea, multi search is the way to go for this.

On Friday, March 9, 2012 at 8:42 PM, egaumer wrote:

The limit filter works at a shard level. Shards are essentially index partitions so a shard does not represent an entire index.

What you want is more classically known as "field collapsing" which is the ability to collapse search results based on a unique key/field (a domain name for instance).

There's been many requests for this feature and I think 0.19 actually provides a solution called "Multi Search". It allows you submit a group of queries as a single request and returns an array of "grouped" matches. With this, you could construct a query against each index and limit the number of results to N.

http://www.elasticsearch.org/guide/reference/api/multi-search.html

This should meet the requirement of field collapsing providing you have a relatively small number of groups. I assume this is the case since each is currently represented by its own index.

Of course you could do this manually at the cost of several round trip requests and having to federate results. With multi-search, the engine handles this for you.

-Eric

On Friday, March 9, 2012 1:08:23 PM UTC-5, Nick Dimiduk wrote:

I believe the limit filter does what you require. http://www.elasticsearch.org/guide/reference/query-dsl/limit-filter.html

-n

On Fri, Mar 9, 2012 at 6:19 AM, maverick <mauricio.alarcon@gmail.com (mailto:mauricio.alarcon@gmail.com)> wrote:

Hi Guys,

I know ES supports cross index searches and we're in love with it, but is there a way to tell ES to return at most "n" hits per index in just one shoot? or do I need to do a search per index?

Our problem is that some documents got higher score, eclipsing the hits from its neighborhood which are also important for this use case.

Cheers


(Mauricio Alarcon) #5

This is just perfect!, Thank you Eric

On Friday, March 9, 2012 1:42:15 PM UTC-5, egaumer wrote:

The limit filter works at a shard level. Shards are essentially index
partitions so a shard does not represent an entire index.

What you want is more classically known as "field collapsing" which is the
ability to collapse search results based on a unique key/field (a domain
name for instance).

There's been many requests for this feature and I think 0.19 actually
provides a solution called "Multi Search". It allows you submit a group of
queries as a single request and returns an array of "grouped" matches. With
this, you could construct a query against each index and limit the number
of results to N.

http://www.elasticsearch.org/guide/reference/api/multi-search.html

This should meet the requirement of field collapsing providing you have a
relatively small number of groups. I assume this is the case since each is
currently represented by its own index.

Of course you could do this manually at the cost of several round trip
requests and having to federate results. With multi-search, the engine
handles this for you.

-Eric

On Friday, March 9, 2012 1:08:23 PM UTC-5, Nick Dimiduk wrote:

I believe the limit filter does what you require.
http://www.elasticsearch.org/guide/reference/query-dsl/limit-filter.html

-n

On Fri, Mar 9, 2012 at 6:19 AM, maverick mauricio.alarcon@gmail.comwrote:

Hi Guys,

I know ES supports cross index searches and we're in love with it, but
is there a way to tell ES to return at most "n" hits per index in just one
shoot? or do I need to do a search per index?

Our problem is that some documents got higher score, eclipsing the hits
from its neighborhood which are also important for this use case.

Cheers


(Mauricio Alarcon) #6

awesome, thanks Shay, I'm still running 0.18 so an update is needed. But
I'll go this way

On Friday, March 9, 2012 2:13:14 PM UTC-5, kimchy wrote:

Yea, multi search is the way to go for this.

On Friday, March 9, 2012 at 8:42 PM, egaumer wrote:

The limit filter works at a shard level. Shards are essentially index
partitions so a shard does not represent an entire index.

What you want is more classically known as "field collapsing" which is the
ability to collapse search results based on a unique key/field (a domain
name for instance).

There's been many requests for this feature and I think 0.19 actually
provides a solution called "Multi Search". It allows you submit a group of
queries as a single request and returns an array of "grouped" matches. With
this, you could construct a query against each index and limit the number
of results to N.

http://www.elasticsearch.org/guide/reference/api/multi-search.html

This should meet the requirement of field collapsing providing you have a
relatively small number of groups. I assume this is the case since each is
currently represented by its own index.

Of course you could do this manually at the cost of several round trip
requests and having to federate results. With multi-search, the engine
handles this for you.

-Eric

On Friday, March 9, 2012 1:08:23 PM UTC-5, Nick Dimiduk wrote:

I believe the limit filter does what you require.
http://www.elasticsearch.org/guide/reference/query-dsl/limit-filter.html

-n

On Fri, Mar 9, 2012 at 6:19 AM, maverick mauricio.alarcon@gmail.comwrote:

Hi Guys,

I know ES supports cross index searches and we're in love with it, but is
there a way to tell ES to return at most "n" hits per index in just one
shoot? or do I need to do a search per index?

Our problem is that some documents got higher score, eclipsing the hits
from its neighborhood which are also important for this use case.

Cheers


(system) #7