Suggester Improvement: Allow to filter specify on which types the suggester should get suggestions from

Hello,

I'm playing with the new Suggester and noticed that it does not allow to
specify on which types it should get suggestions from.
It does allow to specify indices though.

Is this by design?
Is it possible to improve the suggester and add a method setTypes(String...
types) to the class SuggestRequestBuilder?

Cheers,

Stephane

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

hey there,

I think this is by design and IMO there is not much way around this. We are
using Lucene under the hood which provides us lots of info to do
suggestions like term statistics. those statistics are per field and the
type concept is build on top of this. To do this per type without having
sep. fields per type we need to traverse posting lists for each term and
recalculate the statistics at query time which will have a pretty huge
impact on performance. If you have just a bunch of types you can use a
dedicated field per type and if you have a lot of types this won't work
anyway since statistics will be meaningless. so I am not sure if this would
be an improvement?

That said, I think from a usability perspective this would be something we
need to consider but only if the perf impact is not as massive as I have
seen it in the past. This can quickly be a bottleneck here

simon

On Wednesday, March 27, 2013 1:41:43 PM UTC+1, Stephane Bastian wrote:

Hello,

I'm playing with the new Suggester and noticed that it does not allow to
specify on which types it should get suggestions from.
It does allow to specify indices though.

Is this by design?
Is it possible to improve the suggester and add a method
setTypes(String... types) to the class SuggestRequestBuilder?

Cheers,

Stephane

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi Simon,

Thanks for the detailed information. I was pretty sure that this had to do
with stats you were getting from Lucene, but I didn't have time to go
through the code to confirm it.
As you pointed-out, recalculating stats at query time doesn't seem like a
valid option..
I will do what you suggested and use a dedicated field per type
(multi-field mapping), it will get me the same functionality.

Out of curiosity, did you leverage the lucene FST to build the suggester
plugin? I've read somewhere some time ago that Shay was not a big fan of
the FST for suggestion because it was not real-time.
BTW, kudo for the great work on the suggester plugin. It is really awesome!

Stéphane

On Thursday, March 28, 2013 9:16:34 AM UTC+1, simonw wrote:

hey there,

I think this is by design and IMO there is not much way around this. We
are using Lucene under the hood which provides us lots of info to do
suggestions like term statistics. those statistics are per field and the
type concept is build on top of this. To do this per type without having
sep. fields per type we need to traverse posting lists for each term and
recalculate the statistics at query time which will have a pretty huge
impact on performance. If you have just a bunch of types you can use a
dedicated field per type and if you have a lot of types this won't work
anyway since statistics will be meaningless. so I am not sure if this would
be an improvement?

That said, I think from a usability perspective this would be something we
need to consider but only if the perf impact is not as massive as I have
seen it in the past. This can quickly be a bottleneck here

simon

On Wednesday, March 27, 2013 1:41:43 PM UTC+1, Stephane Bastian wrote:

Hello,

I'm playing with the new Suggester and noticed that it does not allow to
specify on which types it should get suggestions from.
It does allow to specify indices though.

Is this by design?
Is it possible to improve the suggester and add a method
setTypes(String... types) to the class SuggestRequestBuilder?

Cheers,

Stephane

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.