Filter search by score

Elastic Search
1

I’d like to perform a search, but only get back search results that have a score greater-than 1 for instance. Is this possible?

2

Hi @AlejandroNextChance, to my knowledge, this is not possible.

3

What is your use case, maybe there is another option?

4

Hi @JasonStoltz, when I perform a search to AppSearch I would like receive only the results that have a score greater-than 1. I didn't find any filter in the documentation to do that.

Do you know if is there any possibility to filter the results by score?

Regards

5

@AlejandroNextChance No, again, I don't think that's possible. App Search provides high recall in searches. So there will usually always be a long tail of search results returned.

You're asking about a particular feature. What I'm asking you though, is what is the underlying problem that you are trying to solve? There may be other ways to solve it.

6

Thank you @JasonStoltz.

I need to return only documents with a score greater-than 1 to avoid send documents with low score.

I'm calling Appsearch service from my server. So, I was thinking to filter when the documents are returned. But I can't do it because that option breaks the Appsearch pagination.

If you know any way to force Appsearch to response only with documents with score grater-than one I will appreciate

Regards

7

Hi @JasonStoltz,

I too think this feature would be a good one. We have huge mixes of fashion items and general store merch that gets searched through, with the broadness of the matches when searching something like "shirt" a few pages down you start to see completely unrelated things like hats and jeans.

The worst thing about this is that we have facets for product type, so when searching for shirt, we also get filters for Dresses, Boots, Hats etc.. completely clogging up the filters areas for what would actually be useful.

Some people would say this is great, as the results are never ending, but others would be annoyed by the irrelevance to their original search.

A simple setting that could be passed with the search query much like filters would fix this I believe.

Cheers

3 Likes
8

That makes sense @Harry_Knowles. You make a great point about the facets.

9

Hi @JasonStoltz,

Is this something that could be implemented/considered in the future?

10

@Harry_Knowles We do have a solution on the roadmap that will address this. It may not be this exact feature, but I think you will find it to be an elegant solution.

11

Could you point me in the right direction to find this roadmap? Or has it not been released yet?

1 Like
12

I don't believe we have a public roadmap.

13

Hi,

I am too is interested in filtering results with a score below X because they tend to be completely irrelevant to the original search.

Like @Harry_Knowles I believe end-users can be confused by the non relevant results. Maybe in a clothes shop it can still be acceptable but I think in anything else than e-commerce it won't.

Imagine a case where you store the periodic table (of elements). You look for "Titanium". It isn't a document in Elasticsearch yet but the search engine returns "Oxygen". As a user with some basic scientific knowledge you are like ... WTF, it isn't at all what I asked for ;-). This is a fictive example, don't try to reproduce, I made it up as a metaphor.

I don't understand why some documents are scored with 0.215 or 0.3 or anything sub 0.7 when there is no correlation at all (no mention of titanium in any other fields, nor word finishing in 'ium' or starting with 'tit' - no pun intended). I have not tested the above, it was just an illustration of what I can see in my use case.

Therefore, having a hard cut-off of "anything below this score ... is probably irrelevant ... then do NOT display it" would be simply awesome.

I know there is curation but we cannot edit all possible searches and curate them manually.

Bonus question: Is there a way to deeply understand the reason of the scoring? I really would like to understand why some unrelated documents are displayed for some of my searches to maybe tweak relevance negatively for some fields... while waiting for this threshold.

Unless there is a way to filter at the Elasticsearch level?

Looking forward for this feature or similar. Otherwise thanks for app-search. It's a cool tool and has potential.

Cheers,
Thomas

2 Likes
14

Thanks @AlejandroNextChance, @Harry_Knowles, and @TomHome for the great feedback! As @JasonStoltz explained, you've all described great examples of the side effects from focusing on recall over precision.

We have been updating App Search's relevance engine to hone results for use cases exactly like the ones you've described. This is a change we want to take the time to get right, so we can't share any more details just yet, but the feedback you've given is great to hear.

We're excited to share more with you when we can. Stay tuned!

Thanks,
Nick

1 Like
15

Thanks @nickchow, looking forward for this release :wink: At a guess, it still sounds far away :slight_smile:

@nickchow is there a document explaining the scoring logic/algo somewhere? Can you explain or point to a doco explaining what you meant by "recall over precision"?

Cheers

1 Like
16

I don' know of a document explaining our scoring logic or algorithm. Precision and recall are information retrieval concepts that you can read about here: https://en.wikipedia.org/wiki/Precision_and_recall

17

Thanks @JasonStoltz for the link. Enlightening. TIL precision and recall :wink:

18

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.