Searching user favorited documents

Elastic Search
1

Hello,

I’m looking into app searches ability to search user favorites. It looks like the only supported way would be to pass the document ids in as a filter with the query (limit 1024 ids). I know I can do multiple queries to artificially extend that limit a bit up to 10x out of the box, but I wanted to check if there was another way that may be more performant?

I don’t have many documents, about a million, and most users will probably be fine with this limit, but we have over a million users. Any ideas are appreciated.

Plain Elasticsearch allows for term filters from another index in query but it doesn’t look like app search has this ability.

Almost forgot we do want to allow a user to create multiple favorites lists in the future.

2

Hi @vortextangent ! Welcome to the Elastic community!

There are two approaches that could work for this case:

  • Up to 32,768 values can be specified in a query filter, when using filter arrays. You can have a look at this in the documentation:

Arrays vs. Objects

Be mindful of the 32 filters limit when building large filtered queries.

Instead of stacking objects, you can use an array when values share a field.

Each array can contain 1024 values.

If need be, you can use 32 full arrays to filter on a total of 32,768 values.

Array behaviour varies depending on whether you are using any , all , or none ...

  • You could add the user ID to each document that the user marks as favorite in a dedicated field. App Search supports array values in fields, so user IDs could be stored in each document and then perform filtering in this field for the specific user ID.

Given that you have millions of users, a proof of concept should be done to check the performance / usefulness of the last approach. This approach is described as well in this post.

Regarding multiple favorites lists, the name of the favorite list can be encoded as part of the user ID to add to the document, for example concatenating the list name or id with the user id.

Please share your findings with us!

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.