How to exclude already-swiped users when using Elasticsearch for a dating app?

Elastic Stack Elasticsearch
1

I’m using Elasticsearch as a matching engine for a dating app.

Right now each user is indexed with:

  • a geo_point

  • filters like age, gender, preferences, activity status

With this I make queries like find users within a certain radius of gender female, etc etc

The issue is that when returning candidates for a user, I must exclude users they already swiped yes/no on (or matched with).

Each user can have thousands of swipes, so adding all those IDs to a must_not terms filter seems like the wrong design.

Is there any way to model this such that elastic returns the users that haven’t been swiped yes? I would appreciate ideas. Or maybe I’m using the wrong tool for the job?

The data about what users have swiped a certain user is stored in dynamo. Maybe I need to replicate that data in elastic and do some joins?

2

Yes, this may end up not scaling well, especially if the database grows and thousands turns into tens of thousands.

Although I do not have any solution that is guaranteed to work I have a suggestion that might be worth testing. It may be possible to store swipes and matches in Elasticsearch as a parent-child relationship where the profile is the parent document and each swipe or match is a child document using a join field. This allows you to perform the exclusion in Elasticsearch without sending huge amounts of IDs but complicates the query syntax, adds some processing overhead and likely at least somewhat longer query latencies.

1 Like
3

This is a hard problem. For users with small exclusion lists one strategy works best however a different strategy works best after a certain size of exclusion list. This also depends on the number of documents matched by your other search criteria.

See here for details and some code you can use.