Hi,
I have a +/- large index of 200M documents, the content inside can be seen as a representation of a "user". One user/ JSON document can get fairly large.
I have a daily cron that does a search for each of the users in the index,
it searches for other "users" that fall in his/ her filter and orders them by activeness/ attractiveness, nearby filters.
I want to keep track on how much a certain user has already been "used" / "seen" in other peoples searches.. so most logical would be to keep track of a field "usage" field that increments each time a user is found/ returned.
This data can be volatile as the counter can be reset every day.
There are a few drawbacks / disadvantages though :
- a complete user document needs to be rewritten each time that field gets incremented /
- searches will go slower as the complete index will be rewritten each day / complete iteration.
- need to manually flush the counter value each day when starting a new iteration..
Would want to use parent/ child for this, but apparently you can't sort on "child_values" in elasticsearch + in a sense this also written to the same index corpus starting from ES 6.0.
What's the best approach in solving something like this?
Write a custom plugin and keep values like this in memory?
Thx
Jayme