ElasticSearch distinct results by multiple fields with pagination

[adolbik (Aliaksandr D)]

I have index with bunch of events (user viewed page, clicked something and etc.). I have a page in the system where I need to display uniq results (only one row per user + event + itemId). It means if we have 7 rows like these

 {'userId': 1, 'event': 'view', 'itemId': "11"}
 {'userId': 1, 'event': 'click', 'itemId': "11"}
 {'userId': 1, 'event': 'view', 'itemId': "11"} // duplicate
 {'userId': 2, 'event': 'view', 'itemId': "12"}
 {'userId': 2, 'event': 'view', 'itemId': "11"}
 {'userId': 3, 'event': 'click', 'itemId': "13"}
 {'userId': 2, 'event': 'view', 'itemId': "12"} // duplicate

I want to get from Elastic Search only uniq rows where combination of userId + event + itemId is uniq

 {'userId': 1, 'event': 'view', 'itemId': "11"}
 {'userId': 1, 'event': 'click', 'itemId': "11"}
 {'userId': 2, 'event': 'view', 'itemId': "12"}
 {'userId': 2, 'event': 'view', 'itemId': "11"}
 {'userId': 3, 'event': 'click', 'itemId': "13"}

Also I need to support pagination for results.

I tried to use buckets aggregation, collapse but these options ether doesn't support pagination or multiple fields aggregation.

[spinscale (Alexander Reelsen)]

Hey,

while you could use field collapsing for this, you may want to do this in a more clean way and take a look at data transforms allowing you to create a continouos process running in the background that collects and groups the latest data resulting in an easy to query index.

Hope this helps!

--Alex

[adolbik (Aliaksandr D)]

Hey Alex,

thanks for response! I will check this options.