ElasticSearch aggregation for recommendation engine

[martsuga]

I need some advice for aggregations.
I am trying to build a recommendation engine based on existing ElasticSearch data but i am out of ideas how to build this query with aggregations.

The easiest way would be to use "significant_terms" but all the examples i found assume a structure like this:

{
    "userId": 123
    "viewedVideos": [112233, 112244]
},
{
    "userId": 124
    "viewedVideos": [112233]
}

...but my current index structure looks like this:

{
    "userId": 123
    "viewedVideoId": 112233
},
{
    "userId": 123
    "viewedVideoId": 112244
},
{
    "userId": 124
    "viewedVideoId": 112233
}

What i want to achieve is that user 124 get a suggestion to view the video 112244 after he has watched 112233 because other user who viewed 112233 also viewed 112244.

[Mark_Harwood]

You have a log-centric index and you need to make an entity-centric index with a document for each user which includes that array of viewings.

This will probably involve some degree of coding on your side.
Generally the transforms API is how we'd pivot log data to an entity-centric index. This particular transformation would require some scripting of the scripted_metric aggregation to gather the video IDs into an array.

Alternatively you can write custom code to do this transformation using the search and index or update APIs.