Updating Elasticsearch Document Popularity by number of times it gets queried

[ankit2]

I was wondering if elasticsearch provides any kind of functionality that lets you keep track & update the documents in indices based on the number of times any document gets queried directly using document_id. Would that be something that would be a part of my query DSL (I am thinking a function to increase popularity count wrapped on my query everytime my API looks for document by document_id, which would mean I would have to provide an extra field in my document/template)? Or would that be something I implement on the application layer/API layer?

If anyone have implemented something similar, insights would be appreciated.

[thiago (Thiago Souza)]

Elasticsearch does not provides this out of the box and should be implemented in application layer.

There is, though, one feature in X-Pack Security that could help implementing this. The plugin has an auditing feature in which you can easily index an event whenever a document GET is issued and the event can go straight to an index. Here is a sample of an auditing event generated by GET /test/test/1:

{
  "_index": ".security_audit_log-2017.09.07",
  "_type": "event",
  "_id": "AV5dxdgAhEw-lasSg8W0",
  "_score": 0,
  "_source": {
    "@timestamp": "2017-09-07T19:16:39.925Z",
    "node_name": null,
    "node_host_name": "127.0.0.1",
    "node_host_address": "127.0.0.1",
    "layer": "rest",
    "event_type": "authentication_success",
    "principal": "elastic",
    "realm": "reserved",
    "request_body": "",
    "origin_type": "rest",
    "origin_address": "127.0.0.1",
    "uri": "/test/test/1"
  }
}

As this is an indexed document, it becomes fairly easy to aggregate on it and find out top documents, for instance.

[ankit2]

thanks @thiago

if anyone runs into this with similar problem, I ended up using _update to search for my document instead of _search.

So _update/document_id with painless script to increment the view counter by one every time a user tries to view product detail via API, would return back the document with updated view. i can then perform sort or other query dsl on that property of documents in that index.

what i didnt know before i posted this was i could get back the updated document after i hit _update endpoint!

[thiago (Thiago Souza)]

That is certainly another possibility, which is how the application layer handling it as I mentioned