I need to trace and aggregate requests of user sessions.
following fields I get on a login request:
- timestamp - session id - function (login) - username - client version number
Following fields I get on a get-data request:
- timestamp - session id - function (get driver data, etc...) - username
My Problem is, I get the client version number only at the login.
Now I need to build aggregations like: top 20 function calls splitted by client version id.
I am using kibana, but my problem in first grade is that I do not know how to merge my data requests.
In SQL I could self-join via session id and then I would have the client version number on all log entries.
I already tried to merge that in logstash:
- storing login string as new type
- when processing a non-login request I query against elasticsearch and get the field client version string by the session id.
- Then I add the field to the event and output the data to elasticsearch.
It was working, but the perfomance impact was way to big to use it in production. I got slower by factor >10.
So how can I solve my problem with elastic stack to see it in kibana?
I am currently on 5.1.2, planing to migrate to 5.6.2 or 6.0.x in near future.
So a scenery which will not be deprecated in v6 is really appreciated