Alternative for sql join in elastic stack?

Elastic Stack Elasticsearch
1

Hi,

I need to trace and aggregate requests of user sessions.

following fields I get on a login request:

- timestamp
- session id
- function (login)
- username
- client version number

Following fields I get on a get-data request:

- timestamp
- session id
- function (get driver data, etc...)
- username

My Problem is, I get the client version number only at the login.

Now I need to build aggregations like: top 20 function calls splitted by client version id.
I am using kibana, but my problem in first grade is that I do not know how to merge my data requests.

In SQL I could self-join via session id and then I would have the client version number on all log entries.

I already tried to merge that in logstash:

  • storing login string as new type
  • when processing a non-login request I query against elasticsearch and get the field client version string by the session id.
  • Then I add the field to the event and output the data to elasticsearch.

It was working, but the perfomance impact was way to big to use it in production. I got slower by factor >10.

So how can I solve my problem with elastic stack to see it in kibana?

I am currently on 5.1.2, planing to migrate to 5.6.2 or 6.0.x in near future.

So a scenery which will not be deprecated in v6 is really appreciated :wink:

Thanks, Andreas

2

Looks like you're needing to consolidate information around a session entity.

Entity centric indexing is a common pattern see https://www.youtube.com/watch?v=yBf7oeJKH2Y

(video comments section includes links to example scripts and data)

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.