How to query 2 logs and show where session.id is missing in one of the logs?

Elastic Stack Elasticsearch
1

Hi,

I have 2 logs (httpd_access and session log).
Both logs have the field session.id.
One httpd call can result in multiple sessionlog entries.

I want to find out the following:

  • which session ids have entries in session log but not in httpd log
  • which session ids have entries in httpd log, but not in session log.

There are thousands of requests, so I cannot do the comparison manually.

How can kibana / elasticsearch help here?

Thanks, Andreas

2

Hi Andreas,
For this sort of analysis it's probably best to combine these two event-centric index contents into a third "entity-centric" index keyed on session ID.
The new dataframes functionality is designed to do the work of fusing the data. You may need to set up an alias that combines the two indices and use that as the source data to perform the transform, using the common session.id field as the join key.