How to make one discover table link to another discover table in Dashboard or elsewhere?

[hsluoyz (Yang Luo)]

I'm using elasticsearch + kibana + logstash + filebeat latest 6.4.1 to collect and analyze web logs. The columns of my log are like:

timestamp, http_method, request_uri, http_status, host, user_agent, client_ip, client_port

I have configured ELK to show my logs in Kibana. But now I want to see my logs in sessions. I hope the log lines can be grouped by session and shown in Kibana's Discover page. In my scenario, the log lines with the same (host, client_ip) belong to the same session.

I hope to have this:

1. Session table

name, client_ip, host
session1, www.google.com, 1.2.3.4
session2, www.bing.com, 5.6.7.8
session3, www.google.com, 4.3.2.1

When I click one of the above session (e.g., session1), I can see all the records of that session in the following 2nd table:

2. Log table

timestamp, http_method, request_uri, http_status, host, user_agent, client_ip, client_port
November 5th 2018, 21:33:17.773, POST, /index.html, 200, www.google.com, chrome 59, 1.2.3.4, 1234
November 5th 2018, 21:33:18.773, POST, /abc.html, 200, www.google.com, chrome 59, 1.2.3.4, 1234
November 5th 2018, 21:33:19.773, POST, /index.html, 404, www.google.com, chrome 59, 1.2.3.4, 5678

I know Elasticsearch does flat indexing, it's not easy to have hierarchy between documents. I'm OK to create separated indices for the above two tables. I know Dashboard can show two Discover tables at the same time. But my question is:

How to link these two tables? When I click one item in the Session table, the Log table will show corresponding contents?

Or is there any other way to fulfill my requirement (view session-based logs easily in Kibana)? Thanks.

[jen-huang]

Hello, you may be able to do this by adding a scripted field to your session index pattern that generates a URL to your log table saved search:

For URL template, put:

http://YOUR_KIBANA_PATH/app/kibana#/discover/YOUR_LOG_TABLE_SAVED_SEARCH_ID?_a=(index:YOUR_LOG_TABLE_INDEX_PATTERN_ID,query:(language:kuery,query:'{{value}}'))

For Script, put:
'host:"'+doc['host'].value+'" AND client_ip:"'+doc['client_ip'].value+'"'

This will string together a URL to your log table saved search with a query applied to filter to the same host and client_ip as the session info. Then this field can be used in Discover to link to the corresponding log table: