How to check kibana user searches

Elastic Stack Kibana
1

We are evaluating ELK stack for log analysis and visualization

ES 2.x,
Logstash2.1
and Kibana 4.

Does anyone happen to know how to track the user search queries made on kibana search bar. I am more interested to know about the non-saved searches from the user.
I understand Kibana stores the search queries , dashboards and visualization in .kibana index.
Is there a way to pull what the user had searched from kibana interface?

Can you help to understand with an example if thats possible to pull out from ES indexes?

2

Highly appreciate any inputs

3

I assume there is no solution to this?

4

While there isn't a specific way to tie into the act of using the search bar in the Kibana UI, you could examine your standard web server access logs to track search requests to elasticsearch. Kibana proxies searches directly to elasticsearch, so you could look at the access logs for your kibana webserver to see the searches to elasticsearch.

5

Thanks for the reply !! We are not using any web server other than nginx to reverse proxy kibana. However, nginx doesnt help to track the user search on the kibana search bar. All i can see is few GET and POST methods

6

Every time a person searches in Kibana, it results in a couple of HTTP requests to the Kibana backend which then proxy to elasticsearch. The request you want to look at is the "msearch" request.

On my local kibana setup, when I filter my data based on the extension "jpg", I get the following request:

POST http://localhost:5601/elasticsearch/_msearch?timeout=0&ignore_unavailable=true&preference=1453303185156
{"index":["logstash-2016.01.20"],"ignore_unavailable":true}
{"highlight":{"pre_tags":["@kibana-highlighted-field@"],"post_tags":["@/kibana-highlighted-field@"],"fields":{"":{}},"require_field_match":false,"fragment_size":2147483647},"query":{"filtered":{"query":{"query_string":{"query":"","analyze_wildcard":true}},"filter":{"bool":{"must":[{"query":{"match":{"extension":{"query":"jpg","type":"phrase"}}}},{"range":{"@timestamp":{"gte":1453302324311,"lte":1453303224311,"format":"epoch_millis"}}}],"must_not":[]}}}},"size":500,"sort":[{"@timestamp":{"order":"desc","unmapped_type":"boolean"}}],"aggs":{"2":{"date_histogram":{"field":"@timestamp","interval":"30s","time_zone":"America/New_York","min_doc_count":0,"extended_bounds":{"min":1453302324308,"max":1453303224309}}}},"fields":["*","_source"],"script_fields":{},"fielddata_fields":["@timestamp","utc_time","relatedContent.article:modified_time","relatedContent.article:published_time"]}

As you can see, that post body includes both the "filters" and the "query" that are used in my search via the search and filter bars in Kibana.

7

We had achieved this audit feature via a proxy servlet - all the requests were intercepted by the proxy to check if its a search query and accordingly audited.

1 Like
8

Thanks Court for the detaield investigation.. I do see POST requests on my nginx access log, however i dont see detailed query parameters logged in the logs.

x.x.x.x - - [20/Jan/2016:10:08:29 -0800] "POST /elasticsearch/_msearch?timeout=0&ignore_unavailable=true&preference=1453314159028 HTTP/1.1" 200 1544 "http://hostname/app/kibana" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.111 Safari/537.36" "-"

9

I've never done it myself, but I'm pretty sure you can configure nginx to log post bodies in access logs.

10

I had to explicitly add request_body to NGINX log_format to gather POST request parameters. I can see the query string now. Thanks much for the pointer.

x.x.x.x - - [20/Jan/2016:19:29:10 -0800] "POST /elasticsearch/_msearch?timeout=0&ignore_unavailable=true&preference=1453347750129 HTTP/1.1" "{\x22index\x22:[\x22test-logs\x22],\x22ignore_unavailable\x22:true}\x0A{\x22size\x22:500,\x22sort\x22:[{\x22@timestamp\x22:{\x22order\x22:\x22desc\x22,\x22unmapped_type\x22:\x22boolean\x22}}],\x22query\x22:{\x22filtered\x22:{\x22query\x22:{\x22query_string\x22:{\x22query\x22:\x22testing\x22,\x22analyze_wildcard\x22:true}},\x22filter\x22:{\x22bool\x22:{\x22must\x22:[{\x22range\x22:{\x22@timestamp\x22:{\x22gte\x22:1448163803080,\x22lte\x22:1453347803080,\x22format\x22:\x22epoch_millis\x22}}}],\x22must_not\x22:[]}}}},\x22highlight\x22:{\x22pre_tags\x22:[\x22@kibana-highlighted-field@\x22],\x22post_tags\x22:[\x22@/kibana-highlighted-field@\x22],\x22fields\x22:{\x22*\x22:{}},\x22require_field_match\x22:false,\x22fragment_size\x22:2147483647},\x22aggs\x22:{\x222\x22:{\x22date_histogram\x22:{\x22field\x22:\x22@timestamp\x22,\x22interval\x22:\x221d\x22,\x22time_zone\x22:\x22GMT\x22,\x22min_doc_count\x22:0,\x22extended_bounds\x22:{\x22min\x22:1448163803080,\x22max\x22:1453347803080}}}},\x22fields\x22:[\x22*\x22,\x22_source\x22],\x22script_fields\x22:{},\x22fielddata_fields\x22:[\x22@timestamp\x22]}\x0A"200 1510 "http://hostname/app/kibana" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.111 Safari/537.36" "-"