I am using Kibana and Elastic versions 5.4.0 with X-Pack enabled.
For auditing purposes I need to log the User and the Query submitted to ES thru the Kibana Interface, but I was unable to perform this task.
I have set the Kibana log to verbose by adjusting the following parameter in the config/kibana.yml:
logging.verbose: true
logging.json: true
I can see the request headers but it doesn't log either the username or the query:
{"type":"response","@timestamp":"2017-05-22T12:12:23Z","tags":[],"pid":2532,"method":"post","statusCode":200,"req":{"url":"/elasticsearch/_msearch","method":"post","headers":{"host":"127.0.0.1:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:53.0) Gecko/20100101 Firefox/53.0","accept":"application/json, text/plain, */*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","content-type":"application/x-ndjson","kbn-version":"5.4.0","referer":"http://127.0.0.1:5601/app/kibana","content-length":"533","dnt":"1","connection":"keep-alive"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"http://127.0.0.1:5601/app/kibana"},"res":{"statusCode":200,"responseTime":22,"contentLength":9},"message":"POST /elasticsearch/_msearch 200 22ms - 9.0B"}
On the Elastic side, I have also tweaked the logging parameters for the X-Pack:
xpack.security.audit.enabled: true
xpack.security.audit.logfile.events.emit_request_body: true
On the access.log I can see the user elastic searching the index shakespeare, but the query is not logged in.
[2017-05-22T13:12:23,593] [transport] [access_granted] origin_type=[rest], origin_address=[127.0.0.1], principal=[elastic], action=[indices:data/read/msearch], request=[MultiSearchRequest]
[2017-05-22T13:12:23,593] [transport] [access_granted] origin_type=[rest], origin_address=[127.0.0.1], principal=[elastic], action=[indices:data/read/search], indices=[shakespeare], request=[SearchRequest]
[2017-05-22T13:12:23,593] [transport] [access_granted] origin_type=[rest], origin_address=[127.0.0.1], principal=[elastic], action=[indices:data/read/search[phase/query]], indices=[shakespeare], request=[ShardSearchTransportRequest]
[2017-05-22T13:12:23,593] [transport] [access_granted] origin_type=[rest], origin_address=[127.0.0.1], principal=[elastic], action=[indices:data/read/search[phase/query]], indices=[shakespeare], request=[ShardSearchTransportRequest]
[2017-05-22T13:12:23,593] [transport] [access_granted] origin_type=[rest], origin_address=[127.0.0.1], principal=[elastic], action=[indices:data/read/search[phase/query]], indices=[shakespeare], request=[ShardSearchTransportRequest]
[2017-05-22T13:12:23,593] [transport] [access_granted] origin_type=[rest], origin_address=[127.0.0.1], principal=[elastic], action=[indices:data/read/search[phase/query]], indices=[shakespeare], request=[ShardSearchTransportRequest]
[2017-05-22T13:12:23,593] [transport] [access_granted] origin_type=[rest], origin_address=[127.0.0.1], principal=[elastic], action=[indices:data/read/search[phase/query]], indices=[shakespeare], request=[ShardSearchTransportRequest]
I did also tried setting the slowlog to log all the queries by setting it to 0ms:
curl -u elastic -XPUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -d '{"index.search.slowlog.threshold.query.debug" : "0s"}'
And I can see the queries reaching the ES log (index_search_slowlog.log):
[2017-05-22T13:12:26,955][DEBUG][index.search.slowlog.query] [1qbRuku] [shakespeare][4] took[117.1micros], took_millis[0], types[], stats[], search_type[QUERY_THEN_FETCH], total_shards[5], source[{"size":500,"query":{"query_string":{"query":"test","fields":[],"use_dis_max":true,"tie_breaker":0.0,"default_operator":"or","auto_generate_phrase_queries":false,"max_determinized_states":10000,"enable_position_increments":true,"fuzziness":"AUTO","fuzzy_prefix_length":0,"fuzzy_max_expansions":50,"phrase_slop":0,"analyze_wildcard":true,"escape":false,"split_on_whitespace":true,"boost":1.0}},"version":true,"_source":{"includes":[],"excludes":[]},"stored_fields":"*","docvalue_fields":[],"script_fields":{},"sort":[{"_score":{"order":"desc"}}],"highlight":{"pre_tags":["@kibana-highlighted-field@"],"post_tags":["@/kibana-highlighted-field@"],"fragment_size":2147483647,"fields":{"*":{"highlight_query":{"query_string":{"query":"test","fields":[],"use_dis_max":true,"tie_breaker":0.0,"default_operator":"or","auto_generate_phrase_queries":false,"max_determinized_states":10000,"enable_position_increments":true,"fuzziness":"AUTO","fuzzy_prefix_length":0,"fuzzy_max_expansions":50,"phrase_slop":0,"analyze_wildcard":true,"escape":false,"split_on_whitespace":true,"all_fields":true,"boost":1.0}}}}}}],
This will indeed log the query terms (hurray!), but unfortunately the User is nowhere to be seen 
Is there any trick I'm missing to log both the User and the query on the same log?
If this is not possible are there any plans to include this feature? I think this makes sense for the X-Pack on the Kibana side, since you would only log one request and on the ES side you have the query showing multiple times (for each shard).
Meanwhile, what would be the best approach to accomplish this? Write a Kibana plugin?
Any help is appreciated!