How to identify specific users who are submitting slowlog queries?

Elastic Stack Elasticsearch
1

I'm trying to create a dashboard that can identify slow queries and the user who submitted them. Was wondering if there were any examples/guides I could follow online as I am a bit stumped on how to go about this (am a beginner to ES).

2

Take a look at Slow Log | Elasticsearch Guide [8.3] | Elastic. Basically unless you are providing information to Elasticsearch about the user making the request, it has no way of identifying them.

3

The X-Opaque-Id http header seems like just what I need, but I was wondering how I'd go about setting that up in Elasticsearch? Only resource I could find was how to set it when using the Python library.

4

Is there a way to set the X-Opaque-Id http header via the ECE GUI?

5

It needs to come from whatever client talks to Elasticsearch, it's not something Elasticsearch can inject.

6

I understand that, right now we have a bunch of settings set up from the ECE GUI via the Elasticsearch.yml file for the whole cluster (as opposed to setting it up manually for each instance individually), so I was wondering if there was a similar method to set up that property for a whole cluster in one place?

7

Another question, as of right now the docs say to set up the X-Opaque-Id http header I would have to updated the search slow log file which is configured in the log4j2.properties file.

Where would I find this file?

8

It says;

You can pass an X-Opaque-Id HTTP header to track the origin of a request in Elasticsearch logs and tasks. If provided, Elasticsearch surfaces the X-Opaque-Id value in the:

Basically, as long as the HTTP request that you are making to Elasticsearch contains this information, it will be added to the existing slow logs. It's not anything that you need to configure in Elasticsearch.

eg if I ran curl -H "Content-Type: application/json" -XGET 0:9200/ from a host called marks-laptop to my cluster and it triggered into the slow log, it wouldn't show any source.
But if I ran curl -H "Content-Type: application/json" -H "X-Opaque-Id: marks-laptop" -XGET 0:9200/ and it triggered into the slow log then it would show up, as I have passed that additional header into the request.

9

Ah, it's entirely a client side feature. Whoever is using it will have to make sure it's included.

Is there a similar feature to implement on Kibana? As that is where I need to get it working and I haven't found any docs related to that.

10

I don't believe there is a way for Kibana sorry. It might be worth raising a feature request on GitHub.

11

We had a recent blog post about tracing kibana that might help with this space.

The feature is still very early though so I'm not sure if you could use it to identify specific users/dashboards yet.

I opened Dashboard-centric APM transaction titles · Issue #134281 · elastic/kibana · GitHub to at least include dashboard names in the transaction titles which should help highlight high impact (latency + throughput) dashboards.

1 Like
12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.