Elasticsearch.slowlog.id is null when sending X-Opaque-Id header

Elastic Stack Elasticsearch
1

Hello,

I have been using slowlogs for awhile, and now I'm trying to get X-Opaque-Id to show up in slowlogs so I can trace slow queries to their source. I believe that if it is set, the header should appear in the output field elasticsearch.slowlog.id, but I'm only seeing this set to null.

First, I followed the instructions here to set index.search.slowlog.include.user to true for an index: Slow Log | Elasticsearch Guide [8.15] | Elastic

I confirmed the new setting was applied by getting the index settings back:

"search": {
    "slowlog": {
        "include": {
            "user": "true"
        },
    "threshold": {
        "fetch": {
            "warn": "1s",
            "trace": "-1",
            "debug": "-1",
            "info": "800ms"
        },
    "query": {
            "warn": "5s",
            "trace": "-1",
            "debug": "-1",
            "info": "2s"
        }
    }
}

Next, I ran a query that exceeds the thresholds I have set and included the header X-Opaque-Id set to headertest. I can see the value getting returned in the query response headers under x-opaque-id, but slowlogs still show elasticsearch.slowlog.id as null.

Are there any other steps I missed to get the header to show up in slowlogs? I tried this on a cluster running 8.15.3 (using ECK 2.13.0).

I also tried this on another cluster running 8.12.0. In that case, I didn't change the slowlog settings since the 8.12 documentation doesn't make note of the new index.search.slowlog.include.user setting (Slow Log | Elasticsearch Guide [8.12] | Elastic), but I still don't see the header being set on the slowlog output.

Thank you!

2

Adding index.search.slowlog.include.user will add user.name to the slowlog and should have not bearing the 'id' value, for example the 'elastic' user or 'elastic/kibana' if you are using Kibana Console/DevTools.

In my case, when I did provide the X-Opaque-Id header via curl, I did see it logged correctly in elasticsearch.slowlog.id in the logs. However, there were also a lot of 'null' values that appeared to be generated by Kibana. Maybe that could be what you are experiencing?

3

Thank you for the reply. Good to know about index.search.slowlog.include.user .

Unfortunately I only see elasticsearch.slowlog.id set to null in the (JSON) log output:

"@timestamp":"2024-11-04T22:06:56.274Z", "log.level": "INFO",  "elasticsearch.slowlog.id":null,"elasticsearch.slowlog.message": ...

I do see the value returned as x-opaque-id in the response header when I run a query from curl or Insomnia, but it's always null in the logs. I am not using Kibana.