Passing requestId in ES logs

Elastic Stack Elasticsearch
1

We use ES in production. We generate our own transactionId / requestId for each request, so that we can trace the query. However, if there is a 400 / 500 error returned by ES (because of client or server error), there is no easy to to stitch the failure in the pipeline with ES failure other than looking at the timestamp in the logs.

Is there anyway to pass our own transactionId / requestId for each request that would be logged in ES logs (e.g. slow logs)?

One post asking similar question: Is there any way of tracking request id?

Thanks so much in advance.

2

Hi @animageofmine,

stats groups may help as I've mentioned in Use of _meta field in elasticsearch 2.4.1?. I'm not sure whether this would help with 400/500s though.

Daniel

3

Thanks Daniel. It does not look like stats group will work for my use case since the requestId is different for each request.

4

Hi @animageofmine,

stats groups are per request. You define them directly in your query. Copying the example that I gave in the other thread verbatim:

GET /_search
{
    "query": {
        "match_all": {}
    },
    "stats" : ["myapplication"]
}

This will show up in the slow query log as:

[2017-05-26 12:12:08,011][INFO ][index.search.slowlog.fetch] [Elven] [old_books][3] took[219.7micros], took_millis[0], types[], stats[myapplication], search_type[QUERY_THEN_FETCH], total_shards[5], source[{"query":{"match_all":{}},"stats":["myapplication"]}], extra_source[],

Daniel

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.