How to send an optional field/parameter in query?

Elastic Stack Elasticsearch
1

I am trying to have a better monitoring over our queries and we have several different systems calling to our elastic . I have logging enabled but is shows the query only. I don't see what system made which query.
So I thought maybe there is a way for our systems to add an extra variable/field/parameter to the body of the query and add their name to it. So this way when I am looking at the logs, I can see what system made what query but this field shouldn't effect the search results.
Any thoughts?

ps: We are using 2.3 with Kibana (without logstash) and Marvel

2

Hi @dijam,

first of all I think it is questionable from an architectural standpoint to tie this information to Elasticsearch. I'd rather have a transparent proxy between each client application and Elasticsearch and have this proxy log the information. Client applications could add a custom HTTP header (e.g. X-Source-System: MY_SYSTEM) and the proxy could log this information if you need it.

Having said that, if all client applications only use request body search, you could probably (mis)use named queries for that. Example:

GET /_search
{
    "query": {
        "match_all": {
            "_name": "some_source_system"
        }
    }
}

Note that this will also change the response format so (a) you need to change all client applications to include this parameter and (b) your clients need to adapt to the changed response format.

However, I'd really urge you to think whether there are better solutions that do not rely on this Elasticsearch feature. Whenever we change the behavior of this feature or may deprecate it (not that there are any plans right now but you never know) this would affect all client applications.

Daniel