Elasticsearch Query Monitoring

Elastic Stack Elasticsearch
1

Hi All,

I am using elasticsearch and looking for a way to get all the queries which are getting executed in the elasticsearch. Basically I want to log all the queries.
I read that one way is that we can decrease the threshold value of the slow log query so that it will start logging into the log file.

But the above approach will result in writing too many logs and I fear it might effect the performance,

Is there a better way to log all the queries?

Thanks in advance,
Vikash

2

You can use Packetbeat to capture the http requests being sent to Elasticsearch.

3

Hey Abdon,

Thanks for the response. I implemented packetbeats but it gave me the below results:

  1. http error codes
  2. http codes
  3. Total number of HTTP transactions

However, it didn't gave me the on-going Elasticsearch search queries that is currently happening on my ES cluster.

Thanks in advance!

4

You need to configure Packetbeat to capture the data that you need. In the packetbeat.yml configuration file, you should set include_body_for to capture the body of the http requests and responses:

- type: http
  # Configure the ports where to listen for HTTP traffic. You can disable
  # the HTTP protocol by commenting out the list of ports.
  ports: [9200]
  include_body_for: ["application/json"]

What you probably also will want to do is remove all requests to any URL that do not contain "_search" (as you're only interested in the queries). To do that, you could configure the following processor in the packetbeat.yml file:

processors:
  - drop_event:
      when:
        not:
          contains:
            url.path: "_search"
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.