Is there any way to monitor elastic search application?

[Rajdeep_Sahoo (Rjdeep Sahoo)]

I am trying to monitor elastic search application and I have found the below ways, and none of them is working for me , please suggest if there is any way to monitor. As part of elastic search application monitoring I want all the features available in an APM dashboard.

i) using elastic apm
Getting error
ERROR co.elastic.apm.agent.configuration.ApmServerConfigurationSource - access denied ("java.net.SocketPermission" "localhost:8200" "connect,resolve")

ii) enabling audit log
Not available for free version , will not get all the metrices.
iii) enabling slow logs
Is it going to cause any performance issue for the elastic search application . will not get all the metrices.

[stephenb (Stephen Brown)]

The proper way to monitor elasticsearch is documented here

In short you will use metricbeat described here

Monitoring of a single cluster is available with a basic license

Monitoring with the Elastic APM agent is not supported at this time.

If you have an App that is using the Elastic Agent and that app calls Elasticsearch you will see Elasticsearch as a dependent service and it show transaction rates and latency.

[Rajdeep_Sahoo (Rjdeep Sahoo)]

Hi @stephenb ,
Can this be achieved with the free version of elastic.. I was going through the documentation, there is it mentioned to enable few xpack property. Can this metric beat monitoring feature be achieved in the free version of elastic.

And is it possible to monitor the search requests using metric beat

[stephenb (Stephen Brown)]

Yes.

@stephenb

Monitoring of a single cluster is available with a basic license

Basic is Free, Basic includes some X-Pack features.

Metricbeat will monitor the rate of indexing and querying it is unclear to me exactly what you mean by "monitor search requests"

Since monitoring a cluster is free and I have provided the link to the documentation I suggest setting it up.

Slow log are available for free.

Audit logging requires a commercial fee based license.

[Rajdeep_Sahoo (Rjdeep Sahoo)]

After enabling xpack elastic for metric beat we have configured the elastic node which we are trying to monitor.
But we are getting error as the basic license does not support multi cluster monitoring.

By saying "monitor search request" I wanted to mention all the params related to search requests we want to check. Like for which index we are getting how many search requests, what is the avg response time of the search requests and if any transaction level details etc.

If we enable slowlog and set the time as 0 ms, we are getting all the queries. But will it cause any performance issue.

[stephenb (Stephen Brown)]

With basic free license you can monitoring a single cluster

To monitor multiple clusters in the same monitoring cluster requires a commercial license.

See Here:

Using slow logs capability for your purpose is typically not recommended.

You can use Elastic APM from your Clients and that will show the Transaction Rate and Response time to/from Elasticsearch for you query and index request.

Typically if you want to log all queries you do that at the client side or with a proxy in front of elasticsearch and then actually ingest those logs to elasticsearch for analysis.