Monitoring ES - Free Tools

Elastic Stack Elasticsearch
1

Hello,

I search a free tools to moniroting my es cluster.

X-pack is not free .... :cry:

Here my short list to important metrics :

  1. Heap
  2. Shards allocation
  3. Disk Usage
  4. Refresh and FLush Latency
  5. Request average
  6. GC
  7. I/O

Do you know free tools for that ?

Thank you for all your helps

2

X-Pack monitoring comes for free with the BASIC license which you can renew annually.

You simply need to

  1. install X-Pack on Elasticsearch and Kibana
  2. register for a free BASIC license at https://register.elastic.co/
  3. Install the license you'll get via email
  4. add the following settings

at the end of your elasticsearch.yml file

xpack.monitoring.enabled: true
xpack.security.enabled: false
xpack.ml.enabled: false
xpack.watcher.enabled: false

at the end of your kibana.yml file

xpack.monitoring.enabled: true
xpack.security.enabled: false
xpack.reporting.enabled: false
xpack.ml.enabled: false
xpack.graph.enabled: false
1 Like
3

Adding to this that this won't even be required with 6.3.0 as it will be shipped with a basic license OOTB.

1 Like
4

You can also use the Logstash http_poller input to collect stats via Elasticsearch and Logstash REST APIs, and build your own monitoring dashboards. This is especially useful for Logstash monitoring as the Pipeline viewer isn't really useful for anything beyond a simple configuration.

5

My own two cents to this is: The xpack basic, even if included, only helps with 1 cluster. It is not much more of a way to see if monitoring looks good enough to pay a lot for it.

imagine a real use case.

I have 1 cluster that has all of my content, this is where I run all of my searches
I have a es cluster for logstash to write metrics of my app too

I would like one monitoring cluster to be able to see both of those clusters.

The above scenario is one of the most basic setups I have seen.

The problem is that, with the way xpack is, you actually need to run two monitoring systems to watch 2 separate clusters. Why? you need to pay for the license to be allowed to monitor multiple clusters in one setup.
How much is that? its like 15k+ per server.

To me, I would rather see something like
xpack, basic/free, could monitor up to 5 clusters for free
or
have a separate, lower cost license just for turning on multiple cluster support. Something like 500/month. Something affordable so you don't have to either go without monitoring, or have to use your own rolled nagios/cactus/pick your free monitor tool to do it, which is no where near as good

6

Having a separate monitoring cluster is not necessarily mandatory. For simple use cases and out of experience, having a single cluster with business data and monitoring data might work perfectly well.

Disclaimer: I'm not an employee of Elastic Inc. :slight_smile:

7

Hi,

did you check Prometheus?

To enable Prometheus to scrape individual ES nodes you will need to install plugin that exposes metrics in Prometheus format, you can use elasticsearch-prometheus-exporter for this.

Author of the Prometheus plugin implemented also dashboard for Grafana, but you can easily customize it or build your own dashboard.

Regards,
Lukáš

1 Like
8

I have lookep for Prometheus - Grafana - Plugins it's work well.

Thanks guys

9

Yep, we had seen Prometheus recently. Nice that it is open source, reads the API, etc. Hoping this keeps getting supported and stays alive.
My IT team is looking into it's viability.

My only gripe as I was saying before was a couple of items
1 - Marvel went away. Too bad, as this was a way easier interface to see multiple nodes together and search times, queues, etc. Now you have to look node by node. Unless somehow has the Kibana dashboards that match what Marvel used to do?

2 - X-Pack Expectations based on previous years of ElasticSearch usage (all of the way back to .2)
a - X-Pack is fine, but now only stores 7 days of content, where Marvel could do more than that.
b - Multi Cluster support still is not on by default, so, no go there
c - If you only have say, 3 average clusters, you would have to setup 3 separate xpack clusters to monitor, or, you can put them into 1 cluster, but you still have to have 3 completely separate xpack setups just to look at 3 different clusters.
If you do not want to do this, well, get ready to pay 15k/yr/server - which is a huge jump in services. Why not a simpler/low usage payment of say 500/month for something like just multi-cluster support in the UI? We don't need everything, just that. Just kind of crazy that the monitoring piece has been over taken by everything else.

Really hoping the open sourcing of XPack will help solve the multi cluster view of monitoring, which is a severe limiting factor in using just Es supported tools.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.