Logstash and x-pack monitoring

[jupp]

hi people, why i can not write my logstash x-pack monitoring data directly in a dedicated monitoring cluster?
greetz jupp

[spinscale (Alexander Reelsen)]

Can you please invest some time and write a more useful error description? This post does neither contain, what you tried so far, nor what has failed, nor what you expected to work. You have not mentioned the version of the stack that you are using, nor how your configuration on the logstash and elasticsearch side looks like.

[jupp]

Hi spinscale,

sorry for my lazy request. I have no error but I am wondering why I have to send my monitoring data to my production-cluster even when I have a dedicated monitoring cluster.

I found this information in the docs:
https://www.elastic.co/guide/en/x-pack/current/monitoring-logstash.html

Configure your Logstash nodes to send metrics to your production Elasticsearch cluster by setting the xpack.monitoring.elasticsearch.url in logstash.yml. If security is enabled, you also need to specify the credentials for the built-in logstash_system user.

Important

Even if you are using a dedicated monitoring cluster, you must configure your Logstash nodes to send metrics to your production cluster. Metrics cannot be sent directly to the monitoring cluster.

When the production cluster is out of order the logstash monitoring data would be inaccessible. Am I right? Would't it be better to send the monitoring-data directly do the dedicated monitoring cluster. I thought this is one of the main benefits when I have a dedicated monitoring cluster.

[bohyun (Bohyun Kim)]

Hey @jupp

You can send your Logstash monitoring data to a dedicated monitoring cluster directly. If you send your Logstash monitoring data to the production cluster, you will see the monitoring data grouped nicely for each cluster. As an example, you may have multiple Elasticsearch clusters and multiple Logstash nodes talking to those clusters, and having a visible group of separation in the monitoring app can help with understanding your setup easily and associate the monitoring data within those groups. All this said, we are improving the wording for clarity and adding some visuals to the linked documentation page, so please stay tuned for that update.

Thanks,
Bohyun

[jupp]

Hi bohyun,

thank you for clarification

So I have to decide between Nice Grouping vs. no Monitoring of Logstash when Production-Cluster is gone.
Am I right?

Greetz Jupp

[bohyun (Bohyun Kim)]

Hi @jupp

It can be a bit more involved than that. Sometimes our users may not be able to set up another cluster for various reasons including lack of resources. By having an ability to send your Logstash monitoring data to your production cluster, you are not flying blindly and will have visibility into your Logstash nodes.

Thanks,
Bohyun

[jupp]

Hi bohuyun,

the Documentation is now clearer. Thank you.