Monitoring Logstash using Metricbeat

[Farid_N]

Hi,

I have 3 instances of Logstash and want to monitor them in Kibana
This is my /etc/metricbeat/modules.d/logstash-xpack.yml configuration:

# Module: logstash
# Docs: https://www.elastic.co/guide/en/beats/metricbeat/7.6/metricbeat-module-logstash.html

- module: logstash
  metricsets:
    - node
    - node_stats
  period: 10s
  hosts: ["http://172.22.129.187:9603","http://172.22.33.186:9601","http://172.22.33.187:9602"]
  username: "elastic"
  password: "MY_PASSWORD"
  xpack.enabled: true

The hhtp.port for every instance of logstash.yml is 9601 , 9602 and 9603
And also the logstash-xpack module is enabled:
metricbeat modules enable logstash-xpack

The output of Elasticsearch in metricbeat.yml is all my 4 ES nodes on port 9200

I have all pipelines in "Stack Monitoring" (all pipelines of 3 instances of Logstash)
But in Nodes section, I just see one instance of Logstash!!!

This is "Pipelines" section which included 3 instances of Logstash:

The weird thing is sometime the name of only Logstash node that shown in "Nodes" changes to another instance name... For example in the first picture above, elk-node101.software:9603 changes to elk-node2.software:9601 automatically!

Could you please help me about this?
I'm using the stack on version 7.16.2

Kind regards

[ylasri (Yassine LASRI)]

Check the node.name in logstash.yml for each instance

[Farid_N]

Thanks for your response

I defined name for every Logstash instance. Now they have node.name in logstash.yml

node.name: elk-lnx1.software
http.host: 0.0.0.0
http.port: 9602

node.name: elk-lnx2.software
http.host: 0.0.0.0
http.port: 9601

node.name: elk-lnx101.software
http.host: 0.0.0.0
http.port: 9603

But in "Nodes" there is just one instance:

[Farid_N]

And if you see, in the last picture I have elk-lnx1.software but in the very first picture I had elk-lnx101.software
It always changes regularly...

[leandrojmp (Leandro Pereira)]

I'm not sure you can monitor 3 different instances this way.

You should use one .yml file per host you want to monitor, or create multiple modules in the same .yml , each one pointing to a different logstash, If i'm not wrong using an array of hosts will work in a load balance way and not querying all the hosts at the same time.

Try to edit your .yml to have something like this:

- module: logstash
  metricsets:
    - node
    - node_stats
  period: 10s
  hosts: ["http://172.22.129.186:9601"]
  username: "elastic"
  password: "MY_PASSWORD"
  xpack.enabled: true

- module: logstash
  metricsets:
    - node
    - node_stats
  period: 10s
  hosts: ["http://172.22.129.187:9602"]
  username: "elastic"
  password: "MY_PASSWORD"
  xpack.enabled: true

- module: logstash
  metricsets:
    - node
    - node_stats
  period: 10s
  hosts: ["http://172.22.129.187:9603"]
  username: "elastic"
  password: "MY_PASSWORD"
  xpack.enabled: true

[Farid_N]

Thanks for your response
I edited logstash-spack.yml as you suggested.
But unfortunately, that doesn't work. I have all of 3 instances' pipelines, but have just one random node...

[ylasri (Yassine LASRI)]

The suggested solution by @leandrojmp should work
I used to use this to monitor multiple instances of beats using the same beat.yml of metricbeat
That should be the same for logstash

[eraste (Eraste AKANDE)]

Metricbeat has two modules: logstash and logstash-xpack.
So,
1- Enable logstash module : metricbeat modules enable logstash
2- Edit etc/metricbeat/modules.d/logstash.yml like this:

• module: logstash
  metricsets:
  • node
  • node_stats
    period: 10s
    hosts:["http://172.22.129.187:9603","http://172.22.33.186:9601","http://172.22.33.187:9602"]
    username: "elastic"
    password: "MY_PASSWORD"