Monitoring Logstash using Metricbeat

Elastic Stack Beats
1

Hi,

I have 3 instances of Logstash and want to monitor them in Kibana
This is my /etc/metricbeat/modules.d/logstash-xpack.yml configuration:

# Module: logstash
# Docs: https://www.elastic.co/guide/en/beats/metricbeat/7.6/metricbeat-module-logstash.html

- module: logstash
  metricsets:
    - node
    - node_stats
  period: 10s
  hosts: ["http://172.22.129.187:9603","http://172.22.33.186:9601","http://172.22.33.187:9602"]
  username: "elastic"
  password: "MY_PASSWORD"
  xpack.enabled: true

The hhtp.port for every instance of logstash.yml is 9601 , 9602 and 9603
And also the logstash-xpack module is enabled:
metricbeat modules enable logstash-xpack

The output of Elasticsearch in metricbeat.yml is all my 4 ES nodes on port 9200

I have all pipelines in "Stack Monitoring" (all pipelines of 3 instances of Logstash)
But in Nodes section, I just see one instance of Logstash!!!

image
image1600×473 36.6 KB

This is "Pipelines" section which included 3 instances of Logstash:

image
image1584×607 56.1 KB

The weird thing is sometime the name of only Logstash node that shown in "Nodes" changes to another instance name... For example in the first picture above, elk-node101.software:9603 changes to elk-node2.software:9601 automatically!

Could you please help me about this?
I'm using the stack on version 7.16.2

Kind regards

2

Check the node.name in logstash.yml for each instance

3

Thanks for your response

I defined name for every Logstash instance. Now they have node.name in logstash.yml

node.name: elk-lnx1.software
http.host: 0.0.0.0
http.port: 9602

node.name: elk-lnx2.software
http.host: 0.0.0.0
http.port: 9601

node.name: elk-lnx101.software
http.host: 0.0.0.0
http.port: 9603

But in "Nodes" there is just one instance:

image
image1598×394 28.5 KB

4

And if you see, in the last picture I have elk-lnx1.software but in the very first picture I had elk-lnx101.software
It always changes regularly...

5

I'm not sure you can monitor 3 different instances this way.

You should use one .yml file per host you want to monitor, or create multiple modules in the same .yml , each one pointing to a different logstash, If i'm not wrong using an array of hosts will work in a load balance way and not querying all the hosts at the same time.

Try to edit your .yml to have something like this:

- module: logstash
  metricsets:
    - node
    - node_stats
  period: 10s
  hosts: ["http://172.22.129.186:9601"]
  username: "elastic"
  password: "MY_PASSWORD"
  xpack.enabled: true

- module: logstash
  metricsets:
    - node
    - node_stats
  period: 10s
  hosts: ["http://172.22.129.187:9602"]
  username: "elastic"
  password: "MY_PASSWORD"
  xpack.enabled: true

- module: logstash
  metricsets:
    - node
    - node_stats
  period: 10s
  hosts: ["http://172.22.129.187:9603"]
  username: "elastic"
  password: "MY_PASSWORD"
  xpack.enabled: true
1 Like
6

Thanks for your response
I edited logstash-spack.yml as you suggested.
But unfortunately, that doesn't work. I have all of 3 instances' pipelines, but have just one random node...

7

The suggested solution by @leandrojmp should work
I used to use this to monitor multiple instances of beats using the same beat.yml of metricbeat
That should be the same for logstash

1 Like
9

Metricbeat has two modules: logstash and logstash-xpack.
So,
1- Enable logstash module : metricbeat modules enable logstash
2- Edit etc/metricbeat/modules.d/logstash.yml like this:

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.