Can't monitor Logstash with Metricbeat

[robertitox (Robert Carna)]

Dear all, I have a 3-node Elasticsearch 7.9.1 cluster. In another server, I have a Logstash 7.9.1 service and Metricbeat with logstash-xpack module enable.

The problem is that in my Kibana --> Stack Monitoring section, I can't see Logstash, entering Setup Mode I can see this:

Please, I show you my configurations in order to get your help:

/etc/logstash/logstash.yml:

path.data: /var/lib/logstash
pipeline.ordered: auto
config.debug: true
http.enabled: true
#HTTP API Settings
http.host: 127.0.0.1
http.port: 9601
#Metrics Settings
http.host: "127.0.0.1"
http.port: 9600
log.level: debug
path.logs: /var/log/logstash
monitoring.enabled: false

/etc/metricbeat/metricbeat.yml:

metricbeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: true
reload.period: 60s
setup.template.settings:
index.number_of_shards: 1
index.codec: best_compression
setup.kibana:
host: "https://box1:5601"
output.elasticsearch:
hosts: ["box1:9200", "box2:9200", "box3:9200"]
protocol: "https"
username: "elastic"
password: "xxx"
ssl.certificate_authorities: ["/etc/ssl/certs/ca.crt"]
processors:

• add_host_metadata: ~
• add_cloud_metadata: ~
• add_docker_metadata: ~
• add_kubernetes_metadata: ~
  monitoring.enabled: false
  monitoring.elasticsearch:

/etc/metricbeat/modules.d:

• module: logstash
  metricsets: ["node", "node_stats"]
  xpack.enabled: true
  period: 10s
  hosts: ["localhost:9600"]

Thanks a lot!!!

[chrisronline (Chris Roberson)]

Hi @robertitox,

Where are your logstash pipelines sending their data? Directly to Elasticsearch, or to another service?

[robertitox (Robert Carna)]

Dear Chris, my logstash pipelines data are sent directly to Elasticsearch.

Thanks and I'll be wainting for your response.

Regards!!!

[chrisronline (Chris Roberson)]

Great.

Two things to check:

1. On the machine running logstash, hit the _node/stats?vertices=true API for Logstash and return the full response

2. Run this query against the monitoring cluster

GET _cat/indices/.monitoring-*

If you see .monitoring-logstash-* indices in the list, then run this query and return the result:

POST .monitoring-logstash-*/_search
{
  "size": 0,
  "aggs": {
    "clusters": {
      "terms": {
        "field": "cluster_uuid",
        "size": 10
      },
      "aggs": {
        "types": {
          "terms": {
            "field": "type",
            "size": 10
          },
          "aggs": {
            "first": {
              "min": {
                "field": "timestamp"
              }
            },
            "last": {
              "max": {
                "field": "timestamp"
              }
            }
          }
        }
      }
    }
  }
}