Can't monitor Logstash with Metricbeat

Dear all, I have a 3-node Elasticsearch 7.9.1 cluster. In another server, I have a Logstash 7.9.1 service and Metricbeat with logstash-xpack module enable.

The problem is that in my Kibana --> Stack Monitoring section, I can't see Logstash, entering Setup Mode I can see this:

Please, I show you my configurations in order to get your help:

/etc/logstash/logstash.yml:

path.data: /var/lib/logstash
pipeline.ordered: auto
config.debug: true
http.enabled: true
#HTTP API Settings
http.host: 127.0.0.1
http.port: 9601
#Metrics Settings
http.host: "127.0.0.1"
http.port: 9600
log.level: debug
path.logs: /var/log/logstash
monitoring.enabled: false

/etc/metricbeat/metricbeat.yml:

metricbeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: true
reload.period: 60s
setup.template.settings:
index.number_of_shards: 1
index.codec: best_compression
setup.kibana:
host: "https://box1:5601"
output.elasticsearch:
hosts: ["box1:9200", "box2:9200", "box3:9200"]
protocol: "https"
username: "elastic"
password: "xxx"
ssl.certificate_authorities: ["/etc/ssl/certs/ca.crt"]
processors:

  • add_host_metadata: ~
  • add_cloud_metadata: ~
  • add_docker_metadata: ~
  • add_kubernetes_metadata: ~
    monitoring.enabled: false
    monitoring.elasticsearch:

/etc/metricbeat/modules.d:

  • module: logstash
    metricsets: ["node", "node_stats"]
    xpack.enabled: true
    period: 10s
    hosts: ["localhost:9600"]

Thanks a lot!!!

Hi @robertitox,

Where are your logstash pipelines sending their data? Directly to Elasticsearch, or to another service?

Dear Chris, my logstash pipelines data are sent directly to Elasticsearch.

Thanks and I'll be wainting for your response.

Regards!!!

Great.

Two things to check:

  1. On the machine running logstash, hit the _node/stats?vertices=true API for Logstash and return the full response

  2. Run this query against the monitoring cluster

GET _cat/indices/.monitoring-*

If you see .monitoring-logstash-* indices in the list, then run this query and return the result:

POST .monitoring-logstash-*/_search
{
  "size": 0,
  "aggs": {
    "clusters": {
      "terms": {
        "field": "cluster_uuid",
        "size": 10
      },
      "aggs": {
        "types": {
          "terms": {
            "field": "type",
            "size": 10
          },
          "aggs": {
            "first": {
              "min": {
                "field": "timestamp"
              }
            },
            "last": {
              "max": {
                "field": "timestamp"
              }
            }
          }
        }
      }
    }
  }
}