Logstash monitoring data not showing in Stack Monitoring

[ffknob (Flávio Knob)]

Hello,

I'm not managing to get my Logstash metrics to be shown in Kibana's Stack Monitoring UI:

Even though it looks like Metricbeat is sending events to the cluster:

GET .monitoring-logstash-7-mb-2020.11.27/_search
{
  "size": 0,
  "aggs": {
    "1": {
      "terms": {
        "field": "cluster_uuid",
        "size": 10
      }
    }
  }
}

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 235,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "1" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "T-qSzUf1QNOYDw6-nGkQbA",
          "doc_count" : 234
        },
        {
          "key" : "M3nD3LkeQaCR9MtyvlpEzw",
          "doc_count" : 1
        }
      ]
    }
  }
}

By the way, cluster T-qSzUf1QNOYDw6-nGkQbA is the one coming from Logstash/Metricbeat, and the other one (M3nD3LkeQaCR9MtyvlpEzw) is the monitoring cluster itself (Elastic Cloud). I manually set the cluster_uuid for one document to the monitoring cluster's uuid guessing that Kibana would only show data for the same uuid... Didn't work.

So, how do I manage to:

1. See my Logstash monitoring data in the Stack Monitoring UI
2. See my Logstash pipelines in the Pipeline Management UI

Cluster: 7.10
Logstash: 7.3
Metricbeat: 7.10

Thank you!

[ffknob (Flávio Knob)]

TL;DR: all monitoring (Elasticsearch, Logstash, Kibana, ...) data is linked to a cluster uuid. The UI filters the data by a cluster uuid. If you are not currently monitoring the cluster to which Kibana and Logstash are linked, you will not see the data in the UI (though they will be indexed in the .monitoring- indices).*

Ok, now I have a better clue on how the Stack Monitoring works. The UI is all filtered by the cluster's uuid. So that image I've included in my original post refers to the monitoring cluster, but my Logstash data is linked to another cluster uuid... one that was not being monitored until now.

Since I've installed a metricbeat with the elasticsearch module pointing to the cluster to which Logstash send data (same cluster, same uuid), the UI started asking me first for which cluster I'd like to see the Stack Monitoring.

I bet my Logstash data would finally appear in the Stack Monitoring for the new monitored cluster, the one that is not the monitoring cluster itself. ...but unfortnutally that cluster has a Basic license, so no luck with that...

Any way, think this at least explains whats's happening.