# Logstash Metrics unavailable on Kibana Stack Monitoring UI **URL:** https://discuss.elastic.co/t/logstash-metrics-unavailable-on-kibana-stack-monitoring-ui/343328 **Category:** Logstash **Tags:** elastic-stack-monitoring **Created:** [September 19, 2023, 8:08am UTC](https://discuss.elastic.co/t/logstash-metrics-unavailable-on-kibana-stack-monitoring-ui/343328 "2023-09-19T08:08:41Z") **Posts on this page:** 10 **Page:** 1
### Author: ![gsekar](https://avatars.discourse-cdn.com/v4/letter/g/ccd318/32.png) [@gsekar](https://discuss.elastic.co/u/gsekar) #### Post date: [September 19, 2023, 8:08am UTC](https://discuss.elastic.co/t/logstash-metrics-unavailable-on-kibana-stack-monitoring-ui/343328/1 "2023-09-19T08:08:41Z")
Hi all Have installed metricbeat to monitor Logstash Nodes. The data stream - .monitoring-logstash-8-mb does get created and am seeing the data in the discover tab. But in the Stack Monitoring page for some reason the data for Logstash does not show up. Facing the same issue for Beats (filebeat) Monitoring too. Have used a similar setup to monitor Kibana and ES nodes and those work fine and am able to see the stats on the monitoring page Any guide on this would be very helpful PS: Running version of the stack is 8.7.1 ---
### Author: ![miltonhultgren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/miltonhultgren/32/100401_2.png) [@miltonhultgren](https://discuss.elastic.co/u/miltonhultgren) #### Post date: [September 19, 2023, 8:59am UTC](https://discuss.elastic.co/t/logstash-metrics-unavailable-on-kibana-stack-monitoring-ui/343328/2 "2023-09-19T08:59:02Z")
Do you have any errors in your Metricbeat logs for the instances that are monitoring Logstash? It could be that it's failing to collect one of the metricsets. In Discover, which metricsets can you see? Since you're on a newer version of Kibana, you have access to the Stack Monitoring [health API](https://github.com/elastic/kibana/blob/v8.9.2/x-pack/plugins/monitoring/server/routes/api/v1/_health/README.md) that can help us debug what data is available. For both Logstash and the Beats, are they all connected to the same Elasticsearch cluster that you are monitoring? ---
### Author: ![gsekar](https://avatars.discourse-cdn.com/v4/letter/g/ccd318/32.png) [@gsekar](https://discuss.elastic.co/u/gsekar) #### Post date: [September 19, 2023, 9:46am UTC](https://discuss.elastic.co/t/logstash-metrics-unavailable-on-kibana-stack-monitoring-ui/343328/3 "2023-09-19T09:46:42Z")
@miltonhultgren Don't see any errors in the metricbeat logs. Below is how the config looks like, currently have disabled Beats but yes they are connected to the same ES cluster. ```auto - module: kibana metricsets: - stats period: 10s hosts: [""] username: "" password: "" xpack.enabled: true - module: elasticsearch period: 10s hosts: [""] username: "" password: "" xpack.enabled: true - module: logstash period: 10s metricsets: - node - node_stats hosts: [""] username: "" password: "" xpack.enabled: true ``` Below is one of the outputs: ```auto { "_index": ".ds-.monitoring-logstash-8-mb-2023.09.19-000001", "_id": "giLIrIoB2hnO2QgUetY4", "_version": 1, "_score": 0, "_source": { "@timestamp": "2023-09-19T09:32:13.970Z", "event": { "dataset": "logstash.node.stats", "module": "logstash", "duration": 58322714 }, "logstash": { "node": { "stats": { "timestamp": "2023-09-19T09:32:14.028Z", "events": { "duration_in_millis": 1026, "in": 0, "filtered": 0, "out": 0 }, "reloads": { "failures": 0, "successes": 0 }, "queue": { "events_count": 0 }, "process": { "open_file_descriptors": 173, "max_file_descriptors": 1048576, "cpu": { "percent": 7 } }, "os": { "cpu": { "percent": 0, "load_average": { "15m": 5.96, "1m": 5.25, "5m": 6.66 } }, "cgroup": { "cpuacct": null, "cpu": { "control_group": "", "cfs_quota_micros": 0, "stat": null } } }, "jvm": { "gc": { "collectors": { "old": { "collection_count": 0, "collection_time_in_millis": 0 }, "young": { "collection_count": 220, "collection_time_in_millis": 4605 } } }, "mem": { "heap_max_in_bytes": 4294967294, "heap_used_in_bytes": 2038912656, "heap_used_percent": 47 }, "uptime_in_millis": 8642993 }, "pipelines": [ { "events": { "queue_push_duration_in_millis": 0, "duration_in_millis": 283, "filtered": 0, "in": 0, "out": 0 }, "reloads": { "successes": 0, "failures": 0 }, "queue": { "events_count": 0, "max_queue_size_in_bytes": 0, "queue_size_in_bytes": 0, "type": "memory" }, "vertices": [ { "pipeline_ephemeral_id": "9a190692-8c9d-4d5b-b2df-f760f83e359a", "queue_push_duration_in_millis": 0, "events_out": 0, "id": "kafka-nonprod" }, { "duration_in_millis": 81, "events_in": 0, "events_out": 0, "id": "173a762c4b4a8401cb50e7091c6372eb665d02155a5d80cd99c94cf34fe0fa22", "pipeline_ephemeral_id": "9a190692-8c9d-4d5b-b2df-f760f83e359a" }, { "events_out": 0, "id": "106c904cd1afa52d0d97fbbcad7cd087950303bad607fc67ad628cf91189595c", "pipeline_ephemeral_id": "9a190692-8c9d-4d5b-b2df-f760f83e359a", "duration_in_millis": 11, "events_in": 0 }, { "events_out": 0, "id": "elastic-nonprod", "pipeline_ephemeral_id": "9a190692-8c9d-4d5b-b2df-f760f83e359a", "cluster_uuid": "MkUomKBLT6aa2MwBprBc_A", "duration_in_millis": 98, "events_in": 0 } ], "id": "nonprodlogs", "hash": "2d28d16bdcbfc5043f824ef368c72c7add4206d00959628775d9e8a7da3c162f", "ephemeral_id": "9a190692-8c9d-4d5b-b2df-f760f83e359a" }, { "hash": "b81067844b9bd7ce93a455f6a867314d0eafd4bf8a265f10fbbd6b49c842ee59", "ephemeral_id": "c01dfe30-1ff5-497e-bd12-dbc8fd555799", "events": { "duration_in_millis": 286, "filtered": 0, "in": 0, "out": 0, "queue_push_duration_in_millis": 0 }, "reloads": { "successes": 0, "failures": 0 }, "queue": { "type": "memory", "events_count": 0, "max_queue_size_in_bytes": 0, "queue_size_in_bytes": 0 }, "vertices": [ { "events_out": 0, "id": "kafka-perf-nonprod", "pipeline_ephemeral_id": "c01dfe30-1ff5-497e-bd12-dbc8fd555799", "queue_push_duration_in_millis": 0 }, { "id": "5cd697a78e4614ce8bad85163662df4f4b544b4f5b26f1f6b91b60aa4390171b", "pipeline_ephemeral_id": "c01dfe30-1ff5-497e-bd12-dbc8fd555799", "duration_in_millis": 68, "events_in": 0, "events_out": 0 }, { "duration_in_millis": 11, "events_in": 0, "events_out": 0, "id": "17ffbaaf92f5cdc150b92f22896e720ad3dc533d904d53293634d4034547d52f", "pipeline_ephemeral_id": "c01dfe30-1ff5-497e-bd12-dbc8fd555799" }, { "cluster_uuid": "MkUomKBLT6aa2MwBprBc_A", "duration_in_millis": 106, "events_in": 0, "events_out": 0, "id": "elastic-perf-nonprod", "pipeline_ephemeral_id": "c01dfe30-1ff5-497e-bd12-dbc8fd555799" } ], "id": "perfcluster" }, { "id": "signetperf", "hash": "a8796c1de82a8534fca361a09b7f93ac23b690570ad9cf8faedff718d22fc420", "ephemeral_id": "aab1a03c-3984-4517-b30d-3539ba3c12b0", "events": { "filtered": 0, "in": 0, "out": 0, "queue_push_duration_in_millis": 0, "duration_in_millis": 174 }, "reloads": { "failures": 0, "successes": 0 }, "queue": { "type": "memory", "events_count": 0, "max_queue_size_in_bytes": 0, "queue_size_in_bytes": 0 }, "vertices": [ { "events_out": 0, "id": "kafka-signet-perf-nonprod", "pipeline_ephemeral_id": "aab1a03c-3984-4517-b30d-3539ba3c12b0", "queue_push_duration_in_millis": 0 }, { "pipeline_ephemeral_id": "aab1a03c-3984-4517-b30d-3539ba3c12b0", "duration_in_millis": 36, "events_in": 0, "events_out": 0, "id": "3f0db88085ac75ecc9e76656a6b6e2e8f5396e391d933339b4cbf0b1499d5e43" }, { "pipeline_ephemeral_id": "aab1a03c-3984-4517-b30d-3539ba3c12b0", "duration_in_millis": 12, "events_in": 0, "events_out": 0, "id": "3ae09666f2cf5668681d20b0957e99106f7eab583678d1894594291737bef1ac" }, { "cluster_uuid": "MkUomKBLT6aa2MwBprBc_A", "duration_in_millis": 58, "events_in": 0, "events_out": 0, "id": "elastic-signet-perf-nonprod", "pipeline_ephemeral_id": "aab1a03c-3984-4517-b30d-3539ba3c12b0" } ] }, { "hash": "da032c952cfa48c81d95b7aefc4508453a3c8c593e05fa0619d901238bda6fae", "ephemeral_id": "a53d534e-e568-492d-a635-a065093a1be4", "events": { "in": 0, "out": 0, "queue_push_duration_in_millis": 0, "duration_in_millis": 264, "filtered": 0 }, "reloads": { "successes": 0, "failures": 0 }, "queue": { "events_count": 0, "max_queue_size_in_bytes": 0, "queue_size_in_bytes": 0, "type": "memory" }, "vertices": [ { "events_out": 0, "id": "kafka-filebeat-metrics-nonprod", "pipeline_ephemeral_id": "a53d534e-e568-492d-a635-a065093a1be4", "queue_push_duration_in_millis": 0 }, { "cluster_uuid": "MkUomKBLT6aa2MwBprBc_A", "duration_in_millis": 141, "events_in": 0, "events_out": 0, "id": "5458cde1d3492f585a53f8348912826bb43c5ec17de43d1b037c28feb995ee17", "pipeline_ephemeral_id": "a53d534e-e568-492d-a635-a065093a1be4" }, { "pipeline_ephemeral_id": "a53d534e-e568-492d-a635-a065093a1be4", "cluster_uuid": "MkUomKBLT6aa2MwBprBc_A", "duration_in_millis": 23, "events_in": 0, "events_out": 0, "id": "ee71743edcff7959c96e78ca85788300924cb785f702453653e9ab60e23987fc" } ], "id": "filebeatmetrics" } ], "logstash": { "host": "logstash-0", "snapshot": false, "http_address": "0.0.0.0:9600", "pipeline": { "batch_size": 250, "workers": 4 }, "uuid": "d7d77c8b-9b20-4c7c-a454-d6516bc6c886", "ephemeral_id": "0c2aa487-acec-44fa-9637-b5650816c011", "name": "logstash-0", "version": "8.7.1", "status": "green" } } }, "cluster": { "id": "nonprod-logs-cluster" }, "elasticsearch": { "cluster": { "id": "nonprod-logs-cluster" } } }, "ecs": { "version": "8.0.0" }, "host": { "name": "metricbeat-8.7.1-7d4ffc985b-pb98d" }, "agent": { "name": "metricbeat-8.7.1-7d4ffc985b-pb98d", "type": "metricbeat", "version": "8.7.1", "ephemeral_id": "d5918dc5-cb84-4d12-aca3-a086041d8b56", "id": "2ee5eb1d-1ef4-42c5-8e6a-a5cfdfb3a7c5" }, "cloud": { "region": "westus2", "provider": "azure", "account": {}, "instance": { "name": "aks-azpool3-35535046-vmss_4", "id": "8818c149-18cf-4a6f-83f0-155271ea3b86" }, "machine": { "type": "Standard_D4s_v3" }, "service": { "name": "Virtual Machines" } }, "service": { "type": "logstash", "name": "logstash", "id": "", "hostname": "logstash-0", "version": "8.7.1", "address": "http://logstash-0:9600/_node/stats" }, "metricset": { "name": "node_stats", "period": 10000 } }, "fields": { "logstash.node.stats.events.out": [ 0 ], "logstash_stats.events.out": [ 0 ], "logstash_stats.queue.events_count": [ 0 ], "logstash.node.stats.jvm.mem.heap_used_percent": [ 47 ], "logstash.cluster.id": [ "nonprod-logs-cluster" ], "logstash.node.stats.timestamp": [ "2023-09-19T09:32:14.028Z" ], "service.type": [ "logstash" ], "logstash.node.stats.process.max_file_descriptors": [ 1048576 ], "logstash.node.stats.events.filtered": [ 0 ], "logstash.node.stats.logstash.pipeline.batch_size": [ 250 ], "logstash.node.stats.logstash.http_address": [ "0.0.0.0:9600" ], "agent.name": [ "metricbeat-8.7.1-7d4ffc985b-pb98d" ], "host.name": [ "metricbeat-8.7.1-7d4ffc985b-pb98d" ], "logstash.node.stats.process.cpu.percent": [ 7 ], "logstash.node.stats.queue.events_count": [ 0 ], "logstash_stats.jvm.mem.heap_max_in_bytes": [ 4294967294 ], "logstash.node.stats.logstash.pipeline.workers": [ 4 ], "logstash.node.stats.logstash.status": [ "green" ], "logstash.node.stats.jvm.mem.heap_max_in_bytes": [ 4294967294 ], "cloud.region": [ "westus2" ], "logstash.node.stats.reloads.successes": [ 0 ], "logstash.node.stats.reloads.failures": [ 0 ], "logstash_stats.logstash.uuid": [ "d7d77c8b-9b20-4c7c-a454-d6516bc6c886" ], "cloud.provider": [ "azure" ], "cloud.machine.type": [ "Standard_D4s_v3" ], "logstash_stats.events.in": [ 0 ], "cluster_uuid": [ "nonprod-logs-cluster" ], "agent.id": [ "2ee5eb1d-1ef4-42c5-8e6a-a5cfdfb3a7c5" ], "cloud.service.name": [ "Virtual Machines" ], "logstash.node.stats.logstash.version": [ "8.7.1" ], "ecs.version": [ "8.0.0" ], "service.address": [ "http://logstash-0:9600/_node/stats" ], "logstash.node.stats.process.open_file_descriptors": [ 173 ], "logstash_stats.events.duration_in_millis": [ 1026 ], "service.hostname": [ "logstash-0" ], "agent.version": [ "8.7.1" ], "logstash.node.stats.events.duration_in_millis": [ 1026 ], "logstash_stats.jvm.mem.heap_used_in_bytes": [ 2038912656 ], "logstash.node.stats.logstash.snapshot": [ false ], "logstash.node.stats.logstash.ephemeral_id": [ "0c2aa487-acec-44fa-9637-b5650816c011" ], "logstash.node.stats.jvm.gc.collectors.young.collection_time_in_millis": [ 4605 ], "service.id": [ "" ], "logstash_stats.process.cpu.percent": [ 7 ], "logstash_stats.os.cpu.load_average.5m": [ 6.6601562 ], "logstash.node.stats.os.cgroup.cpu.control_group": [ "" ], "logstash.node.stats.os.cpu.percent": [ 0 ], "cloud.instance.id": [ "8818c149-18cf-4a6f-83f0-155271ea3b86" ], "agent.type": [ "metricbeat" ], "logstash_stats.timestamp": [ "2023-09-19T09:32:13.970Z" ], "logstash_stats.os.cpu.load_average.1m": [ 5.25 ], "event.module": [ "logstash" ], "logstash.node.stats.logstash.name": [ "logstash-0" ], "logstash_stats.os.cpu.load_average.15m": [ 5.9609375 ], "logstash.node.stats.jvm.mem.heap_used_in_bytes": [ 2038912656 ], "logstash.node.stats.pipelines": [ { "events.duration_in_millis": [ 283 ], "reloads.failures": [ 0 ], "vertices": [ { "id": [ "kafka-nonprod" ], "events_out": [ 0 ], "pipeline_ephemeral_id": [ "9a190692-8c9d-4d5b-b2df-f760f83e359a" ], "queue_push_duration_in_millis": [ 0 ] }, { "duration_in_millis": [ 81 ], "id": [ "173a762c4b4a8401cb50e7091c6372eb665d02155a5d80cd99c94cf34fe0fa22" ], "events_out": [ 0 ], "pipeline_ephemeral_id": [ "9a190692-8c9d-4d5b-b2df-f760f83e359a" ], "events_in": [ 0 ] }, { "duration_in_millis": [ 11 ], "id": [ "106c904cd1afa52d0d97fbbcad7cd087950303bad607fc67ad628cf91189595c" ], "events_out": [ 0 ], "pipeline_ephemeral_id": [ "9a190692-8c9d-4d5b-b2df-f760f83e359a" ], "events_in": [ 0 ] }, { "events_out": [ 0 ], "cluster_uuid": [ "MkUomKBLT6aa2MwBprBc_A" ], "pipeline_ephemeral_id": [ "9a190692-8c9d-4d5b-b2df-f760f83e359a" ], "events_in": [ 0 ], "duration_in_millis": [ 98 ], "id": [ "elastic-nonprod" ] } ], "queue.events_count": [ 0 ], "queue.queue_size_in_bytes": [ 0 ], "queue.max_queue_size_in_bytes": [ 0 ], "events.filtered": [ 0 ], "queue.type": [ "memory" ], "events.queue_push_duration_in_millis": [ 0 ], "events.in": [ 0 ], "reloads.successes": [ 0 ], "id": [ "nonprodlogs" ], "ephemeral_id": [ "9a190692-8c9d-4d5b-b2df-f760f83e359a" ], "hash": [ "2d28d16bdcbfc5043f824ef368c72c7add4206d00959628775d9e8a7da3c162f" ], "events.out": [ 0 ] }, { "events.duration_in_millis": [ 286 ], "reloads.failures": [ 0 ], "vertices": [ { "id": [ "kafka-perf-nonprod" ], "events_out": [ 0 ], "pipeline_ephemeral_id": [ "c01dfe30-1ff5-497e-bd12-dbc8fd555799" ], "queue_push_duration_in_millis": [ 0 ] }, { "duration_in_millis": [ 68 ], "id": [ "5cd697a78e4614ce8bad85163662df4f4b544b4f5b26f1f6b91b60aa4390171b" ], "events_out": [ 0 ], "pipeline_ephemeral_id": [ "c01dfe30-1ff5-497e-bd12-dbc8fd555799" ], "events_in": [ 0 ] }, { "duration_in_millis": [ 11 ], "id": [ "17ffbaaf92f5cdc150b92f22896e720ad3dc533d904d53293634d4034547d52f" ], "events_out": [ 0 ], "pipeline_ephemeral_id": [ "c01dfe30-1ff5-497e-bd12-dbc8fd555799" ], "events_in": [ 0 ] }, { "events_out": [ 0 ], "cluster_uuid": [ "MkUomKBLT6aa2MwBprBc_A" ], "pipeline_ephemeral_id": [ "c01dfe30-1ff5-497e-bd12-dbc8fd555799" ], "events_in": [ 0 ], "duration_in_millis": [ 106 ], "id": [ "elastic-perf-nonprod" ] } ], "queue.events_count": [ 0 ], "queue.queue_size_in_bytes": [ 0 ], "queue.max_queue_size_in_bytes": [ 0 ], "events.filtered": [ 0 ], "queue.type": [ "memory" ], "events.queue_push_duration_in_millis": [ 0 ], "events.in": [ 0 ], "reloads.successes": [ 0 ], "id": [ "perfcluster" ], "ephemeral_id": [ "c01dfe30-1ff5-497e-bd12-dbc8fd555799" ], "hash": [ "b81067844b9bd7ce93a455f6a867314d0eafd4bf8a265f10fbbd6b49c842ee59" ], "events.out": [ 0 ] }, { "events.duration_in_millis": [ 174 ], "reloads.failures": [ 0 ], "vertices": [ { "id": [ "kafka-signet-perf-nonprod" ], "events_out": [ 0 ], "pipeline_ephemeral_id": [ "aab1a03c-3984-4517-b30d-3539ba3c12b0" ], "queue_push_duration_in_millis": [ 0 ] }, { "duration_in_millis": [ 36 ], "id": [ "3f0db88085ac75ecc9e76656a6b6e2e8f5396e391d933339b4cbf0b1499d5e43" ], "events_out": [ 0 ], "pipeline_ephemeral_id": [ "aab1a03c-3984-4517-b30d-3539ba3c12b0" ], "events_in": [ 0 ] }, { "duration_in_millis": [ 12 ], "id": [ "3ae09666f2cf5668681d20b0957e99106f7eab583678d1894594291737bef1ac" ], "events_out": [ 0 ], "pipeline_ephemeral_id": [ "aab1a03c-3984-4517-b30d-3539ba3c12b0" ], "events_in": [ 0 ] }, { "events_out": [ 0 ], "cluster_uuid": [ "MkUomKBLT6aa2MwBprBc_A" ], "pipeline_ephemeral_id": [ "aab1a03c-3984-4517-b30d-3539ba3c12b0" ], "events_in": [ 0 ], "duration_in_millis": [ 58 ], "id": [ "elastic-signet-perf-nonprod" ] } ], "queue.events_count": [ 0 ], "queue.queue_size_in_bytes": [ 0 ], "queue.max_queue_size_in_bytes": [ 0 ], "events.filtered": [ 0 ], "queue.type": [ "memory" ], "events.queue_push_duration_in_millis": [ 0 ], "events.in": [ 0 ], "reloads.successes": [ 0 ], "id": [ "signetperf" ], "ephemeral_id": [ "aab1a03c-3984-4517-b30d-3539ba3c12b0" ], "hash": [ "a8796c1de82a8534fca361a09b7f93ac23b690570ad9cf8faedff718d22fc420" ], "events.out": [ 0 ] }, { "events.duration_in_millis": [ 264 ], "reloads.failures": [ 0 ], "vertices": [ { "id": [ "kafka-filebeat-metrics-nonprod" ], "events_out": [ 0 ], "pipeline_ephemeral_id": [ "a53d534e-e568-492d-a635-a065093a1be4" ], "queue_push_duration_in_millis": [ 0 ] }, { "events_out": [ 0 ], "cluster_uuid": [ "MkUomKBLT6aa2MwBprBc_A" ], "pipeline_ephemeral_id": [ "a53d534e-e568-492d-a635-a065093a1be4" ], "events_in": [ 0 ], "duration_in_millis": [ 141 ], "id": [ "5458cde1d3492f585a53f8348912826bb43c5ec17de43d1b037c28feb995ee17" ] }, { "events_out": [ 0 ], "cluster_uuid": [ "MkUomKBLT6aa2MwBprBc_A" ], "pipeline_ephemeral_id": [ "a53d534e-e568-492d-a635-a065093a1be4" ], "events_in": [ 0 ], "duration_in_millis": [ 23 ], "id": [ "ee71743edcff7959c96e78ca85788300924cb785f702453653e9ab60e23987fc" ] } ], "queue.events_count": [ 0 ], "queue.queue_size_in_bytes": [ 0 ], "queue.max_queue_size_in_bytes": [ 0 ], "events.filtered": [ 0 ], "queue.type": [ "memory" ], "events.queue_push_duration_in_millis": [ 0 ], "events.in": [ 0 ], "reloads.successes": [ 0 ], "id": [ "filebeatmetrics" ], "ephemeral_id": [ "a53d534e-e568-492d-a635-a065093a1be4" ], "hash": [ "da032c952cfa48c81d95b7aefc4508453a3c8c593e05fa0619d901238bda6fae" ], "events.out": [ 0 ] } ], "timestamp": [ "2023-09-19T09:32:13.970Z" ], "logstash_stats.logstash.version": [ "8.7.1" ], "logstash.node.stats.os.cpu.load_average.15m": [ 5.9609375 ], "service.name": [ "logstash" ], "logstash.node.stats.jvm.gc.collectors.old.collection_count": [ 0 ], "logstash.node.stats.jvm.gc.collectors.old.collection_time_in_millis": [ 0 ], "logstash.node.stats.jvm.gc.collectors.young.collection_count": [ 220 ], "metricset.period": [ 10000 ], "logstash_stats.os.cgroup.cpu.cfs_quota_micros": [ 0 ], "logstash.node.stats.logstash.host": [ "logstash-0" ], "logstash.node.stats.jvm.uptime_in_millis": [ 8642993 ], "logstash.node.stats.events.in": [ 0 ], "logstash.node.stats.logstash.uuid": [ "d7d77c8b-9b20-4c7c-a454-d6516bc6c886" ], "metricset.name": [ "node_stats" ], "event.duration": [ 58322714 ], "logstash.node.stats.os.cpu.load_average.1m": [ 5.25 ], "@timestamp": [ "2023-09-19T09:32:13.970Z" ], "logstash.node.stats.os.cgroup.cpu.cfs_quota_micros": [ 0 ], "service.version": [ "8.7.1" ], "logstash.node.stats.os.cpu.load_average.5m": [ 6.6601562 ], "agent.ephemeral_id": [ "d5918dc5-cb84-4d12-aca3-a086041d8b56" ], "logstash_stats.jvm.uptime_in_millis": [ 8642993 ], "event.dataset": [ "logstash.node.stats" ], "logstash.elasticsearch.cluster.id": [ "nonprod-logs-cluster" ], "cloud.instance.name": [ "aks-azpool3-35535046-vmss_4" ] } } ``` ---
### Author: ![miltonhultgren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/miltonhultgren/32/100401_2.png) [@miltonhultgren](https://discuss.elastic.co/u/miltonhultgren) #### Post date: [September 19, 2023, 9:53am UTC](https://discuss.elastic.co/t/logstash-metrics-unavailable-on-kibana-stack-monitoring-ui/343328/4 "2023-09-19T09:53:00Z")
> [@gsekar](#): > > `nonprod-logs-cluster` Did you specify this as the cluster ID in the logstash.yml? Is there a matching Elasticsearch cluster for this? I think the Stack Monitoring UI won't know where to display that data otherwise. What about the result from the Health API? ---
### Author: ![gsekar](https://avatars.discourse-cdn.com/v4/letter/g/ccd318/32.png) [@gsekar](https://discuss.elastic.co/u/gsekar) #### Post date: [September 19, 2023, 10:03am UTC](https://discuss.elastic.co/t/logstash-metrics-unavailable-on-kibana-stack-monitoring-ui/343328/5 "2023-09-19T10:03:38Z")
Thanks @miltonhultgren that gave me an idea, actually had set monitoring.cluster\_uuid: "nonprod-logs-cluster" but then realized the actual UUID was different a randomly generated id. ---
### Author: ![gsekar](https://avatars.discourse-cdn.com/v4/letter/g/ccd318/32.png) [@gsekar](https://discuss.elastic.co/u/gsekar) #### Post date: [September 19, 2023, 10:06am UTC](https://discuss.elastic.co/t/logstash-metrics-unavailable-on-kibana-stack-monitoring-ui/343328/6 "2023-09-19T10:06:56Z")
For Beats the approach we follow is push the output to kafka -\> logstash and then read it into ES. The datastream that get's created is not of the format .monitoring-beats-8-mb. But of the format metricbeat-\*. Will the Stack Monitoring still be able to display the data? ---
### Author: ![miltonhultgren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/miltonhultgren/32/100401_2.png) [@miltonhultgren](https://discuss.elastic.co/u/miltonhultgren) #### Post date: [September 19, 2023, 11:22am UTC](https://discuss.elastic.co/t/logstash-metrics-unavailable-on-kibana-stack-monitoring-ui/343328/7 "2023-09-19T11:22:54Z")
Right, so for Logstash, please align the ID with the randomly generated ID of the Elasticsearch cluster. For the Beats data, it has to end up in `.monitoring-*` for the Stack Monitoring UI to use it, so you'll need to figure out how to route the data to the right data stream. I'm not sure how that is done, what does your Logstash output look like today? ---
### Author: ![gsekar](https://avatars.discourse-cdn.com/v4/letter/g/ccd318/32.png) [@gsekar](https://discuss.elastic.co/u/gsekar) #### Post date: [September 19, 2023, 11:28am UTC](https://discuss.elastic.co/t/logstash-metrics-unavailable-on-kibana-stack-monitoring-ui/343328/8 "2023-09-19T11:28:36Z")
Got it thanks again, for beats we have just routed the logs directly to ES for now and things seem fine. ---
### Author: ![Jason\_Paralta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jason_paralta/32/118856_2.png) [@Jason\_Paralta](https://discuss.elastic.co/u/Jason_Paralta) #### Post date: [October 5, 2023, 3:01pm UTC](https://discuss.elastic.co/t/logstash-metrics-unavailable-on-kibana-stack-monitoring-ui/343328/9 "2023-10-05T15:01:56Z")
Hello @gsekar , Could you please help with your config file through which you are able to monitor beats and logstash. I need to implement a monitoring dashboard displaying all beats- metrcicbeat,filebeat and logstash with there hostname and status if up or down in kibana table visual. I am new to this stack and confused about XPACK settings ,where and what settings need to be done. Below table is requirement and also would like to monitor the same on kibana UI if possible. ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/5/f5f2e7b04769ed82c524d4692750680f16db9ea0.png) The idea is to show for example out of 10 filebeat on respective host if 3 goes down, then need to show the hostname along with its status up or down, same applies for rest beats and logstash.The documentation is bit confusing to implement for me. Your help would be highly appreciated. Many Thanx. ---
### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system) #### Post date: [November 2, 2023, 3:02pm UTC](https://discuss.elastic.co/t/logstash-metrics-unavailable-on-kibana-stack-monitoring-ui/343328/10 "2023-11-02T15:02:47Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.