# X-pack logstash not showing in Kibana **URL:** \ **Category:** Kibana\ **Tags:** elastic-stack-monitoring\ **Created:** [November 7, 2018, 8:00pm UTC](https://discuss.elastic.co/t/x-pack-logstash-not-showing-in-kibana/155783 "2018-11-07T20:00:10Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![frankfoti](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frankfoti/32/26965_2.png) [@frankfoti](https://discuss.elastic.co/u/frankfoti)\ **Post date:** [November 7, 2018, 8:00pm UTC](https://discuss.elastic.co/t/x-pack-logstash-not-showing-in-kibana/155783/1 "2018-11-07T20:00:10Z")
Using a remote elasticsearch monitoring cluster for x-pack. xpack enabled on logstash node. Elasticsearch and kibana are showing in monitoring fine. The logstash index is created in the remote monitoring elasticsearch. I see no errors in logs but the monitoring tab in kibana does not show logstash All elastic stack modules are 6.3. ---
**Author:** ![frankfoti](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frankfoti/32/26965_2.png) [@frankfoti](https://discuss.elastic.co/u/frankfoti)\ **Post date:** [November 9, 2018, 12:53pm UTC](https://discuss.elastic.co/t/x-pack-logstash-not-showing-in-kibana/155783/2 "2018-11-09T12:53:49Z")
The index is getting created but still not visible from Kibana. I am able to change and manage uuid and node name, Any Ideas? { ...... "ephemeral\_id": "6a5f16c3-b415-4c1b-bc30-9ee44a352d79", "host": "cloudcontrol-logstash", "uuid": "ffotitest123456", "http\_address": "127.0.0.1:9602", "name": "metricbeat\_5401", "version": "6.3.0", "snapshot": false, "status": "green" }, "events": { "in": 121, "filtered": 121, "out": 121, "duration\_in\_millis": 2161 } } }, "sort": [ 1541767370643 ] } ---
**Author:** ![frankfoti](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frankfoti/32/26965_2.png) [@frankfoti](https://discuss.elastic.co/u/frankfoti)\ **Post date:** [November 9, 2018, 3:34pm UTC](https://discuss.elastic.co/t/x-pack-logstash-not-showing-in-kibana/155783/3 "2018-11-09T15:34:12Z")
Does X-Pack management need to be enabled?. Here is the logstash.yml: # ------------ X-Pack Settings (not applicable for OSS build)-------------- # # X-Pack Monitoring # [https://www.elastic.co/guide/en/logstash/current/monitoring-logstash.html](https://www.elastic.co/guide/en/logstash/current/monitoring-logstash.html) xpack.monitoring.enabled: true #xpack.monitoring.elasticsearch.username: logstash\_system #xpack.monitoring.elasticsearch.password: password xpack.monitoring.elasticsearch.url: ["[http://10.200.2.163:9200](http://10.200.2.163:9200)"] #xpack.monitoring.elasticsearch.ssl.ca: ["/path/to/ca.crt"] #xpack.monitoring.elasticsearch.ssl.truststore.path: path/to/file #xpack.monitoring.elasticsearch.ssl.truststore.password: password #xpack.monitoring.elasticsearch.ssl.keystore.path: /path/to/file #xpack.monitoring.elasticsearch.ssl.keystore.password: password #xpack.monitoring.elasticsearch.ssl.verification\_mode: certificate #xpack.monitoring.elasticsearch.sniffing: false xpack.monitoring.collection.interval: 10s xpack.monitoring.collection.pipeline.details.enabled: true # # X-Pack Management # [https://www.elastic.co/guide/en/logstash/current/logstash-centralized-pipeline-management.html](https://www.elastic.co/guide/en/logstash/current/logstash-centralized-pipeline-management.html) #xpack.management.enabled: true #xpack.management.pipeline.id: ["main", "apache\_logs"] #xpack.management.elasticsearch.username: logstash\_admin\_user #xpack.management.elasticsearch.password: password #xpack.management.elasticsearch.url: ["[https://10.200.2.163:9200](https://10.200.2.163:9200)"] #xpack.management.elasticsearch.ssl.ca: ["/path/to/ca.crt"] #xpack.management.elasticsearch.ssl.truststore.path: /path/to/file #xpack.management.elasticsearch.ssl.truststore.password: password #xpack.management.elasticsearch.ssl.keystore.path: /path/to/file #xpack.management.elasticsearch.ssl.keystore.password: password #xpack.management.elasticsearch.sniffing: false #xpack.management.logstash.poll\_interval: 5s # ---
**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\ **Post date:** [November 9, 2018, 7:47pm UTC](https://discuss.elastic.co/t/x-pack-logstash-not-showing-in-kibana/155783/4 "2018-11-09T19:47:48Z")
You do not need `xpack.management.*` to be enabled in `logstash.yml` for Logstash X-Pack Monitoring to work. Your `xpack.monitoring.*` settings look okay. A couple of follow up questions: 1. When you start up Logstash, do you see any error messages in the log? Would it be possible for you to paste this log here after masking any sensitive details in it? 2. Do you have X-Pack Security enabled? If you do, then you'll need to uncomment the `xpack.monitoring.elasticsearch.username` and `xpack.monitoring.elasticsearch.password` settings in your `logstash.yml` as well and set them to appropriate values. More on that here: [https://www.elastic.co/guide/en/logstash/6.x/configuring-logstash.html](https://www.elastic.co/guide/en/logstash/6.x/configuring-logstash.html). Thanks, Shaunak ---
**Author:** ![frankfoti](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frankfoti/32/26965_2.png) [@frankfoti](https://discuss.elastic.co/u/frankfoti)\ **Post date:** [November 9, 2018, 8:07pm UTC](https://discuss.elastic.co/t/x-pack-logstash-not-showing-in-kibana/155783/5 "2018-11-09T20:07:47Z")
No X-Pack security enabled yet. Logstash restarted and here is the log. The index does get sent to the remote monitoring elasticsearch. Thanks [2018-11-09T15:01:32,392][INFO][logstash.pipeline] Pipeline has terminated {:pipeline\_id=\>".monitoring-logstash", :thread=\>"#\"} [2018-11-09T15:01:34,150][INFO][logstash.pipeline] Pipeline has terminated {:pipeline\_id=\>"main", :thread=\>"#\"} [2018-11-09T15:04:06,305][WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified [2018-11-09T15:04:11,649][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"6.3.0"} [2018-11-09T15:04:30,028][WARN][logstash.outputs.elasticsearch] You are using a deprecated config setting "document\_type" set in elasticsearch. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. Document types are being deprecated in Elasticsearch 6.0, and removed entirely in 7.0. You should avoid this feature If you have any questions about this, please visit the #logstash channel on freenode irc. {:name=\>"document\_type", :plugin=\>\[[http://10.200.2.163:9200](http://10.200.2.163:9200)], bulk\_path=\>"/\_xpack/monitoring/\_bulk?system\_id=logstash&system\_api\_version=2&interval=1s", manage\_template=\>false, document\_type=\>"%{[@metadata][document\_type]}", sniffing=\>false, id=\>"8391c1b4dab58805d1cb82dade9703aad5b9a4f5307bde6ec18ac2e129147a81", enable\_metric=\>true, codec=\>\"plain\_44e6158d-0697-40fc-b5af-a28068af3c3b", enable\_metric=\>true, charset=\>"UTF-8"\>, workers=\>1, template\_name=\>"logstash", template\_overwrite=\>false, doc\_as\_upsert=\>false, script\_type=\>"inline", script\_lang=\>"painless", script\_var\_name=\>"event", scripted\_upsert=\>false, retry\_initial\_interval=\>2, retry\_max\_interval=\>64, retry\_on\_conflict=\>1, action=\>"index", ssl\_certificate\_verification=\>true, sniffing\_delay=\>5, timeout=\>60, pool\_max=\>1000, pool\_max\_per\_route=\>100, resurrect\_delay=\>5, validate\_after\_inactivity=\>10000, http\_compression=\>false\>} [2018-11-09T15:04:30,135][INFO][logstash.pipeline] Starting pipeline {:pipeline\_id=\>"main", "pipeline.workers"=\>8, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>50} [2018-11-09T15:04:30,137][INFO][logstash.pipeline] Starting pipeline {:pipeline\_id=\>".monitoring-logstash", "pipeline.workers"=\>1, "pipeline.batch.size"=\>2, "pipeline.batch.delay"=\>50} [2018-11-09T15:04:34,174][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=\>{:removed=\>, :added=\>[[http://10.200.2.163:9200/](http://10.200.2.163:9200/)]}} [2018-11-09T15:04:34,173][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=\>{:removed=\>, :added=\>[[http://cloudcontrol-elasticsearch1.office.re.local:9200/](http://cloudcontrol-elasticsearch1.office.re.local:9200/)]}} [2018-11-09T15:04:34,190][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://10.200.2.163:9200/](http://10.200.2.163:9200/), :path=\>"/"} [2018-11-09T15:04:34,194][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://cloudcontrol-elasticsearch1.office.re.local:9200/](http://cloudcontrol-elasticsearch1.office.re.local:9200/), :path=\>"/"} [2018-11-09T15:04:35,637][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=\>"[http://cloudcontrol-elasticsearch1.office.re.local:9200/](http://cloudcontrol-elasticsearch1.office.re.local:9200/)"} [2018-11-09T15:04:35,642][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=\>"[http://10.200.2.163:9200/](http://10.200.2.163:9200/)"} [2018-11-09T15:04:36,334][INFO][logstash.outputs.elasticsearch] ES Output version determined {:es\_version=\>6} [2018-11-09T15:04:36,335][INFO][logstash.outputs.elasticsearch] ES Output version determined {:es\_version=\>6} [2018-11-09T15:04:36,336][WARN][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>6} [2018-11-09T15:04:36,341][WARN][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>6} [2018-11-09T15:04:36,366][INFO][logstash.outputs.elasticsearch] Using mapping template from {:path=\>nil} [2018-11-09T15:04:36,377][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::ElasticSearch", :hosts=\>["[http://10.200.2.163:9200](http://10.200.2.163:9200)"]} [2018-11-09T15:04:36,409][INFO][logstash.outputs.elasticsearch] Attempting to install template {:manage\_template=\>{"template"=\>"logstash-_", "version"=\>60001, "settings"=\>{"index.refresh\_interval"=\>"5s"}, "mappings"=\>{"default"=\>{"dynamic\_templates"=\>[{"message\_field"=\>{"path\_match"=\>"message", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false}}}, {"string\_fields"=\>{"match"=\>"_", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false, "fields"=\>{"keyword"=\>{"type"=\>"keyword", "ignore\_above"=\>256}}}}}], "properties"=\>{"@timestamp"=\>{"type"=\>"date"}, "@version"=\>{"type"=\>"keyword"}, "geoip"=\>{"dynamic"=\>true, "properties"=\>{"ip"=\>{"type"=\>"ip"}, "location"=\>{"type"=\>"geo\_point"}, "latitude"=\>{"type"=\>"half\_float"}, "longitude"=\>{"type"=\>"half\_float"}}}}}}}} ---
**Author:** ![frankfoti](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frankfoti/32/26965_2.png) [@frankfoti](https://discuss.elastic.co/u/frankfoti)\ **Post date:** [November 9, 2018, 8:08pm UTC](https://discuss.elastic.co/t/x-pack-logstash-not-showing-in-kibana/155783/6 "2018-11-09T20:08:04Z")
[2018-11-09T15:04:37,534][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::ElasticSearch", :hosts=\>["[//cloudcontrol-elasticsearch1.office.re.local:9200](https://cloudcontrol-elasticsearch1.office.re.local:9200)"]} [2018-11-09T15:04:37,712][INFO][logstash.licensechecker.licensereader] Elasticsearch pool URLs updated {:changes=\>{:removed=\>, :added=\>[[http://10.200.2.163:9200/](http://10.200.2.163:9200/)]}} [2018-11-09T15:04:37,716][INFO][logstash.licensechecker.licensereader] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://10.200.2.163:9200/](http://10.200.2.163:9200/), :path=\>"/"} [2018-11-09T15:04:38,368][WARN][logstash.licensechecker.licensereader] Restored connection to ES instance {:url=\>"[http://10.200.2.163:9200/](http://10.200.2.163:9200/)"} [2018-11-09T15:04:38,374][INFO][logstash.licensechecker.licensereader] ES Output version determined {:es\_version=\>6} [2018-11-09T15:04:38,375][WARN][logstash.licensechecker.licensereader] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>6} [2018-11-09T15:04:40,146][INFO][logstash.pipeline] Pipeline started successfully {:pipeline\_id=\>".monitoring-logstash", :thread=\>"#\"} [2018-11-09T15:04:41,072][INFO][logstash.inputs.beats] Beats inputs: Starting input listener {:address=\>"0.0.0.0:5041"} [2018-11-09T15:04:41,095][INFO][logstash.pipeline] Pipeline started successfully {:pipeline\_id=\>"main", :thread=\>"#\"} [2018-11-09T15:04:41,699][INFO][org.logstash.beats.Server] Starting server on port: 5041 [2018-11-09T15:04:41,702][INFO][logstash.agent] Pipelines running {:count=\>2, :running\_pipelines=\>[:main, :".monitoring-logstash"], :non\_running\_pipelines=\>} [2018-11-09T15:04:41,743][INFO][logstash.inputs.metrics] Monitoring License OK [2018-11-09T15:04:45,235][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9602} ---
**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\ **Post date:** [November 9, 2018, 8:27pm UTC](https://discuss.elastic.co/t/x-pack-logstash-not-showing-in-kibana/155783/7 "2018-11-09T20:27:01Z")
Thanks, logs look okay too. Can you run the following ES query on your **remote monitoring** Elasticsearch cluster and report the results here? ```auto POST .monitoring-logstash-*/_search { "size": 0, "aggs": { "type": { "terms": { "field": "type", "size": 10 }, "aggs": { "cluster_uuid": { "terms": { "field": "cluster_uuid", "size": 10 } } } } } } ``` ---
**Author:** ![bigphil](https://avatars.discourse-cdn.com/v4/letter/b/65b543/32.png) [@bigphil](https://discuss.elastic.co/u/bigphil)\ **Post date:** [November 10, 2018, 5:50am UTC](https://discuss.elastic.co/t/x-pack-logstash-not-showing-in-kibana/155783/8 "2018-11-10T05:50:24Z")
I had a similar issue with my Logstash nodes not showing up in monitoring. Turns out it was a formatting issue in the logstash yaml indentation. All settings before xpack were one space from beginning of line, column 1, xpack settings started at beginning of line, column 0. After changing the formatting so all indentation matched and restarting Logstash the node appeared in monitoring. ---
**Author:** ![frankfoti](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frankfoti/32/26965_2.png) [@frankfoti](https://discuss.elastic.co/u/frankfoti)\ **Post date:** [November 12, 2018, 2:18pm UTC](https://discuss.elastic.co/t/x-pack-logstash-not-showing-in-kibana/155783/9 "2018-11-12T14:18:34Z")
Logstash is creating the index docs which appear correct. I am using the remote elasticsearch monitoring cluster. I did manually adjust the URL to include: "cluster\_uuid": "eCSSS5C\_TsuIpqFn9LLaBg" Still stuck on this. [http://10.200.2.161:5601/app/monitoring#/logstash?\_g=(cluster\_uuid:'eCSSS5C\_TsuIpqFn9LLaBg')](http://10.200.2.161:5601/app/monitoring#/logstash?_g=(cluster_uuid:'eCSSS5C_TsuIpqFn9LLaBg')) { "\_index": ".monitoring-logstash-6-2018.11.09", "\_type": "doc", "\_id": "i4Pq-mYBkjQ9k5j1-Gwh", "\_score": null, "\_source": { "cluster\_uuid": "eCSSS5C\_TsuIpqFn9LLaBg", "timestamp": "2018-11-09T23:59:54.908Z", "interval\_ms": 1000, "type": "logstash\_state", "source\_node": { "uuid": "zrr42fh5RO6nuLkSKz8Ayw", "host": "10.200.2.163", "transport\_address": "10.200.2.163:9300", "ip": "10.200.2.163", "name": "node-1", "timestamp": "2018-11-09T23:59:54.908Z" }, ---
**Author:** ![frankfoti](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frankfoti/32/26965_2.png) [@frankfoti](https://discuss.elastic.co/u/frankfoti)\ **Post date:** [November 12, 2018, 3:03pm UTC](https://discuss.elastic.co/t/x-pack-logstash-not-showing-in-kibana/155783/10 "2018-11-12T15:03:56Z")
Added X-Pack to the monitoring cluster (trial) and appears to be working correctly now. Not sure why that was required but moving forward now. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 10, 2018, 3:17pm UTC](https://discuss.elastic.co/t/x-pack-logstash-not-showing-in-kibana/155783/11 "2018-12-10T15:17:05Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.