# Kibana complaining that "metricbeat-\*" is not indexed **URL:** \ **Category:** Beats\ **Tags:** metricbeat\ **Created:** [November 21, 2016, 8:46pm UTC](https://discuss.elastic.co/t/kibana-complaining-that-metricbeat-is-not-indexed/66783 "2016-11-21T20:46:40Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![walterfore](https://avatars.discourse-cdn.com/v4/letter/w/22d042/32.png) [@walterfore](https://discuss.elastic.co/u/walterfore)\ **Post date:** [November 21, 2016, 8:46pm UTC](https://discuss.elastic.co/t/kibana-complaining-that-metricbeat-is-not-indexed/66783/1 "2016-11-21T20:46:40Z")
Hi - although I have created an index manually called metricbeat, when I try to bring up any visualizations associated with metricbeat or perform a discovery on metricbeat-\* I received the following error in Kibana (4.4.1) [index\_not\_found\_exception] no such index, with: {"index":"[metricbeat-\*]"} I have metricbeat logging to logstash which in turn is piping the data over to elasticsearch. Has anyone ran into this issue and do you have a resolution. I have also successfully uploaded the beats-dashboards-5.0.0/ metricbeats json. ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [November 21, 2016, 10:46pm UTC](https://discuss.elastic.co/t/kibana-complaining-that-metricbeat-is-not-indexed/66783/2 "2016-11-21T22:46:34Z")
> [@walterfore](#): > > I have created an index manually called metricbeat An index called `metricbeat` will not be matched by the pattern `metricbeat-*`. I recommend following the [Getting Started](https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-getting-started.html) guide for Metricbeat. There is a [section](https://www.elastic.co/guide/en/beats/libbeat/5.0/logstash-installation.html#logstash-setup) in the docs that describes how to use Metricbeat with Logstash and what the LS config should be. When using Metricbeat with LS you also need to manually install the Metricbeat index template (this is the getting started guide). ---
**Author:** ![walterfore](https://avatars.discourse-cdn.com/v4/letter/w/22d042/32.png) [@walterfore](https://discuss.elastic.co/u/walterfore)\ **Post date:** [November 22, 2016, 7:16pm UTC](https://discuss.elastic.co/t/kibana-complaining-that-metricbeat-is-not-indexed/66783/3 "2016-11-22T19:16:09Z")
Looks like it did create the index metricbeat-\*. I can see it in kibana but when I try to view or build a visual against it it throws that error back at me that I listed in the thread. If I execute http://:9200/\_cat/indices?pretty it show "metricbeat" but it shows no file count, it has a zero beside it. I was able to run the curl XPUT command and load the templete, I was able to successfully upload the beats/metricbeats dashboards, index patterns, search, and visualization(s) via the import script. It's just not populating any information into logstash that I can tell and then of course, the error I listed pops up say the index is not found. I seems like up until that point I had pressed all the right buttons but it's not populating the index with anything and/or kibana just want recognize it even though it shows everything listed correctly. ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [November 22, 2016, 7:20pm UTC](https://discuss.elastic.co/t/kibana-complaining-that-metricbeat-is-not-indexed/66783/4 "2016-11-22T19:20:35Z")
Please share your Metricbeat and Logstash configs. And can you post what's in the Metricbeat log output. ---
**Author:** ![walterfore](https://avatars.discourse-cdn.com/v4/letter/w/22d042/32.png) [@walterfore](https://discuss.elastic.co/u/walterfore)\ **Post date:** [November 22, 2016, 7:59pm UTC](https://discuss.elastic.co/t/kibana-complaining-that-metricbeat-is-not-indexed/66783/5 "2016-11-22T19:59:18Z")
Thanks Andrew: Here is a snippet of the metricbeat output running: 2016/11/22 18:24:31.011025 beat.go:264: INFO Home path: [/home/ag03655/metricbeat-5.0.0-linux-x86\_64] Config path: [/home/ag03655/metricbeat-5.0.0-linux-x86\_64] Data path: [/home/ag03655/metricbeat-5.0.0-linux-x86\_64/data] Logs path: [/home/ag03655/metricbeat-5.0.0-linux-x86\_64/logs] 2016/11/22 18:24:31.011086 beat.go:174: INFO Setup Beat: metricbeat; Version: 5.0.0 2016/11/22 18:24:31.011145 logp.go:219: INFO Metrics logging every 30s 2016/11/22 18:24:31.011178 logstash.go:90: INFO Max Retries set to: 3 2016/11/22 18:24:31.011274 outputs.go:106: INFO Activated logstash as output plugin. 2016/11/22 18:24:31.011364 publish.go:291: INFO Publisher name: ag03sdcla00801.dcsouth.tenn 2016/11/22 18:24:31.011636 async.go:63: INFO Flush Interval set to: 1s 2016/11/22 18:24:31.011650 async.go:64: INFO Max Bulk Size set to: 2048 2016/11/22 18:24:31.011735 metricbeat.go:25: INFO Register [ModuleFactory:[system], MetricSetFactory:[apache/status, haproxy/info, haproxy/stat, mongodb/status, mysql/status, nginx/stubstatus, postgresql/activity, postgresql/bgwriter, postgresql/database, redis/info, redis/keyspace, system/core, system/cpu, system/diskio, system/filesystem, system/fsstat, system/load, system/memory, system/network, system/process, zookeeper/mntr]] 2016/11/22 18:24:31.113507 beat.go:204: INFO metricbeat start running. 2016/11/22 18:25:01.011417 logp.go:230: INFO Non-zero metrics in the last 30s: fetches.system-memory.success=3 fetches.system-load.events=3 fetches.system-memory.events=3 fetches.system-process.events=821 fetches.system-cpu.events=3 fetches.system-filesystem.success=3 libbeat.publisher.published\_events=965 fetches.system-filesystem.events=126 fetches.system-network.events=9 fetches.system-process.success=3 libbeat.logstash.publish.write\_bytes=74459 libbeat.logstash.published\_and\_acked\_events=965 fetches.system-load.success=3 libbeat.publisher.messages\_in\_worker\_queues=965 fetches.system-cpu.success=3 libbeat.logstash.call\_count.PublishEvents=3 libbeat.logstash.publish.read\_bytes=384 fetches.system-network.success=3 2016/11/22 18:25:31.011411 logp.go:230: INFO Non-zero metrics in the last 30s: libbeat.publisher.published\_events=966 fetches.system-process.success=3 libbeat.logstash.publish.write\_bytes=71640 libbeat.logstash.publish.read\_bytes=150 fetches.system-cpu.events=3 fetches.system-load.events=3 fetches.system-load.success=3 fetches.system-filesystem.events=126 libbeat.logstash.call\_count.PublishEvents=3 fetches.system-filesystem.success=3 fetches.system-network.success=3 fetches.system-network.events=9 fetches.system-process.events=822 libbeat.logstash.published\_and\_acked\_events=966 fetches.system-cpu.success=3 fetches.system-memory.events=3 fetches.system-memory.success=3 libbeat.publisher.messages\_in\_worker\_queues=966 I looks like it's sending data but I could be mistaken. My LogStash config file is: input { file { type =\> "syslog" path =\> ["/var/log/secure", "/var/log/messages"] exclude =\> ["\*.gz"] } udp { port =\> 25826 buffer\_size =\> 1452 codec =\> collectd { } } beats { port =\> 5044 } file { type =\> "jboss" path =\> ["/var/log/jbossas/eportalhost/servers/\*/server.log"] start\_position =\> beginning codec =\> multiline { pattern =\> "^%{TIMESTAMP\_ISO8601} " negate =\> true what =\> "previous" } } file { type =\> "jboss\_host" path =\> ["/var/log/jbossas/eportalhost/host-controller.log"] start\_position =\> beginning codec =\> multiline { pattern =\> "^%{TIMESTAMP\_ISO8601} " negate =\> true what =\> "previous" } } file { type =\> "jboss\_process" path =\> ["/var/log/jbossas/eportalhost/process-controller.log"] start\_position =\> beginning } file { type =\> "logstash\_error" path =\> ["/var/log/logstash/logstash.err"] start\_position =\> beginning } file { type =\> "logstash\_error" path =\> ["/var/log/logstash/logstash.err"] start\_position =\> beginning } file { type =\> "jon\_agent" path =\> ["/var/log/jboss-on/agent/agent.log"] start\_position =\> beginning } } output { stdout { } redis { host =\> [":6379", ":6379"] shuffle\_hosts =\> true data\_type =\> "list" key =\> "logstash" } So, as you can see, I may have left out the fact that logstash is pushing to two REDIS servers and from the REDIS servers there is a central standalone logstash server pulling everything from REDIS and pushing to elasticsearch. That can be a little complicated to try and visualize, i know. This was in place when I came on board and I really do not understand the importance of REDIS in the mix. But that is a different Topic of discussion I suppose. ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [November 22, 2016, 9:20pm UTC](https://discuss.elastic.co/t/kibana-complaining-that-metricbeat-is-not-indexed/66783/6 "2016-11-22T21:20:24Z")
> [@walterfore](#): > > libbeat.logstash.published\_and\_acked\_events=966 It looks like the data is being ack'ed by that Logstash. What's the config look like on the Logstash that pulls from Redis? (if you surround the config with three backticks at the start and end it will format nicely). ---
**Author:** ![walterfore](https://avatars.discourse-cdn.com/v4/letter/w/22d042/32.png) [@walterfore](https://discuss.elastic.co/u/walterfore)\ **Post date:** [November 22, 2016, 9:39pm UTC](https://discuss.elastic.co/t/kibana-complaining-that-metricbeat-is-not-indexed/66783/7 "2016-11-22T21:39:40Z")
Cool - You're going to love the simplicity of the centralized logstash standalone server that pull from the REDIS servers: ```auto input { redis { host => "" type => "redis-input" port => "6379" data_type => "list" key => "logstash" } redis { host => "" type => "redis-input" port => "6379" data_type => "list" key => "logstash" } # syslog { # type => syslog # port => 5514 # } } output { stdout { } elasticsearch { hosts => [":9200"] user => "" password => "" } } ``` So it is basically pulling All the info from REDIS and finally piping it over to our ElasticSearch server. Im not sure if I know what you mean by (if you surround the config with three backticks at the start and end it will format nicely). But it sounds promising. ---
**Author:** ![walterfore](https://avatars.discourse-cdn.com/v4/letter/w/22d042/32.png) [@walterfore](https://discuss.elastic.co/u/walterfore)\ **Post date:** [November 22, 2016, 9:41pm UTC](https://discuss.elastic.co/t/kibana-complaining-that-metricbeat-is-not-indexed/66783/8 "2016-11-22T21:41:29Z")
scratch that syslog { type =\> syslog port =\> 5514 } That is actually commented out so the format of this messaging system thinks I meant to make it bold because of the pound sign ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [November 22, 2016, 9:48pm UTC](https://discuss.elastic.co/t/kibana-complaining-that-metricbeat-is-not-indexed/66783/9 "2016-11-22T21:48:02Z")
Based on the elasticsearch output config you posted, the events are not being routed to the proper index. The default index is "logstash-%{+YYYY.MM.dd}". Can you print out one of the Metricbeat events coming from redis with: ```auto output { stdout { codec => rubydebug { metadata => true } } } ``` Then I can advise you on a conditional output configuration. ---
**Author:** ![walterfore](https://avatars.discourse-cdn.com/v4/letter/w/22d042/32.png) [@walterfore](https://discuss.elastic.co/u/walterfore)\ **Post date:** [November 22, 2016, 9:56pm UTC](https://discuss.elastic.co/t/kibana-complaining-that-metricbeat-is-not-indexed/66783/10 "2016-11-22T21:56:42Z")
Awesome. Would I add that entry into the Centralized Logstash config - or where would be the best place to have that execute. Im not 100% sure if that would replace the current "output" entry in the shipper.conf file for logstash and/or what server it needs to execute on ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [November 22, 2016, 10:03pm UTC](https://discuss.elastic.co/t/kibana-complaining-that-metricbeat-is-not-indexed/66783/11 "2016-11-22T22:03:58Z")
Just briefly modify the config on the Logstash instance that pulls from Redis. And post one of the Metricbeat events that's coming through Redis. ```auto output { stdout { codec => rubydebug { metadata => true } } elasticsearch { hosts => [":9200"] user => "" password => "" } } ``` ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [November 22, 2016, 10:17pm UTC](https://discuss.elastic.co/t/kibana-complaining-that-metricbeat-is-not-indexed/66783/12 "2016-11-22T22:17:14Z")
I think this should work. It basically routes all Metricbeat events to the metricbeat-YYYY.MM.dd index based on the fact that the `type` is `metricsets`. I don't think the events in Redis will have `@metadata` so we can't use that to route on (that's why I was asking for an event to be output). ```auto output { stdout { } if [type] == "metricsets" { elasticsearch { hosts => "elasticsearch:9200" manage_template => false index => "metricbeat-%{+YYYY.MM.dd}" document_type => "%{[type]}" user => "" password => "" } } else { elasticsearch { hosts => ["elasticsearch:9200"] user => "" password => "" } } } ``` FYI: Metricbeat can write directly to Redis if you want to bypass that first round of Logstash servers. See [Redis output](https://www.elastic.co/guide/en/beats/metricbeat/5.0/redis-output.html). ---
**Author:** ![walterfore](https://avatars.discourse-cdn.com/v4/letter/w/22d042/32.png) [@walterfore](https://discuss.elastic.co/u/walterfore)\ **Post date:** [November 22, 2016, 10:22pm UTC](https://discuss.elastic.co/t/kibana-complaining-that-metricbeat-is-not-indexed/66783/13 "2016-11-22T22:22:19Z")
ok it's in there now and I restarted logstash. How can I view the event that comes through? It did however give me a different error when I tried to view the metricbeat-\* index: Courier Fetch: [index\_not\_found\_exception] no such index, with: {"index":"[metricbeat-\*]"} Also, I will try your suggested config change with the if condition and let you know. ---
**Author:** ![walterfore](https://avatars.discourse-cdn.com/v4/letter/w/22d042/32.png) [@walterfore](https://discuss.elastic.co/u/walterfore)\ **Post date:** [November 22, 2016, 10:43pm UTC](https://discuss.elastic.co/t/kibana-complaining-that-metricbeat-is-not-indexed/66783/14 "2016-11-22T22:43:02Z")
I'll have to revisit this in the morning Andrew, I locked up the logstash system id with the new config settings. (I think) . I may have already locked it out before and didnt know it from an eariler run. I'll touch base tomorrow. Thank you for you help so far on this. W ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 20, 2016, 10:43pm UTC](https://discuss.elastic.co/t/kibana-complaining-that-metricbeat-is-not-indexed/66783/15 "2016-12-20T22:43:28Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.