# .marvel-es-YYYY.MM.DD index deleted HOW-TO create it again **URL:** \ **Category:** Kibana\ **Tags:** elastic-stack-monitoring\ **Created:** [November 6, 2015, 8:53am UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959 "2015-11-06T08:53:05Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![nskalis](https://avatars.discourse-cdn.com/v4/letter/n/ebca7d/32.png) [@nskalis](https://discuss.elastic.co/u/nskalis)\ **Post date:** [November 6, 2015, 8:53am UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959/1 "2015-11-06T08:53:05Z")
Dear All, i recently installed Elasticsearch 2.0 and Marvel 2.0 i was experimenting with Curator and I accidentally delete the .marvel-es-YYY.MM.DD index ``` health status index pri rep docs.count docs.deleted store.size pri.store.size yellow open .kibana 1 1 3 1 26.7kb 26.7kb yellow open .marvel-es-data 1 1 1 0 3.6kb 3.6kb ``` i have only 1 configuration change in elasticsearch.yml ``` marvel.agent.interval: 1m ``` but i cannot see any statistics on the marvel page because the corresponding index is simple not re-created, even waiting for 1 day to pass i cannot see the index created. below are some screenshots from Kibana and Marvel. could you please advise on how to troubleshoot this issue ? ![](https://us1.discourse-cdn.com/elastic/original/2X/8/870239fb4c14bae86b962a50079e785254aae99d.png) ![](https://us1.discourse-cdn.com/elastic/original/2X/0/0a8bec296ed0a868301ff6647f06036af8f72208.png) Nikos ---
**Author:** ![tanguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tanguy/32/6030_2.png) [@tanguy](https://discuss.elastic.co/u/tanguy)\ **Post date:** [November 9, 2015, 10:46am UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959/2 "2015-11-09T10:46:07Z")
Hi Nikos, Did you configure marvel to index data locally or remotely using Http? Unfortunately I wasn't able to reproduce your issue on my laptop. This is what I tested: I installed both ES 2.0 and Marvel 2.0 with default configuration (ie marvel indexes data in the same cluster being monitored). After a while, I can see marvel indices corectly created: ```auto GET /_cat/indices?v health status index pri rep docs.count docs.deleted store.size pri.store.size yellow open .marvel-es-2015.11.09 1 1 8 0 20.6kb 20.6kb yellow open .marvel-es-data 1 1 1 0 3.2kb 3.2kb ``` So I deleted the .marvel-es-YYYY-MM-DD index: ```auto DELETE /.marvel-es-2015.11.09 { "acknowledged" : true } ``` And the elasticsearch logs reported: ```auto [INFO][cluster.metadata] [General Orwell Taylor] [.marvel-es-2015.11.09] deleting index ``` After few seconds, the logs show that the index has been re created: ```auto [INFO][cluster.metadata] [General Orwell Taylor] [.marvel-es-2015.11.09] creating index, cause [auto(bulk api)], templates [.marvel-es], shards [1]/[1], mappings [shards, _default_, index_stats, index_recovery, cluster_state, cluster_info, cluster_stats, node_stats, indices_stats] [2015-11-09 11:31:36,911][INFO][cluster.metadata] [General Orwell Taylor] [.marvel-es-2015.11.09] update_mapping [node_stats] [2015-11-09 11:31:36,928][INFO][cluster.metadata] [General Orwell Taylor] [.marvel-es-2015.11.09] update_mapping [cluster_stats] ``` So does GET /\_cat/indices?v: ```auto health status index pri rep docs.count docs.deleted store.size pri.store.size yellow open .marvel-es-2015.11.09 1 1 4 0 20.6kb 20.6kb yellow open .marvel-es-data 1 1 1 0 3.2kb 3.2kb ``` I also tested with Marvel configured to export data using HTTP to another elasticsearch cluster and it also works. Can you please check your logs to see if there is an error that can help us to find the issue? Thanks -- Tanguy ---
**Author:** ![nskalis](https://avatars.discourse-cdn.com/v4/letter/n/ebca7d/32.png) [@nskalis](https://discuss.elastic.co/u/nskalis)\ **Post date:** [November 9, 2015, 11:11am UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959/3 "2015-11-09T11:11:53Z")
Thank you Tanguy for looking into this. Here are the logs when I restart elasticsearch ``` [root@nl-ams02c-ispdat01 ~]# tail -f /var/log/elasticsearch/*.log & [1] 37618 [root@nl-ams02c-ispdat01 ~]# systemctl stop elasticsearch [2015-11-09 12:06:06,174][INFO][node] [nl-ams02c-ispdat01] stopping ... [2015-11-09 12:06:06,315][INFO][node] [nl-ams02c-ispdat01] stopped [2015-11-09 12:06:06,315][INFO][node] [nl-ams02c-ispdat01] closing ... [2015-11-09 12:06:06,330][INFO][node] [nl-ams02c-ispdat01] closed [root@nl-ams02c-ispdat01 ~]# systemctl start elasticsearch [root@nl-ams02c-ispdat01 ~]# [2015-11-09 12:06:22,039][INFO][node] [nl-ams02c-ispdat01] version[2.0.0], pid[37635], build[de54438/2015-10-22T08:09:48Z] [2015-11-09 12:06:22,039][INFO][node] [nl-ams02c-ispdat01] initializing ... [2015-11-09 12:06:22,301][INFO][plugins] [nl-ams02c-ispdat01] loaded [license, marvel], sites [kopf, hq] [2015-11-09 12:06:22,337][INFO][env] [nl-ams02c-ispdat01] using [1] data paths, mounts [[/usr (/dev/mapper/centos-usr)]], net usable_space [1.2tb], net total_space [1.2tb], spins? [possibly], types [xfs] [2015-11-09 12:06:23,460][INFO][marvel.agent.settings] [nl-ams02c-ispdat01] marvel setting [marvel.agent.interval : 1m] updated [2015-11-09 12:06:24,586][INFO][node] [nl-ams02c-ispdat01] initialized [2015-11-09 12:06:24,586][INFO][node] [nl-ams02c-ispdat01] starting ... [2015-11-09 12:06:24,898][INFO][transport] [nl-ams02c-ispdat01] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}, {[::1]:9300} [2015-11-09 12:06:24,908][INFO][discovery] [nl-ams02c-ispdat01] IP-spotlight/EBST-cZUQa-jGhnnBkhd0g [2015-11-09 12:06:27,954][INFO][cluster.service] [nl-ams02c-ispdat01] new_master {nl-ams02c-ispdat01}{EBST-cZUQa-jGhnnBkhd0g}{127.0.0.1}{127.0.0.1:9300}{tag=exabgp, group=IP-spotlight}, reason: zen-disco-join(elected_as_master, [0] joins received) [2015-11-09 12:06:28,026][WARN][common.network] [nl-ams02c-ispdat01] publish address: {0.0.0.0} is a wildcard address, falling back to first non-loopback: {fe80::569f:35ff:fe0a:99c0} [2015-11-09 12:06:28,026][INFO][http] [nl-ams02c-ispdat01] publish_address {[fe80::569f:35ff:fe0a:99c0]:9200}, bound_addresses {[::]:9200} [2015-11-09 12:06:28,028][INFO][node] [nl-ams02c-ispdat01] started [2015-11-09 12:06:28,095][INFO][license.plugin.core] [nl-ams02c-ispdat01] license [b568290d-5957-4ab7-bbd4-58fde74bb508] - valid [2015-11-09 12:06:28,097][ERROR][license.plugin.core] [nl-ams02c-ispdat01] # # License will expire on [Monday, December 07, 2015]. If you have a new license, please update it. # Otherwise, please reach out to your support contact. # # Commercial plugins operate with reduced functionality on license expiration: # - marvel # - The agent will stop collecting cluster and indices metrics [2015-11-09 12:06:28,120][INFO][gateway] [nl-ams02c-ispdat01] recovered [3] indices into cluster_state [root@nl-ams02c-ispdat01 ~]# kill -SIGTERM 37618 [1]+ Terminated tail -f /var/log/elasticsearch/*.log ``` i deleted also the ".marvel-es-data" index as well, and re-install marvel also, but in vain. ``` health status index pri rep docs.count docs.deleted store.size pri.store.size yellow open .kibana 1 1 3 0 24.3kb 24.3kb green open 1 0 819859 0 131.9mb 131.9mb green open 1 0 1921373 0 284.4mb 284.4mb ``` thank you a lot for looking into this. i really appreciate it. i can provide a team viewer session if necessary in order to troubleshoot the issue. Nikos ---
**Author:** ![tanguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tanguy/32/6030_2.png) [@tanguy](https://discuss.elastic.co/u/tanguy)\ **Post date:** [November 9, 2015, 11:33am UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959/4 "2015-11-09T11:33:41Z")
I don't see anything strange in the log. Can you please set Marvel to debug level? To do that, you need to edit the logging.yml file located in the elasticsearch config directory and add the line `marvel: DEBUG` under the `logger` entry: ```auto logger: ... marvel: DEBUG ``` Then you can restart and wait few minutes so that Marvel will collect some data (it collects every minute with your config) Thanks ---
**Author:** ![nskalis](https://avatars.discourse-cdn.com/v4/letter/n/ebca7d/32.png) [@nskalis](https://discuss.elastic.co/u/nskalis)\ **Post date:** [November 9, 2015, 12:37pm UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959/5 "2015-11-09T12:37:40Z")
``` [root@nl-ams02c-ispdat01 elasticsearch]# [2015-11-09 13:31:39,729][INFO][node] [nl-ams02c-ispdat01] version[2.0.0], pid[47891], build[de54438/2015-10-22T08:09:48Z] [2015-11-09 13:31:39,730][INFO][node] [nl-ams02c-ispdat01] initializing ... [2015-11-09 13:31:39,992][INFO][plugins] [nl-ams02c-ispdat01] loaded [license, marvel], sites [kopf, hq] [2015-11-09 13:31:40,026][INFO][env] [nl-ams02c-ispdat01] using [1] data paths, mounts [[/usr (/dev/mapper/centos-usr)]], net usable_space [1.2tb], net total_space [1.2tb], spins? [possibly], types [xfs] [2015-11-09 13:31:41,034][INFO][marvel.agent.settings] [nl-ams02c-ispdat01] marvel setting [marvel.agent.interval : 1m] updated [2015-11-09 13:31:42,181][INFO][node] [nl-ams02c-ispdat01] initialized [2015-11-09 13:31:42,182][INFO][node] [nl-ams02c-ispdat01] starting ... [2015-11-09 13:31:42,182][DEBUG][marvel.agent.collector.cluster] [nl-ams02c-ispdat01] starting collector [cluster-info-collector] [2015-11-09 13:31:42,182][DEBUG][marvel.agent.collector.cluster] [nl-ams02c-ispdat01] starting collector [cluster-state-collector] [2015-11-09 13:31:42,182][DEBUG][marvel.agent.collector.indices] [nl-ams02c-ispdat01] starting collector [indices-stats-collector] [2015-11-09 13:31:42,183][DEBUG][marvel.agent.collector.shards] [nl-ams02c-ispdat01] starting collector [shards-collector] [2015-11-09 13:31:42,183][DEBUG][marvel.agent.collector.indices] [nl-ams02c-ispdat01] starting collector [index-stats-collector] [2015-11-09 13:31:42,183][DEBUG][marvel.agent.collector.cluster] [nl-ams02c-ispdat01] starting collector [cluster-stats-collector] [2015-11-09 13:31:42,183][DEBUG][marvel.agent.collector.indices] [nl-ams02c-ispdat01] starting collector [index-recovery-collector] [2015-11-09 13:31:42,183][DEBUG][marvel.agent.collector.node] [nl-ams02c-ispdat01] starting collector [node-stats-collector] [2015-11-09 13:31:42,478][INFO][transport] [nl-ams02c-ispdat01] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}, {[::1]:9300} [2015-11-09 13:31:42,487][INFO][discovery] [nl-ams02c-ispdat01] IP-spotlight/QYc0hiKnTQ2AbL-0ZtjV3Q [2015-11-09 13:31:45,532][INFO][cluster.service] [nl-ams02c-ispdat01] new_master {nl-ams02c-ispdat01}{QYc0hiKnTQ2AbL-0ZtjV3Q}{127.0.0.1}{127.0.0.1:9300}{tag=exabgp, group=IP-spotlight}, reason: zen-disco-join(elected_as_master, [0] joins received) [2015-11-09 13:31:45,541][DEBUG][marvel.agent.exporter.local] local exporter [default_local] - waiting until gateway has recovered from disk [2015-11-09 13:31:45,612][WARN][common.network] [nl-ams02c-ispdat01] publish address: {0.0.0.0} is a wildcard address, falling back to first non-loopback: {fe80::569f:35ff:fe0a:99c0} [2015-11-09 13:31:45,613][INFO][http] [nl-ams02c-ispdat01] publish_address {[fe80::569f:35ff:fe0a:99c0]:9200}, bound_addresses {[::]:9200} [2015-11-09 13:31:45,614][INFO][node] [nl-ams02c-ispdat01] started [2015-11-09 13:31:45,667][INFO][license.plugin.core] [nl-ams02c-ispdat01] license [SNIP!] - valid [2015-11-09 13:31:45,669][ERROR][license.plugin.core] [nl-ams02c-ispdat01] # # License will expire on [Monday, December 07, 2015]. If you have a new license, please update it. # Otherwise, please reach out to your support contact. # # Commercial plugins operate with reduced functionality on license expiration: # - marvel # - The agent will stop collecting cluster and indices metrics [2015-11-09 13:31:45,672][DEBUG][marvel.agent.exporter.local] local exporter [default_local] - currently installed marvel template version [2.0.0] is up-to-date [2015-11-09 13:31:45,673][DEBUG][marvel.agent.exporter.local] local exporter [default_local] - started! [2015-11-09 13:31:45,690][INFO][gateway] [nl-ams02c-ispdat01] recovered [3] indices into cluster_state [2015-11-09 13:31:46,013][DEBUG][marvel.agent.exporter.local] local exporter [default_local] - currently installed marvel template version [2.0.0] is up-to-date [2015-11-09 13:31:46,116][DEBUG][marvel.agent.exporter.local] local exporter [default_local] - currently installed marvel template version [2.0.0] is up-to-date [2015-11-09 13:31:46,138][DEBUG][marvel.agent.exporter.local] local exporter [default_local] - currently installed marvel template version [2.0.0] is up-to-date [2015-11-09 13:31:46,155][DEBUG][marvel.agent.exporter.local] local exporter [default_local] - currently installed marvel template version [2.0.0] is up-to-date [root@nl-ams02c-ispdat01 elasticsearch]# date Mon Nov 9 13:32:34 CET 2015 [root@nl-ams02c-ispdat01 elasticsearch]# date Mon Nov 9 13:34:02 CET 2015 ``` ---
**Author:** ![nskalis](https://avatars.discourse-cdn.com/v4/letter/n/ebca7d/32.png) [@nskalis](https://discuss.elastic.co/u/nskalis)\ **Post date:** [November 9, 2015, 12:38pm UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959/6 "2015-11-09T12:38:22Z")
hmmm.. it seems that it doesn't even try to gather any data if i am not mistaken. any idea why ? ---
**Author:** ![tanguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tanguy/32/6030_2.png) [@tanguy](https://discuss.elastic.co/u/tanguy)\ **Post date:** [November 9, 2015, 4:08pm UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959/7 "2015-11-09T16:08:22Z")
No... Can you try again but with `TRACE` level instead of `DEBUG`? Hopefully we'll see something... ---
**Author:** ![nskalis](https://avatars.discourse-cdn.com/v4/letter/n/ebca7d/32.png) [@nskalis](https://discuss.elastic.co/u/nskalis)\ **Post date:** [November 9, 2015, 7:23pm UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959/8 "2015-11-09T19:23:10Z")
``` [2015-11-09 20:18:46,451][DEBUG][marvel.agent.exporter.local] local exporter [default_local] - currently installed marvel template version [2.0.0] is up-to-date [2015-11-09 20:18:46,451][DEBUG][marvel.agent.exporter.local] local exporter [default_local] - started! [2015-11-09 20:18:46,470][INFO][gateway] [nl-ams02c-ispdat01] recovered [3] indices into cluster_state [2015-11-09 20:18:46,798][DEBUG][marvel.agent.exporter.local] local exporter [default_local] - currently installed marvel template version [2.0.0] is up-to-date [2015-11-09 20:18:46,907][DEBUG][marvel.agent.exporter.local] local exporter [default_local] - currently installed marvel template version [2.0.0] is up-to-date [2015-11-09 20:18:46,926][DEBUG][marvel.agent.exporter.local] local exporter [default_local] - currently installed marvel template version [2.0.0] is up-to-date [2015-11-09 20:18:46,929][DEBUG][marvel.agent.exporter.local] local exporter [default_local] - currently installed marvel template version [2.0.0] is up-to-date [2015-11-09 20:19:42,961][TRACE][marvel.agent] [nl-ams02c-ispdat01] collecting data - collectors [cluster-info-collector,cluster-state-collector,indices-stats-collector,shards-collector,index-stats-collector,cluster-stats-collector,index-recovery-collector,node-stats-collector] [2015-11-09 20:19:42,962][TRACE][marvel.agent.collector.cluster] [nl-ams02c-ispdat01] collector [cluster-info-collector] - collecting data... [2015-11-09 20:19:43,009][TRACE][marvel.agent] [nl-ams02c-ispdat01] bulk [all] - adding [1] collected docs from [cluster-info-collector] collector [2015-11-09 20:19:43,041][TRACE][marvel.agent.collector.cluster] [nl-ams02c-ispdat01] collector [cluster-state-collector] - collecting data... [2015-11-09 20:19:43,042][TRACE][marvel.agent] [nl-ams02c-ispdat01] bulk [all] - adding [1] collected docs from [cluster-state-collector] collector [2015-11-09 20:19:43,056][TRACE][marvel.agent.collector.indices] [nl-ams02c-ispdat01] collector [indices-stats-collector] - collecting data... [2015-11-09 20:19:43,077][TRACE][marvel.agent] [nl-ams02c-ispdat01] bulk [all] - adding [1] collected docs from [indices-stats-collector] collector [2015-11-09 20:19:43,088][TRACE][marvel.agent.collector.shards] [nl-ams02c-ispdat01] collector [shards-collector] - collecting data... [2015-11-09 20:19:43,089][TRACE][marvel.agent] [nl-ams02c-ispdat01] bulk [all] - adding [4] collected docs from [shards-collector] collector [2015-11-09 20:19:43,091][TRACE][marvel.agent.collector.indices] [nl-ams02c-ispdat01] collector [index-stats-collector] - collecting data... [2015-11-09 20:19:43,095][TRACE][marvel.agent] [nl-ams02c-ispdat01] bulk [all] - adding [3] collected docs from [index-stats-collector] collector [2015-11-09 20:19:43,099][TRACE][marvel.agent.collector.cluster] [nl-ams02c-ispdat01] collector [cluster-stats-collector] - collecting data... [2015-11-09 20:19:43,102][TRACE][marvel.agent] [nl-ams02c-ispdat01] bulk [all] - adding [1] collected docs from [cluster-stats-collector] collector [2015-11-09 20:19:43,103][TRACE][marvel.agent.collector.indices] [nl-ams02c-ispdat01] collector [index-recovery-collector] - collecting data... [2015-11-09 20:19:43,104][TRACE][marvel.agent] [nl-ams02c-ispdat01] bulk [all] - adding [1] collected docs from [index-recovery-collector] collector [2015-11-09 20:19:43,114][TRACE][marvel.agent.collector.node] [nl-ams02c-ispdat01] collector [node-stats-collector] - collecting data... [2015-11-09 20:19:43,131][TRACE][marvel.agent] [nl-ams02c-ispdat01] bulk [all] - adding [1] collected docs from [node-stats-collector] collector [2015-11-09 20:19:43,136][TRACE][marvel.agent.exporter.local] exporter [default_local] - exporting data... ``` ---
**Author:** ![nskalis](https://avatars.discourse-cdn.com/v4/letter/n/ebca7d/32.png) [@nskalis](https://discuss.elastic.co/u/nskalis)\ **Post date:** [November 9, 2015, 7:24pm UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959/9 "2015-11-09T19:24:19Z")
this pattern of log messages keeps repeating every 1min. now i got even more confused. ``` [2015-11-09 20:18:46,451][DEBUG][marvel.agent.exporter.local] local exporter [default_local] - currently installed marvel template version [2.0.0] is up-to-date [2015-11-09 20:18:46,451][DEBUG][marvel.agent.exporter.local] local exporter [default_local] - started! [2015-11-09 20:18:46,470][INFO][gateway] [nl-ams02c-ispdat01] recovered [3] indices into cluster_state [2015-11-09 20:18:46,798][DEBUG][marvel.agent.exporter.local] local exporter [default_local] - currently installed marvel template version [2.0.0] is up-to-date [2015-11-09 20:18:46,907][DEBUG][marvel.agent.exporter.local] local exporter [default_local] - currently installed marvel template version [2.0.0] is up-to-date [2015-11-09 20:18:46,926][DEBUG][marvel.agent.exporter.local] local exporter [default_local] - currently installed marvel template version [2.0.0] is up-to-date [2015-11-09 20:18:46,929][DEBUG][marvel.agent.exporter.local] local exporter [default_local] - currently installed marvel template version [2.0.0] is up-to-date [2015-11-09 20:19:42,961][TRACE][marvel.agent] [nl-ams02c-ispdat01] collecting data - collectors [cluster-info-collector,cluster-state-collector,indices-stats-collector,shards-collector,index-stats-collector,cluster-stats-collector,index-recovery-collector,node-stats-collector] [2015-11-09 20:19:42,962][TRACE][marvel.agent.collector.cluster] [nl-ams02c-ispdat01] collector [cluster-info-collector] - collecting data... [2015-11-09 20:19:43,009][TRACE][marvel.agent] [nl-ams02c-ispdat01] bulk [all] - adding [1] collected docs from [cluster-info-collector] collector [2015-11-09 20:19:43,041][TRACE][marvel.agent.collector.cluster] [nl-ams02c-ispdat01] collector [cluster-state-collector] - collecting data... [2015-11-09 20:19:43,042][TRACE][marvel.agent] [nl-ams02c-ispdat01] bulk [all] - adding [1] collected docs from [cluster-state-collector] collector [2015-11-09 20:19:43,056][TRACE][marvel.agent.collector.indices] [nl-ams02c-ispdat01] collector [indices-stats-collector] - collecting data... [2015-11-09 20:19:43,077][TRACE][marvel.agent] [nl-ams02c-ispdat01] bulk [all] - adding [1] collected docs from [indices-stats-collector] collector [2015-11-09 20:19:43,088][TRACE][marvel.agent.collector.shards] [nl-ams02c-ispdat01] collector [shards-collector] - collecting data... [2015-11-09 20:19:43,089][TRACE][marvel.agent] [nl-ams02c-ispdat01] bulk [all] - adding [4] collected docs from [shards-collector] collector [2015-11-09 20:19:43,091][TRACE][marvel.agent.collector.indices] [nl-ams02c-ispdat01] collector [index-stats-collector] - collecting data... [2015-11-09 20:19:43,095][TRACE][marvel.agent] [nl-ams02c-ispdat01] bulk [all] - adding [3] collected docs from [index-stats-collector] collector [2015-11-09 20:19:43,099][TRACE][marvel.agent.collector.cluster] [nl-ams02c-ispdat01] collector [cluster-stats-collector] - collecting data... [2015-11-09 20:19:43,102][TRACE][marvel.agent] [nl-ams02c-ispdat01] bulk [all] - adding [1] collected docs from [cluster-stats-collector] collector [2015-11-09 20:19:43,103][TRACE][marvel.agent.collector.indices] [nl-ams02c-ispdat01] collector [index-recovery-collector] - collecting data... [2015-11-09 20:19:43,104][TRACE][marvel.agent] [nl-ams02c-ispdat01] bulk [all] - adding [1] collected docs from [index-recovery-collector] collector [2015-11-09 20:19:43,114][TRACE][marvel.agent.collector.node] [nl-ams02c-ispdat01] collector [node-stats-collector] - collecting data... [2015-11-09 20:19:43,131][TRACE][marvel.agent] [nl-ams02c-ispdat01] bulk [all] - adding [1] collected docs from [node-stats-collector] collector [2015-11-09 20:19:43,136][TRACE][marvel.agent.exporter.local] exporter [default_local] - exporting data... [2015-11-09 20:20:43,140][TRACE][marvel.agent] [nl-ams02c-ispdat01] collecting data - collectors [cluster-info-collector,cluster-state-collector,indices-stats-collector,shards-collector,index-stats-collector,cluster-stats-collector,index-recovery-collector,node-stats-collector] ``` ---
**Author:** ![tanguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tanguy/32/6030_2.png) [@tanguy](https://discuss.elastic.co/u/tanguy)\ **Post date:** [November 10, 2015, 8:24pm UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959/10 "2015-11-10T20:24:17Z")
According to the trace, it works correctly. Every minute marvel collects a bunch of metrics and exports them locally ie on node nl-ams02c-ispdat01. A `curl -XGET 'http://:9200/.marvel-es-2015*/_search` should return some documents... ---
**Author:** ![nskalis](https://avatars.discourse-cdn.com/v4/letter/n/ebca7d/32.png) [@nskalis](https://discuss.elastic.co/u/nskalis)\ **Post date:** [November 10, 2015, 11:07pm UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959/11 "2015-11-10T23:07:42Z")
hmm.. no documents returned. can i ask how to completely delete Marvel ? i did `bin/plugin remove` but re-installed it but didn't help.. ---
**Author:** ![tanguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tanguy/32/6030_2.png) [@tanguy](https://discuss.elastic.co/u/tanguy)\ **Post date:** [November 12, 2015, 8:14am UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959/12 "2015-11-12T08:14:37Z")
To uninstall the marvel plugin, run `bin/plugin remove marvel-agent`. You'll need to delete the `.marvel-es-*` indices manually. Then you can try to reinstall it again to see if it works better. Right now I can't say why it does not index data because everything looks good in traces and I did not manage to reproduce the issue. ---
**Author:** ![tanguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tanguy/32/6030_2.png) [@tanguy](https://discuss.elastic.co/u/tanguy)\ **Post date:** [November 18, 2015, 8:10am UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959/13 "2015-11-18T08:10:57Z")
@nskalis Did you manage to uninstall / reinstall marvel? ---
**Author:** ![nskalis](https://avatars.discourse-cdn.com/v4/letter/n/ebca7d/32.png) [@nskalis](https://discuss.elastic.co/u/nskalis)\ **Post date:** [November 18, 2015, 12:27pm UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959/14 "2015-11-18T12:27:25Z")
hi @tanguy . i had done this in the past. but unfortunately it didn't help. i'll do it again, but i have to do it during the weekend this time. thank you for the support. it is much appreciated. i'll let you know. ---
**Author:** ![tanguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tanguy/32/6030_2.png) [@tanguy](https://discuss.elastic.co/u/tanguy)\ **Post date:** [November 18, 2015, 12:40pm UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959/15 "2015-11-18T12:40:11Z")
Sure. I'd like to understand what's going on in your cluster and be sure that there's no bug around. In case you reproduce it and if there's not much data in your cluster, do you think you could send me your config/data so that I can debug it locally? Thanks ---
**Author:** ![nskalis](https://avatars.discourse-cdn.com/v4/letter/n/ebca7d/32.png) [@nskalis](https://discuss.elastic.co/u/nskalis)\ **Post date:** [November 18, 2015, 1:02pm UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959/16 "2015-11-18T13:02:05Z")
data would be a bit difficult to share. am gathering IP prefixes from 13 route reflectors (and countries). and that is a bit of sensitive data from technical perspective. but we can have a team viewer session, i'll pass over control to you, and like that you 'll have the setup and the data. and we can look into that together, if that is OK with you. ---
**Author:** ![tanguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tanguy/32/6030_2.png) [@tanguy](https://discuss.elastic.co/u/tanguy)\ **Post date:** [November 19, 2015, 7:42pm UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959/17 "2015-11-19T19:42:35Z")
Thanks for the proposal but what is the most helpful for me would be to debug locally and check line by line with source code what's going on. Lastly, can you - still in DEBUG log level - update the sampling interval from 30 sec to 60 sec: ```auto curl -XPUT localhost:9200/_cluster/settings -d '{"transient" : {"marvel.agent.interval" : "30s" }}' ``` Updating this setting may trigger index re creation. ---
**Author:** ![nskalis](https://avatars.discourse-cdn.com/v4/letter/n/ebca7d/32.png) [@nskalis](https://discuss.elastic.co/u/nskalis)\ **Post date:** [November 22, 2015, 8:17am UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959/18 "2015-11-22T08:17:27Z")
hi @tanguy i removed all plugins i removed the .kibana index i installed the new Kibana 4.2.1, Sense, Marvel but i do not see the .marvel\* indices created i also applied the transient settings suggested but no luck. ``` Waiting for Marvel Data It appears that we have not received any data for this cluster. Marvel data is sent every 10 seconds by the agent plugin; once the data appears this page will automatically redirect to the overview page. ``` and enabled logging in DEBUG mode, the output is : ---
**Author:** ![nskalis](https://avatars.discourse-cdn.com/v4/letter/n/ebca7d/32.png) [@nskalis](https://discuss.elastic.co/u/nskalis)\ **Post date:** [November 22, 2015, 8:19am UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959/19 "2015-11-22T08:19:31Z")
# cat /var/log/elasticsearch/\*.log | grep -i marvel # - marvel [2015-11-22 08:57:53,525][INFO][plugins] [nl-ams02c-ispdat01] loaded [license, marvel], sites [kopf, hq] [2015-11-22 08:57:54,587][INFO][marvel.agent.settings] [nl-ams02c-ispdat01] marvel setting [marvel.agent.interval : 1m] updated # - marvel [2015-11-22 09:00:19,138][INFO][rest.suppressed] /.marvel-es-data/cluster\_info/\_search Params: {index=.marvel-es-data, type=cluster\_info} [.marvel-es-data] IndexNotFoundException[no such index] [2015-11-22 09:00:29,236][INFO][rest.suppressed] /.marvel-es-data/cluster\_info/\_search Params: {index=.marvel-es-data, type=cluster\_info} [.marvel-es-data] IndexNotFoundException[no such index] [2015-11-22 09:00:49,254][INFO][rest.suppressed] /.marvel-es-data/cluster\_info/\_search Params: {index=.marvel-es-data, type=cluster\_info} [.marvel-es-data] IndexNotFoundException[no such index] [2015-11-22 09:01:22,461][INFO][plugins] [nl-ams02c-ispdat01] loaded [license, marvel], sites [kopf, hq] [2015-11-22 09:01:23,506][INFO][marvel.agent.settings] [nl-ams02c-ispdat01] marvel setting [marvel.agent.interval : 1m] updated # - marvel [2015-11-22 09:01:49,570][INFO][rest.suppressed] /.marvel-es-data/cluster\_info/\_search Params: {index=.marvel-es-data, type=cluster\_info} [.marvel-es-data] IndexNotFoundException[no such index] [2015-11-22 09:02:01,173][INFO][rest.suppressed] /.marvel-es-data/cluster\_info/\_search Params: {index=.marvel-es-data, type=cluster\_info} [.marvel-es-data] IndexNotFoundException[no such index] [2015-11-22 09:02:42,197][INFO][rest.suppressed] /.marvel-es-data/cluster\_info/\_search Params: {index=.marvel-es-data, type=cluster\_info} [.marvel-es-data] IndexNotFoundException[no such index] [2015-11-22 09:03:42,356][INFO][rest.suppressed] /.marvel-es-data/cluster\_info/\_search Params: {index=.marvel-es-data, type=cluster\_info} [.marvel-es-data] IndexNotFoundException[no such index] [2015-11-22 09:04:42,527][INFO][rest.suppressed] /.marvel-es-data/cluster\_info/\_search Params: {index=.marvel-es-data, type=cluster\_info} [.marvel-es-data] IndexNotFoundException[no such index] [2015-11-22 09:05:43,575][INFO][rest.suppressed] /.marvel-es-data/cluster\_info/\_search Params: {index=.marvel-es-data, type=cluster\_info} [.marvel-es-data] IndexNotFoundException[no such index] [2015-11-22 09:06:44,878][INFO][rest.suppressed] /.marvel-es-data/cluster\_info/\_search Params: {index=.marvel-es-data, type=cluster\_info} [.marvel-es-data] IndexNotFoundException[no such index] [2015-11-22 09:07:24,157][INFO][plugins] [nl-ams02c-ispdat01] loaded [license, marvel], sites [kopf, hq] [2015-11-22 09:07:25,204][INFO][marvel.agent.settings] [nl-ams02c-ispdat01] marvel setting [marvel.agent.interval : 1m] updated [2015-11-22 09:07:25,730][DEBUG][marvel.agent.collector.cluster] [nl-ams02c-ispdat01] starting collector [cluster-info-collector] [2015-11-22 09:07:25,730][DEBUG][marvel.agent.collector.cluster] [nl-ams02c-ispdat01] starting collector [cluster-state-collector] [2015-11-22 09:07:25,730][DEBUG][marvel.agent.collector.indices] [nl-ams02c-ispdat01] starting collector [indices-stats-collector] [2015-11-22 09:07:25,730][DEBUG][marvel.agent.collector.shards] [nl-ams02c-ispdat01] starting collector [shards-collector] [2015-11-22 09:07:25,730][DEBUG][marvel.agent.collector.indices] [nl-ams02c-ispdat01] starting collector [index-stats-collector] [2015-11-22 09:07:25,730][DEBUG][marvel.agent.collector.cluster] [nl-ams02c-ispdat01] starting collector [cluster-stats-collector] [2015-11-22 09:07:25,730][DEBUG][marvel.agent.collector.indices] [nl-ams02c-ispdat01] starting collector [index-recovery-collector] [2015-11-22 09:07:25,730][DEBUG][marvel.agent.collector.node] [nl-ams02c-ispdat01] starting collector [node-stats-collector] [2015-11-22 09:07:29,099][DEBUG][marvel.agent.exporter.local] local exporter [default\_local] - waiting until gateway has recovered from disk # - marvel ---
**Author:** ![nskalis](https://avatars.discourse-cdn.com/v4/letter/n/ebca7d/32.png) [@nskalis](https://discuss.elastic.co/u/nskalis)\ **Post date:** [November 22, 2015, 8:20am UTC](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959/20 "2015-11-22T08:20:27Z")
[2015-11-22 09:07:29,209][DEBUG][marvel.agent.exporter.local] local exporter [default\_local] - currently installed marvel template version [2.0.0] is up-to-date [2015-11-22 09:07:29,209][DEBUG][marvel.agent.exporter.local] local exporter [default\_local] - started! [2015-11-22 09:07:29,454][DEBUG][marvel.agent.exporter.local] local exporter [default\_local] - currently installed marvel template version [2.0.0] is up-to-date [2015-11-22 09:07:29,574][DEBUG][marvel.agent.exporter.local] local exporter [default\_local] - currently installed marvel template version [2.0.0] is up-to-date [2015-11-22 09:07:52,969][INFO][rest.suppressed] /.marvel-es-data/cluster\_info/\_search Params: {index=.marvel-es-data, type=cluster\_info} [.marvel-es-data] IndexNotFoundException[no such index] [2015-11-22 09:09:12,558][INFO][marvel.agent.settings] [nl-ams02c-ispdat01] marvel setting [marvel.agent.interval : 30s] updated [2015-11-22 09:09:12,559][INFO][marvel.agent] [nl-ams02c-ispdat01] sampling interval updated to [30s] [2015-11-22 09:09:12,559][DEBUG][marvel.agent.exporter.local] local exporter [default\_local] - currently installed marvel template version [2.0.0] is up-to-date [2015-11-22 09:09:28,421][INFO][rest.suppressed] /.marvel-es-data/cluster\_info/\_search Params: {index=.marvel-es-data, type=cluster\_info} [.marvel-es-data] IndexNotFoundException[no such index] [2015-11-22 09:09:31,721][INFO][rest.suppressed] /.marvel-es-data/cluster\_info/\_search Params: {index=.marvel-es-data, type=cluster\_info} [.marvel-es-data] IndexNotFoundException[no such index] [2015-11-22 09:09:48,552][INFO][rest.suppressed] /.marvel-es-data/cluster\_info/\_search Params: {index=.marvel-es-data, type=cluster\_info} [.marvel-es-data] IndexNotFoundException[no such index] [2015-11-22 09:09:50,873][INFO][rest.suppressed] /.marvel-es-data/cluster\_info/\_search Params: {index=.marvel-es-data, type=cluster\_info} [.marvel-es-data] IndexNotFoundException[no such index] [2015-11-22 09:09:52,069][INFO][rest.suppressed] /.marvel-es-data/cluster\_info/\_search Params: {index=.marvel-es-data, type=cluster\_info} [.marvel-es-data] IndexNotFoundException[no such index] [2015-11-22 09:09:53,069][INFO][rest.suppressed] /.marvel-es-data/cluster\_info/\_search Params: {index=.marvel-es-data, type=cluster\_info} [.marvel-es-data] IndexNotFoundException[no such index] [2015-11-22 09:10:15,640][INFO][rest.suppressed] /.marvel-es-data/cluster\_info/\_search Params: {index=.marvel-es-data, type=cluster\_info} [.marvel-es-data] IndexNotFoundException[no such index] [2015-11-22 09:10:19,649][DEBUG][marvel.agent] [nl-ams02c-ispdat01] worker shutdown [2015-11-22 09:10:19,649][DEBUG][marvel.agent.collector.cluster] [nl-ams02c-ispdat01] stopping collector [cluster-info-collector] [2015-11-22 09:10:19,650][DEBUG][marvel.agent.collector.cluster] [nl-ams02c-ispdat01] stopping collector [cluster-state-collector] [2015-11-22 09:10:19,650][DEBUG][marvel.agent.collector.indices] [nl-ams02c-ispdat01] stopping collector [indices-stats-collector] [2015-11-22 09:10:19,651][DEBUG][marvel.agent.collector.shards] [nl-ams02c-ispdat01] stopping collector [shards-collector] [2015-11-22 09:10:19,651][DEBUG][marvel.agent.collector.indices] [nl-ams02c-ispdat01] stopping collector [index-stats-collector] [2015-11-22 09:10:19,651][DEBUG][marvel.agent.collector.cluster] [nl-ams02c-ispdat01] stopping collector [cluster-stats-collector] [2015-11-22 09:10:19,651][DEBUG][marvel.agent.collector.indices] [nl-ams02c-ispdat01] stopping collector [index-recovery-collector] [2015-11-22 09:10:19,651][DEBUG][marvel.agent.collector.node] [nl-ams02c-ispdat01] stopping collector [node-stats-collector] [2015-11-22 09:10:29,226][INFO][plugins] [nl-ams02c-ispdat01] loaded [license, marvel], sites [kopf, hq] [2015-11-22 09:10:30,282][INFO][marvel.agent.settings] [nl-ams02c-ispdat01] marvel setting [marvel.agent.interval : 1m] updated [2015-11-22 09:10:30,849][DEBUG][marvel.agent.collector.cluster] [nl-ams02c-ispdat01] starting collector [cluster-info-collector] [2015-11-22 09:10:30,849][DEBUG][marvel.agent.collector.cluster] [nl-ams02c-ispdat01] starting collector [cluster-state-collector] [2015-11-22 09:10:30,849][DEBUG][marvel.agent.collector.indices] [nl-ams02c-ispdat01] starting collector [indices-stats-collector] [2015-11-22 09:10:30,849][DEBUG][marvel.agent.collector.shards] [nl-ams02c-ispdat01] starting collector [shards-collector] [2015-11-22 09:10:30,849][DEBUG][marvel.agent.collector.indices] [nl-ams02c-ispdat01] starting collector [index-stats-collector] [2015-11-22 09:10:30,849][DEBUG][marvel.agent.collector.cluster] [nl-ams02c-ispdat01] starting collector [cluster-stats-collector] [Next page](https://discuss.elastic.co/t/marvel-es-yyyy-mm-dd-index-deleted-how-to-create-it-again/33959.md?page=2)