# Marvel agent and index.mapper.dynamic: false

**URL:** <https://discuss.elastic.co/t/marvel-agent-and-index-mapper-dynamic-false/50969>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-monitoring\
**Created:** [May 25, 2016, 4:49pm UTC](https://discuss.elastic.co/t/marvel-agent-and-index-mapper-dynamic-false/50969 "2016-05-25T16:49:05Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![f0x](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f0x/32/9947_2.png) [@f0x](https://discuss.elastic.co/u/f0x)\
**Post date:** [May 25, 2016, 4:49pm UTC](https://discuss.elastic.co/t/marvel-agent-and-index-mapper-dynamic-false/50969/1 "2016-05-25T16:49:05Z")

</div>

Hello,  
I'm trying to upgrade to Marvel 2.3.3 with Elasticsearch 2.3.3 but marvel-agent seems not able to create its indices.  
These are the errors logs:

```
[2016-05-25 17:16:01,960][ERROR][marvel.agent] [APPS] background thread had an uncaught exception
ElasticsearchException[failed to flush exporter bulks]
        at org.elasticsearch.marvel.agent.exporter.ExportBulk$Compound.flush(ExportBulk.java:104)
        at org.elasticsearch.marvel.agent.exporter.ExportBulk.close(ExportBulk.java:53)
        at org.elasticsearch.marvel.agent.AgentService$ExportingWorker.run(AgentService.java:201)
        at java.lang.Thread.run(Thread.java:745)
        Suppressed: ElasticsearchException[failed to flush [default_local] exporter bulk]; nested: ElasticsearchException[failure in bulk execution:
[0]: index [.marvel-es-1-2016.05.25], type [cluster_state], id [null], message [[.marvel-es-1-2016.05.25] IndexNotFoundException[no such index]]
[1]: index [.marvel-es-1-2016.05.25], type [node], id [null], message [[.marvel-es-1-2016.05.25] IndexNotFoundException[no such index]]
[2]: index [.marvel-es-data-1], type [node], id [NVxAuBdcSIG_UCGVti32Dg], message [[.marvel-es-data-1] IndexNotFoundException[no such index]]
[3]: index [.marvel-es-1-2016.05.25], type [indices_stats], id [null], message [[.marvel-es-1-2016.05.25] IndexNotFoundException[no such index]]

```

I have only 1 server in my cluster...and as you can see, the marvel indices are not created.

I also set a custom template for marvel indices:

```
curl -XGET http://localhost:9200/_template/custom_marvel?pretty
{
  "custom_marvel" : {
    "order" : 1,
    "template" : ".marvel*",
    "settings" : {
      "index" : {
        "number_of_replicas" : "0",
        "mapper" : {
          "dynamic" : "true"
        }
      }
    },
    "mappings" : { },
    "aliases" : { }
  }
}

```

but this doesn't fix the problem.

The only solution seems to comment the index.mapper.dynamic in the main elasticsearch configuration... and, in this situation, the marvel indices are created:

```
green open .marvel-2016.05.25 1 0 13815 0 20.1mb 20.1mb
green open .marvel-es-1-2016.05.25 1 0 29 3 87.8kb 87.8kb

```

...but I need this option.

Any idea how to fix the problem?

This is my elasticsearch.yml:

```
cluster.name: Test
node.name: APPS
path.data: /data/elasticsearch
bootstrap.mlockall: true
action.auto_create_index: .marvel-*, kibana-int
index.mapper.dynamic: false
index.number_of_shards: 1
index.number_of_replicas: 0
marvel.history.duration: 7

discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts:
  - "apps.test.dev"
```

---

<div class="post-metadata">

**Author:** ![bohyun](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bohyun/32/10087_2.png) [@bohyun](https://discuss.elastic.co/u/bohyun)\
**Post date:** [June 2, 2016, 4:33am UTC](https://discuss.elastic.co/t/marvel-agent-and-index-mapper-dynamic-false/50969/2 "2016-06-02T04:33:20Z")

</div>

Hi Francesco -

It looks like you created a custom template for Marvel but in our [Configuring Marvel](https://www.elastic.co/guide/en/marvel/2.3/configuration.html#config-marvel-indices) documentation, we state the following:

**Only set the number\_of\_shards and number\_of\_replicas in the settings section. Overriding other Marvel template settings could cause your Marvel dashboards to stop working correctly.**

With that said, could you share with us how you created the template (i.e. CURL command)?

Also, the following does not look right:

**action.auto\_create\_index: .marvel-\*, kibana-int**

as you need to have '+' before the index names to allow those indices ('-' is to disallow). Please refer to [Automatic Index Creation](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#index-creation) page for more information.

Thanks,  
Bohyun

---

<div class="post-metadata">

**Author:** ![f0x](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f0x/32/9947_2.png) [@f0x](https://discuss.elastic.co/u/f0x)\
**Post date:** [June 2, 2016, 2:45pm UTC](https://discuss.elastic.co/t/marvel-agent-and-index-mapper-dynamic-false/50969/3 "2016-06-02T14:45:29Z")

</div>

Hi Bohyun,  
this configuration was from an elasticsearch 1.6/1.7 installation (where the marvel indices were correctly created) upgraded to 2.3.3 (where I have the problem).

Btw, I started from a clean situation in a vagrant machine with elasticsearch 2.3 and marvel-agent (ubuntu 14.04 and es deb package). With no options set in elasticsearch.yml, the marvel indices are created.  
If I add these options:  
action.auto\_create\_index: +.marvel-\*, +kibana-int  
index.mapper.dynamic: false  
I have the same errors pasted in the first post.

I suppose the problem is the index.mapper.dynamic set to false.

This happens with or without the custom template for marvel indices (where I'm trying to disable the option for marvel indices, hope withour errors) that I can set with:

```
 curl -XPUT http://localhost:9200/_template/custom_marvel -d '{"template":".marvel*","order":1,"settings":{"number_of_replicas": 0,"mapper.dynamic":true}}'

```

Finally, my question 🙂  
can I use the marvel agent with the index.mapper.dynamic set to false in the main elasticsearch configuration (if this is the problem as it seems)?

Thanks,  
Francesco

---

<div class="post-metadata">

**Author:** ![bohyun](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bohyun/32/10087_2.png) [@bohyun](https://discuss.elastic.co/u/bohyun)\
**Post date:** [June 3, 2016, 1:37am UTC](https://discuss.elastic.co/t/marvel-agent-and-index-mapper-dynamic-false/50969/4 "2016-06-03T01:37:53Z")

</div>

Hi Francesco -

The option index.mapper.dynamic is used to disable automatic type creation, and setting the index.mapper.dynamic to false will disable automatic mapping. It looks like to continue in using the template you have, it logically makes sense to set that to false.

I would still like to point out that it is not recommended to override the number\_of\_shards and number\_of\_replicas in Marvel templates.

Thanks,  
Bohyun

---

<div class="post-metadata">

**Author:** ![Damien\_Alexandre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/damien_alexandre/32/21566_2.png) [@Damien\_Alexandre](https://discuss.elastic.co/u/Damien_Alexandre)\
**Post date:** [September 5, 2016, 9:54am UTC](https://discuss.elastic.co/t/marvel-agent-and-index-mapper-dynamic-false/50969/6 "2016-09-05T09:54:16Z")

</div>

I didn't find anything about this on the documentation,  
but on Elasticsearch 2.2, having marvel and `index.mapper.dynamic: false` was working fine.

After upgrading to 2.4, I had to remove this option, because Marvel was going nuts, only throwing IndexNotFoundException.

---

<div class="post-metadata">

**Author:** ![pickypg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pickypg/32/62409_2.png) [@pickypg](https://discuss.elastic.co/u/pickypg)\
**Post date:** [September 5, 2016, 4:55pm UTC](https://discuss.elastic.co/t/marvel-agent-and-index-mapper-dynamic-false/50969/7 "2016-09-05T16:55:39Z")

</div>

Marvel, particularly 2.x, benefits a good amount from dynamic mappings. We explicitly add [most of] what we use to the mappings, but we also send a bunch of data that we don't currently use.

I have not tried it, but you can probably get around this and still disable dynamic mappings globally by tweaking the Marvel templates' `settings` to explicitly include `"index.mapper.dynamic": true`.

It's something that we should probably add too. Thanks for calling it out.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:42pm UTC](https://discuss.elastic.co/t/marvel-agent-and-index-mapper-dynamic-false/50969/8 "2017-07-06T13:42:03Z")

</div>