# Marvel plugin incompatible with action.auto\_create\_index: false and index.mapper.dynamic: false configuration settings

**URL:** <https://discuss.elastic.co/t/marvel-plugin-incompatible-with-action-auto-create-index-false-and-index-mapper-dynamic-false-configuration-settings/15477>\
**Category:** Elasticsearch\
**Created:** [January 29, 2014, 3:41pm UTC](https://discuss.elastic.co/t/marvel-plugin-incompatible-with-action-auto-create-index-false-and-index-mapper-dynamic-false-configuration-settings/15477 "2014-01-29T15:41:16Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dane\_Bertram](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dane_bertram/32/1840_2.png) [@Dane\_Bertram](https://discuss.elastic.co/u/Dane_Bertram)\
**Post date:** [January 29, 2014, 3:41pm UTC](https://discuss.elastic.co/t/marvel-plugin-incompatible-with-action-auto-create-index-false-and-index-mapper-dynamic-false-configuration-settings/15477/1 "2014-01-29T15:41:16Z")

</div>

Environment: I'm running ES 0.90.10 on a Windows 7 Pro box (my dev machine).

_Steps:_

1. Add the following to your [es directory]/config/elasticsearch.yml  
_before_ installing marvel:

action.auto\_create\_index: false  
index.mapper.dynamic: false

1. Install marvel plugin
2. Restart your ES cluster
3. Navigate to [http://localhost:9200/\_plugin/marvel](http://localhost:9200/_plugin/marvel)

_Expected:_

Either a working marvel dashboard or a very explicit warning/error message  
that marvel failed to create its index.

Bonus: detect the settings above and give specific remedies in the  
notification (remove those settings, use the auto\_create\_index whitelist to  
allow marvel to create it's own indices, etc.).

_Actual:_

1. Empty marvel dashboard (page renders just fine, just no data to  
display), with a short-lived message at the top of the dashboard: "No  
results There were no results because no indices were found that match your  
selected time span"
2. The following error repeating in your ES logs 4 times every second:

[2014-01-29 10:21:11,639][ERROR][marvel.agent.exporter] error sending  
data  
java.io.FileNotFoundException: [http://localhost:9200/\_bulk](http://localhost:9200/_bulk)  
at  
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1624)  
at  
org.elasticsearch.marvel.agent.exporter.ESExporter.sendCloseExportingConnection(ESExporter.java:232)  
at  
org.elasticsearch.marvel.agent.exporter.ESExporter.exportXContent(ESExporter.java:252)  
at  
org.elasticsearch.marvel.agent.exporter.ESExporter.exportClusterStats(ESExporter.java:172)  
at  
org.elasticsearch.marvel.agent.AgentService$ExportingWorker.exportClusterStats(AgentService.java:214)  
at  
org.elasticsearch.marvel.agent.AgentService$ExportingWorker.run(AgentService.java:180)  
at java.lang.Thread.run(Thread.java:744)

This error is a symptom of marvel not being able to create it's own index  
due to action.auto\_create\_index: false in our elasticsearch.yml

My next step was to turn off the action.auto\_create\_index: false setting in  
elasticsearch.yml and re-start the cluster.

This resulted in marvel being able to create it's index, but still failing  
to index any data.

1. Same issue with the marvel dashboard (no data, no warnings/errors)
2. Different error message in the ES logs 4 times/second:

[2014-01-29 10:24:54,793][DEBUG][action.bulk] [Princess  
Python] [.marvel-2014.01.29][0] failed to execute  
bulk item (index) index  
{[.marvel-2014.01.29][indices\_stats][5eVda-3SRBK2TKmjHzFpBA],  
source[{"@timestamp":"2014-01-29T1  
5:24:54.786Z","cluster\_name":"elasticsearch","\_source\_node":{"id":"LBDVUL\_eRt27pnVCno7NBQ","name":"Princess  
Python","tra  
nsport\_address":"inet[[dane.hq.fogcreek.com/10.1.1.167:9300](http://dane.hq.fogcreek.com/10.1.1.167:9300)]","ip":"10.1.1.167","host":"[dane.hq.fogcreek.com](http://dane.hq.fogcreek.com)","ip\_port":"  
10.1.1.167:9300"},"primaries":{"docs":{"count":15249,"deleted":720},"store":{"size\_in\_bytes":5790385,"throttle\_time\_in\_m  
illis":2},"indexing":{"index\_total":37,"index\_time\_in\_millis":430,"index\_current":0,"delete\_total":0,"delete\_time\_in\_mil  
lis":0,"delete\_current":0},"get":{"total":0,"time\_in\_millis":0,"exists\_total":0,"exists\_time\_in\_millis":0,"missing\_total  
":0,"missing\_time\_in\_millis":0,"current":0},"search":{"open\_contexts":0,"query\_total":120,"query\_time\_in\_millis":353,"qu  
ery\_current":0,"fetch\_total":96,"fetch\_time\_in\_millis":20,"fetch\_current":0},"merges":{"current":0,"current\_docs":0,"cur  
rent\_size\_in\_bytes":0,"total":1,"total\_time\_in\_millis":262,"total\_docs":27,"total\_size\_in\_bytes":1007730},"refresh":{"to  
tal":19,"total\_time\_in\_millis":1049},"flush":{"total":0,"total\_time\_in\_millis":0},"warmer":{"current":0,"total":20,"tota  
l\_time\_in\_millis":26},"filter\_cache":{"memory\_size\_in\_bytes":2200,"evictions":0},"id\_cache":{"memory\_size\_in\_bytes":0},"  
fielddata":{"memory\_size\_in\_bytes":174036,"evictions":0},"completion":{"size\_in\_bytes":0},"segments":{"count":38,"memory  
_in\_bytes":0}},"total":{"docs":{"count":15249,"deleted":720},"store":{"size\_in\_bytes":5790385,"throttle\_time\_in\_millis":  
2},"indexing":{"index\_total":37,"index\_time\_in\_millis":430,"index\_current":0,"delete\_total":0,"delete\_time\_in\_millis":0,  
"delete\_current":0},"get":{"total":0,"time\_in\_millis":0,"exists\_total":0,"exists\_time\_in\_millis":0,"missing\_total":0,"mi  
ssing\_time\_in\_millis":0,"current":0},"search":{"open\_contexts":0,"query\_total":120,"query\_time\_in\_millis":353,"query\_cur  
rent":0,"fetch\_total":96,"fetch\_time\_in\_millis":20,"fetch\_current":0},"merges":{"current":0,"current\_docs":0,"current\_si  
ze\_in\_bytes":0,"total":1,"total\_time\_in\_millis":262,"total\_docs":27,"total\_size\_in\_bytes":1007730},"refresh":{"total":19  
,"total\_time\_in\_millis":1049},"flush":{"total":0,"total\_time\_in\_millis":0},"warmer":{"current":0,"total":20,"total\_time_  
in\_millis":26},"filter\_cache":{"memory\_size\_in\_bytes":2200,"evictions":0},"id\_cache":{"memory\_size\_in\_bytes":0},"fieldda  
ta":{"memory\_size\_in\_bytes":174036,"evictions":0},"completion":{"size\_in\_bytes":0},"segments":{"count":38,"memory\_in\_byt  
es":0}}}]}  
org.elasticsearch.indices.TypeMissingException: [.marvel-2014.01.29]  
type[indices\_stats] missing: trying to auto create  
mapping, but dynamic mapping is disabled  
at  
org.elasticsearch.index.mapper.MapperService.documentMapperWithAutoCreate(MapperService.java:343)  
at  
org.elasticsearch.index.shard.service.InternalIndexShard.prepareCreate(InternalIndexShard.java:352)  
at  
org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:402)

```
    at 

```

org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:  
156)  
at  
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationActi  
on.performOnPrimary(TransportShardReplicationOperationAction.java:556)  
at  
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationActi  
on$1.run(TransportShardReplicationOperationAction.java:426)  
at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:744)

Clearly that one's due to dynamic type mappings being disabled.

If you remove both of those configuration options, marvel appears to run  
quite happily. 🙂

_Possible solutions:_

1. Remove the offending configuration options! 😛

This isn't really an option for us, because we rely on these two  
configuration settings in our production environment to help ensure  
spurious indices aren't created accidentally and to ensure our type  
mappings aren't accidentally corrupted due to extra json data being  
submitted for indexing (also to help void incorrect type sniffing).

1. marvel could explicitly create it's indices and update its type mappings  
rather than relying on auto index creation and dynamic type mappings

To me, this feels like the "most correct" way to handle this. Clearly  
marvel knows what index/indices it wants to have. And if it were to  
explicitly check & create them (rather than relying on auto index  
creation), it could also specify its type mapping instead of relying on  
dynamic type mapping. A little tedious to define it explicitly, sure, but  
then marvel would be compatible with a wider set of configurations  
out-of-the-box.

1. marvel could check for these potentially conflicting settings and notify  
you during plugin startup (or the ES the logs or the marvel plugin UI) of  
the problem along with a workaround (use the auto\_create\_index whitelist,  
etc.)?

2. Other solutions I'm missing/not aware of?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f530a591-ec3c-4304-80c4-94960ff698eb%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f530a591-ec3c-4304-80c4-94960ff698eb%40googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Dane\_Bertram](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dane_bertram/32/1840_2.png) [@Dane\_Bertram](https://discuss.elastic.co/u/Dane_Bertram)\
**Post date:** [January 29, 2014, 4:00pm UTC](https://discuss.elastic.co/t/marvel-plugin-incompatible-with-action-auto-create-index-false-and-index-mapper-dynamic-false-configuration-settings/15477/2 "2014-01-29T16:00:27Z")

</div>

Ah, I see the index auto creation setting is mentioned in the  
docs: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/marvel/current/#_other_relevant_elasticsearch_settings)

The dynamic type mapping setting should probably be added there as well,  
but I still feel like the plugin should be more explicit/helpful when it  
runs into this type of problem.

On Wednesday, 29 January 2014 10:41:16 UTC-5, Dane Bertram wrote:

> Environment: I'm running ES 0.90.10 on a Windows 7 Pro box (my dev  
> machine).
> 
> _Steps:_
> 
> 1. Add the following to your [es directory]/config/elasticsearch.yml  
> _before_ installing marvel:
> 
> action.auto\_create\_index: false  
> index.mapper.dynamic: false
> 
> 1. Install marvel plugin
> 2. Restart your ES cluster
> 3. Navigate to [http://localhost:9200/\_plugin/marvel](http://localhost:9200/_plugin/marvel)
> 
> _Expected:_
> 
> Either a working marvel dashboard or a very explicit warning/error message  
> that marvel failed to create its index.
> 
> Bonus: detect the settings above and give specific remedies in the  
> notification (remove those settings, use the auto\_create\_index whitelist to  
> allow marvel to create it's own indices, etc.).
> 
> _Actual:_
> 
> 1. Empty marvel dashboard (page renders just fine, just no data to  
> display), with a short-lived message at the top of the dashboard: "No  
> results There were no results because no indices were found that match your  
> selected time span"
> 2. The following error repeating in your ES logs 4 times every second:
> 
> [2014-01-29 10:21:11,639][ERROR][marvel.agent.exporter] error sending  
> data  
> java.io.FileNotFoundException: [http://localhost:9200/\_bulk](http://localhost:9200/_bulk)  
> at  
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1624)  
> at  
> org.elasticsearch.marvel.agent.exporter.ESExporter.sendCloseExportingConnection(ESExporter.java:232)  
> at  
> org.elasticsearch.marvel.agent.exporter.ESExporter.exportXContent(ESExporter.java:252)  
> at  
> org.elasticsearch.marvel.agent.exporter.ESExporter.exportClusterStats(ESExporter.java:172)  
> at  
> org.elasticsearch.marvel.agent.AgentService$ExportingWorker.exportClusterStats(AgentService.java:214)  
> at  
> org.elasticsearch.marvel.agent.AgentService$ExportingWorker.run(AgentService.java:180)  
> at java.lang.Thread.run(Thread.java:744)
> 
> This error is a symptom of marvel not being able to create it's own index  
> due to action.auto\_create\_index: false in our elasticsearch.yml
> 
> My next step was to turn off the action.auto\_create\_index: false setting  
> in elasticsearch.yml and re-start the cluster.
> 
> This resulted in marvel being able to create it's index, but still failing  
> to index any data.
> 
> 1. Same issue with the marvel dashboard (no data, no warnings/errors)
> 2. Different error message in the ES logs 4 times/second:
> 
> [2014-01-29 10:24:54,793][DEBUG][action.bulk] [Princess  
> Python] [.marvel-2014.01.29][0] failed to execute  
> bulk item (index) index  
> {[.marvel-2014.01.29][indices\_stats][5eVda-3SRBK2TKmjHzFpBA],  
> source[{"@timestamp":"2014-01-29T1  
> 5:24:54.786Z","cluster\_name":"elasticsearch","\_source\_node":{"id":"LBDVUL\_eRt27pnVCno7NBQ","name":"Princess  
> Python","tra  
> nsport\_address":"inet[[dane.hq.fogcreek.com/10.1.1.167:9300](http://dane.hq.fogcreek.com/10.1.1.167:9300)  
> ]","ip":"10.1.1.167","host":"[dane.hq.fogcreek.com](http://dane.hq.fogcreek.com)","ip\_port":"  
> 10.1.1.167:9300  
> "},"primaries":{"docs":{"count":15249,"deleted":720},"store":{"size\_in\_bytes":5790385,"throttle\_time\_in\_m
> 
> illis":2},"indexing":{"index\_total":37,"index\_time\_in\_millis":430,"index\_current":0,"delete\_total":0,"delete\_time\_in\_mil
> 
> lis":0,"delete\_current":0},"get":{"total":0,"time\_in\_millis":0,"exists\_total":0,"exists\_time\_in\_millis":0,"missing\_total
> 
> ":0,"missing\_time\_in\_millis":0,"current":0},"search":{"open\_contexts":0,"query\_total":120,"query\_time\_in\_millis":353,"qu
> 
> ery\_current":0,"fetch\_total":96,"fetch\_time\_in\_millis":20,"fetch\_current":0},"merges":{"current":0,"current\_docs":0,"cur
> 
> rent\_size\_in\_bytes":0,"total":1,"total\_time\_in\_millis":262,"total\_docs":27,"total\_size\_in\_bytes":1007730},"refresh":{"to
> 
> tal":19,"total\_time\_in\_millis":1049},"flush":{"total":0,"total\_time\_in\_millis":0},"warmer":{"current":0,"total":20,"tota
> 
> l\_time\_in\_millis":26},"filter\_cache":{"memory\_size\_in\_bytes":2200,"evictions":0},"id\_cache":{"memory\_size\_in\_bytes":0},"
> 
> fielddata":{"memory\_size\_in\_bytes":174036,"evictions":0},"completion":{"size\_in\_bytes":0},"segments":{"count":38,"memory
> 
> \_in\_bytes":0}},"total":{"docs":{"count":15249,"deleted":720},"store":{"size\_in\_bytes":5790385,"throttle\_time\_in\_millis":
> 
> 2},"indexing":{"index\_total":37,"index\_time\_in\_millis":430,"index\_current":0,"delete\_total":0,"delete\_time\_in\_millis":0,
> 
> "delete\_current":0},"get":{"total":0,"time\_in\_millis":0,"exists\_total":0,"exists\_time\_in\_millis":0,"missing\_total":0,"mi
> 
> ssing\_time\_in\_millis":0,"current":0},"search":{"open\_contexts":0,"query\_total":120,"query\_time\_in\_millis":353,"query\_cur
> 
> rent":0,"fetch\_total":96,"fetch\_time\_in\_millis":20,"fetch\_current":0},"merges":{"current":0,"current\_docs":0,"current\_si
> 
> ze\_in\_bytes":0,"total":1,"total\_time\_in\_millis":262,"total\_docs":27,"total\_size\_in\_bytes":1007730},"refresh":{"total":19
> 
> ,"total\_time\_in\_millis":1049},"flush":{"total":0,"total\_time\_in\_millis":0},"warmer":{"current":0,"total":20,"total\_time\_
> 
> in\_millis":26},"filter\_cache":{"memory\_size\_in\_bytes":2200,"evictions":0},"id\_cache":{"memory\_size\_in\_bytes":0},"fieldda
> 
> ta":{"memory\_size\_in\_bytes":174036,"evictions":0},"completion":{"size\_in\_bytes":0},"segments":{"count":38,"memory\_in\_byt  
> es":0}}}]}  
> org.elasticsearch.indices.TypeMissingException: [.marvel-2014.01.29]  
> type[indices\_stats] missing: trying to auto create  
> mapping, but dynamic mapping is disabled  
> at  
> org.elasticsearch.index.mapper.MapperService.documentMapperWithAutoCreate(MapperService.java:343)  
> at  
> org.elasticsearch.index.shard.service.InternalIndexShard.prepareCreate(InternalIndexShard.java:352)  
> at  
> org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:402)
> 
> ```
> at 
> 
> ```
> 
> org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:  
> 156)  
> at  
> org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationActi  
> on.performOnPrimary(TransportShardReplicationOperationAction.java:556)  
> at  
> org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationActi  
> on$1.run(TransportShardReplicationOperationAction.java:426)  
> at  
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> at  
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> at java.lang.Thread.run(Thread.java:744)
> 
> Clearly that one's due to dynamic type mappings being disabled.
> 
> If you remove both of those configuration options, marvel appears to run  
> quite happily. 🙂
> 
> _Possible solutions:_
> 
> 1. Remove the offending configuration options! 😛
> 
> This isn't really an option for us, because we rely on these two  
> configuration settings in our production environment to help ensure  
> spurious indices aren't created accidentally and to ensure our type  
> mappings aren't accidentally corrupted due to extra json data being  
> submitted for indexing (also to help void incorrect type sniffing).
> 
> 1. marvel could explicitly create it's indices and update its type  
> mappings rather than relying on auto index creation and dynamic type  
> mappings
> 
> To me, this feels like the "most correct" way to handle this. Clearly  
> marvel knows what index/indices it wants to have. And if it were to  
> explicitly check & create them (rather than relying on auto index  
> creation), it could also specify its type mapping instead of relying on  
> dynamic type mapping. A little tedious to define it explicitly, sure, but  
> then marvel would be compatible with a wider set of configurations  
> out-of-the-box.
> 
> 1. marvel could check for these potentially conflicting settings and  
> notify you during plugin startup (or the ES the logs or the marvel plugin  
> UI) of the problem along with a workaround (use the auto\_create\_index  
> whitelist, etc.)?
> 
> 2. Other solutions I'm missing/not aware of?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c520e6e8-1750-4c64-880c-3c330f8f8771%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c520e6e8-1750-4c64-880c-3c330f8f8771%40googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Dane\_Bertram](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dane_bertram/32/1840_2.png) [@Dane\_Bertram](https://discuss.elastic.co/u/Dane_Bertram)\
**Post date:** [January 29, 2014, 4:01pm UTC](https://discuss.elastic.co/t/marvel-plugin-incompatible-with-action-auto-create-index-false-and-index-mapper-dynamic-false-configuration-settings/15477/3 "2014-01-29T16:01:40Z")

</div>

Ah, I set the index auto create setting is mentioned in the docs for  
marvel:  
[http://www.elasticsearch.org/guide/en/marvel/current/#\_other\_relevant\_elasticsearch\_settings](http://www.elasticsearch.org/guide/en/marvel/current/#_other_relevant_elasticsearch_settings)

The dynamic type mapping setting should also probably be mentioned there,  
but I still feel like the plugin could/should be more helpful when it runs  
into this type of problem.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/4e2e2c06-34d0-429a-924a-5a8678021dab%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4e2e2c06-34d0-429a-924a-5a8678021dab%40googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:54am UTC](https://discuss.elastic.co/t/marvel-plugin-incompatible-with-action-auto-create-index-false-and-index-mapper-dynamic-false-configuration-settings/15477/4 "2017-07-06T01:54:09Z")

</div>