# Failed to update master on updated mapping for index ( unable to create new native thread )

**URL:** <https://discuss.elastic.co/t/failed-to-update-master-on-updated-mapping-for-index-unable-to-create-new-native-thread/13925>\
**Category:** Elasticsearch\
**Created:** [October 11, 2013, 12:17pm UTC](https://discuss.elastic.co/t/failed-to-update-master-on-updated-mapping-for-index-unable-to-create-new-native-thread/13925 "2013-10-11T12:17:52Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![onthefloorr](https://avatars.discourse-cdn.com/v4/letter/o/ee7513/32.png) [@onthefloorr](https://discuss.elastic.co/u/onthefloorr)\
**Post date:** [October 11, 2013, 12:17pm UTC](https://discuss.elastic.co/t/failed-to-update-master-on-updated-mapping-for-index-unable-to-create-new-native-thread/13925/1 "2013-10-11T12:17:52Z")

</div>

Hi there,

We have the next problem (or misconfiguration) with elasticsearch :

We have installed the latest version of elasticsearch ( 0.90.5 ) from deb  
package, and we used it at multiple clients, and this type of error only  
occurs on this one customer's system, and doesn't occur for other customers.  
The only difference beetween the servers of clients is the configuration of  
elasticsearch, so I thought that the configuration file may be wrong.

Here is the working configuration althought sometimes an OutOfMemoryError  
execption is occurred.. but that's an another topic..

* * *

cluster.name: "xxxxxxxx"  
thrift.port: 9500

index.cache.field.type: soft  
index.cache.field.max\_size: 50000

indices.store.throttle.type: merge  
indices.store.throttle.max\_bytes\_per\_sec: 5mb

indices.memory.index\_buffer\_size: 50%  
index.refresh\_interval: 30  
index.translog.flush\_threshold\_ops: 50000  
index.store.compress.stored: true

# Search pool

threadpool.search.type: fixed  
threadpool.search.size: 50  
threadpool.search.queue\_size: 100

# Bulk pool

threadpool.bulk.type: fixed  
threadpool.bulk.size: 50  
threadpool.bulk.queue\_size: 300

# Index pool

## threadpool.index.type: fixed threadpool.index.size: 50 threadpool.index.queue\_size: 100

Here is the other configuration which contains bigger values because the  
memory is much more (120GB) than the others, and we give 30GB to  
elasticsearch.

* * *

cluster.name: xxx  
thrift.port: 9500  
bootstrap.mlockall: true  
index.cache.field.type: soft  
index.cache.field.max\_size: 200000  
index.cache.field.expire: 15m  
indices.store.throttle.type: merge  
indices.store.throttle.max\_bytes\_per\_sec: 25mbcustomers  
indices.memory.index\_buffer\_size: 30%  
index.refresh\_interval: 30  
index.translog.flush\_threshold\_ops: 200000  
index.store.compress.stored: true  
index.merge.policy.max\_merged\_segment: 10g  
indices.fielddata.cache.size: 30%  
indices.fielddata.cache.expire: 20m  
index.merge.policy.max\_merge\_size: 5g

#Search pool  
threadpool.search.type: fixed  
threadpool.search.size: 10000  
threadpool.search.queue\_size: 50000

#Bulk pool  
threadpool.bulk.type: fixed  
threadpool.bulk.size: 10000  
threadpool.bulk.queue\_size: 30000

## #Index pool threadpool.index.type: fixed threadpool.index.size: 10000 threadpool.index.queue\_size: 50000

I don't know what is causing the OutOfMemoryError exception.

[2013-10-11 10:58:29,409][WARN][action.bulk] [Rama-Tut]  
failed to update master on updated mapping for index  
[normalized-2013-10-11], type [normalized] and source  
[{"normalized":{"properties":{"\_\_uidentifer":{"type":"long"},"\_day":{"type":"date","format":"dateOptionalTime"},"\_event\_id":{"type":"long"},"\_gparent\_id":{"type":"long"},"\_container\_id":{"type":"long"},"\_subcat\_id":{"type":"long"},"\_time\_is":{"type":"date","format":"dateOptionalTime"},"action":{"type":"string"},"date":{"type":"date","format":"dateOptionalTime"},"thesource":{"type":"string"},"message":{"type":"string","analyzer":"standard"},"service":{"type":"string"},"username":{"type":"string"}}}}]  
java.lang.OutOfMemoryError: unable to create new native thread  
at java.lang.Thread.start0(Native Method)  
at java.lang.Thread.start(Thread.java:693)  
at  
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)  
at  
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1360)  
at  
org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor.execute(PrioritizedEsThreadPoolExecutor.java:95)  
at  
org.elasticsearch.cluster.service.InternalClusterService.submitStateUpdateTask(InternalClusterService.java:237)  
at  
org.elasticsearch.cluster.metadata.MetaDataMappingService.updateMapping(MetaDataMappingService.java:281)  
at  
org.elasticsearch.cluster.action.index.MappingUpdatedAction.masterOperation(MappingUpdatedAction.java:79)  
at  
org.elasticsearch.cluster.action.index.MappingUpdatedAction.masterOperation(MappingUpdatedAction.java:45)  
at  
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$2.run(TransportMasterNodeOperationAction.java:144)  
at  
org.elasticsearch.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:293)  
at  
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction.innerExecute(TransportMasterNodeOperationAction.java:140)  
at  
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction.doExecute(TransportMasterNodeOperationAction.java:94)  
at  
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction.doExecute(TransportMasterNodeOperationAction.java:42)  
at  
org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:61)  
at  
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction.execute(TransportMasterNodeOperationAction.java:89)  
at  
org.elasticsearch.action.bulk.TransportShardBulkAction.updateMappingOnMaster(TransportShardBulkAction.java:612)  
at  
org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:339)  
at  
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:521)  
at  
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:419)  
at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:724)

I would be very happy if someone there is who has had a similar propblem  
with the elasticsearch, and explain it what would I have done with it.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [October 11, 2013, 4:22pm UTC](https://discuss.elastic.co/t/failed-to-update-master-on-updated-mapping-for-index-unable-to-create-new-native-thread/13925/2 "2013-10-11T16:22:15Z")

</div>

You have a very high threadpool size settings for fixed. This will exhaust  
all the threads a JVM can create and it will OOM then.

Jörg  
Am 11.10.2013 14:17 schrieb "onthefloorr" [onthefloorr@gmail.com](mailto:onthefloorr@gmail.com):

> Hi there,
> 
> We have the next problem (or misconfiguration) with elasticsearch :
> 
> We have installed the latest version of elasticsearch ( 0.90.5 ) from deb  
> package, and we used it at multiple clients, and this type of error only  
> occurs on this one customer's system, and doesn't occur for other customers.  
> The only difference beetween the servers of clients is the configuration  
> of elasticsearch, so I thought that the configuration file may be wrong.
> 
> Here is the working configuration althought sometimes an OutOfMemoryError  
> execption is occurred.. but that's an another topic..
> 
> * * *
> 
> cluster.name: "xxxxxxxx"  
> thrift.port: 9500
> 
> index.cache.field.type: soft  
> index.cache.field.max\_size: 50000
> 
> indices.store.throttle.type: merge  
> indices.store.throttle.max\_bytes\_per\_sec: 5mb
> 
> indices.memory.index\_buffer\_size: 50%  
> index.refresh\_interval: 30  
> index.translog.flush\_threshold\_ops: 50000  
> index.store.compress.stored: true
> 
> # Search pool
> 
> threadpool.search.type: fixed  
> threadpool.search.size: 50  
> threadpool.search.queue\_size: 100
> 
> # Bulk pool
> 
> threadpool.bulk.type: fixed  
> threadpool.bulk.size: 50  
> threadpool.bulk.queue\_size: 300
> 
> # Index pool
> 
> ## threadpool.index.type: fixed threadpool.index.size: 50 threadpool.index.queue\_size: 100
> 
> Here is the other configuration which contains bigger values because the  
> memory is much more (120GB) than the others, and we give 30GB to  
> elasticsearch.
> 
> * * *
> 
> cluster.name: xxx  
> thrift.port: 9500  
> bootstrap.mlockall: true  
> index.cache.field.type: soft  
> index.cache.field.max\_size: 200000  
> index.cache.field.expire: 15m  
> indices.store.throttle.type: merge  
> indices.store.throttle.max\_bytes\_per\_sec: 25mbcustomers  
> indices.memory.index\_buffer\_size: 30%  
> index.refresh\_interval: 30  
> index.translog.flush\_threshold\_ops: 200000  
> index.store.compress.stored: true  
> index.merge.policy.max\_merged\_segment: 10g  
> indices.fielddata.cache.size: 30%  
> indices.fielddata.cache.expire: 20m  
> index.merge.policy.max\_merge\_size: 5g
> 
> #search pool  
> threadpool.search.type: fixed  
> threadpool.search.size: 10000  
> threadpool.search.queue\_size: 50000
> 
> #Bulk pool  
> threadpool.bulk.type: fixed  
> threadpool.bulk.size: 10000  
> threadpool.bulk.queue\_size: 30000
> 
> ## #Index pool threadpool.index.type: fixed threadpool.index.size: 10000 threadpool.index.queue\_size: 50000
> 
> I don't know what is causing the OutOfMemoryError exception.
> 
> [2013-10-11 10:58:29,409][WARN][action.bulk] [Rama-Tut]  
> failed to update master on updated mapping for index  
> [normalized-2013-10-11], type [normalized] and source  
> [{"normalized":{"properties":{"\_\_uidentifer":{"type":"long"},"\_day":{"type":"date","format":"dateOptionalTime"},"\_event\_id":{"type":"long"},"\_gparent\_id":{"type":"long"},"\_container\_id":{"type":"long"},"\_subcat\_id":{"type":"long"},"\_time\_is":{"type":"date","format":"dateOptionalTime"},"action":{"type":"string"},"date":{"type":"date","format":"dateOptionalTime"},"thesource":{"type":"string"},"message":{"type":"string","analyzer":"standard"},"service":{"type":"string"},"username":{"type":"string"}}}}]  
> java.lang.OutOfMemoryError: unable to create new native thread  
> at java.lang.Thread.start0(Native Method)  
> at java.lang.Thread.start(Thread.java:693)  
> at  
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)  
> at  
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1360)  
> at  
> org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor.execute(PrioritizedEsThreadPoolExecutor.java:95)  
> at  
> org.elasticsearch.cluster.service.InternalClusterService.submitStateUpdateTask(InternalClusterService.java:237)  
> at  
> org.elasticsearch.cluster.metadata.MetaDataMappingService.updateMapping(MetaDataMappingService.java:281)  
> at  
> org.elasticsearch.cluster.action.index.MappingUpdatedAction.masterOperation(MappingUpdatedAction.java:79)  
> at  
> org.elasticsearch.cluster.action.index.MappingUpdatedAction.masterOperation(MappingUpdatedAction.java:45)  
> at  
> org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$2.run(TransportMasterNodeOperationAction.java:144)  
> at  
> org.elasticsearch.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:293)  
> at  
> org.elasticsearch.action.support.master.TransportMasterNodeOperationAction.innerExecute(TransportMasterNodeOperationAction.java:140)  
> at  
> org.elasticsearch.action.support.master.TransportMasterNodeOperationAction.doExecute(TransportMasterNodeOperationAction.java:94)  
> at  
> org.elasticsearch.action.support.master.TransportMasterNodeOperationAction.doExecute(TransportMasterNodeOperationAction.java:42)  
> at  
> org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:61)  
> at  
> org.elasticsearch.action.support.master.TransportMasterNodeOperationAction.execute(TransportMasterNodeOperationAction.java:89)  
> at  
> org.elasticsearch.action.bulk.TransportShardBulkAction.updateMappingOnMaster(TransportShardBulkAction.java:612)  
> at  
> org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:339)  
> at  
> org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:521)  
> at  
> org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:419)  
> at  
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> at  
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> at java.lang.Thread.run(Thread.java:724)
> 
> I would be very happy if someone there is who has had a similar propblem  
> with the elasticsearch, and explain it what would I have done with it.
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\
**Post date:** [October 11, 2013, 9:21pm UTC](https://discuss.elastic.co/t/failed-to-update-master-on-updated-mapping-for-index-unable-to-create-new-native-thread/13925/3 "2013-10-11T21:21:45Z")

</div>

is this a typo of did you really fix the bulk threadpool to 10000 threads?  
This is waaaay to high, you should maybe just stick to the defaults we set  
here, if you set the size that high you will keep all those threads around  
which can take a lot of resources. I don't think you should go higher than  
the number of cores in the box for bulk as well as indexing?

Can you elaborate a bit on the hardware like how many cores does the box  
have?

simon

On Friday, October 11, 2013 2:17:52 PM UTC+2, onthefloorr wrote:

> Hi there,
> 
> We have the next problem (or misconfiguration) with elasticsearch :
> 
> We have installed the latest version of elasticsearch ( 0.90.5 ) from deb  
> package, and we used it at multiple clients, and this type of error only  
> occurs on this one customer's system, and doesn't occur for other customers.  
> The only difference beetween the servers of clients is the configuration  
> of elasticsearch, so I thought that the configuration file may be wrong.
> 
> Here is the working configuration althought sometimes an OutOfMemoryError  
> execption is occurred.. but that's an another topic..
> 
> * * *
> 
> cluster.name: "xxxxxxxx"  
> thrift.port: 9500
> 
> index.cache.field.type: soft  
> index.cache.field.max\_size: 50000
> 
> indices.store.throttle.type: merge  
> indices.store.throttle.max\_bytes\_per\_sec: 5mb
> 
> indices.memory.index\_buffer\_size: 50%  
> index.refresh\_interval: 30  
> index.translog.flush\_threshold\_ops: 50000  
> index.store.compress.stored: true
> 
> # Search pool
> 
> threadpool.search.type: fixed  
> threadpool.search.size: 50  
> threadpool.search.queue\_size: 100
> 
> # Bulk pool
> 
> threadpool.bulk.type: fixed  
> threadpool.bulk.size: 50  
> threadpool.bulk.queue\_size: 300
> 
> # Index pool
> 
> ## threadpool.index.type: fixed threadpool.index.size: 50 threadpool.index.queue\_size: 100
> 
> Here is the other configuration which contains bigger values because the  
> memory is much more (120GB) than the others, and we give 30GB to  
> elasticsearch.
> 
> * * *
> 
> cluster.name: xxx  
> thrift.port: 9500  
> bootstrap.mlockall: true  
> index.cache.field.type: soft  
> index.cache.field.max\_size: 200000  
> index.cache.field.expire: 15m  
> indices.store.throttle.type: merge  
> indices.store.throttle.max\_bytes\_per\_sec: 25mbcustomers  
> indices.memory.index\_buffer\_size: 30%  
> index.refresh\_interval: 30  
> index.translog.flush\_threshold\_ops: 200000  
> index.store.compress.stored: true  
> index.merge.policy.max\_merged\_segment: 10g  
> indices.fielddata.cache.size: 30%  
> indices.fielddata.cache.expire: 20m  
> index.merge.policy.max\_merge\_size: 5g
> 
> #search pool  
> threadpool.search.type: fixed  
> threadpool.search.size: 10000  
> threadpool.search.queue\_size: 50000
> 
> #Bulk pool  
> threadpool.bulk.type: fixed  
> threadpool.bulk.size: 10000  
> threadpool.bulk.queue\_size: 30000
> 
> ## #Index pool threadpool.index.type: fixed threadpool.index.size: 10000 threadpool.index.queue\_size: 50000
> 
> I don't know what is causing the OutOfMemoryError exception.
> 
> [2013-10-11 10:58:29,409][WARN][action.bulk] [Rama-Tut]  
> failed to update master on updated mapping for index  
> [normalized-2013-10-11], type [normalized] and source  
> [{"normalized":{"properties":{"\_\_uidentifer":{"type":"long"},"\_day":{"type":"date","format":"dateOptionalTime"},"\_event\_id":{"type":"long"},"\_gparent\_id":{"type":"long"},"\_container\_id":{"type":"long"},"\_subcat\_id":{"type":"long"},"\_time\_is":{"type":"date","format":"dateOptionalTime"},"action":{"type":"string"},"date":{"type":"date","format":"dateOptionalTime"},"thesource":{"type":"string"},"message":{"type":"string","analyzer":"standard"},"service":{"type":"string"},"username":{"type":"string"}}}}]  
> java.lang.OutOfMemoryError: unable to create new native thread  
> at java.lang.Thread.start0(Native Method)  
> at java.lang.Thread.start(Thread.java:693)  
> at  
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)  
> at  
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1360)  
> at  
> org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor.execute(PrioritizedEsThreadPoolExecutor.java:95)  
> at  
> org.elasticsearch.cluster.service.InternalClusterService.submitStateUpdateTask(InternalClusterService.java:237)  
> at  
> org.elasticsearch.cluster.metadata.MetaDataMappingService.updateMapping(MetaDataMappingService.java:281)  
> at  
> org.elasticsearch.cluster.action.index.MappingUpdatedAction.masterOperation(MappingUpdatedAction.java:79)  
> at  
> org.elasticsearch.cluster.action.index.MappingUpdatedAction.masterOperation(MappingUpdatedAction.java:45)  
> at  
> org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$2.run(TransportMasterNodeOperationAction.java:144)  
> at  
> org.elasticsearch.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:293)  
> at  
> org.elasticsearch.action.support.master.TransportMasterNodeOperationAction.innerExecute(TransportMasterNodeOperationAction.java:140)  
> at  
> org.elasticsearch.action.support.master.TransportMasterNodeOperationAction.doExecute(TransportMasterNodeOperationAction.java:94)  
> at  
> org.elasticsearch.action.support.master.TransportMasterNodeOperationAction.doExecute(TransportMasterNodeOperationAction.java:42)  
> at  
> org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:61)  
> at  
> org.elasticsearch.action.support.master.TransportMasterNodeOperationAction.execute(TransportMasterNodeOperationAction.java:89)  
> at  
> org.elasticsearch.action.bulk.TransportShardBulkAction.updateMappingOnMaster(TransportShardBulkAction.java:612)  
> at  
> org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:339)  
> at  
> org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:521)  
> at  
> org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:419)  
> at  
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> at  
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> at java.lang.Thread.run(Thread.java:724)
> 
> I would be very happy if someone there is who has had a similar propblem  
> with the elasticsearch, and explain it what would I have done with it.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![onthefloorr](https://avatars.discourse-cdn.com/v4/letter/o/ee7513/32.png) [@onthefloorr](https://discuss.elastic.co/u/onthefloorr)\
**Post date:** [October 12, 2013, 4:15pm UTC](https://discuss.elastic.co/t/failed-to-update-master-on-updated-mapping-for-index-unable-to-create-new-native-thread/13925/4 "2013-10-12T16:15:30Z")

</div>

Thank you for your answers.

I also think that the problem was caused by too high a value of the  
threads, and I haven't read the documentation of elasticsearch completely..  
I thought that the size of the {threadpool.size} is the same as the size of  
the queue (threadpool.queue\_size).

In any case, I will try it with a smaller value next week.

Thanks again for your help and have a great day .

On Friday, October 11, 2013 2:17:52 PM UTC+2, onthefloorr wrote:

> Hi there,
> 
> We have the next problem (or misconfiguration) with elasticsearch :
> 
> We have installed the latest version of elasticsearch ( 0.90.5 ) from deb  
> package, and we used it at multiple clients, and this type of error only  
> occurs on this one customer's system, and doesn't occur for other customers.  
> The only difference beetween the servers of clients is the configuration  
> of elasticsearch, so I thought that the configuration file may be wrong.
> 
> Here is the working configuration althought sometimes an OutOfMemoryError  
> execption is occurred.. but that's an another topic..
> 
> * * *
> 
> cluster.name: "xxxxxxxx"  
> thrift.port: 9500
> 
> index.cache.field.type: soft  
> index.cache.field.max\_size: 50000
> 
> indices.store.throttle.type: merge  
> indices.store.throttle.max\_bytes\_per\_sec: 5mb
> 
> indices.memory.index\_buffer\_size: 50%  
> index.refresh\_interval: 30  
> index.translog.flush\_threshold\_ops: 50000  
> index.store.compress.stored: true
> 
> # Search pool
> 
> threadpool.search.type: fixed  
> threadpool.search.size: 50  
> threadpool.search.queue\_size: 100
> 
> # Bulk pool
> 
> threadpool.bulk.type: fixed  
> threadpool.bulk.size: 50  
> threadpool.bulk.queue\_size: 300
> 
> # Index pool
> 
> ## threadpool.index.type: fixed threadpool.index.size: 50 threadpool.index.queue\_size: 100
> 
> Here is the other configuration which contains bigger values because the  
> memory is much more (120GB) than the others, and we give 30GB to  
> elasticsearch.
> 
> * * *
> 
> cluster.name: xxx  
> thrift.port: 9500  
> bootstrap.mlockall: true  
> index.cache.field.type: soft  
> index.cache.field.max\_size: 200000  
> index.cache.field.expire: 15m  
> indices.store.throttle.type: merge  
> indices.store.throttle.max\_bytes\_per\_sec: 25mbcustomers  
> indices.memory.index\_buffer\_size: 30%  
> index.refresh\_interval: 30  
> index.translog.flush\_threshold\_ops: 200000  
> index.store.compress.stored: true  
> index.merge.policy.max\_merged\_segment: 10g  
> indices.fielddata.cache.size: 30%  
> indices.fielddata.cache.expire: 20m  
> index.merge.policy.max\_merge\_size: 5g
> 
> #search pool  
> threadpool.search.type: fixed  
> threadpool.search.size: 10000  
> threadpool.search.queue\_size: 50000
> 
> #Bulk pool  
> threadpool.bulk.type: fixed  
> threadpool.bulk.size: 10000  
> threadpool.bulk.queue\_size: 30000
> 
> ## #Index pool threadpool.index.type: fixed threadpool.index.size: 10000 threadpool.index.queue\_size: 50000
> 
> I don't know what is causing the OutOfMemoryError exception.
> 
> [2013-10-11 10:58:29,409][WARN][action.bulk] [Rama-Tut]  
> failed to update master on updated mapping for index  
> [normalized-2013-10-11], type [normalized] and source  
> [{"normalized":{"properties":{"\_\_uidentifer":{"type":"long"},"\_day":{"type":"date","format":"dateOptionalTime"},"\_event\_id":{"type":"long"},"\_gparent\_id":{"type":"long"},"\_container\_id":{"type":"long"},"\_subcat\_id":{"type":"long"},"\_time\_is":{"type":"date","format":"dateOptionalTime"},"action":{"type":"string"},"date":{"type":"date","format":"dateOptionalTime"},"thesource":{"type":"string"},"message":{"type":"string","analyzer":"standard"},"service":{"type":"string"},"username":{"type":"string"}}}}]  
> java.lang.OutOfMemoryError: unable to create new native thread  
> at java.lang.Thread.start0(Native Method)  
> at java.lang.Thread.start(Thread.java:693)  
> at  
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)  
> at  
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1360)  
> at  
> org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor.execute(PrioritizedEsThreadPoolExecutor.java:95)  
> at  
> org.elasticsearch.cluster.service.InternalClusterService.submitStateUpdateTask(InternalClusterService.java:237)  
> at  
> org.elasticsearch.cluster.metadata.MetaDataMappingService.updateMapping(MetaDataMappingService.java:281)  
> at  
> org.elasticsearch.cluster.action.index.MappingUpdatedAction.masterOperation(MappingUpdatedAction.java:79)  
> at  
> org.elasticsearch.cluster.action.index.MappingUpdatedAction.masterOperation(MappingUpdatedAction.java:45)  
> at  
> org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$2.run(TransportMasterNodeOperationAction.java:144)  
> at  
> org.elasticsearch.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:293)  
> at  
> org.elasticsearch.action.support.master.TransportMasterNodeOperationAction.innerExecute(TransportMasterNodeOperationAction.java:140)  
> at  
> org.elasticsearch.action.support.master.TransportMasterNodeOperationAction.doExecute(TransportMasterNodeOperationAction.java:94)  
> at  
> org.elasticsearch.action.support.master.TransportMasterNodeOperationAction.doExecute(TransportMasterNodeOperationAction.java:42)  
> at  
> org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:61)  
> at  
> org.elasticsearch.action.support.master.TransportMasterNodeOperationAction.execute(TransportMasterNodeOperationAction.java:89)  
> at  
> org.elasticsearch.action.bulk.TransportShardBulkAction.updateMappingOnMaster(TransportShardBulkAction.java:612)  
> at  
> org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:339)  
> at  
> org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:521)  
> at  
> org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:419)  
> at  
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> at  
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> at java.lang.Thread.run(Thread.java:724)
> 
> I would be very happy if someone there is who has had a similar propblem  
> with the elasticsearch, and explain it what would I have done with it.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:12am UTC](https://discuss.elastic.co/t/failed-to-update-master-on-updated-mapping-for-index-unable-to-create-new-native-thread/13925/5 "2017-07-06T02:12:46Z")

</div>