# Elastic rejected execution of coordinating operation **URL:** \ **Category:** Elasticsearch\ **Created:** [August 6, 2021, 2:39pm UTC](https://discuss.elastic.co/t/elastic-rejected-execution-of-coordinating-operation/280656 "2021-08-06T14:39:29Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![mug3n451](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mug3n451/32/92640_2.png) [@mug3n451](https://discuss.elastic.co/u/mug3n451)\ **Post date:** [August 6, 2021, 2:39pm UTC](https://discuss.elastic.co/t/elastic-rejected-execution-of-coordinating-operation/280656/1 "2021-08-06T14:39:29Z")
Hi, i have a problem with elasticsearch on kubernetes. here the problem: ```auto "current_state" : "unassigned", "unassigned_info" : { "reason" : "ALLOCATION_FAILED", "at" : "2021-08-06T14:28:31.995Z", "failed_allocation_attempts" : 3, "details" : """failed shard on node [wsTDaACvQTOQdBheyzOGdg]: failed recovery, failure org.elasticsearch.indices.recovery.RecoveryFailedException: [inetsi-2021.08.04][2]: Recovery failed from {elasticsearch-master-1}{LY-jylbCRGyw7p73VAu5Xg}{zmy_D534S6i-GxawJsuTPw}{10.42.9.38}{10.42.9.38:9300}{dim}{xpack.installed=true} into {elasticsearch-master-2}{wsTDaACvQTOQdBheyzOGdg}{IpvOjjpYSx-YocAsjKi1vA}{10.42.7.60}{10.42.7.60:9300}{dim}{xpack.installed=true} at org.elasticsearch.indices.recovery.PeerRecoveryTargetService$RecoveryResponseHandler.handleException(PeerRecoveryTargetService.java:632) at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1174) at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1174) at org.elasticsearch.transport.InboundHandler.lambda$handleException$3(InboundHandler.java:313) at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:678) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.lang.Thread.run(Thread.java:831) Caused by: org.elasticsearch.transport.RemoteTransportException: [elasticsearch-master-1][10.42.9.38:9300][internal:index/shard/recovery/start_recovery] Caused by: org.elasticsearch.transport.RemoteTransportException: [elasticsearch-master-1][10.42.9.38:9300][indices:admin/seq_no/retention_lease_sync[p]] Caused by: org.elasticsearch.common.util.concurrent.EsRejectedExecutionException: rejected execution of primary operation [coordinating_and_primary_bytes=2206343514, replica_bytes=1370085522, all_bytes=3576429036, primary_operation_bytes=0, max_coordinating_and_primary_bytes=3221225472] at org.elasticsearch.index.IndexingPressure.markPrimaryOperationStarted(IndexingPressure.java:120) at org.elasticsearch.action.support.replication.TransportWriteAction.checkPrimaryLimits(TransportWriteAction.java:111) at org.elasticsearch.action.support.replication.TransportWriteAction.checkPrimaryLimits(TransportWriteAction.java:50) at org.elasticsearch.action.support.replication.TransportReplicationAction.handlePrimaryRequest(TransportReplicationAction.java:288) at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$1.doRun(SecurityServerTransportInterceptor.java:230) at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:301) at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:61) at org.elasticsearch.transport.TransportService.sendLocalRequest(TransportService.java:792) at org.elasticsearch.transport.TransportService$3.sendRequest(TransportService.java:114) at org.elasticsearch.transport.TransportService.sendRequestInternal(TransportService.java:736) at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor.sendWithUser(SecurityServerTransportInterceptor.java:147) at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$1.lambda$sendRequest$0(SecurityServerTransportInterceptor.java:99) at org.elasticsearch.xpack.core.security.SecurityContext.executeAsUser(SecurityContext.java:139) at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$1.sendRequest(SecurityServerTransportInterceptor.java:99) at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:650) at org.elasticsearch.transport.TransportService.sendChildRequest(TransportService.java:701) at org.elasticsearch.transport.TransportService.sendChildRequest(TransportService.java:687) at org.elasticsearch.index.seqno.RetentionLeaseSyncAction.sync(RetentionLeaseSyncAction.java:106) at org.elasticsearch.index.seqno.RetentionLeaseSyncer.sync(RetentionLeaseSyncer.java:38) at org.elasticsearch.index.shard.IndexShard.lambda$new$0(IndexShard.java:361) at org.elasticsearch.index.seqno.ReplicationTracker.cloneRetentionLease(ReplicationTracker.java:341) at org.elasticsearch.index.seqno.ReplicationTracker.cloneLocalPeerRecoveryRetentionLease(ReplicationTracker.java:506) at org.elasticsearch.index.shard.IndexShard.cloneLocalPeerRecoveryRetentionLease(IndexShard.java:2797) at org.elasticsearch.indices.recovery.RecoverySourceHandler.lambda$createRetentionLease$29(RecoverySourceHandler.java:575) at org.elasticsearch.indices.recovery.RecoverySourceHandler.lambda$runUnderPrimaryPermit$19(RecoverySourceHandler.java:367) at org.elasticsearch.common.util.CancellableThreads.execute(CancellableThreads.java:82) at org.elasticsearch.indices.recovery.RecoverySourceHandler.runUnderPrimaryPermit(RecoverySourceHandler.java:345) at org.elasticsearch.indices.recovery.RecoverySourceHandler.createRetentionLease(RecoverySourceHandler.java:563) at org.elasticsearch.indices.recovery.RecoverySourceHandler.lambda$phase1$23(RecoverySourceHandler.java:520) at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:134) at org.elasticsearch.common.util.concurrent.ListenableFuture.notifyListenerDirectly(ListenableFuture.java:113) at org.elasticsearch.common.util.concurrent.ListenableFuture.done(ListenableFuture.java:100) at org.elasticsearch.common.util.concurrent.BaseFuture.set(BaseFuture.java:133) at org.elasticsearch.common.util.concurrent.ListenableFuture.onResponse(ListenableFuture.java:139) at org.elasticsearch.action.StepListener.innerOnResponse(StepListener.java:52) at org.elasticsearch.action.NotifyOnceListener.onResponse(NotifyOnceListener.java:29) at org.elasticsearch.indices.recovery.MultiChunkTransfer.onCompleted(MultiChunkTransfer.java:142) at org.elasticsearch.indices.recovery.MultiChunkTransfer.handleItems(MultiChunkTransfer.java:107) at org.elasticsearch.indices.recovery.MultiChunkTransfer$1.write(MultiChunkTransfer.java:67) at org.elasticsearch.common.util.concurrent.AsyncIOProcessor.processList(AsyncIOProcessor.java:97) at org.elasticsearch.common.util.concurrent.AsyncIOProcessor.drainAndProcessAndRelease(AsyncIOProcessor.java:85) at org.elasticsearch.common.util.concurrent.AsyncIOProcessor.put(AsyncIOProcessor.java:73) at org.elasticsearch.indices.recovery.MultiChunkTransfer.addItem(MultiChunkTransfer.java:78) at org.elasticsearch.indices.recovery.MultiChunkTransfer.lambda$handleItems$3(MultiChunkTransfer.java:113) at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:134) at org.elasticsearch.action.ActionListener$RunBeforeActionListener.onResponse(ActionListener.java:387) at org.elasticsearch.action.ActionListener$MappedActionListener.onResponse(ActionListener.java:101) at org.elasticsearch.action.ActionListener$RunBeforeActionListener.onResponse(ActionListener.java:387) at org.elasticsearch.action.ActionListener$RunBeforeActionListener.onResponse(ActionListener.java:387) at org.elasticsearch.action.support.RetryableAction$RetryingListener.onResponse(RetryableAction.java:133) at org.elasticsearch.action.ActionListener$RunBeforeActionListener.onResponse(ActionListener.java:387) at org.elasticsearch.action.ActionListenerResponseHandler.handleResponse(ActionListenerResponseHandler.java:43) at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleResponse(TransportService.java:1164) at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleResponse(TransportService.java:1164) at org.elasticsearch.transport.InboundHandler.doHandleResponse(InboundHandler.java:287) at org.elasticsearch.transport.InboundHandler.lambda$handleResponse$1(InboundHandler.java:275) at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:678) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.lang.Thread.run(Thread.java:831) ``` elasticsearch is configured in this way: esJavaOpts: "-Xmx30g -Xms30g" 3 nodes with master-data roles 3 shards for index with 1 replica ```auto shards disk.indices disk.used disk.avail disk.total disk.percent host ip node 35 2.8gb 6.5gb 91.3gb 97.9gb 6 10.42.7.60 10.42.7.60 elasticsearch-master-2 33 2.6gb 7.5gb 90.4gb 97.9gb 7 10.42.9.38 10.42.9.38 elasticsearch-master-1 35 2.5gb 6.3gb 91.5gb 97.9gb 6 10.42.10.139 10.42.10.139 elasticsearch-master-0 3 UNASSIGNED ``` I don't understand why it gives me this error. the server is not under pressure in RAM or CPU. The indexs is not so big: ```auto inetsi-2021.08.05 2 p STARTED 7000501 451.3mb 10.42.7.60 elasticsearch-master-2 inetsi-2021.08.05 2 r INITIALIZING 10.42.9.38 elasticsearch-master-1 inetsi-2021.08.05 1 r STARTED 4994013 340.8mb 10.42.10.139 elasticsearch-master-0 inetsi-2021.08.05 1 p STARTED 5476452 351.5mb 10.42.9.38 elasticsearch-master-1 inetsi-2021.08.05 0 p STARTED 5670786 363.2mb 10.42.10.139 elasticsearch-master-0 inetsi-2021.08.05 0 r STARTED 5272284 357mb 10.42.9.38 elasticsearch-master-1 ``` What do i have to do, to fix this problem? thanks Cristian ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 3, 2021, 2:39pm UTC](https://discuss.elastic.co/t/elastic-rejected-execution-of-coordinating-operation/280656/2 "2021-09-03T14:39:48Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.