# Getting ReceiveTimeoutTransportException at Index creation, Indexing and Repo Creation

**URL:** <https://discuss.elastic.co/t/getting-receivetimeouttransportexception-at-index-creation-indexing-and-repo-creation/359072>\
**Category:** Elasticsearch\
**Created:** [May 8, 2024, 2:00pm UTC](https://discuss.elastic.co/t/getting-receivetimeouttransportexception-at-index-creation-indexing-and-repo-creation/359072 "2024-05-08T14:00:05Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![dharunkumar](https://avatars.discourse-cdn.com/v4/letter/d/ee59a6/32.png) [@dharunkumar](https://discuss.elastic.co/u/dharunkumar)\
**Post date:** [May 8, 2024, 2:00pm UTC](https://discuss.elastic.co/t/getting-receivetimeouttransportexception-at-index-creation-indexing-and-repo-creation/359072/1 "2024-05-08T14:00:05Z")

</div>

Hi Team,

while creating new Index shard acknowledgement fails, due to this mapping is not updated to the Index and status is yellow. Indexing fails due to ReceiveTimeoutTransportException. Also, Repo creation fails due to the same Exception.

Cluster Details:  
Data size - 60 GB  
Node count - 1  
Indices count - 94  
Heap committed - 5 GB

Node Hot Threads:

```auto
0.0% (0s out of 500ms) cpu usage by thread 'Attach Listener'
  unique snapshot
  unique snapshot
  unique snapshot
  unique snapshot
  unique snapshot
  unique snapshot
  unique snapshot
  unique snapshot
  unique snapshot
  unique snapshot

 0.0% (0s out of 500ms) cpu usage by thread 'Wrapper-Control-Event-Monitor'
  10/10 snapshots sharing following 2 elements
    java.lang.Thread.sleep(Native Method)
    org.tanukisoftware.wrapper.WrapperManager$3.run(WrapperManager.java:1074)

 0.0% (0s out of 500ms) cpu usage by thread 'Wrapper-Connection'
  10/10 snapshots sharing following 9 elements
    java.net.SocketInputStream.socketRead0(Native Method)
    java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
    java.net.SocketInputStream.read(SocketInputStream.java:171)
    java.net.SocketInputStream.read(SocketInputStream.java:141)
    java.net.SocketInputStream.read(SocketInputStream.java:224)
    java.io.DataInputStream.readByte(DataInputStream.java:265)
    org.tanukisoftware.wrapper.WrapperManager.handleBackend(WrapperManager.java:5891)
    org.tanukisoftware.wrapper.WrapperManager.run(WrapperManager.java:6332)
    java.lang.Thread.run(Thread.java:748)

 0.0% (0s out of 500ms) cpu usage by thread 'DestroyJavaVM'
  unique snapshot
  unique snapshot
  unique snapshot
  unique snapshot
  unique snapshot
  unique snapshot
  unique snapshot
  unique snapshot
  unique snapshot
  unique snapshot

 0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[keepAlive/5.6.4]'
  10/10 snapshots sharing following 8 elements
    sun.misc.Unsafe.park(Native Method)
    java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:837)
    java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:999)
    java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1308)
    java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
    org.elasticsearch.bootstrap.Bootstrap$1.run(Bootstrap.java:84)
    java.lang.Thread.run(Thread.java:748)

 0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[int_desktop-7jo7479][[timer]]'
  10/10 snapshots sharing following 2 elements
    java.lang.Thread.sleep(Native Method)
    org.elasticsearch.threadpool.ThreadPool$CachedTimeThread.run(ThreadPool.java:536)

 0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[int_desktop-7jo7479][clusterService#updateTask][T#1]'
  10/10 snapshots sharing following 16 elements
    org.elasticsearch.index.shard.IndexShard.updateRoutingEntry(IndexShard.java:380)
    org.elasticsearch.indices.cluster.IndicesClusterStateService.updateShard(IndicesClusterStateService.java:557)
    org.elasticsearch.indices.cluster.IndicesClusterStateService.createOrUpdateShards(IndicesClusterStateService.java:521)
    org.elasticsearch.indices.cluster.IndicesClusterStateService.applyClusterState(IndicesClusterStateService.java:204)
    org.elasticsearch.cluster.service.ClusterService.callClusterStateAppliers(ClusterService.java:814)
    org.elasticsearch.cluster.service.ClusterService.publishAndApplyChanges(ClusterService.java:768)
    org.elasticsearch.cluster.service.ClusterService.runTasks(ClusterService.java:587)
    org.elasticsearch.cluster.service.ClusterService$ClusterServiceTaskBatcher.run(ClusterService.java:263)
    org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150)
    org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188)
    org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:569)
    org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:247)
    org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:210)
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    java.lang.Thread.run(Thread.java:748)

 0.0% (0s out of 500ms) cpu usage by thread 'threadDeathWatcher-2-1'
  10/10 snapshots sharing following 4 elements
    java.lang.Thread.sleep(Native Method)
    io.netty.util.ThreadDeathWatcher$Watcher.run(ThreadDeathWatcher.java:152)
    io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
    java.lang.Thread.run(Thread.java:748)

 0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[int_desktop-7jo7479][generic][T#7]'
  10/10 snapshots sharing following 31 elements
    sun.nio.fs.WindowsNativeDispatcher.CreateFile0(Native Method)
    sun.nio.fs.WindowsNativeDispatcher.CreateFile(WindowsNativeDispatcher.java:71)
    sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:302)
    sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFactory.java:162)
    sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:110)
    java.nio.channels.FileChannel.open(FileChannel.java:287)
    java.nio.channels.FileChannel.open(FileChannel.java:335)
    org.elasticsearch.index.translog.Translog$$Lambda$1448/1714002180.open(Unknown Source)
    org.elasticsearch.index.translog.ChannelFactory.open(ChannelFactory.java:33)
    org.elasticsearch.index.translog.TranslogWriter.create(TranslogWriter.java:92)
    org.elasticsearch.index.translog.Translog.createWriter(Translog.java:379)
    org.elasticsearch.index.translog.Translog.<init>(Translog.java:203)
    org.elasticsearch.index.engine.InternalEngine.openTranslog(InternalEngine.java:272)
    org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:160)
    org.elasticsearch.index.engine.InternalEngineFactory.newReadWriteEngine(InternalEngineFactory.java:25)
    org.elasticsearch.index.shard.IndexShard.newEngine(IndexShard.java:1602)
    org.elasticsearch.index.shard.IndexShard.createNewEngine(IndexShard.java:1584)
    org.elasticsearch.index.shard.IndexShard.internalPerformTranslogRecovery(IndexShard.java:1027)
    org.elasticsearch.index.shard.IndexShard.performTranslogRecovery(IndexShard.java:987)
    org.elasticsearch.index.shard.StoreRecovery.internalRecoverFromStore(StoreRecovery.java:360)
    org.elasticsearch.index.shard.StoreRecovery.lambda$recoverFromStore$0(StoreRecovery.java:90)
    org.elasticsearch.index.shard.StoreRecovery$$Lambda$1444/521597373.run(Unknown Source)
    org.elasticsearch.index.shard.StoreRecovery.executeRecovery(StoreRecovery.java:257)
    org.elasticsearch.index.shard.StoreRecovery.recoverFromStore(StoreRecovery.java:88)
    org.elasticsearch.index.shard.IndexShard.recoverFromStore(IndexShard.java:1236)
    org.elasticsearch.index.shard.IndexShard.lambda$startRecovery$1(IndexShard.java:1484)
    org.elasticsearch.index.shard.IndexShard$$Lambda$1443/1274477608.run(Unknown Source)
    org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:569)
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    java.lang.Thread.run(Thread.java:748)

 0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[int_desktop-7jo7479][flush][T#1]'
  10/10 snapshots sharing following 12 elements
    org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3006)
    org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3244)
    org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3207)
    org.elasticsearch.index.engine.InternalEngine.commitIndexWriter(InternalEngine.java:1576)
    org.elasticsearch.index.engine.InternalEngine.flush(InternalEngine.java:1062)
    org.elasticsearch.index.shard.IndexShard.flush(IndexShard.java:777)
    org.elasticsearch.index.shard.IndexShard$2.doRun(IndexShard.java:1748)
    org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638)
    org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    java.lang.Thread.run(Thread.java:748)

 0.0% (0s out of 500ms) cpu usage by thread 'elasticsearch[int_desktop-7jo7479][refresh][T#2]'
  10/10 snapshots sharing following 41 elements
    sun.nio.fs.WindowsNativeDispatcher.CreateFile0(Native Method)
    sun.nio.fs.WindowsNativeDispatcher.CreateFile(WindowsNativeDispatcher.java:71)
    sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:302)
    sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFactory.java:162)
    sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:225)
    java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
    java.nio.file.Files.newOutputStream(Files.java:216)
    org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:413)
    org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:409)
    org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
    org.apache.lucene.store.RateLimitedFSDirectory.createOutput(RateLimitedFSDirectory.java:40)
    org.apache.lucene.store.FilterDirectory.createOutput(FilterDirectory.java:73)
    org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
    org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
    org.apache.lucene.codecs.lucene53.Lucene53NormsConsumer.<init>(Lucene53NormsConsumer.java:46)
    org.apache.lucene.codecs.lucene53.Lucene53NormsFormat.normsConsumer(Lucene53NormsFormat.java:77)
    org.apache.lucene.index.DefaultIndexingChain.writeNorms(DefaultIndexingChain.java:299)
    org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:136)
    org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:444)
    org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:539)
    org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:653)
    org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:445)
    org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:291)
    org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:266)
    org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:256)
    org.apache.lucene.index.FilterDirectoryReader.doOpenIfChanged(FilterDirectoryReader.java:104)
    org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140)
    org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:156)
    org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:58)
    org.apache.lucene.search.ReferenceManager.doMaybeRefresh(ReferenceManager.java:176)
    org.apache.lucene.search.ReferenceManager.maybeRefreshBlocking(ReferenceManager.java:253)
    org.elasticsearch.index.engine.InternalEngine.refresh(InternalEngine.java:909)
    org.elasticsearch.index.shard.IndexShard.refresh(IndexShard.java:632)
    org.elasticsearch.index.IndexService.maybeRefreshEngine(IndexService.java:690)
    org.elasticsearch.index.IndexService.access$400(IndexService.java:92)
    org.elasticsearch.index.IndexService$AsyncRefreshTask.runInternal(IndexService.java:832)
    org.elasticsearch.index.IndexService$BaseAsyncTask.run(IndexService.java:743)
    org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:569)
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    java.lang.Thread.run(Thread.java:748)

```

Please help us to identify the possible root cause.

Thanks and Regards  
Dharun Kumar

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [May 8, 2024, 6:21pm UTC](https://discuss.elastic.co/t/getting-receivetimeouttransportexception-at-index-creation-indexing-and-repo-creation/359072/2 "2024-05-08T18:21:22Z")

</div>

> [@dharunkumar](#):
>
> ```auto
> org.tanukisoftware.wrapper.WrapperManager.handleBackend(WrapperManager.java:5891)
> org.tanukisoftware.wrapper.WrapperManager.run(WrapperManager.java:6332)
> 
> ```

You're running Elasticsearch in some strange wrapper thing it seems. This isn't a supported environment. You'll need to run Elasticsearch as described in the manual instead.

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [May 8, 2024, 6:22pm UTC](https://discuss.elastic.co/t/getting-receivetimeouttransportexception-at-index-creation-indexing-and-repo-creation/359072/3 "2024-05-08T18:22:19Z")

</div>

> [@dharunkumar](#):
>
> `keepAlive/5.6.4`

Also you're using a version so old it's almost prehistoric. Upgrade to a supported version (at least 7.17.x) ASAP.

---

<div class="post-metadata">

**Author:** ![dharunkumar](https://avatars.discourse-cdn.com/v4/letter/d/ee59a6/32.png) [@dharunkumar](https://discuss.elastic.co/u/dharunkumar)\
**Post date:** [May 21, 2024, 10:53am UTC](https://discuss.elastic.co/t/getting-receivetimeouttransportexception-at-index-creation-indexing-and-repo-creation/359072/4 "2024-05-21T10:53:07Z")

</div>

Thanks for the analysis,

Actually we have been using tanuki wrapper, it doesn't seem like it is causing the issue.

can you please help me with some other possible root cause ?

Also, we are planning upgrade to latest ES (it is in pipeline).

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [May 21, 2024, 11:08am UTC](https://discuss.elastic.co/t/getting-receivetimeouttransportexception-at-index-creation-indexing-and-repo-creation/359072/5 "2024-05-21T11:08:23Z")

</div>

> [@dharunkumar](#):
>
> Actually we have been using tanuki wrapper, it doesn't seem like it is causing the issue.

Sorry, it is not supported to run Elasticsearch within this wrapper. You'll either need to reproduce the problem without it, or contact the Tanuki developers for further assistance.

I have no other ideas for troubleshooting such an old version. Once you've upgraded, we can likely help further.

---

<div class="post-metadata">

**Author:** ![dharunkumar](https://avatars.discourse-cdn.com/v4/letter/d/ee59a6/32.png) [@dharunkumar](https://discuss.elastic.co/u/dharunkumar)\
**Post date:** [May 22, 2024, 10:34am UTC](https://discuss.elastic.co/t/getting-receivetimeouttransportexception-at-index-creation-indexing-and-repo-creation/359072/6 "2024-05-22T10:34:09Z")

</div>

I understand your concern. since, our data size is huge. due to Indices compatibility, we are currently upgrading to 6.5.4 and so on in future.

we have some finding from analyzing hot Threads, Please help us to find the possible root cause.

```auto
15.6% (78.1ms out of 500ms) cpu usage by thread 'elasticsearch[int_desktop-7jo7479][clusterService#updateTask][T#1]'

```

**ClusterService.executeTask** method is waiting for some time, due to  
**WindowsNativeDispatcher.MoveFileEx0**. Also, we have noticed bulk request threads waiting because of **WindowsNativeDispatcher.CreateFile0**

```auto
18.8% (93.7ms out of 500ms) cpu usage by thread 'elasticsearch[int_desktop-7jo7479][bulk][T#4]'
  10/10 snapshots sharing following 60 elements
    sun.nio.fs.WindowsNativeDispatcher.CreateFile0(Native Method)
    sun.nio.fs.WindowsNativeDispatcher.CreateFile(WindowsNativeDispatcher.java:71)
    sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:302)
    sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFactory.java:162)
    sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:225)
    java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
    java.nio.file.Files.newOutputStream(Files.java:216)
    org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:413)
    org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:409)
    org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
    org.apache.lucene.store.RateLimitedFSDirectory.createOutput(RateLimitedFSDirectory.java:40)
    org.apache.lucene.store.FilterDirectory.createOutput(FilterDirectory.java:73)
    org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
    org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
    org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:117)
    org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
    org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
    org.apache.lucene.index.StoredFieldsConsumer.initStoredFieldsWriter(StoredFieldsConsumer.java:39)
    org.apache.lucene.index.StoredFieldsConsumer.startDocument(StoredFieldsConsumer.java:46)
    org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:364)
    org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:398)
    org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:232)
    org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:478)
    org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1571)
    org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1316)
    org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:662)
    org.elasticsearch.index.engine.InternalEngine.indexIntoLucene(InternalEngine.java:606)
    org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:504)
    org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:557)
    org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:546)
    org.elasticsearch.action.bulk.TransportShardBulkAction.executeIndexRequestOnPrimary(TransportShardBulkAction.java:492)
    org.elasticsearch.action.bulk.TransportShardBulkAction.executeBulkItemRequest(TransportShardBulkAction.java:146)
    org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:115)
    org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:70)
    org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:975)
    org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:944)
    org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:113)
    org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:345)
    org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:270)
    org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:924)
    org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:921)
    org.elasticsearch.index.shard.IndexShardOperationsLock.acquire(IndexShardOperationsLock.java:151)
    org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationLock(IndexShard.java:1659)
    org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryShardReference(TransportReplicationAction.java:933)
    org.elasticsearch.action.support.replication.TransportReplicationAction.access$500(TransportReplicationAction.java:92)
    org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:291)
    org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
    org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:266)
    org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:248)
    com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceivedDecorate(SearchGuardSSLRequestHandler.java:178)
    com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:107)
    com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:92)
    com.floragunn.searchguard.SearchGuardPlugin$4$1.messageReceived(SearchGuardPlugin.java:423)
    org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69)
    org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:654)
    org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638)
    org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    java.lang.Thread.run(Thread.java:748)

15.6% (78.1ms out of 500ms) cpu usage by thread 'elasticsearch[int_desktop-7jo7479][clusterService#updateTask][T#1]'
  10/10 snapshots sharing following 45 elements
    sun.nio.fs.WindowsNativeDispatcher.MoveFileEx0(Native Method)
    sun.nio.fs.WindowsNativeDispatcher.MoveFileEx(WindowsNativeDispatcher.java:327)
    sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:293)
    sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287)
    java.nio.file.Files.move(Files.java:1395)
    org.apache.logging.log4j.core.appender.rolling.action.FileRenameAction.moveFile(FileRenameAction.java:178)
    org.apache.logging.log4j.core.appender.rolling.action.FileRenameAction.execute(FileRenameAction.java:121)
    org.apache.logging.log4j.core.appender.rolling.action.FileRenameAction.execute(FileRenameAction.java:68)
    org.apache.logging.log4j.core.appender.rolling.RollingFileManager.rollover(RollingFileManager.java:509)
    org.apache.logging.log4j.core.appender.rolling.RollingFileManager.rollover(RollingFileManager.java:392)
    org.apache.logging.log4j.core.appender.rolling.RollingFileManager.checkRollover(RollingFileManager.java:308)
    org.apache.logging.log4j.core.appender.RollingFileAppender.append(RollingFileAppender.java:311)
    org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:161)
    org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:134)
    org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:125)
    org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:89)
    org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:542)
    org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:500)
    org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:483)
    org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:388)
    org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63)
    org.apache.logging.log4j.core.Logger.logMessage(Logger.java:153)
    org.apache.logging.log4j.spi.ExtendedLoggerWrapper.logMessage(ExtendedLoggerWrapper.java:222)
    org.elasticsearch.common.logging.PrefixLogger.logMessage(PrefixLogger.java:102)
    org.apache.logging.log4j.spi.AbstractLogger.log(AbstractLogger.java:2117)
    org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2205)
    org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2159)
    org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2142)
    org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2040)
    org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1907)
    org.apache.logging.log4j.spi.AbstractLogger.info(AbstractLogger.java:1449)
    org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.applyRequest(MetaDataMappingService.java:332)
    org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.execute(MetaDataMappingService.java:230)
    org.elasticsearch.cluster.service.ClusterService.executeTasks(ClusterService.java:634)
    org.elasticsearch.cluster.service.ClusterService.calculateTaskOutputs(ClusterService.java:612)
    org.elasticsearch.cluster.service.ClusterService.runTasks(ClusterService.java:571)
    org.elasticsearch.cluster.service.ClusterService$ClusterServiceTaskBatcher.run(ClusterService.java:263)
    org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150)
    org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188)
    org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:569)
    org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:247)
    org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:210)
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    java.lang.Thread.run(Thread.java:748)

 9.4% (46.8ms out of 500ms) cpu usage by thread 'pool-2-thread-1'
  10/10 snapshots sharing following 16 elements
    sun.nio.fs.WindowsNativeDispatcher.CreateFile0(Native Method)
    sun.nio.fs.WindowsNativeDispatcher.CreateFile(WindowsNativeDispatcher.java:71)
    sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:302)
    sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFactory.java:162)
    sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:225)
    java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
    java.nio.file.Files.newOutputStream(Files.java:216)
    com.zoho.za.dae.elastic.ESHealthMonitor.fsHealthCheck(ESHealthMonitor.java:62)
    com.zoho.za.dae.elastic.ESHealthMonitor.run(ESHealthMonitor.java:52)
    java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    java.lang.Thread.run(Thread.java:748)

 0.0% (0s out of 500ms) cpu usage by thread 'Wrapper-Connection'
  10/10 snapshots sharing following 9 elements
    java.net.SocketInputStream.socketRead0(Native Method)
    java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
    java.net.SocketInputStream.read(SocketInputStream.java:171)
    java.net.SocketInputStream.read(SocketInputStream.java:141)
    java.net.SocketInputStream.read(SocketInputStream.java:224)
    java.io.DataInputStream.readByte(DataInputStream.java:265)
    org.tanukisoftware.wrapper.WrapperManager.handleBackend(WrapperManager.java:5891)
    org.tanukisoftware.wrapper.WrapperManager.run(WrapperManager.java:6332)
    java.lang.Thread.run(Thread.java:748)

```

from this it seems, due to disk slowness the threads are waiting that results in cluster event timeout.

Please let us know that the above finding could be the possible reason. Also, kindly share the other possible reasons (if any).

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [May 22, 2024, 5:39pm UTC](https://discuss.elastic.co/t/getting-receivetimeouttransportexception-at-index-creation-indexing-and-repo-creation/359072/7 "2024-05-22T17:39:37Z")

</div>

Seems like a reasonable guess although this code has changed beyond recognition in the 5½ years since 6.5.4 was released and I'm not planning on digging back through history to do a proper analysis.

> [@dharunkumar](#):
>
> ```auto
> com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceivedDecorate(SearchGuardSSLRequestHandler.java:178)
> com.floragunn.searchguard.transport.SearchGuardRequestHandler.messageReceivedDecorate(SearchGuardRequestHandler.java:107)
> com.floragunn.searchguard.ssl.transport.SearchGuardSSLRequestHandler.messageReceived(SearchGuardSSLRequestHandler.java:92)
> com.floragunn.searchguard.SearchGuardPlugin$4$1.messageReceived(SearchGuardPlugin.java:423)
> 
> ```

You're using the third-party SearchGuard plugin too it seems, perhaps this could also be related? This hasn't been necessary for a very long time, security features were added to the core product ages ago.