# Initializing shards everytime!
**URL:** https://discuss.elastic.co/t/initializing-shards-everytime/26388
**Category:** Elasticsearch
**Created:** [July 28, 2015, 6:45am UTC](https://discuss.elastic.co/t/initializing-shards-everytime/26388 "2015-07-28T06:45:57Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@it2](https://discuss.elastic.co/u/it2)
#### Post date: [July 28, 2015, 6:45am UTC](https://discuss.elastic.co/t/initializing-shards-everytime/26388/1 "2015-07-28T06:45:57Z")
Hello everyone!
I have an elasticsearch cluster: 1 master node with logstash on board and 3 data nodes.
And everytime in kopf i see next picture: [http://prntscr.com/7xv67x](http://prntscr.com/7xv67x)
What is the reason of this ?
Please give me some advice. Thanks!
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [July 28, 2015, 6:47am UTC](https://discuss.elastic.co/t/initializing-shards-everytime/26388/2 "2015-07-28T06:47:15Z")
Looks like you may have heap pressure, how much data in the cluster, how much heap?
---
### Author:  [@it2](https://discuss.elastic.co/u/it2)
#### Post date: [July 28, 2015, 6:57am UTC](https://discuss.elastic.co/t/initializing-shards-everytime/26388/3 "2015-07-28T06:57:09Z")
Look here:
> **[Screenshot](https://prnt.sc/7xv9pd)**
>
> Captured with Lightshot
---
### Author:  [@it2](https://discuss.elastic.co/u/it2)
#### Post date: [July 28, 2015, 1:03pm UTC](https://discuss.elastic.co/t/initializing-shards-everytime/26388/4 "2015-07-28T13:03:27Z")
Any advices, guys?
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [July 28, 2015, 9:40pm UTC](https://discuss.elastic.co/t/initializing-shards-everytime/26388/5 "2015-07-28T21:40:01Z")
Are you experiencing high GC?
I'd suggest seeing if you can increase the heap, or reduce the amount of data in the cluster and see if that helps.
---
### Author:  [@it2](https://discuss.elastic.co/u/it2)
#### Post date: [July 29, 2015, 10:29am UTC](https://discuss.elastic.co/t/initializing-shards-everytime/26388/6 "2015-07-29T10:29:01Z")
I will try it,
i also see that my swap is overloaded on the data nodes. Maybe i need to do something with that?
free -h
total used free shared buffers cached
Mem: 15G 15G 584M 0B 38M 4,0G
-/+ buffers/cache: 11G 4,6G
Swap: 1,9G 1,9G 0B
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [July 29, 2015, 10:49am UTC](https://discuss.elastic.co/t/initializing-shards-everytime/26388/7 "2015-07-29T10:49:52Z")
Disable swap! [https://www.elastic.co/guide/en/elasticsearch/guide/current/heap-sizing.html#\_swapping\_is\_the\_death\_of\_performance](https://www.elastic.co/guide/en/elasticsearch/guide/current/heap-sizing.html#_swapping_is_the_death_of_performance)
---
### Author:  [@it2](https://discuss.elastic.co/u/it2)
#### Post date: [July 29, 2015, 12:08pm UTC](https://discuss.elastic.co/t/initializing-shards-everytime/26388/8 "2015-07-29T12:08:30Z")
I disable it!
Now there is no initializing shards, cluster is green, but heap is still big:
> **[Screenshot](https://prnt.sc/7yd813)**
>
> Captured with Lightshot
> **[Screenshot](https://prnt.sc/7yd8kn)**
>
> Captured with Lightshot
Am i still need to increase it ?
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [July 29, 2015, 9:30pm UTC](https://discuss.elastic.co/t/initializing-shards-everytime/26388/9 "2015-07-29T21:30:53Z")
If it's still high, yes.
---
### Author:  [@dmugyr](https://discuss.elastic.co/u/dmugyr)
#### Post date: [November 30, 2015, 6:38pm UTC](https://discuss.elastic.co/t/initializing-shards-everytime/26388/10 "2015-11-30T18:38:54Z")
i have similar problem with initializing. Using elasticsearch 2.0, heap size 10gb
Log:
`[2015-11-30 20:27:31,111][WARN][cluster.action.shard] [orion] [indexer.queue][0] received shard failed for [indexer.queue][0], node[EHA2PbjXTsmipmdGppfHUQ], [P], v[7], s[INITIALIZING], a[id=OwlEK\_iWTnGAAa\_zKMrhkA], unassigned\_info[[reason=ALLOCATION\_FAILED], at[2015-11-30T18:27:26.913Z], details[failed recovery, failure IndexShardRecoveryException[failed to recovery from gateway]; nested: EngineCreationFailureException[failed to recover from translog]; nested: EngineException[failed to recover from translog]; nested: ElasticsearchException[unexpected exception reading from translog snapshot of /usr/local/elasticsearch/data/elasticsearch/nodes/0/indices/indexer.queue/0/translog/translog-666.tlog]; nested: EOFException[read past EOF. pos [217667] length: [4] end: [217667]]; ]], indexUUID [Qn7a3si5TlmWT5jQZRUQPQ], message [failed recovery], failure [IndexShardRecoveryException[failed to recovery from gateway]; nested: EngineCreationFailureException[failed to recover from translog]; nested: EngineException[failed to recover from translog]; nested: ElasticsearchException[unexpected exception reading from translog snapshot of /usr/local/elasticsearch/data/elasticsearch/nodes/0/indices/indexer.queue/0/translog/translog-666.tlog]; nested: EOFException[read past EOF. pos [217667] length: [4] end: [217667]]; ]
[indexer.queue][[indexer.queue][0]] IndexShardRecoveryException[failed to recovery from gateway]; nested: EngineCreationFailureException[failed to recover from translog]; nested: EngineException[failed to recover from translog]; nested: ElasticsearchException[unexpected exception reading from translog snapshot of /usr/local/elasticsearch/data/elasticsearch/nodes/0/indices/indexer.queue/0/translog/translog-666.tlog]; nested: EOFException[read past EOF. pos [217667] length: [4] end: [217667]];
at org.elasticsearch.index.shard.StoreRecoveryService.recoverFromStore(StoreRecoveryService.java:258)
at org.elasticsearch.index.shard.StoreRecoveryService.access$100(StoreRecoveryService.java:60)
at org.elasticsearch.index.shard.StoreRecoveryService$1.run(StoreRecoveryService.java:133)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: [indexer.queue][[indexer.queue][0]] EngineCreationFailureException[failed to recover from translog]; nested: EngineException[failed to recover from translog]; nested: ElasticsearchException[unexpected exception reading from translog snapshot of /usr/local/elasticsearch/data/elasticsearch/nodes/0/indices/indexer.queue/0/translog/translog-666.tlog]; nested: EOFException[read past EOF. pos [217667] length: [4] end: [217667]];
at org.elasticsearch.index.engine.InternalEngine.(InternalEngine.java:157)
at org.elasticsearch.index.engine.InternalEngineFactory.newReadWriteEngine(InternalEngineFactory.java:25)
at org.elasticsearch.index.shard.IndexShard.newEngine(IndexShard.java:1349)
at org.elasticsearch.index.shard.IndexShard.createNewEngine(IndexShard.java:1344)
at org.elasticsearch.index.shard.IndexShard.internalPerformTranslogRecovery(IndexShard.java:889)
at org.elasticsearch.index.shard.IndexShard.performTranslogRecovery(IndexShard.java:866)
at org.elasticsearch.index.shard.StoreRecoveryService.recoverFromStore(StoreRecoveryService.java:249)
... 5 more
Caused by: [indexer.queue][[indexer.queue][0]] EngineException[failed to recover from translog]; nested: ElasticsearchException[unexpected exception reading from translog snapshot of /usr/local/elasticsearch/data/elasticsearch/nodes/0/indices/indexer.queue/0/translog/translog-666.tlog]; nested: EOFException[read past EOF. pos [217667] length: [4] end: [217667]];
at org.elasticsearch.index.engine.InternalEngine.recoverFromTranslog(InternalEngine.java:233)
at org.elasticsearch.index.engine.InternalEngine.(InternalEngine.java:154)
... 11 more
Caused by: ElasticsearchException[unexpected exception reading from translog snapshot of /usr/local/elasticsearch/data/elasticsearch/nodes/0/indices/indexer.queue/0/translog/translog-666.tlog]; nested: EOFException[read past EOF. pos [217667] length: [4] end: [217667]];
at org.elasticsearch.index.translog.TranslogReader.readSize(TranslogReader.java:102)
at org.elasticsearch.index.translog.TranslogReader.access$000(TranslogReader.java:46)
at org.elasticsearch.index.translog.TranslogReader$ReaderSnapshot.readOperation(TranslogReader.java:297)
at org.elasticsearch.index.translog.TranslogReader$ReaderSnapshot.next(TranslogReader.java:290)
at org.elasticsearch.index.translog.MultiSnapshot.next(MultiSnapshot.java:70)
at org.elasticsearch.index.engine.InternalEngine.recoverFromTranslog(InternalEngine.java:219)
... 12 more
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [November 30, 2015, 7:05pm UTC](https://discuss.elastic.co/t/initializing-shards-everytime/26388/11 "2015-11-30T19:05:58Z")
@dmugyr Please create your own thread for this.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 11:34pm UTC](https://discuss.elastic.co/t/initializing-shards-everytime/26388/12 "2017-07-05T23:34:44Z")