# \[logstash-2015.07.21\]\[4\] failed to create engine

**URL:** <https://discuss.elastic.co/t/logstash-2015-07-21-4-failed-to-create-engine/25992>\
**Category:** Elasticsearch\
**Created:** [July 21, 2015, 2:18pm UTC](https://discuss.elastic.co/t/logstash-2015-07-21-4-failed-to-create-engine/25992 "2015-07-21T14:18:58Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![wdijkerman](https://avatars.discourse-cdn.com/v4/letter/w/3ec8ea/32.png) [@wdijkerman](https://discuss.elastic.co/u/wdijkerman)\
**Post date:** [July 21, 2015, 2:18pm UTC](https://discuss.elastic.co/t/logstash-2015-07-21-4-failed-to-create-engine/25992/1 "2015-07-21T14:18:58Z")

</div>

Hi,

We have an data corruption issue in Elasticsearch. We only have 1 node. We use it for our logstash setup.  
The machine crashed and we needed to run an filesystem check (and it repaired a lot of inodes and a lot of other stuff, before we could use it.

When finally started, we see the following error messages:  
[2015-07-21 16:05:30,327][WARN][cluster.action.shard] [loghost] [logstash-2015.07.21][4] received shard failed for [logstash-2015.07.21][4], node[ps5dpZM3SIaXe-kxPG82Kg], [P], s[INITIALIZING], indexUUID [fJVMIfuKSP6VTxsVuLi0Ag], reason [shard failure [failed recovery][IndexShardGatewayRecoveryException[[logstash-2015.07.21][4] failed recovery]; nested: EngineCreationFailureException[[logstash-2015.07.21][4] failed to create engine]; nested: NoSuchFileException[/var/lib/elasticsearch//nodes/0/indices/logstash-2015.07.21/4/index/segments\_1r]; ]]  
[2015-07-21 16:05:40,327][WARN][indices.cluster] [loghost] [[logstash-2015.07.21][4]] marking and sending shard failed due to [failed recovery]  
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: [logstash-2015.07.21][4] failed recovery  
at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:162)  
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:745)  
Caused by: org.elasticsearch.index.engine.EngineCreationFailureException: [logstash-2015.07.21][4] failed to create engine  
at org.elasticsearch.index.engine.InternalEngine.(InternalEngine.java:153)  
at org.elasticsearch.index.engine.InternalEngineFactory.newReadWriteEngine(InternalEngineFactory.java:32)  
at org.elasticsearch.index.shard.IndexShard.newEngine(IndexShard.java:1351)  
at org.elasticsearch.index.shard.IndexShard.createNewEngine(IndexShard.java:1346)  
at org.elasticsearch.index.shard.IndexShard.prepareForTranslogRecovery(IndexShard.java:866)  
at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:233)  
at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:112)  
... 3 more  
Caused by: java.nio.file.NoSuchFileException: /var/lib/elasticsearch//nodes/0/indices/logstash-2015.07.21/4/index/segments\_1r  
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)  
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)  
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)  
at sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)  
at java.nio.channels.FileChannel.open(FileChannel.java:287)  
at java.nio.channels.FileChannel.open(FileChannel.java:334)  
at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:81)  
at org.apache.lucene.store.FileSwitchDirectory.openInput(FileSwitchDirectory.java:172)  
at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:80)  
at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:80)  
at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:80)  
at org.elasticsearch.index.store.Store$StoreDirectory.openInput(Store.java:733)  
at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:113)  
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:341)  
at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:462)  
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:923)  
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:769)  
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:458)  
at org.apache.lucene.index.IndexWriter.(IndexWriter.java:794)  
at org.elasticsearch.index.engine.InternalEngine.createWriter(InternalEngine.java:1134)  
at org.elasticsearch.index.engine.InternalEngine.(InternalEngine.java:151)  
... 9 more  
[2015-07-21 16:05:40,331][WARN][cluster.action.shard] [loghost] [logstash-2015.07.21][4] received shard failed for [logstash-2015.07.21][4], node[ps5dpZM3SIaXe-kxPG82Kg], [P], s[INITIALIZING], indexUUID [fJVMIfuKSP6VTxsVuLi0Ag], reason [shard failure [failed recovery][IndexShardGatewayRecoveryException[[logstash-2015.07.21][4] failed recovery]; nested: EngineCreationFailureException[[logstash-2015.07.21][4] failed to create engine]; nested: NoSuchFileException[/var/lib/elasticsearch//nodes/0/indices/logstash-2015.07.21/4/index/segments\_1r]; ]]

All shards are UNASSIGNED:  
[root@loghost:~]# curl localhost:9200/\_cat/shards  
logstash-2015.07.21 4 r UNASSIGNED  
logstash-2015.07.21 4 p UNASSIGNED  
logstash-2015.07.21 0 p UNASSIGNED  
logstash-2015.07.21 0 r UNASSIGNED  
logstash-2015.07.21 3 p UNASSIGNED  
logstash-2015.07.21 3 r UNASSIGNED  
logstash-2015.07.21 1 p UNASSIGNED  
logstash-2015.07.21 1 r UNASSIGNED  
logstash-2015.07.21 2 p UNASSIGNED  
logstash-2015.07.21 2 r UNASSIGNED

[root@loghost:~]# curl -XGETs 'localhost:9200/\_cat/indices?v'  
health status index pri rep docs.count docs.deleted store.size pri.store.size  
red open kibana-int 5 1  
red open logstash-2015.07.16 5 1  
red open logstash-2015.07.17 5 1  
red open logstash-2015.07.20 5 1  
red open logstash-2015.07.19 5 1  
red open logstash-2015.07.21 5 1  
red open logstash-2015.07.18 5 1

I've tried to look on the www to rebuild indixes, but none of them work.

Anybody have ideas?  
CentOS 6, x86\_64  
Elasticsearch 1.6.0 (6GB jvm XMX)  
JVM: jdk1.7.0\_75

Kind regards,  
Werner

---

<div class="post-metadata">

**Author:** ![colings86](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/colings86/32/44960_2.png) [@colings86](https://discuss.elastic.co/u/colings86)\
**Post date:** [July 21, 2015, 2:59pm UTC](https://discuss.elastic.co/t/logstash-2015-07-21-4-failed-to-create-engine/25992/2 "2015-07-21T14:59:29Z")

</div>

> [@wdijkerman](#):
>
> NoSuchFileException[/var/lib/elasticsearch//nodes/0/indices/logstash-2015.07.21/4/index/segments\_1r]; ]]

It looks like the crash or the file system check corrupted your index files as there are required files missing. Unfortunately without replica shards to copy the data from there is no way to repair your index and retrieve your data. You will need to delete the index and re-index the data from its original source.

---

<div class="post-metadata">

**Author:** ![wdijkerman](https://avatars.discourse-cdn.com/v4/letter/w/3ec8ea/32.png) [@wdijkerman](https://discuss.elastic.co/u/wdijkerman)\
**Post date:** [July 21, 2015, 3:15pm UTC](https://discuss.elastic.co/t/logstash-2015-07-21-4-failed-to-create-engine/25992/3 "2015-07-21T15:15:06Z")

</div>

Bummer..

Thank you for your answer.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 12:00am UTC](https://discuss.elastic.co/t/logstash-2015-07-21-4-failed-to-create-engine/25992/4 "2017-07-06T00:00:08Z")

</div>


