# ES lost the file lock while running

**URL:** <https://discuss.elastic.co/t/es-lost-the-file-lock-while-running/84233>\
**Category:** Elasticsearch\
**Created:** [May 2, 2017, 10:05am UTC](https://discuss.elastic.co/t/es-lost-the-file-lock-while-running/84233 "2017-05-02T10:05:27Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![xuhaibo](https://avatars.discourse-cdn.com/v4/letter/x/34f0e0/32.png) [@xuhaibo](https://discuss.elastic.co/u/xuhaibo)\
**Post date:** [May 2, 2017, 10:05am UTC](https://discuss.elastic.co/t/es-lost-the-file-lock-while-running/84233/1 "2017-05-02T10:05:27Z")

</div>

Ask you have not encountered under the ES operation of the file lock is lost, and can not lead to the problem, I encountered several times. Every time you see the file lock is lost and then the new index begins to go wrong

[2017-04-30T12:07:26,705][WARN][o.e.e.NodeEnvironment] [es\_infra\_bjza\_1] lock assertion failed  
java.nio.file.NoSuchFileException: /data/elasticsearch/data/es\_infra\_bjza/nodes/0/node.lock  
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) ~[?:?]  
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:?]  
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:?]  
at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) ~[?:?]  
at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) ~[?:?]  
at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) ~[?:?]  
at java.nio.file.Files.readAttributes(Files.java:1737) ~[?:1.8.0\_111]  
at org.apache.lucene.store.NativeFSLockFactory$NativeFSLock.ensureValid(NativeFSLockFactory.java:177) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]  
at org.elasticsearch.env.NodeEnvironment.assertEnvIsLocked(NodeEnvironment.java:901) [elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.env.NodeEnvironment.nodePaths(NodeEnvironment.java:753) [elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.monitor.fs.FsProbe.stats(FsProbe.java:55) [elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.monitor.fs.FsService.stats(FsService.java:60) [elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.monitor.fs.FsService.access$200(FsService.java:33) [elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.monitor.fs.FsService$FsInfoCache.refresh(FsService.java:78) [elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.monitor.fs.FsService$FsInfoCache.refresh(FsService.java:67) [elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.common.util.SingleObjectCache.getOrRefresh(SingleObjectCache.java:54) [elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.monitor.fs.FsService.stats(FsService.java:55) [elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.node.service.NodeService.stats(NodeService.java:114) [elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.action.admin.cluster.node.stats.TransportNodesStatsAction.nodeOperation(TransportNodesStatsAction.java:77) [elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.action.admin.cluster.node.stats.TransportNodesStatsAction.nodeOperation(TransportNodesStatsAction.java:42) [elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.action.support.nodes.TransportNodesAction.nodeOperation(TransportNodesAction.java:145) [elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.action.support.nodes.TransportNodesAction$NodeTransportHandler.messageReceived(TransportNodesAction.java:270) [elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.action.support.nodes.TransportNodesAction$NodeTransportHandler.messageReceived(TransportNodesAction.java:266) [elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) [elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.transport.TransportService$6.doRun(TransportService.java:577) [elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:527) [elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.1.1.jar:5.1.1]  
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0\_111]  
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0\_111]  
at java.lang.Thread.run(Thread.java:745) [?:1.8.0\_111]  
[2017-04-30T12:07:26,706][DEBUG][o.e.a.a.c.n.s.TransportNodesStatsAction] [es\_infra\_bjza\_1] failed to execute on node [-bZFv7AcSc6hK94oEmnTYA]  
org.elasticsearch.transport.RemoteTransportException: [es\_infra\_bjza\_1][10.120.6.246:9300][cluster:monitor/nodes/stats[n]]  
Caused by: java.lang.IllegalStateException: environment is not locked  
at org.elasticsearch.env.NodeEnvironment.assertEnvIsLocked(NodeEnvironment.java:904) ~[elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.env.NodeEnvironment.nodePaths(NodeEnvironment.java:753) ~[elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.monitor.fs.FsProbe.stats(FsProbe.java:55) ~[elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.monitor.fs.FsService.stats(FsService.java:60) ~[elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.monitor.fs.FsService.access$200(FsService.java:33) ~[elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.monitor.fs.FsService$FsInfoCache.refresh(FsService.java:78) ~[elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.monitor.fs.FsService$FsInfoCache.refresh(FsService.java:67) ~[elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.common.util.SingleObjectCache.getOrRefresh(SingleObjectCache.java:54) ~[elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.monitor.fs.FsService.stats(FsService.java:55) ~[elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.node.service.NodeService.stats(NodeService.java:114) ~[elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.action.admin.cluster.node.stats.TransportNodesStatsAction.nodeOperation(TransportNodesStatsAction.java:77) ~[elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.action.admin.cluster.node.stats.TransportNodesStatsAction.nodeOperation(TransportNodesStatsAction.java:42) ~[elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.action.support.nodes.TransportNodesAction.nodeOperation(TransportNodesAction.java:145) ~[elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.action.support.nodes.TransportNodesAction$NodeTransportHandler.messageReceived(TransportNodesAction.java:270) ~[elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.action.support.nodes.TransportNodesAction$NodeTransportHandler.messageReceived(TransportNodesAction.java:266) ~[elasticsearch-5.1.1.jar:5.1.1]  
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.1.1.jar:5.1.1]

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 2, 2017, 10:16am UTC](https://discuss.elastic.co/t/es-lost-the-file-lock-while-running/84233/2 "2017-05-02T10:16:34Z")

</div>

What sort of filesystem is this on?

---

<div class="post-metadata">

**Author:** ![xuhaibo](https://avatars.discourse-cdn.com/v4/letter/x/34f0e0/32.png) [@xuhaibo](https://discuss.elastic.co/u/xuhaibo)\
**Post date:** [May 3, 2017, 2:33am UTC](https://discuss.elastic.co/t/es-lost-the-file-lock-while-running/84233/3 "2017-05-03T02:33:34Z")

</div>

ext4

---

<div class="post-metadata">

**Author:** ![xuhaibo](https://avatars.discourse-cdn.com/v4/letter/x/34f0e0/32.png) [@xuhaibo](https://discuss.elastic.co/u/xuhaibo)\
**Post date:** [May 3, 2017, 2:49am UTC](https://discuss.elastic.co/t/es-lost-the-file-lock-while-running/84233/4 "2017-05-03T02:49:28Z")

</div>

My ES cluster is running on the aws server. Data storage is a log service that requires periodic cleanup of data

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 31, 2017, 2:50am UTC](https://discuss.elastic.co/t/es-lost-the-file-lock-while-running/84233/5 "2017-05-31T02:50:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
