OverlappingFileException Issue

Elastic Stack Elasticsearch
1

hi,

i have restarted elasticsearch this morning, but sadly i meet the OverlappingFileException as follows:

[2017-12-04T20:37:09,783][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [node-1-60] uncaught exception in thread [main]
    org.elasticsearch.bootstrap.StartupException: java.nio.channels.OverlappingFileLockException
    	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-5.6.4.jar:5.6.4]
    	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:123) ~[elasticsearch-5.6.4.jar:5.6.4]
    	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:70) ~[elasticsearch-5.6.4.jar:5.6.4]
    	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:134) ~[elasticsearch-5.6.4.jar:5.6.4]
    	at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-5.6.4.jar:5.6.4]
    	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) ~[elasticsearch-5.6.4.jar:5.6.4]
    	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) ~[elasticsearch-5.6.4.jar:5.6.4]
    Caused by: java.nio.channels.OverlappingFileLockException
    	at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255) ~[?:?]
    	at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152) ~[?:?]
    	at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1108) ~[?:?]
    	at java.nio.channels.FileChannel.tryLock(FileChannel.java:1155) ~[?:1.8.0_151]
    	at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:114) ~[lucene-core-6.6.1.jar:6.6.1 9aa465a89b64ff2dabe7b4d50c472de32c298683 - varunthacker - 2017-08-29 21:54:39]
    	at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41) ~[lucene-core-6.6.1.jar:6.6.1 9aa465a89b64ff2dabe7b4d50c472de32c298683 - varunthacker - 2017-08-29 21:54:39]
    	at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45) ~[lucene-core-6.6.1.jar:6.6.1 9aa465a89b64ff2dabe7b4d50c472de32c298683 - varunthacker - 2017-08-29 21:54:39]
    	at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:226) ~[elasticsearch-5.6.4.jar:5.6.4]
    	at org.elasticsearch.node.Node.<init>(Node.java:265) ~[elasticsearch-5.6.4.jar:5.6.4]
    	at org.elasticsearch.node.Node.<init>(Node.java:245) ~[elasticsearch-5.6.4.jar:5.6.4]
    	at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:233) ~[elasticsearch-5.6.4.jar:5.6.4]
    	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:233) ~[elasticsearch-5.6.4.jar:5.6.4]
    	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:342) ~[elasticsearch-5.6.4.jar:5.6.4]
    	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:132) ~[elasticsearch-5.6.4.jar:5.6.4]
    	... 6 more

and then i tried
./bin/elasticsearch -E logger.org.elasticsearch.env.NodeEnvironment=trace
[2017-12-04T21:08:58,441][INFO ][o.e.n.Node ] [node-1-60] initializing ...
[2017-12-04T21:08:58,471][TRACE][o.e.e.NodeEnvironment ] [node-1-60] obtaining node lock on /d03/es-data/nodes/0 ...
[2017-12-04T21:08:58,474][TRACE][o.e.e.NodeEnvironment ] [node-1-60] obtaining node lock on /d04/es-data/nodes/0 ...
[2017-12-04T21:08:58,475][TRACE][o.e.e.NodeEnvironment ] [node-1-60] obtaining node lock on /d05/es-data/nodes/0 ...
[2017-12-04T21:08:58,477][TRACE][o.e.e.NodeEnvironment ] [node-1-60] obtaining node lock on /d06/es-data/nodes/0 ...
[2017-12-04T21:08:58,488][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [node-1-60] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.nio.channels.OverlappingFileLockException
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-5.6.4.jar:5.6.4]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:123) ~[elasticsearch-5.6.4.jar:5.6.4]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:70) ~[elasticsearch-5.6.4.jar:5.6.4]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:134) ~[elasticsearch-5.6.4.jar:5.6.4]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-5.6.4.jar:5.6.4]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) ~[elasticsearch-5.6.4.jar:5.6.4]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) ~[elasticsearch-5.6.4.jar:5.6.4]

Ok, it is beacause the data file is already locked.
but i tried : ps -ef | grep java
i am sure there is no process running another elasticsearch instance
ps -ef | grep java
elk 64594 59516 0 21:09 pts/2 00:00:00 grep java

i also removed all /data/node/*/node.lock, but it not work.

and i removed all data directory, and mkdir the new directory, using different name.
i do this for trying to give elasticsearch new data directory, and get rid of the lock problem.

but same OverlappingFileException when i try to run elasticsearch.

or how to find which process is lock this file? and how to release the lock manually?
Can anyone who can help me? thx sooo much!

Finally, i solve this problem.
after i see the log, it always failed to set lock on /d06/es-data/...
and i remove this directory in configuration file. it then works!!!

something maybe wrong with my harddisk /d06

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.