In the past few days I have seen in two different occasions a
MergeException: java.io.FileNotFoundException: _fg_1.del
details gisted: https://gist.github.com/3002612
If you look carefully the date and time of the supposedly FileNotFound is
the same as the error log, 2012-06-23 03:19, the mount is configured with
noatime, so it's likely the date is the creation date, (ie the file is
created at the same time that the check is failing), I guess it could be a
concurrency race condition issue.
This was seen in two different configurations:
The HDDs are SSD, and the index that failed aren't exactly heavy loaded (I
have other indices with more bulk adds for example).
It seems the merge policy thread dies, and a couple of hours later the
whole ES server dies with "java.io.IOException: Too many open files".
I would appreciate some help and ideas with this.