Broken index

Every 4-7 days one of my indexes get broken.
ES: 5.6.8
OS: Debian 9.3 4.9.0-6-amd64

curl -XGET http://hostname:9200/_cat/indices | grep red
red    delivered HotrN1eGQpmxTkbJN0oL0g 5 1 36618688  1476436   1.8gb   1.8gb

ls /var/lib/elasticsearch/nodes/0/indices/HotrN1eGQpmxTkbJN0oL0g/
0  1  2  3  4  _state

I have checked every of 5 indexes like this:
java -cp lucene-core-6.6.1.jar -ea:org.apache.lucene... org.apache.lucene.index.CheckIndex /var/lib/elasticsearch/nodes/0/indices/HotrN1eGQpmxTkbJN0oL0g/0/index/ -verbose -exorcise

No problems were detected with this index.

Can you please suggest me something ?

The problem was in broken server. I don't know what exactly but they changed everything except disks and I have never see again such problems.

I would check the Log files to look for error messages. There are only 2 events I know that might corrupt the index but I am sure there are many more.

  • Your files system filled up
  • You lost the ability to write to the partition

At least those are the common ones for me, but I would look at the elastic logfiles to see first what is happening

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.