After searching everywhere, I couldn't find the purpose of state files.
We started off with version 2.2 and then went to 5.6 and then at 6.8 atm.
Below is the scenario we frequently run into:
Elasticsearch continuously restarts due to state files either being empty or corrupted.
We do not know what's the reason behind the state file being empty or corruption.
What's interesting is, we end up deleting the state files and Elasticsearch gets initialized with no issues.
So, if the state files can be deleted, why does ES need them?
Are we losing any information by deleting those state files?
If ES finds itself in a state where there are empty state files or corrupted state files, why doesn't it fix it by itself? based on individual scenarios?
What does ES do with the data present in a state file?
I also noticed, when ES runs into an exception related to state files, it prints one at a time.
What I mean by that is, once I delete a corrupted state file, ES re-initializes and then throws an error about another state file.
Why doesn't it print all the state files which are corrupted?
Is there any way we can find out all the corrupted/empty state files?
Appreciate your time and help.