Elasticsearch pod going into crashloopbackoff state due to no space left on the device

Elastic Stack Elasticsearch
1

Caused by: org.Elasticsearch.gateway.WriteStateException: failed to write state to the first location tmp file /usr/share/Elasticsearch/data/nodes/0/node-1.st.tmp",
"at org.Elasticsearch.gateway.MetaDataStateFormat.writeStateToFirstLocation(MetaDataStateFormat.java:127) ~[Elasticsearch-7.4.0.jar:7.4.0]",
"at org.Elasticsearch.gateway.MetaDataStateFormat.write(MetaDataStateFormat.java:243) ~[Elasticsearch-7.4.0.jar:7.4.0]",
"at org.Elasticsearch.gateway.MetaDataStateFormat.writeAndCleanup(MetaDataStateFormat.java:185) ~[Elasticsearch-7.4.0.jar:7.4.0]",
"at org.Elasticsearch.env.NodeEnvironment.loadOrCreateNodeMetaData(NodeEnvironment.java:431) ~[Elasticsearch-7.4.0.jar:7.4.0]",
"at org.Elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:304) ~[Elasticsearch-7.4.0.jar:7.4.0]",
"at org.Elasticsearch.node.Node.(Node.java:275) ~[Elasticsearch-7.4.0.jar:7.4.0]",
"at org.Elasticsearch.node.Node.(Node.java:255) ~[Elasticsearch-7.4.0.jar:7.4.0]",
"at org.Elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:221) ~[Elasticsearch-7.4.0.jar:7.4.0]",
"at org.Elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) ~[Elasticsearch-7.4.0.jar:7.4.0]",
"at org.Elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[Elasticsearch-7.4.0.jar:7.4.0]",
"at org.Elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[Elasticsearch-7.4.0.jar:7.4.0]",
"... 6 more",
"Caused by: java.io.IOException: No space left on device",
"at

It seems like the PV is fully consumed for one of the 2 replica pods in the configuration. What is the best way to handle such situation. Please help.

2

Welcome to our community! :smiley:

Are you using ECK here?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.