Unable to run logstash pod on kubernetes


I am trying to deploy elk stack on Kubernetes cluster (running on my laptop) using helm chart. Elastic search and kibana have deployed successfully, however, logstash pod is crashing ans showing "CrashLoopBackOff" errors.

Below aare the logs (kubectl logs logstash)

2020/05/18 17:02:28 Setting 'config.reload.automatic' from environment.
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties
[2020-05-18T17:02:57,140][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<ArgumentError: Path "/usr/share/logstash/data" must be a writable directory. It is not writable.>, :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/settings.rb:447:in validate'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:229:in validate_value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:140:in block in validate_all'", "org/jruby/RubyHash.java:1419:in each'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:139:in validate_all'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:278:in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in run'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:237:in run'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in run'", "/usr/share/logstash/lib/bootstrap/environment.rb:73:in '"]}
[2020-05-18T17:02:57,161][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

below is the event ( kubectl describe pod logstash)

             node.kubernetes.io/unreachable:NoExecute for 300s

Type Reason Age From Message

Warning Unhealthy 31m (x20 over 96m) kubelet, worker Liveness probe failed: Get dial tcp connect: connection refused
Warning BackOff 111s (x395 over 94m) kubelet, worker Back-off restarting failed container

Please help me as i am new in this elk.


Hi @Abhinav_Shrivastava,
Can you provide the full command that you are using to deploy Logstash chart?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.