Hello,
I am trying to deploy elk stack on Kubernetes cluster (running on my laptop) using helm chart. Elastic search and kibana have deployed successfully, however, logstash pod is crashing ans showing "CrashLoopBackOff" errors.
Below aare the logs (kubectl logs logstash)
2020/05/18 17:02:28 Setting 'config.reload.automatic' from environment.
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties
[2020-05-18T17:02:57,140][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<ArgumentError: Path "/usr/share/logstash/data" must be a writable directory. It is not writable.>, :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/settings.rb:447:in validate'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:229:in validate_value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:140:in block in validate_all'", "org/jruby/RubyHash.java:1419:in each'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:139:in validate_all'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:278:in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in run'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:237:in run'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in run'", "/usr/share/logstash/lib/bootstrap/environment.rb:73:in '"]}
[2020-05-18T17:02:57,161][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
below is the event ( kubectl describe pod logstash)
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
Warning Unhealthy 31m (x20 over 96m) kubelet, worker Liveness probe failed: Get http://192.168.171.68:9600/: dial tcp 192.168.171.68:9600: connect: connection refused
Warning BackOff 111s (x395 over 94m) kubelet, worker Back-off restarting failed container
Please help me as i am new in this elk.
Regards