# Logstash dont use \*.conf files **URL:** \ **Category:** Logstash\ **Created:** [May 26, 2020, 8:21am UTC](https://discuss.elastic.co/t/logstash-dont-use-conf-files/234277 "2020-05-26T08:21:06Z")\ **Posts on this page:** 9\ **Page:** 2
**Author:** ![ahiyaz](https://avatars.discourse-cdn.com/v4/letter/a/f17d59/32.png) [@ahiyaz](https://discuss.elastic.co/u/ahiyaz)\ **Post date:** [May 30, 2020, 12:13pm UTC](https://discuss.elastic.co/t/logstash-dont-use-conf-files/234277/21 "2020-05-30T12:13:18Z")
> [@ptamba](#): > > /usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash" attached indent preformatted text by 4 spaces root@logstashsapir:~# su - logstash logstash@logstashsapir:~$ /usr/share/logstash/bin/logstash "--path.settings" "/e tc/logstash" logstash@logstashsapir:~$ /usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash" OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/usr/share/logstash/logstash-core/lib/jars/jruby-complete-9.2.11.1.jar) to method sun.nio.ch.NativeThread.signal(long) WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties [2020-05-30T12:12:22,983][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"7.7.0"} [2020-05-30T12:12:27,499][INFO][org.reflections.Reflections] Reflections took 93 ms to scan 1 urls, producing 21 keys and 41 values [2020-05-30T12:12:31,663][ERROR][logstash.outputs.lumberjack] Invalid setting for lumberjack output plugin: output { lumberjack { # This setting must be a path # File does not exist or cannot be opened /usr/share/logstash/keys/TrustExternalCARoot.crt ssl\_certificate =\> "/usr/share/logstash/keys/TrustExternalCARoot.crt" ... } } [2020-05-30T12:12:31,750][ERROR][logstash.agent] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"Java::JavaLang::IllegalStateException", :message=\>"Unable to configure plugins: (ConfigurationError) Something is wrong with your configuration.", :backtrace=\>["org.logstash.config.ir.CompiledPipeline.(CompiledPipeline.java:126)", "org.logstash.execution.JavaBasePipelineExt.initialize(JavaBasePipelineExt.java:80)", "org.logstash.execution.JavaBasePipelineExt$INVOKER$i$1$0$initialize.call(JavaBasePipelineExt$INVOKER$i$1$0$initialize.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:837)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1169)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuperSplatArgs(IRRuntimeHelpers.java:1156)", "org.jruby.ir.targets.InstanceSuperInvokeSite.invoke(InstanceSuperInvokeSite.java:39)", "usr.share.logstash.logstash\_minus\_core.lib.logstash.java\_pipeline.RUBY$method$initialize$0(/usr/share/logstash/logstash-core/lib/logstash/java\_pipeline.rb:43)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:82)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:86)", "org.jruby.RubyClass.newInstance(RubyClass.java:939)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "usr.share.logstash.logstash\_minus\_core.lib.logstash.pipeline\_action.create.RUBY$method$execute$0(/usr/share/logstash/logstash-core/lib/logstash/pipeline\_action/create.rb:52)", "usr.share.logstash.logstash\_minus\_core.lib.logstash.pipeline\_action.create.RUBY$method$execute$0$ **VARARGS** (/usr/share/logstash/logstash-core/lib/logstash/pipeline\_action/create.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:82)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "usr.share.logstash.logstash\_minus\_core.lib.logstash.agent.RUBY$block$converge\_state$2(/usr/share/logstash/logstash-core/lib/logstash/agent.rb:342)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)", "org.jruby.runtime.Block.call(Block.java:139)", "org.jruby.RubyProc.call(RubyProc.java:318)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)", "java.base/java.lang.Thread.run(Thread.java:834)"]} warning: thread "Converge PipelineAction::Create" terminated with exception (report\_on\_exception is true): LogStash::Error: Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create
` create at org/logstash/execution/ConvergeResultExt.java:129 add at org/logstash/execution/ConvergeResultExt.java:57 converge\_state at /usr/share/logstash/logstash-core/lib/logstash/agent.rb:355 [2020-05-30T12:12:31,768][ERROR][logstash.agent] An exception happened when converging configuration {:exception=\>LogStash::Error, :message=\>"Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create
`", :backtrace=\>["org/logstash/execution/ConvergeResultExt.java:129:in `create'", "org/logstash/execution/ConvergeResultExt.java:57:in `add'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:355:in `block in converge_state'"]} [2020-05-30T12:12:31,869][FATAL][logstash.runner] An unexpected error occurred! {:error=>#, :backtrace=>["org/logstash/execution/ConvergeResultExt.java:129:in `create'", "org/logstash/execution/ConvergeResultExt.java:57:in `add'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:355:in `block in converge\_state'"]} [2020-05-30T12:12:31,925][ERROR][org.logstash.Logstash] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit indent preformatted text by 4 spaces ---
**Author:** ![ptamba](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@ptamba](https://discuss.elastic.co/u/ptamba)\ **Post date:** [May 30, 2020, 12:44pm UTC](https://discuss.elastic.co/t/logstash-dont-use-conf-files/234277/22 "2020-05-30T12:44:13Z")
> [@ahiyaz](#): > > # This setting must be a path > > # File does not exist or cannot be opened /usr/share/logstash/keys/TrustExternalCARoot.crt > > ssl\_certificate =\> "/usr/share/logstash/keys/TrustExternalCARoot.crt" > ... using root account , give access to logstash user to that file. `chmod o+r /usr/share/logstash/keys/TrustExternalCARoot.crt` ---
**Author:** ![ahiyaz](https://avatars.discourse-cdn.com/v4/letter/a/f17d59/32.png) [@ahiyaz](https://discuss.elastic.co/u/ahiyaz)\ **Post date:** [May 30, 2020, 3:09pm UTC](https://discuss.elastic.co/t/logstash-dont-use-conf-files/234277/23 "2020-05-30T15:09:42Z")
> [@ahiyaz](#): > > /usr/share/logstash/bin/logstash "--path.settings" "/e tc/logstash" hi i did give access to logstash and tried loading logstash both systemctl loading my config file and without config file using cli indent preformatted text by 4 spaces but no success root@logstashsapir:~# chmod o+r /usr/share/logstash/keys/TrustExternalCARoot.crt root@logstashsapir:~# systemctl stop logstash root@logstashsapir:~# systemctl start logstash root@logstashsapir:~# systemctl status logstash ? logstash.service - logstash Loaded: loaded (/etc/systemd/system/logstash.service; enabled; vendor preset: enabled) Active: active (running) since Sat 2020-05-30 15:00:55 UTC; 8s ago Main PID: 8402 (java) Tasks: 14 (limit: 2318) CGroup: /system.slice/logstash.service mq8402 /usr/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.awt.hea May 30 15:00:55 logstashsapir systemd[1]: Started logstash. May 30 15:00:55 logstashsapir logstash[8402]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be re May 30 15:00:59 logstashsapir logstash[8402]: WARNING: An illegal reflective access operation has occurred May 30 15:00:59 logstashsapir logstash[8402]: WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/usr/share/logstash/logstash- May 30 15:00:59 logstashsapir logstash[8402]: WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules May 30 15:00:59 logstashsapir logstash[8402]: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations May 30 15:00:59 logstashsapir logstash[8402]: WARNING: All illegal access operations will be denied in a future release root@logstashsapir:~# netstat -taupn | grep ESTABLISHED tcp 0 36 172.16.0.131:22 10.212.134.200:57291 ESTABLISHED 8272/sshd: logstash root@logstashsapir:~# root@logstashsapir:~# root@logstashsapir:~# root@logstashsapir:~# root@logstashsapir:~# su - logstash logstash@logstashsapir:~$ /usr/share/logstash/bin/logstash "--path.settings" "/e tc/logstash" OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/usr/share/logstash/logstash-core/lib/jars/jruby-complete-9.2.11.1.jar) to method sun.nio.ch.NativeThread.signal(long) WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release WARNING: Could not find logstash.yml which is typically located in $LS\_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults Could not find log4j2 configuration at path /e tc/logstash/log4j2.properties. Using default config which logs errors to the console [FATAL] 2020-05-30 15:04:53.330 [main] runner - An unexpected error occurred! {:error=\>#\, :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/settings.rb:528:in `validate'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:288:in `validate\_value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:199:in `block in validate_all'", "org/jruby/RubyHash.java:1415:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:198:in `validate_all'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:305:in `execute'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:263:in `run'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in `run'", "/usr/share/logstash/lib/bootstrap/environment.rb:88:in `'"]} [ERROR] 2020-05-30 15:04:53.354 [main] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit indent preformatted text by 4 spaces ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [May 30, 2020, 3:11pm UTC](https://discuss.elastic.co/t/logstash-dont-use-conf-files/234277/24 "2020-05-30T15:11:28Z")
> [@ahiyaz](#): > > Path "/usr/share/logstash/data/queue" must be a writable directory. It is not writable. What is unclear about that? ---
**Author:** ![ahiyaz](https://avatars.discourse-cdn.com/v4/letter/a/f17d59/32.png) [@ahiyaz](https://discuss.elastic.co/u/ahiyaz)\ **Post date:** [May 30, 2020, 3:24pm UTC](https://discuss.elastic.co/t/logstash-dont-use-conf-files/234277/25 "2020-05-30T15:24:03Z")
ive tried that before the re-installation. chmod logstash u+x /usr/share/logstash/keys and this ive just tried chmod o+r /usr/share/logstash/keys/TrustExternalCARoot.crt do you mean of something else? ---
**Author:** ![ahiyaz](https://avatars.discourse-cdn.com/v4/letter/a/f17d59/32.png) [@ahiyaz](https://discuss.elastic.co/u/ahiyaz)\ **Post date:** [May 30, 2020, 3:40pm UTC](https://discuss.elastic.co/t/logstash-dont-use-conf-files/234277/26 "2020-05-30T15:40:38Z")
> [@ahiyaz](#): > > /usr/share/logstash/data/queue nowi see its a different dir let me try that ---
**Author:** ![ahiyaz](https://avatars.discourse-cdn.com/v4/letter/a/f17d59/32.png) [@ahiyaz](https://discuss.elastic.co/u/ahiyaz)\ **Post date:** [May 30, 2020, 4:03pm UTC](https://discuss.elastic.co/t/logstash-dont-use-conf-files/234277/27 "2020-05-30T16:03:42Z")
> [@ahiyaz](#): > > /usr/share/logstash/data/queue ive just executed these two commands but still, and it starts working via cli (user logstash) but not via systemctl ``` indent preformatted text by 4 spaces ``` sudo chmod 775 /usr/share/logstash/data/queue sudo chmod 775 /usr/share/logstash/keys/ logstash@logstashsapir:~$ /usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash" OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/usr/share/logstash/logstash-core/lib/jars/jruby-complete-9.2.11.1.jar) to method sun.nio.ch.NativeThread.signal(long) WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties [2020-05-30T15:50:03,416][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"7.7.0"} [2020-05-30T15:50:08,017][INFO][org.reflections.Reflections] Reflections took 102 ms to scan 1 urls, producing 21 keys and 41 values [2020-05-30T15:50:15,130][WARN][org.logstash.instrument.metrics.gauge.LazyDelegatingGauge][main] A gauge metric of an unknown type (org.jruby.RubyArray) has been created for key: cluster\_uuids. This may result in invalid serialization. It is recommended to log an issue to the responsible developer/development team. [2020-05-30T15:50:15,179][INFO][logstash.javapipeline][main] Starting pipeline {:pipeline\_id=\>"main", "pipeline.workers"=\>1, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>50, "pipeline.max\_inflight"=\>125, "pipeline.sources"=\>["/etc/logstash/conf.d/sflowlog.conf"], :thread=\>"#\"} [2020-05-30T15:50:16,907][INFO][logstash.javapipeline][main] Pipeline started {"pipeline.id"=\>"main"} [2020-05-30T15:50:17,176][INFO][logstash.agent] Pipelines running {:count=\>1, :running\_pipelines=\>[:main], :non\_running\_pipelines=\>} [2020-05-30T15:50:17,219][INFO][logstash.inputs.udp][main][cfeabc89a9145c2e5083c84cd1cda4cd042fb5db41e7f4ceac3abee10d2deba8] Starting UDP listener {:address=\>"0.0.0.0:6343"} [2020-05-30T15:50:17,366][INFO][logstash.inputs.udp][main][cfeabc89a9145c2e5083c84cd1cda4cd042fb5db41e7f4ceac3abee10d2deba8] UDP listener started {:address=\>"0.0.0.0:6343", :receive\_buffer\_bytes=\>"106496", :queue\_size=\>"2000"} [2020-05-30T15:50:17,734][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600} indent preformatted text by 4 spaces listening on right ports as configuration indent preformatted text by 4 spaces netstat -taupn | grep ESTABLISHED tcp 0 0 172.16.0.131:22 10.212.134.200:57291 ESTABLISHED 8272/sshd: logstash tcp 0 36 172.16.0.131:22 10.212.134.200:57814 ESTABLISHED 11012/sshd: logstas tcp6 0 0 172.16.0.131:40210 23.22.183.192:5006 ESTABLISHED 11629/java root@logstashsapir:~# netstat -unlp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name udp 0 0 127.0.0.53:53 0.0.0.0:\* 966/systemd-resolve udp 0 0 172.16.0.131:68 0.0.0.0:\* 947/systemd-network udp 0 0 0.0.0.0:6343 0.0.0.0:\* 11629/java indent preformatted text by 4 spaces but not with systemctl indent preformatted text by 4 spaces systemctl start logstash root@logstashsapir:~# netstat -taupn | grep ESTABLISHED tcp 0 0 172.16.0.131:22 10.212.134.200:57291 ESTABLISHED 8272/sshd: logstash tcp 0 36 172.16.0.131:22 10.212.134.200:57814 ESTABLISHED 11012/sshd: logstas root@logstashsapir:~# netstat -taupn | grep ESTABLISHED indent preformatted text by 4 spaces ---
**Author:** ![ahiyaz](https://avatars.discourse-cdn.com/v4/letter/a/f17d59/32.png) [@ahiyaz](https://discuss.elastic.co/u/ahiyaz)\ **Post date:** [May 30, 2020, 4:31pm UTC](https://discuss.elastic.co/t/logstash-dont-use-conf-files/234277/28 "2020-05-30T16:31:59Z")
running like this make it work just take a while. ``` indent preformatted text by 4 spaces ``` systemctl start logstash.service indent preformatted text by 4 spaces seems like make the dir writable did the work thank you guys for all the help!!!! do you know what went wrong with my installation? because i did it by the guide, the only differences were the plugins: sflow and [logz.io](http://logz.io) version of the lumberjack. thank you again for all your trouble ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 27, 2020, 4:32pm UTC](https://discuss.elastic.co/t/logstash-dont-use-conf-files/234277/29 "2020-06-27T16:32:05Z")
Closed on June 27, 2020, 4:32pm UTC This topic was automatically closed 28 days after the last reply. New replies are no longer allowed. [Previous page](https://discuss.elastic.co/t/logstash-dont-use-conf-files/234277.md?page=1)