Hi,
I am creating 3 node cluster with ELK also installing filebeat on another server with predefined nginx logs to pass. While using Elasticsearch as an output in filebeat.yml file I am able to pass all logs successfully to elasticsearch and then to kibana. Then I changed the output to Logstash in filebeat.yml then I am getting an error,
Here's my test.conf file in /etc/logstash/conf.d folder
input {
_ beats {_
_ port => 5044_
_ }_
}
output {
_ elasticsearch {_
_ hosts => ["10.1.1.47:9200"]_
_ index => "nginx-%{+YYYY.MM.dd}"_
_ }_
}
I have added output in filebeat.yml as
output.logstash:
_ hosts: ["10.1.1 I :5044"]_
After that when i do this,
/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/test.conf
gives me this error,
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
then i added --path.settings as,
/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/test.conf --path.settings /etc/logstash
then i am getting this logs in /var/log/logstash/logstash-plain.log
[root@master centos]# tail -f /var/log/logstash/logstash-plain.log
_ at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73) [?:?]_
_ at org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:132) [?:?]_
_ at org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:148) [?:?]_
_ at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:73) [?:?]_
_ at org.jruby.runtime.Block.call(Block.java:124) [?:?]_
_ at org.jruby.RubyProc.call(RubyProc.java:289) [?:?]_
_ at org.jruby.RubyProc.call(RubyProc.java:246) [?:?]_
_ at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:104) [?:?]_
_ at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]_
[2017-12-05T12:42:11,997][DEBUG][logstash.pipeline ] Pushing flush onto pipeline {:pipeline_id=>"main", :thread=>"#<Thread:0x53c8049c@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:290 sleep>"}
[2017-12-05T12:42:13,928][ERROR][logstash.pipeline ] A plugin had an unrecoverable error. Will restart this plugin.
_ Pipeline_id:main_
_ Plugin: <LogStash::Inputs::Beats port=>5044, id=>"9bb67ac3ef5bf1f5c137746a3980352ccd9c43b1e497717a4e5bad712b1e5b4a", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_6b561c66-6e66-4b4f-80ee-a6967d9379fb", enable_metric=>true, charset=>"UTF-8">, host=>"0.0.0.0", ssl=>false, ssl_verify_mode=>"none", include_codec_tag=>true, ssl_handshake_timeout=>10000, tls_min_version=>1, tls_max_version=>1.2, cipher_suites=>["TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256"], client_inactivity_timeout=>60>_
_ Error: event executor terminated_
_ Exception: Java::JavaUtilConcurrent::RejectedExecutionException_
org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:148) [?:?]_
_ at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:73) [?:?]_
_ at org.jruby.runtime.Block.call(Block.java:124) [?:?]_
_ at org.jruby.RubyProc.call(RubyProc.java:289) [?:?]_
_ at org.jruby.RubyProc.call(RubyProc.java:246) [?:?]_
_ at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:104) [?:?]_
_ at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]_
[2017-12-05T12:42:14,930][ERROR][io.netty.util.concurrent.DefaultPromise.rejectedExecution] Failed to submit a listener notification task. Event loop shut down?
java.util.concurrent.RejectedExecutionException: event executor terminated
_ at io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:840) ~[netty-all-4.1.3.Final.jar:4.1.3.Final]_
_ at io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:342) ~[netty-all-4.1.3.Final.jar:4.1.3.Final]_
_ at io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:335) ~[netty-all-4.1.3.Final.jar:4.1.3.Final]_
_ at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:765) ~[netty-all-4.1.3.Final.jar:4.1.3.Final]_
_ at io.netty.util.concurrent.DefaultPromise.safeExecute(DefaultPromise.java:767) [netty-all-4.1.3.Final.jar:4.1.3.Final]_
_ at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:435) [netty-all-4.1.3.Final.jar:4.1.3.Final]_
_ at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:111) [netty-all-4.1.3.Final.jar:4.1.3.Final]_
_ at io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82) [netty-all-4.1.3.Final.jar:4.1.3.Final]_
_ at io.netty.channel.AbstractChannel$CloseFuture.setClosed(AbstractChannel.java:1062) [netty-all-4.1.3.Final.jar:4.1.3.Final]_
_ at io.netty.channel.AbstractChannel$AbstractUnsafe.register(AbstractChannel.java:486) [netty-all-4.1.3.Final.jar:4.1.3.Final]_
_ at io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:80) [netty-all-4.1.3.Final.jar:4.1.3.Final]_
_ at io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:74) [netty-all-4.1.3.Final.jar:4.1.3.Final]_
_ at io.netty.channel.MultithreadEventLoopGroup.register(MultithreadEventLoopGroup.java:85) [netty-all-4.1.3.Final.jar:4.1.3.Final]_
_ at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:330) [netty-all-4.1.3.Final.jar:4.1.3.Final]_
_ at io.netty.bootstrap.AbstractBootstrap.doBind(AbstractBootstrap.java:281) [netty-all-4.1.3.Final.jar:4.1.3.Final]_
_ at io.netty.bootstrap.AbstractBootstrap.bind(AbstractBootstrap.java:277) [netty-all-4.1.3.Final.jar:4.1.3.Final]_
_ at io.netty.bootstrap.AbstractBootstrap.bind(AbstractBootstrap.java:259) [netty-all-4.1.3.Final.jar:4.1.3.Final]_
_ at org.logstash.beats.Server.listen(Server.java:66) [logstash-input-beats-5.0.2.jar:?]_
_ at sun.reflect.GeneratedMethodAccessor30.invoke(Unknown Source) ~[?:?]_
_ at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_151]_
_ at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_151]_
any help is appreciated. Thank you