# Logstash TCP and Syslog Plugin Error

**URL:** <https://discuss.elastic.co/t/logstash-tcp-and-syslog-plugin-error/330722>\
**Category:** Logstash\
**Created:** [April 25, 2023, 9:37am UTC](https://discuss.elastic.co/t/logstash-tcp-and-syslog-plugin-error/330722 "2023-04-25T09:37:39Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![hanna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hanna/32/109117_2.png) [@hanna](https://discuss.elastic.co/u/hanna)\
**Post date:** [April 25, 2023, 9:37am UTC](https://discuss.elastic.co/t/logstash-tcp-and-syslog-plugin-error/330722/1 "2023-04-25T09:37:39Z")

</div>

Hello,

I'm experiencing a Logstash error with the syslog input plugin. The input plugin for my pipeline keeps crashing with the message `Force-closing a channel whose registration task was not accepted by an event loop` and `A plugin had an unrecoverable error. Will restart this plugin.`  
This results in only the udp port being opened but not both tcp and udp (which should be the normal behaviour if I'm not mistaken).  
There is no other pipeline that is configured to use this port, so there shouldn't be any conflicts.

This is the error I get from logstash:

```auto
[2023-04-25T10:58:19,696][INFO][logstash.inputs.tcp][my_log_source][d443af5bc08bf4f9af84081172b5891f490cb8e57db0a7e2575e719f4143fbc6] Starting tcp input listener {:address=>"192.168.0.35:49166", :ssl_enable=>false}
[2023-04-25T10:58:19,697][WARN][io.netty.channel.AbstractChannel][my_log_source][d443af5bc08bf4f9af84081172b5891f490cb8e57db0a7e2575e719f4143fbc6] Force-closing a channel whose registration task was not accepted by an event loop: [id: 0x7b1445e1]
java.util.concurrent.RejectedExecutionException: event executor terminated
        at io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:934) ~[netty-common-4.1.87.Final.jar:4.1.87.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:351) ~[netty-common-4.1.87.Final.jar:4.1.87.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:344) ~[netty-common-4.1.87.Final.jar:4.1.87.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:836) ~[netty-common-4.1.87.Final.jar:4.1.87.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor.execute0(SingleThreadEventExecutor.java:827) ~[netty-common-4.1.87.Final.jar:4.1.87.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:817) ~[netty-common-4.1.87.Final.jar:4.1.87.Final]
        at io.netty.channel.AbstractChannel$AbstractUnsafe.register(AbstractChannel.java:483) [netty-transport-4.1.87.Final.jar:4.1.87.Final]
        at io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:89) [netty-transport-4.1.87.Final.jar:4.1.87.Final]
        at io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:83) [netty-transport-4.1.87.Final.jar:4.1.87.Final]
        at io.netty.channel.MultithreadEventLoopGroup.register(MultithreadEventLoopGroup.java:86) [netty-transport-4.1.87.Final.jar:4.1.87.Final]
        at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:323) [netty-transport-4.1.87.Final.jar:4.1.87.Final]
        at io.netty.bootstrap.AbstractBootstrap.doBind(AbstractBootstrap.java:272) [netty-transport-4.1.87.Final.jar:4.1.87.Final]
        at io.netty.bootstrap.AbstractBootstrap.bind(AbstractBootstrap.java:268) [netty-transport-4.1.87.Final.jar:4.1.87.Final]
        at io.netty.bootstrap.AbstractBootstrap.bind(AbstractBootstrap.java:253) [netty-transport-4.1.87.Final.jar:4.1.87.Final]
        at org.logstash.tcp.InputLoop.run(InputLoop.java:86) [logstash-input-tcp-6.3.2.jar:?]
        at jdk.internal.reflect.GeneratedMethodAccessor153.invoke(Unknown Source) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
        at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:442) [jruby.jar:?]
        at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:306) [jruby.jar:?]
        at org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:32) [jruby.jar:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.logstash_minus_input_minus_tcp_minus_6_dot_3_dot_2_minus_java.lib.logstash.inputs.tcp.RUBY$method$run$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-input-tcp-6.3.2-java/lib/logstash/inputs/tcp.rb:167) [?:?]
        at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:165) [jruby.jar:?]
        at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:185) [jruby.jar:?]
        at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:218) [jruby.jar:?]
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:173) [jruby.jar:?]
        at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:316) [jruby.jar:?]
        at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72) [jruby.jar:?]
        at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:86) [jruby.jar:?]
        at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:201) [jruby.jar:?]
        at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:188) [jruby.jar:?]
        at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:218) [jruby.jar:?]
        at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:372) [jruby.jar:?]
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:175) [jruby.jar:?]
        at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:316) [jruby.jar:?]
        at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72) [jruby.jar:?]
        at org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:116) [jruby.jar:?]
        at org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:136) [jruby.jar:?]
        at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:66) [jruby.jar:?]
        at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58) [jruby.jar:?]
        at org.jruby.runtime.Block.call(Block.java:143) [jruby.jar:?]
        at org.jruby.RubyProc.call(RubyProc.java:309) [jruby.jar:?]
        at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:107) [jruby.jar:?]
        at java.lang.Thread.run(Thread.java:833) [?:?]
[2023-04-25T10:58:19,698][ERROR][logstash.javapipeline][my_log_source][d443af5bc08bf4f9af84081172b5891f490cb8e57db0a7e2575e719f4143fbc6] A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:my_log_source
  Plugin: <LogStash::Inputs::Tcp host=>"192.168.0.35", id=>"d443af5bc08bf4f9af84081172b5891f490cb8e57db0a7e2575e719f4143fbc6", port=>49166, tags=>["my_log_source"], enable_metric=>true, codec=><LogStash::Codecs::Line id=>"line_ec431109-d7f2-4cb3-91d9-ae831163ee41", enable_metric=>true, charset=>"UTF-8", delimiter=>"\n">, mode=>"server", proxy_protocol=>false, ssl_enable=>false, ssl_verify=>true, ssl_key_passphrase=><password>, tcp_keep_alive=>false, dns_reverse_lookup_enabled=>true>
  Error: event executor terminated
  Exception: Java::JavaUtilConcurrent::RejectedExecutionException
  Stack: io.netty.util.concurrent.SingleThreadEventExecutor.reject(io/netty/util/concurrent/SingleThreadEventExecutor.java:934)
io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(io/netty/util/concurrent/SingleThreadEventExecutor.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor.addTask(io/netty/util/concurrent/SingleThreadEventExecutor.java:344)
io.netty.util.concurrent.SingleThreadEventExecutor.execute(io/netty/util/concurrent/SingleThreadEventExecutor.java:836)
io.netty.util.concurrent.SingleThreadEventExecutor.execute0(io/netty/util/concurrent/SingleThreadEventExecutor.java:827)
io.netty.util.concurrent.SingleThreadEventExecutor.execute(io/netty/util/concurrent/SingleThreadEventExecutor.java:817)
io.netty.channel.AbstractChannel$AbstractUnsafe.register(io/netty/channel/AbstractChannel.java:483)
io.netty.channel.SingleThreadEventLoop.register(io/netty/channel/SingleThreadEventLoop.java:89)
io.netty.channel.SingleThreadEventLoop.register(io/netty/channel/SingleThreadEventLoop.java:83)
io.netty.channel.MultithreadEventLoopGroup.register(io/netty/channel/MultithreadEventLoopGroup.java:86)
io.netty.bootstrap.AbstractBootstrap.initAndRegister(io/netty/bootstrap/AbstractBootstrap.java:323)
io.netty.bootstrap.AbstractBootstrap.doBind(io/netty/bootstrap/AbstractBootstrap.java:272)
io.netty.bootstrap.AbstractBootstrap.bind(io/netty/bootstrap/AbstractBootstrap.java:268)
io.netty.bootstrap.AbstractBootstrap.bind(io/netty/bootstrap/AbstractBootstrap.java:253)
org.logstash.tcp.InputLoop.run(org/logstash/tcp/InputLoop.java:86)
jdk.internal.reflect.GeneratedMethodAccessor153.invoke(jdk/internal/reflect/GeneratedMethodAccessor153)
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(jdk/internal/reflect/DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:568)
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:442)
org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:306)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.logstash_minus_input_minus_tcp_minus_6_dot_3_dot_2_minus_java.lib.logstash.inputs.tcp.run(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-input-tcp-6.3.2-java/lib/logstash/inputs/tcp.rb:167)
RUBY.inputworker(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:414)
RUBY.start_input(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:405)
org.jruby.RubyProc.call(org/jruby/RubyProc.java:309)
java.lang.Thread.run(java/lang/Thread.java:833)

```

The problem also occurs with other pipelines, as if they were taking turns, and there is always at least one pipeline, that throws these errors. Sometimes it also happens with the tcp input plugin, but never with udp. I've already tried restarting the server (Ubuntu 22.04 lts) but the issue persists.

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [April 25, 2023, 11:02am UTC](https://discuss.elastic.co/t/logstash-tcp-and-syslog-plugin-error/330722/2 "2023-04-25T11:02:43Z")

</div>

Which LS version do you run? 7.x, 8.x?

There was an issue with [the Netty lib](https://github.com/elastic/logstash/issues/11908).

---

<div class="post-metadata">

**Author:** ![hanna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hanna/32/109117_2.png) [@hanna](https://discuss.elastic.co/u/hanna)\
**Post date:** [April 25, 2023, 1:20pm UTC](https://discuss.elastic.co/t/logstash-tcp-and-syslog-plugin-error/330722/3 "2023-04-25T13:20:39Z")

</div>

I am using the latest logstash version 8.7.0.  
According to the github issue this shouldn't be a problem anymore with the current version. The error seems to be the same though...

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [April 25, 2023, 7:47pm UTC](https://discuss.elastic.co/t/logstash-tcp-and-syslog-plugin-error/330722/4 "2023-04-25T19:47:13Z")

</div>

Are you using syslog or pure tcp plugin?  
According to the log, _logstash.inputs.tcp_, you are using the input tcp plugin.  
Try with syslog, tcp and udp plugins.  
Since only TCP is making problems, try with: `tcp_keep_alive=>true`, default is false.  
Also check will help `pipeline.workers: 1`

---

<div class="post-metadata">

**Author:** ![hanna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hanna/32/109117_2.png) [@hanna](https://discuss.elastic.co/u/hanna)\
**Post date:** [May 9, 2023, 1:26pm UTC](https://discuss.elastic.co/t/logstash-tcp-and-syslog-plugin-error/330722/5 "2023-05-09T13:26:55Z")

</div>

It happens with the tcp and the syslog input plugin. The udp Plugin works fine.  
I tried setting _pipeline.workers_ to a lower value than the default and the problem seems to be fixed now.  
Thank you @Rios!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 6, 2023, 1:27pm UTC](https://discuss.elastic.co/t/logstash-tcp-and-syslog-plugin-error/330722/6 "2023-06-06T13:27:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.