# Logstash could not receive log after several minutes

**URL:** <https://discuss.elastic.co/t/logstash-could-not-receive-log-after-several-minutes/297462>\
**Category:** Logstash\
**Created:** [February 17, 2022, 9:19am UTC](https://discuss.elastic.co/t/logstash-could-not-receive-log-after-several-minutes/297462 "2022-02-17T09:19:37Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Stevenssssss](https://avatars.discourse-cdn.com/v4/letter/s/97f17d/32.png) [@Stevenssssss](https://discuss.elastic.co/u/Stevenssssss)\
**Post date:** [February 17, 2022, 9:19am UTC](https://discuss.elastic.co/t/logstash-could-not-receive-log-after-several-minutes/297462/1 "2022-02-17T09:19:37Z")

</div>

Hi, my logstash is able to receive the log when the service is started. However after several minutes, it could not receive the log. I could not find any error message in syslog or logstash-plain.log.

I then run the /usr/share/logstash/bin/logstash --debug -f \<config\_folder\> and get the error log as below (did not stop the service yet):

0001\_input\_windows\_logstash.conf

```auto
input {
  tcp {
    port => 5001
    codec => json_lines
  }

```

9001\_output\_windows\_logstash.conf

```auto
output {
  stdout {
    codec => rubydebug
  }
  rabbitmq {
    automatic_recovery => true
    exchange => "windows"
    exchange_type => "direct"
    host => <host>
    port => "5671"
    key => <key>
    user =><user>
    password => <password>
    durable => true
    persistent => true
    ssl => true
  }

```

error log

```auto
[ERROR] 2022-02-17 08:48:32.404 [[main]<tcp] javapipeline - A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:main
  Plugin: <LogStash::Inputs::Tcp codec=><LogStash::Codecs::JSONLines id=>"json_lines_dc2f227c-160a-4370-af02-87042efccbf9", enable_metric=>true, charset=>"UTF-8", delimiter=>"\n">, port=>5001, id=>"fc09b927c564b634fec8eb6e660087c1c021630a81ac5f445137b5b635867160", enable_metric=>true, host=>"0.0.0.0", mode=>"server", proxy_protocol=>false, ssl_enable=>false, ssl_verify=>true, ssl_key_passphrase=><password>, tcp_keep_alive=>false, dns_reverse_lookup_enabled=>true>
  Error: Address already in use
  Exception: Java::JavaNet::BindException
  Stack: sun.nio.ch.Net.bind0(Native Method)
sun.nio.ch.Net.bind(sun/nio/ch/Net.java:455)
sun.nio.ch.Net.bind(sun/nio/ch/Net.java:447)
sun.nio.ch.ServerSocketChannelImpl.bind(sun/nio/ch/ServerSocketChannelImpl.java:227)
io.netty.channel.socket.nio.NioServerSocketChannel.doBind(io/netty/channel/socket/nio/NioServerSocketChannel.java:134)
io.netty.channel.AbstractChannel$AbstractUnsafe.bind(io/netty/channel/AbstractChannel.java:562)
io.netty.channel.DefaultChannelPipeline$HeadContext.bind(io/netty/channel/DefaultChannelPipeline.java:1334)
io.netty.channel.AbstractChannelHandlerContext.invokeBind(io/netty/channel/AbstractChannelHandlerContext.java:506)
io.netty.channel.AbstractChannelHandlerContext.bind(io/netty/channel/AbstractChannelHandlerContext.java:491)
io.netty.channel.DefaultChannelPipeline.bind(io/netty/channel/DefaultChannelPipeline.java:973)
io.netty.channel.AbstractChannel.bind(io/netty/channel/AbstractChannel.java:260)
io.netty.bootstrap.AbstractBootstrap$2.run(io/netty/bootstrap/AbstractBootstrap.java:356)
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(io/netty/util/concurrent/AbstractEventExecutor.java:164)
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(io/netty/util/concurrent/SingleThreadEventExecutor.java:472)
io.netty.channel.nio.NioEventLoop.run(io/netty/channel/nio/NioEventLoop.java:500)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(io/netty/util/concurrent/SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(io/netty/util/internal/ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(io/netty/util/concurrent/FastThreadLocalRunnable.java:30)
java.lang.Thread.run(java/lang/Thread.java:829)
[DEBUG] 2022-02-17 08:48:33.408 [[main]<tcp] tcp - Closing {:plugin=>"LogStash::Inputs::Tcp"}
[DEBUG] 2022-02-17 08:48:34.478 [pool-5-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2022-02-17 08:48:34.483 [pool-5-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2022-02-17 08:48:36.423 [[main]<tcp] pluginmetadata - Removing metadata for plugin fc09b927c564b634fec8eb6e660087c1c021630a81ac5f445137b5b635867160
[INFO] 2022-02-17 08:48:36.424 [[main]<tcp] tcp - Starting tcp input listener {:address=>"0.0.0.0:5001", :ssl_enable=>false}
[WARN] 2022-02-17 08:48:36.429 [[main]<tcp] AbstractChannel - Force-closing a channel whose registration task was not accepted by an event loop: [id: 0xdb8cbeb6]
java.util.concurrent.RejectedExecutionException: event executor terminated
        at io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:926) ~[logstash-input-tcp-6.2.1.jar:?]
        at io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:353) ~[logstash-input-tcp-6.2.1.jar:?]
        at io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:346) ~[logstash-input-tcp-6.2.1.jar:?]
        at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:828) ~[logstash-input-tcp-6.2.1.jar:?]
        at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:818) ~[logstash-input-tcp-6.2.1.jar:?]
        at io.netty.channel.AbstractChannel$AbstractUnsafe.register(AbstractChannel.java:483) [logstash-input-tcp-6.2.1.jar:?]
        at io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:87) [logstash-input-tcp-6.2.1.jar:?]
        at io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:81) [logstash-input-tcp-6.2.1.jar:?]
        at io.netty.channel.MultithreadEventLoopGroup.register(MultithreadEventLoopGroup.java:86) [logstash-input-tcp-6.2.1.jar:?]
        at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:323) [logstash-input-tcp-6.2.1.jar:?]
        at io.netty.bootstrap.AbstractBootstrap.doBind(AbstractBootstrap.java:272) [logstash-input-tcp-6.2.1.jar:?]
        at io.netty.bootstrap.AbstractBootstrap.bind(AbstractBootstrap.java:268) [logstash-input-tcp-6.2.1.jar:?]
        at io.netty.bootstrap.AbstractBootstrap.bind(AbstractBootstrap.java:253) [logstash-input-tcp-6.2.1.jar:?]
        at org.logstash.tcp.InputLoop.run(InputLoop.java:86) [logstash-input-tcp-6.2.1.jar:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:441) [jruby-complete-9.2.19.0.jar:?]
        at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:305) [jruby-complete-9.2.19.0.jar:?]
        at org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:32) [jruby-complete-9.2.19.0.jar:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_tcp_minus_6_dot_2_dot_1_minus_java.lib.logstash.inputs.tcp.RUBY$method$run$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-tcp-6.2.1-java/lib/logstash/inputs/tcp.rb:160) [jruby-complete-9.2.19.0.jar:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_tcp_minus_6_dot_2_dot_1_minus_java.lib.logstash.inputs.tcp.RUBY$method$run$0$ __VARARGS__ (/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-tcp-6.2.1-java/lib/logstash/inputs/tcp.rb) [jruby-complete-9.2.19.0.jar:?]
        at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80) [jruby-complete-9.2.19.0.jar:?]
        at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70) [jruby-complete-9.2.19.0.jar:?]
        at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207) [jruby-complete-9.2.19.0.jar:?]
        at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$inputworker$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:405) [jruby-complete-9.2.19.0.jar:?]
        at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$inputworker$0$ __VARARGS__ (/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb) [jruby-complete-9.2.19.0.jar:?]
        at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80) [jruby-complete-9.2.19.0.jar:?]
        at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70) [jruby-complete-9.2.19.0.jar:?]
        at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207) [jruby-complete-9.2.19.0.jar:?]
        at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$block$start_input$1(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:396) [jruby-complete-9.2.19.0.jar:?]
        at org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138) [jruby-complete-9.2.19.0.jar:?]
        at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58) [jruby-complete-9.2.19.0.jar:?]
        at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52) [jruby-complete-9.2.19.0.jar:?]
        at org.jruby.runtime.Block.call(Block.java:139) [jruby-complete-9.2.19.0.jar:?]
        at org.jruby.RubyProc.call(RubyProc.java:318) [jruby-complete-9.2.19.0.jar:?]
        at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105) [jruby-complete-9.2.19.0.jar:?]
        at java.lang.Thread.run(Thread.java:829) [?:?]

```

I have restarted the service with debug mode. I will paste the log once the logstash fails to receive the incoming log again.

Thanks.

---

<div class="post-metadata">

**Author:** ![Stevenssssss](https://avatars.discourse-cdn.com/v4/letter/s/97f17d/32.png) [@Stevenssssss](https://discuss.elastic.co/u/Stevenssssss)\
**Post date:** [February 17, 2022, 9:37am UTC](https://discuss.elastic.co/t/logstash-could-not-receive-log-after-several-minutes/297462/2 "2022-02-17T09:37:02Z")

</div>

After several minutes, the debug log will have such output as below:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/4/747cbf3c6f0ab89687555230510ab3ce24161a2c.png)

It will stop the log ingestion and keep giving out the repeated output:

```auto
[DEBUG] 2022-02-17 09:32:05.223 [pool-6-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2022-02-17 09:32:05.223 [pool-6-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2022-02-17 09:32:10.227 [pool-5-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2022-02-17 09:32:10.227 [pool-5-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2022-02-17 09:32:15.232 [pool-6-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2022-02-17 09:32:15.232 [pool-6-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2022-02-17 09:32:20.236 [pool-5-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2022-02-17 09:32:20.236 [pool-5-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2022-02-17 09:32:25.240 [pool-6-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2022-02-17 09:32:25.241 [pool-6-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2022-02-17 09:32:30.244 [pool-5-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2022-02-17 09:32:30.245 [pool-5-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2022-02-17 09:32:35.248 [pool-6-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2022-02-17 09:32:35.249 [pool-6-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2022-02-17 09:32:40.253 [pool-5-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2022-02-17 09:32:40.253 [pool-5-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2022-02-17 09:32:45.258 [pool-6-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2022-02-17 09:32:45.258 [pool-6-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2022-02-17 09:32:50.262 [pool-5-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2022-02-17 09:32:50.262 [pool-5-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2022-02-17 09:32:55.266 [pool-6-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2022-02-17 09:32:55.266 [pool-6-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 17, 2022, 9:37am UTC](https://discuss.elastic.co/t/logstash-could-not-receive-log-after-several-minutes/297462/3 "2022-03-17T09:37:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.