UDP listener died >>IOError: closed stream

Elastic Stack Logstash
1

hi,
the disk filled up in ELK server on 23/1 , I dealt with that by deleteing indices and then installed curator so the same situation would
not arise again.

However since 23/1 no new indices are created, even though I have restarted ELK and the centos server.Hence no input of syslogs into kibana.

Below is an error output with UDP listerner died error.

This error message with this exception has not already been dealt with on discuss.elastic.co.

[root@logstash1 logstash]# cat logstash.log-20170125
{:timestamp=>"2017-01-24T11:30:36.419000+0000", :message=>"Pipeline main started"}
{:timestamp=>"2017-01-24T12:47:18.358000+0000", :message=>"SIGTERM received. Shutting down the agent.", :level=>:warn}
{:timestamp=>"2017-01-24T12:47:18.363000+0000", :message=>"stopping pipeline", :id=>"main"}
{:timestamp=>"2017-01-24T12:47:18.746000+0000", :message=>"UDP listener died", :exception=>#<IOError: closed stream>, :backtrace=>["org/jruby/RubyIO.java:3705:in select'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-udp-2.0.5/lib/logstash/inputs/udp.rb:77:inudp_listener'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-udp-2.0.5/lib/logstash/inputs/udp.rb:50:in run'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/pipeline.rb:342:ininputworker'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/pipeline.rb:336:in `start_input'"], :level=>:warn}
{:timestamp=>"2017-01-24T12:47:20.019000+0000", :message=>"Pipeline main has been shutdown"}
{:timestamp=>"2017-01-24T12:48:18.697000+0000", :message=>"Pipeline main started"}
[root@logstash1 logstash]#

[root@logstash1 inputs]# curator_cli show_indices --header
.kibana
logstash-2017.01.23
[root@logstash1 inputs]#

[root@logstash1 logstash]# netstat -nlu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
udp 0 0 127.0.0.1:3030 0.0.0.0:*
udp 0 0 127.0.0.1:323 0.0.0.0:*
udp 0 0 0.0.0.0:514 0.0.0.0:*
udp6 0 0 ::1:323 :::*
[root@logstash1 logstash]#
[root@logstash1 logstash]# netstat -an | grep 514
udp 0 0 0.0.0.0:514 0.0.0.0:*
[root@logstash1 logstash]#
[root@logstash1 logstash]#

Can anyone assist ?

br,
Daragh

2

syslog messages are still coming into the server, since I can see them in TCP dump today.

3

kibana version 4.6.1 build 10146
logstash version 2.4.0
elasticsearch version 2.4.0

4

Please post your Logstash log as preformatted text. Please also consider upgrading Logstash since 2.4.0 has a bug preventing the error message from being logged.

It looks like SIGTERM comes before the "UDP listener died" error. Are you sure the UDP problem isn't a consequence of the SIGTERM signal rather than the cause of it?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.