Filebeat - last handler in the pipeline did not handle the exception

[irom77]

I am getting below error with filebeat/logstash.

[2018-09-04T12:05:09,164][WARN ][io.netty.channel.DefaultChannelPipeline] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.

See my config below. I use port 5045, instead default one

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - C:\Program Files (x86)\Palo Alto Networks\User-ID Agent\UaDebug.log
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true
  reload.period: 10s
setup.template.settings:
  index.number_of_shards: 3
name: dc
tags: ["user-id"]
env: production
setup.kibana:
output.logstash: 
  hosts: ["1.1.1.1:5045"]

input {
        beats {
                port => 5045
                type => "userid"             
        }
}

filter {

}
output {
        if [type] == "userid" {
                elasticsearch {
                        hosts => "elasticsearch:9200"
                        index => "index-pauserid-%{+YYYY.MM.dd}"
                }                
        }
}

[RobBavey]

Hi @irom77,

Is there anything else in the log file - any more log entries either above or below that warning? How regularly are you seeing the warning?

The warning is likely benign, but if there is any more information, I can take a look to make sure.

Thanks,

Rob

[irom77]

There was this....:
[2018-09-04T12:36:36,907][INFO ][org.logstash.beats.BeatsHandler] [local: 0.0.0.0:5045, remote: 10.41.1.151:64771] Handling exception: Connection timed out
logstash_1 | [2018-09-04T12:36:36,908][WARN ][io.netty.channel.DefaultChannelPipeline] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
logstash_1 | java.io.IOException: Connection timed out
logstash_1 | at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[?:1.8.0_171]
logstash_1 | at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[?:1.8.0_171]
logstash_1 | at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[?:1.8.0_171]
logstash_1 | at sun.nio.ch.IOUtil.read(IOUtil.java:192) ~[?:1.8.0_171]
logstash_1 | at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) ~[?:1.8.0_171]
logstash_1 | at io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:288) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
logstash_1 | at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1108) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
logstash_1 | at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:345) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
logstash_1 | at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:126) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
logstash_1 | at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
logstash_1 | at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
logstash_1 | at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
logstash_1 | at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459) [netty-all-4.1.18.Final.jar:4.1.18.Final]
logstash_1 | at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-all-4.1.18.Final.jar:4.1.18.Final]
logstash_1 | at io.netty.util.concurrent.FastThreadLocalRunnable

[RobBavey]

Hi @irom77

Looks like you are having some connectivity issues between your beats node and Logstash. How often do you see this message in your logstash logs? Are there any errors in the beats logs too?

Thanks,

Rob