Latest filebeat binary failing

Logs are flowing fine to logstash-2.4.1 if I am using filebeat-1.2.3 binary. However as part of upgradation when I am trying latest filebeat binary 5.1.2. Filebeat is failing with error

2016-12-26T03:42:46-08:00 DBG connect
2016-12-26T03:42:46-08:00 DBG Try to publish 2002 events to logstash with window size 1
2016-12-26T03:42:46-08:00 DBG handle error: EOF
2016-12-26T03:42:46-08:00 DBG closing
2016-12-26T03:42:46-08:00 DBG 0 events out of 2002 events sent to logstash. Continue sending
2016-12-26T03:42:46-08:00 DBG close connection
2016-12-26T03:42:46-08:00 ERR Failed to publish events caused by: EOF
2016-12-26T03:42:46-08:00 INFO Error publishing events (retrying): EOF
2016-12-26T03:42:46-08:00 DBG close connection
2016-12-26T03:42:46-08:00 DBG send fail

I looked at the version matching matrix and it says filebeat-5.1 will work with logstash-2.4.1. Error looks like a network issue, but if I use 1.2.3 binary its working good. Is there any configuration specifications required specially for the latest filebeat?.

What logstash-input-beats plugin version do you have? Try to upgrade it, please.

I am using logstash-2.4.1 with recent 3.1.2 logstash-input-beats plugin. I am using 5.1.2 filebeat binary. Following is my logstash config.

input {
beats {
port => 443
ssl => true
ssl_certificate_authorities => ["/tmp/cacert_filebeat.pem"]
ssl_certificate => "/tmp/"
ssl_key => "/tmp/"
ssl_verify_mode => "force_peer"
filter {
output {
stdout { }
elasticsearch {
hosts => [ "" ]
index => "heartbeatlogs_%{+YYYY.MM.dd}"

Following is my filebeat config.

- /var/log/heartbeat.log
hostip: ""
cloudname: ""
document_type: heartbeat_Etc/GMT+0
close_older: 5m

hosts: [""]
loadbalance: true
certificate_authorities: ["/sc/filebeat/certs/cacert_logstash.pem"]
certificate: "/sc/filebeat/certs/fbt-zsn.crt"
certificate_key: "/sc/filebeat/certs/fbt-zsn.key"

to_syslog: false
to_files: true
level: debug
path: /sc/filebeat
name: auditlog.log
rotateeverybytes: 10485760
keepfiles: 2

This is a config issue. Its working fine with the following config modification

hosts: [""]
ssl.certificate_authorities: ["/sc/filebeat/certs/cacert_logstash.pem"]
ssl.certificate: "/sc/filebeat/certs/fbt-zsn.crt"
ssl.key: "/sc/filebeat/certs/fbt-zsn.key"


I'd recommend to update the logstash-input-beats plugin to the more recent version. The 3.x plugin version is a complete rewrite and there have been some critical bug fixes since version 3.1.2.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.