Filebeat doesn't send logs to logstash in client server architecture


(Josh Star Long) #1

Hi all,

I just set up filebeat following the documentation. But I can't have logs of my cluster in logstash. I use a client-server architecture (i.e a client with Filebeat configured and installed and a server with logstash and elasticsearch) The client and the server are not in the same subnet so I used domain name to reach logstash in the Server instance.

Here is my /etc/filebeat/filebeat.yml config file:

`filebeat:
  prospectors:
    -
      paths:
        - "/var/log/*.log"

    input_type: log

output:
  logstash:
    hosts: ["filebeatdomainname.com:5044"]
    index: indexname

shipper:

logging:
  files:
  rotateeverybytes: 10485760 # = 10MB`

And on logstash side, I just configured a logstash.conf file under /etc/logstash/conf.d/logstash.conf:

`input {
 beats {
   port => 5044
}
}

output {
elasticsearch {
hosts => "172.31.62.244:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}`

Please, Were I am wrong?

Thanks in advance!


(Steffen Siering) #2

Any log output from filebeat/logstash?


(Josh Star Long) #3

Yes, Any log.


(Josh Star Long) #4

I means no logs in logstash and no index in elasticsearch


(Steffen Siering) #5

can you post filebeat log output?


(Steffen Siering) #6

Check out filebeat troubleshooting/faq: https://www.elastic.co/guide/en/beats/filebeat/current/faq.html#filebeat-not-collecting-lines


(Josh Star Long) #7

I don't know where are filebeat log output. I expected it to be on /var/log/filebeat but I don't have it here. Please, do you know where I can see them?


(Josh Star Long) #8

Hey @steffens, Here is what I have in the logs

2016/03/20 17:26:23.768081 output.go:87: DBG output worker: publish 177 events 2016/03/20 17:26:23.768272 client.go:136: DBG Try to publish 177 events to logstash with window size 10 2016/03/20 17:26:23.890194 client.go:95: DBG close connection 2016/03/20 17:26:23.890362 client.go:114: DBG 0 events out of 177 events sent to logstash. Continue sending ... 2016/03/20 17:26:23.890398 single.go:76: INFO Error publishing events (retrying): EOF 2016/03/20 17:26:23.890423 single.go:152: INFO send fail 2016/03/20 17:26:23.890447 single.go:159: INFO backoff retry: 1s 2016/03/20 17:26:24.241759 reader.go:138: DBG End of file reached: /var/log/auth.log; Backoff now. 2016/03/20 17:26:24.241951 reader.go:138: DBG End of file reached: /var/log/cloud-init-output.log; Backoff now. 2016/03/20 17:26:24.242127 reader.go:138: DBG End of file reached: /var/log/boot.log; Backoff now. 2016/03/20 17:26:24.242734 reader.go:138: DBG End of file reached: /var/log/dpkg.log; Backoff now. 2016/03/20 17:26:24.242906 reader.go:138: DBG End of file reached: /var/log/cloud-init.log; Backoff now. 2016/03/20 17:26:24.243992 reader.go:138: DBG End of file reached: /var/log/kern.log; Backoff now. 2016/03/20 17:26:24.890672 client.go:90: DBG connect 2016/03/20 17:26:25.012786 client.go:136: DBG Try to publish 177 events to logstash with window size 5 2016/03/20 17:26:25.136129 client.go:95: DBG close connection 2016/03/20 17:26:25.136279 client.go:114: DBG 0 events out of 177 events sent to logstash. Continue sending ... 2016/03/20 17:26:25.136319 single.go:76: INFO Error publishing events (retrying): EOF 2016/03/20 17:26:25.136345 single.go:152: INFO send fail


(Josh Star Long) #9

Issue fixed. Thank you for your tip. After reading deeply the logs I saw that Filebeat was not able to communicate directly with logstash. So in logstash logs I saw a warning. I updated the beats plugin and everything works fine. Thanks again!!!!


#10

yo dawg. same problem, that link is rotten now though. what was the secret sauce?


(Andrew Kroh) #11

Here's the updated link: https://www.elastic.co/guide/en/beats/filebeat/current/faq.html


(system) #12