Docker gelf udp to logstash-elasticsearch-kibana

I need someone to please verify my simple config because I'm not seeing the logs in Kibana like I thought. It's very simple; here's what I have. I would think the container I run on the daemon; whatever it logs would end up in Kibana.

Server1 - docker daemon configured with log driver to udp gelf messages to Server2
Server2 - logstash, elasticsearch, kibana all running and communicating with each other.

I'm running RHEL 7 on both servers(VM's).

GELF+logstash ->I verified all the communication SEEMS to be correct. I can manually send a UDP message from server1 and have the logs in logstash on server2 complain it can't parse the UDP message (because I don't send the manual UDP message in GELF format). But that at least tells me server1 and 2 can interact via UDP on the port I specified.

elasticsearch -> I can hit that and get back that generic JSON response in the brower

kibana -> I have the index to look for logstash messages and at one point I saw some logs in kibana. I haven't changed anything but I don't see any logs.

However I don't see any plain docker logs in kibana (whatever those would be; I've never seen any, not even daemon start-up messages) and when I run a container on server1 whatever it logs never ends up in server2's kibana.

Can someone at least read this and verify I have the basic setup of docker and gelf etc correct if all my ports and routes are working?

docker config =
ExecStart=/usr/bin/dockerd --log-driver gelf --log-opt gelf-address=udp://server2:8202

logstash config (on server2):
input {
gelf { port => 8202 }
output {
elasticsearch { hosts => ["localhost:8200"]}

elasticsearch { hosts => ["localhost:8200"]}

ES normally listens on port 9200.

Thanks for pointing that out! Apologies for not including the relevant elasticsearch.yml config. Here it is:
http.port: 8200

I'm glad you didn't see anything wrong with the other config items I have though. That's a good sign!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.