We are using file beat version 5.6 to tail logs from all the docker containers of a machine and sending them to logstash.
Here is the config we have for file beat.
- input_type: log
Earlier we were using swarm as a orchestration tool with that each node was having 40-50 containers, the above configuration was working fine , we didn't see any issues with that.
After we moved to kubernetes the number of containers on the node is doubled. With that the file beat is stop harvesting or sending logs to logstash from few containers. We restart the filebeat on the specific node to send the logs to logstash again.After some time we are seeing the same issue again.
filebeat registry timestamp says it is tailed all the logs.