FileBeat harvester doesn't work on the defined path with 777 permission

I was deployed ELK on local host and filebeat on remote host in order to fetch log from remote to local by using docker-compose for ELK and docker run for filebeat.

The flow will be like Filebeat(remote) -> Logstash (local) -> Elasticsearch(local) -> kibana(local)

The connection between Logstash and filebeat is great. Also the connection among ELK stack themselves.

At first, I've set multiple input paths in filebeat

  • /var/log/*.log
  • /var/log/message/.*log
  • /deploy/tomcat/log/.*log

from these configuration, Kibana is able to discover a message but only for the first two paths. the /deploy/tomcat/log/.*log. So, I gave the permission (recursively) 777 to this path. Unfortunately, Filebeat harvester still doesn't work and keep saying

Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":860},"total":{"ticks":1550,"time":{"ms":4},"value":1550},"user":{"ticks":690,"time":{"ms":4}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral_id":"8877057c-bb0f-4363-b935-4db908c030e9","uptime":{"ms":4770025}},"memstats":{"gc_next":7600432,"memory_alloc":4448944,"memory_total":56104688},"runtime":{"goroutines":17}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.36,"15":0.46,"5":0.45,"norm":{"1":0.0225,"15":0.0288,"5":0.0281}}}}}}

Here's my filebeat configuration

   - type: log
     enabled: true
       - "/deploy/tomcat/logs/*.log"

    hosts: ["http://x.x.x.x:5000"]

logging.level: debug

(my Logstash is running with port 5000)

What should I do next ? I've tried to change the configuration but result remain the same.

In your post you say the path is /deploy/tomcat/log/*.log but in your config your refer to /deploy/tomcat/logs/*.log

If this was not a typo in this post, thats your error.

The excerpt from the filebeat you posted by the way is not an error message, its just some metrics he sends.