FileBeat harvester doesn't work on the defined path with 777 permission

I was deployed ELK on local host and filebeat on remote host in order to fetch log from remote to local by using docker-compose for ELK and docker run for filebeat.

The flow will be like Filebeat(remote) -> Logstash (local) -> Elasticsearch(local) -> kibana(local)

The connection between Logstash and filebeat is great. Also the connection among ELK stack themselves.

At first, I've set multiple input paths in filebeat

  • /var/log/*.log
  • /var/log/message/.*log
  • /deploy/tomcat/log/.*log

from these configuration, Kibana is able to discover a message but only for the first two paths. the /deploy/tomcat/log/.*log. So, I gave the permission (recursively) 777 to this path. Unfortunately, Filebeat harvester still doesn't work and keep saying

Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":860},"total":{"ticks":1550,"time":{"ms":4},"value":1550},"user":{"ticks":690,"time":{"ms":4}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral_id":"8877057c-bb0f-4363-b935-4db908c030e9","uptime":{"ms":4770025}},"memstats":{"gc_next":7600432,"memory_alloc":4448944,"memory_total":56104688},"runtime":{"goroutines":17}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.36,"15":0.46,"5":0.45,"norm":{"1":0.0225,"15":0.0288,"5":0.0281}}}}}}

Here's my filebeat configuration

   - type: log
     enabled: true
       - "/deploy/tomcat/logs/*.log"

    hosts: ["http://x.x.x.x:5000"]

logging.level: debug

(my Logstash is running with port 5000)

What should I do next ? I've tried to change the configuration but result remain the same.

In your post you say the path is /deploy/tomcat/log/*.log but in your config your refer to /deploy/tomcat/logs/*.log

If this was not a typo in this post, thats your error.

The excerpt from the filebeat you posted by the way is not an error message, its just some metrics he sends.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.