Filebeat doesn't harvest

Elastic Stack Beats
1

Hello,

I've been facing a probem since a while now. My filebeat plugin does not harvest the fields that i ask him to harvest in my conf file. I'm using filebeat 7.6.0

My conf:

filebeat.inputs:
- type: log
  paths:
    - /var/log/user.log
    - /var/log/slapd.log
output.kafka:
  hosts: ["kafka1:9092"]
  topic: 'log'
  partition.round_robin:
    reachable_only: false
  required_acks: 1
  compression: gzip
  max_message_bytes: 10000000

The log output:

|2020-02-14T07:55:58.664Z|INFO|crawler/crawler.go:72|Loading Inputs: 1,
|2020-02-14T07:55:58.665Z|INFO|log/input.go:152|Configured paths: [/var/log/user.log /var/log/slapd.log],|
|2020-02-14T07:55:58.665Z|INFO|input/input.go:114|Starting input of type: log; ID: 6297130742941599674 ,|
|2020-02-14T07:55:58.665Z|INFO|crawler/crawler.go:106|Loading and starting Inputs completed. Enabled inputs: 1,
|2020-02-14T07:56:00.664Z	INFO	[monitoring]	log/log.go:145	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":250,"time":{"ms":8}},"total":{"ticks":390,"time":{"ms":16},"value":390},"user":{"ticks":140,"time":{"ms":8}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":6},"info":{"ephemeral_id":"a601336e-8252-460f-9a25-f05dad5851b2","uptime":{"ms":480275}},"memstats":{"gc_next":8594432,"memory_alloc":5169696,"memory_total":17158072},"runtime":{"goroutines":20}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.21,"15":1.06,"5":0.88,"norm":{"1":0.105,"15":0.53,"5":0.44}}}}}}

I've been trying to solve it by making it harvest various files. But without success so far. I always get this "filebeat":{"harvester":{"open_files":0,"running":0}}

Thanks!

2

The most likely problem if you aren't seeing any harvesters is that Filebeat can't see the target files, either because of a typo or a permissions issue. If you're on linux or mac you might try running (with the same user you use to run Filebeat) head /var/log/user.log to confirm that the file exists and is readable from the path given.

3

Thank for the reply.

I tried the head /var/log/user.log. I was able to see inside the file with the same user as the one I'm launching my docker-compose with (I switched to root so that I'm sure there is no permission issue) But no luck so far, still 0 file under harvest.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.