Hi,
I installed filebeat-full
using the brew command from the documentation. I changed the config file(present at /usr/local/etc/filebeat/filebeat.yml
). I started the service using brew services start filebeat-full
. But I am not seeing logs getting harvested and pushed into Kafka. Checked the logs at /usr/local/var/log/filebeat
. There are no errors.
But when I start as a foreground process using filebeat -e -d "*"
, I am able to see that the events are getting published. And I am able to see them in Kibana.
I am interested in running this as a daemon. I don't want to have to start this manually each time a mac restarts or if someone accidentally stops this process.
Mac OS version: 10.15.5
Filebeat version from homebrew: 7.8.0
Filebeat configuration file :
#=========================== Filebeat prospectors =============================
filebeat.inputs:
-
paths:
- /Users/myuser/Desktop/Nightly/a/*.log
ignore_older: 48h
close_inactive: 5m
clean_inactive: 49h
multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
multiline.negate: true
multiline.match: after
processors:
- add_fields:
fields:
kafka_topic: mykafkatopic
#================================ Outputs =====================================
#------------------------------- Kafka output ----------------------------------
output.kafka:
enabled: true
hosts: ["Kafka.mydomain.COM:9092"]
topic: '%{[fields.kafka_topic]}'
partition.hash:
reachable_only: false
#================================ Logging =====================================
logging:
to_files: true
files:
rotateeverybytes: 10485760 # = 10MB
selectors: ["*"]
level: debug
Please help me in debugging this issue.
Thanks,
Venkat.