Filebeat not monitoring data

Hi Friends,

I set up Elasticsearch in our Lab, there in filebeat.yml I given the following configuration

filebeat:
paths:
- /var/log/*
document_type: syslog
scan_frequency: 10s

logstash:
# The Logstash hosts
hosts: ["...:5044"]

even though Scan_frequency:10s, Filebeat is not Monitoring files.
It just Adding the files at very first time when it see new file path is given.

Could anyone help me to sort it out.

Please post full logs, i.e. start Filebeat with -v -d "*".

(I suspect -v is superfluous when -d is specified but it won't hurt.)

Hi Magnus, Please find the below lines are in my Filebeat.yml file.

filebeat:
prospectors:
paths:
- /var/log/*
type: log
paths:
- /var/log/cron
type: cron_log
document_type: syslog
scan_frequency: 10s
output:
elasticsearch:
hosts: ["localhost:9200"]
logstash:
hosts: ["ip_add:5044"] // here output as Logstash
shipper:
logging:
files:

this is total file , means remaining all lines contains # front of them.

File is adding, I can discover through kibana.
but, later changes in files are not monitoring by Filebeat.

I asked for logs, not your configuration (although it's good to have the full configuration, but please make sure you format it as code since YAML is indentation-sensitive and this forum strips leading whitespace).

why have you configured both, elasticsearch and logstash output? This on purpose?

Hi Magnus, I would like to monitor syslog. and I didn't found syslog so I given the path like,

paths:

  • /var/log/*

in that folder all the below log files are there. problem is eventhough scan_frequency is 10s , it is not monitoring.

anaconda.log, audit, boot.log, btmp, cron, cups, dmesg, dracut.log, gdm, httpd, kern, maillog, mcelog, messages, mysqld.log, pm-powersave.log, sa-update.log, secure, spooler, tallylog, up2date, wtmp, Xorg.0.log, yum.log

Hi Steffens,

there is no purpose for putting output both as elasticsearch and logstash. It is accidentally forget to comment one.