Filebeat not monitoring data

Elastic Stack Beats
1

Hi Friends,

I set up Elasticsearch in our Lab, there in filebeat.yml I given the following configuration

filebeat:
paths:
- /var/log/*
document_type: syslog
scan_frequency: 10s

logstash:
# The Logstash hosts
hosts: ["...:5044"]

even though Scan_frequency:10s, Filebeat is not Monitoring files.
It just Adding the files at very first time when it see new file path is given.

Could anyone help me to sort it out.

2

Please post full logs, i.e. start Filebeat with -v -d "*".

(I suspect -v is superfluous when -d is specified but it won't hurt.)

3

Hi Magnus, Please find the below lines are in my Filebeat.yml file.

filebeat:
prospectors:
paths:
- /var/log/*
type: log
paths:
- /var/log/cron
type: cron_log
document_type: syslog
scan_frequency: 10s
output:
elasticsearch:
hosts: ["localhost:9200"]
logstash:
hosts: ["ip_add:5044"] // here output as Logstash
shipper:
logging:
files:

this is total file , means remaining all lines contains # front of them.

File is adding, I can discover through kibana.
but, later changes in files are not monitoring by Filebeat.

4

I asked for logs, not your configuration (although it's good to have the full configuration, but please make sure you format it as code since YAML is indentation-sensitive and this forum strips leading whitespace).

5

why have you configured both, elasticsearch and logstash output? This on purpose?

6

Hi Magnus, I would like to monitor syslog. and I didn't found syslog so I given the path like,

paths:

  • /var/log/*

in that folder all the below log files are there. problem is eventhough scan_frequency is 10s , it is not monitoring.

anaconda.log, audit, boot.log, btmp, cron, cups, dmesg, dracut.log, gdm, httpd, kern, maillog, mcelog, messages, mysqld.log, pm-powersave.log, sa-update.log, secure, spooler, tallylog, up2date, wtmp, Xorg.0.log, yum.log

7

Hi Steffens,

there is no purpose for putting output both as elasticsearch and logstash. It is accidentally forget to comment one.