current state:
elk 6.8.5 setup, which seems to be parsing and visualizing... however:
/etc/filebeat/filebeat.yml has the following ;
I only want filebeat to look at 1 logfile (/var/log/network):
"- type: log
- /var/log/network
exclude_files: ['.gz$']
"
/etc/filebeat/modules.d/system.yml has the following: "
"- module: system
syslog:
enabled: true
var.paths: ["/var/log/network"]
"
Issues I'm seeing:
when I restart filebeat, I keep seeing this error:
ERROR instance/beat.go:906 Exiting: Can only start an input when all related states are finished: {Id:4458412-64512 Finished:false Fileinfo:0xc420582680 Source:/var/log/network Offset:0 Timestamp:2019-12-02 12:28:40.849396001 -0800 PST m=+0.049508312 TTL:-1ns Type:log Meta:map FileStateOS:4458412-64512}
I've tried removing the entry for "source":"/var/log/network"... from /var/lib/filebeat/registry but error still repeats...
when I look at Kibana discover console, [fileset][name] is syslog as expected
but [source] is "/var/log/syslog" instead of "/var/log/network "
how do I fix these issues? I don't recall anywhere else to specify/limit the logfile to be checked by filebeat
has something to do with /etc/filebeat/modules.d/system.yml
if I leave this commented, filebeats starts up
#var.paths: [/var/log/network]
however it starts looking at these instead of just the single file
2019-12-02T16:18:12.637-0800 INFO log/input.go:148 Configured paths: [/var/log/messages* /var/log/syslog*]
2019-12-02T16:18:12.637-0800 INFO crawler/crawler.go:106 Loading and starting Inputs completed. Enabled inputs: 1
2019-12-02T16:18:12.637-0800 INFO cfgfile/reload.go:150 Config reloader started
2019-12-02T16:18:12.640-0800 INFO log/input.go:148 Configured paths: [/var/log/auth.log* /var/log/secure*]
2019-12-02T16:18:12.643-0800 INFO log/input.go:148 Configured paths: [/var/log/messages* /var/log/syslog*]
2019-12-02T16:18:12.643-0800 INFO input/input.go:114 Starting input of type: log; ID: 11691604117334136852
2019-12-02T16:18:12.643-0800 INFO input/input.go:114 Starting input of type: log; ID: 15959826002125184592
2019-12-02T16:18:12.643-0800 INFO cfgfile/reload.go:205 Loading of config files completed.
2019-12-02T16:18:12.644-0800 INFO log/harvester.go:255 Harvester started for file: /var/log/syslog
2019-12-02T16:18:12.644-0800 INFO log/harvester.go:255 Harvester started for file: /var/log/auth.log
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.