Filebeat not collect file which has being marked inactived. But the file is always update intime.
We use filebeat to collect application's log to logstash.
config:
filebeat.inputs:
- type: log
enabled: true
paths:
- /home/xxxlogs/logs/sys/*/*.log
- /home/xxxlogs/logs/sys/*.log
max_bytes: 1048576
ignore_older: 168h
multiline.pattern: '^20\d{2}'
multiline.negate: true
multiline.match: after
multiline.max_lines: 500
multiline.timeout: 5
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
reload.period: 10s
queue.mem:
events: 200
flush.min_events: 100
output.logstash:
hosts: ['x.x.x.x:5048']
bulk_max_size: 100
http:
enabled: true
host: localhost
port: 5066
# filebeat logs setting
logging.level: info
logging.to_files: true
logging.files:
path: /home/xxxlogs/applogs/filebeat
name: filebeat.log
keepfiles: 7
permissions: 0644
rotateeverybytesedit: 209715200
in filebeat-log:
We found the log was be marked inactived after close_inactive time .
2020-05-12T00:00:09.101+0800 INFO log/harvester.go:251 Harvester started for file: /home/xxxlogs/logs/sys/xxx/http-server.log"
2020-05-12T00:05:05.664+0800 INFO log/harvester.go:276 File is inactive: /home/xxxlogs/logs/sys/xxx/http-server.log. Closing because close_inactive of 5m0s reached.
but actual file are always being updated...
After we restart the application.
The log in file unexpectedly can be collected. but the filebeat-log has no "Harvester started for file: " record.
No rotate log event occured！