Problem description：
More than 2000 files are generated per hour in the multi-level directory， I only need to log in an hour, so I'm configuration ignore_older: 1 h, but after an hour, which is 1 hour before the file is not sent. How can I solve this problem?
config file
filebeat.prospectors:
- input_type: log
paths:
- /vol2/ZT2M/*/*/*.log.*
scan_frequency: 10s
ignore_older: 1h
close_eof: true
close_timeout: 3h
close_interval: 5m
close_removed: true
clean_removed: true
clean_inactive: 3h
exclude_lines: ["存储至", "压缩档案数据成功"]
include_lines: ['^\d{6}-\d{2}:\d{2}:\d{2}']
output.logstash:
hosts: ["127.0.0.1:5044"]
Pictures show
The logs are getting less and less from 13:00 to 13:59.