I'm using Filebeat to process a big number of logfiles and I've got the feeling that the registry is not able to cope with the amount. The config I am using has worked the first time and the startup happens immediately when I delete the registry file before starting. For obvious reasons, I can't do this in production.
OS: Ubuntu 16.04
Filebeat version 6.3.2 (amd64), libbeat 6.3.2 [45a9a9e1561b6c540e94211ebe03d18abcacae55 built 2018-07-20 04:18:19 +0000 UTC]
The startup log is as follows:
2018-08-23T11:11:45.915Z INFO instance/beat.go:315 filebeat start running.
2018-08-23T11:11:45.915Z INFO registrar/registrar.go:117 Loading registrar data from /var/lib/filebeat/registry
2018-08-23T11:11:45.915Z INFO [monitoring] log/log.go:97 Starting metrics logging every 30s
2018-08-23T11:11:46.570Z INFO registrar/registrar.go:124 States Loaded from registrar: 100668
2018-08-23T11:11:46.570Z INFO crawler/crawler.go:48 Loading Inputs: 1
From that point, I'm only getting metrics printed out - I've been waiting for 30 minutes but the loading didn't finish.
I'm unable to reduce the number of individual files and I've also reduced the ignore_older and clean_inactive time.
My config looks as follows:
paths: - /var/lib/jenkins/logs/* - /var/lib/jenkins/jobs/*/branches/*/builds/*/*.log - /var/lib/jenkins/jobs/*/builds/*/*.log symlinks: false exclude_files: [ 'lastStableBuild', 'lastSuccessfulBuild' ] close_inactive: 5s scan_frequency: 10s ignore_older: 24h clean_inactive: 25h
Thank you in advance.