Hi,
I'm facing a couple of problem when running filebeat (both alpha 3 and 4) on a windows server 2003.
I'm trying to send the content of a log file which is written continuously and it's rotated often.
This is how the rotation works: after reaching a certain size, the file "logxxx.log" get renamed in "logxxx-timestamp.log" and a new "logxxx.log" is created.
As the file is rotated very often (and I am only on testing, the real environment creates a lot of log file with the same logic) I would like to controll the registry size, so this is my prospector config:
- input_type: log
# Paths that should be crawled and fetched. Glob based paths.
paths:
#- /var/log/*.log
- yyy\logxxx.log
exclude_lines: ["^\\s*$"]
include_lines: ["^([a-z]|[A-Z])"]
ignore_older: 1m
document_type: log_xxx
registry:
clean_inactive: 2m
I'm not sure if this is a working config, but I tried everything! The registry won't be cleaned, even if in the log file of filebeat I found many entries like these:
DBG State removed for yyy\logxxx.log because of older: %!s(MISSING)
Moreover, after running for a couple of hour, filebeat crashes and I get this error which I can't understand, but I guess is related with the registry clean up:
panic: runtime error: slice bounds out of range
goroutine 16 [running]:
panic(0x8c3a80, 0x1125a010)
/usr/local/go/src/runtime/panic.go:481 +0x326
github.com/elastic/beats/filebeat/input/file.(*States).Cleanup(0x113b6f60, 0xf84
75800, 0xd)
/go/src/github.com/elastic/beats/filebeat/input/file/state.go:93 +0x388
github.com/elastic/beats/filebeat/prospector.(*ProspectorLog).Run(0x113b37c0)
/go/src/github.com/elastic/beats/filebeat/prospector/prospector_log.go:5
3 +0x84
github.com/elastic/beats/filebeat/prospector.(*Prospector).Run(0x113b8420)
/go/src/github.com/elastic/beats/filebeat/prospector/prospector.go:129 +
0x290
github.com/elastic/beats/filebeat/crawler.(*Crawler).Start.func1(0x113de0f0, 0x0
, 0x113b8420)
/go/src/github.com/elastic/beats/filebeat/crawler/crawler.go:71 +0x102
created by github.com/elastic/beats/filebeat/crawler.(*Crawler).Start
/go/src/github.com/elastic/beats/filebeat/crawler/crawler.go:72 +0x41d
And these are the last filebeat log lines:
2016-08-01T18:27:41+02:00 INFO Registry file updated. 94 states written.
2016-08-01T18:27:41+02:00 INFO Run prospector
2016-08-01T18:27:41+02:00 DBG Start next scan
2016-08-01T18:27:41+02:00 DBG Check file for harvesting: yyy\logxxx.log
2016-08-01T18:27:41+02:00 DBG Update existing file for harvesting: yyy\logxxx.log, offset: 1350352
2016-08-01T18:27:41+02:00 DBG No updates needed, file yyy\logxxx.log is already harvested.
2016-08-01T18:27:41+02:00 DBG State removed for yyy\logxxx.log because of older: %!s(MISSING)
2016-08-01T18:27:41+02:00 DBG State removed for yyy\logxxx.log because of older: %!s(MISSING)
2016-08-01T18:27:41+02:00 DBG State removed for yyy\logxxx.log because of older: %!s(MISSING)
Thank you for your attention!
Gabriele