Filebeat Service fails with no error in log file

[ori.rubinfeld]

Hi,

I have Filebeat installed on Windows Servers.
After a while the Service starts to fail, with Registry file of 70MB(+)
I am using the log level of debug, and did not receive any error message.

Do you know of any limitation to the Registry file size ?
This is Version 1.2.3 of Filebeat.

Also seeing lots of messages like:
ERR Stop Harvesting. Unexpected file opening error: Error creating file '%!s(*uint16=0x14a52ff0)': The system cannot find the file specified
What does it means ?

Thanks,

Ori

[ruflin]

As your registry file is 70MB+ I assume you have a lot of new files created every day. Can you share some more details on how new files are created, naming and what kind of log rotation you use? In version 5.0 we are going to introduce new config options to clean up the registry file from files which were not touched or do not exist anymore to handle the problem of growing registry files. For more details see: https://github.com/elastic/beats/issues/1600 clean_removed and clean_inactive are currently available in the most recent nightly build: https://beats-nightlies.s3.amazonaws.com/index.html?prefix=filebeat/

The second issue is somehow strange and I don't think it is related to the registry file size. It seems like a file was removed before the harvester started. How do you remove files and when?

Can you test the nightly build and see if in general your problems disappear?

[ori.rubinfeld]

Hi Ruflin,

The File rotation is by size (10M, 1M, etc).
When rotating a sequence number is added to the name of the old file.

app.log - Current Active
app.log.1
app.log.2
app.log.3 - Oldest

Ori

[ruflin]

How many files are created / rotated per minute / hour? How many files are around in total? When are files removed?

[ruflin]

Lets also close this discussion in favor of Fatal error: runtime: cannot map pages in arena address space

I'm not sure why you opened 3 different discussion topics

[ori.rubinfeld]

OK, it can be closed.