Limit havesters for logging *.eml

(Javier Leal) #1

Hello. I want to scan files from directory /email/*.eml

So, the question is if I will have any problem in the future due to the amount of .eml files in /email/ could it be very big. We know that an .eml file is logged just on the time is created and never more is logged again. So, could it this behaviour throw any type of error on the future with memory related or another issue?


(Javier Leal) #2

Will "close_inactive" option avoid any memory problem since the haverster for each file is closed after 5min?

(Pier-Hugues Pellerin) #3

@nacesprin I think what you want is close_eof so we close the harvester as soon as we have read the file. You can also use harvester_limit to limit the number of concurrent harvesters.

(Javier Leal) #4

Thanks @pierhugues. Your reply is very useful, but I want to know whether with the "close_inactive" option is sufficient since only a few of .eml files are created a day. I understand "close_inactive" free the harvester for each file from memory past 5 minutes of inactivity (default).

I dont want to use "close_eof" due to the prospector file for .eml files is the same than others *.log files which are constantly updating.

(Pier-Hugues Pellerin) #5

@nacesprin What about creating a custom prospector only for *.eml? You can use wildcards when you define a path, so you could have one prospector watching *.log and the other *.eml with different strategies.

(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.