I see that max_size for the output file has not been implemented yet. I am finding a way where Logstash will write to an output file and close the file which then can serve as the indication that it is done with the output file. Is there a way to get this behavior ?
I am using the time format for the output file (%{+YYYY-MM-dd-HH-mm-ss}). But then it looks to me that this file can be re-opened multiple times (depending on the time of the incoming log ?) and there was no definite time where one can know that Logstash is done with it. Am i missing something ?
What does "done" even mean in this context? How can Logstash possibly know when it won't receive any more messages that should be routed to the file in question?
Logstash's file output will continue writing to a file until it has been idle for a configurable amount of time, i.e. it'll basically garbage collect its open files to avoid using up all file descriptors.
I need a way to specify "done". max_size would be a good way. In the absence of that, what else can be used ?
What is that configurable time ? The problem is that if there are no activities, the file remains open and not closed. A file is closed only when there is some other activity and this file happens to remain inactive (based on my observations).
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.