I am new to this group. I have been playing around with Logstash for about a couple of weeks. I was surprised to see that Logstash does not provide the feature where in, it reads a log file once and forgets about it. But it seems to have a very strong affinity to the files it has read. I am pretty sure there has been a need where you get thousands of log files and you just read them and forget it.
Reading through the logs, I see many folks have asked the same question. I have not found a satisfactory answer. I have decided to do some log rotation myself outside of logstash but then logstash still remembers the offsets in memory and even if i rotate through to an older file, it reads from the offsets from where it left off last time. I could work around this by truncating the file before it gets written to and hoping that logstash notices the truncation first before any new content gets written to. Obviously, this is very racy. If there is not enough time between truncation and new content, it would read from where it left off.
Can someone tell me a reliable way to do this log rotation ? I have sincedb_path set to NULL. Any clues would be appreciated ?