Hello,
I've started a pipeline in with file input in read mode, and configured sincedb_clean_after to 2 hours, and close_older to 2 minutes (input files are added every minute).
Initially the since db was empty. But since then it keeps growing and growing, whereas I would expect that it stays rather small since the records are expunged every 2 hours right ??
Well, it does not seem to be the case, I opened the file and found many old records, some have 6 fields and many w/o that same 6th field (a short excerpt):
6473924466549094 0 39 15801 1538729585.759023 /data_nfs/fdm/LOGS/val/2018/09/30/FOR-SERVICE/zsbwatsov02/FDM_history_2018-09-30-20-09.log.gz
4785074605861734 0 39 18262 1538729575.9758039 /data_nfs/fdm/LOGS/val/2018/09/28/FOR-SERVICE/zsbwatsov02/FDM_history_2018-09-28-20-56.log.gz
5066549584457954 0 39 41308 1538729640.504036
8162774328299343 0 39 21106 1538729640.761678
What are these 5-field records ? do they correspond to records inactivated thanks to the sincedb_clean_after setting ? Shouldn't inactive records be expunged from the sincedb ?
Thanks for the clarifications...