i am using file-beat to pull the logs to log-stash and from there to gray-log, its working fine with .csv files and when i am sending .txt files facing issue
example : if one error logged at 6:00 AM that log get pushed to grey-log and another error logged at 9:00 AM and appended on the same log file its pushing older log also to grey-log is there a way to stop that presently i am using below configuration please suggest if any thing missed
ignore_older: 5m
close_inactive: 5m
close_renamed: true
close_removed: true
close_eof: true
clean_inactive: 10m
clean_removed: true
#exclude_lines: ["^|"]
#exclude_lines: ['^(.?|)$']
#exclude_files: ["_Info.txt$","_Debug.txt$","MemCache_Error.txt$","UpdateChatInQMemcache_Error.txt$"]
#multiline.pattern: '^%{TIMESTAMP_ISO8601}'
multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
multiline.negate: true
multiline.match: after