Question about read mode vs tail mode

I'm processing files that comes from an external server; so once copied, files are static and no new content will be added into the file.
Originally I was reading the files in tail mode but I've experienced some issues processing some lines of the files ( were not ingested into ES.) I have a complex logstash configuration with several translate filters + large dictionaries for enrichment.

I made a change from tail to read mode and I have noticed that:

  • Files are processed one at a time (In tail mode, If I have 3 files there were open till files aged at the specified time in close_older)
  • Processing is much faster.

Is the performance significant affected then when using tail mode?
Thank you

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.