How can we read log files with size greater than zero

[Jeetu19 (Jitendra)]

Hi
we have situation where abinitio jobs creating .err files every day. but if there is "no error" file size of zero byte.
If there is error then file size is greater than zero.

Almost more than 20000 thousand error files is creating everyday but mostly with size zero byte.
you can see in screenshot.

i don't want put watch on files with size zero.

Can you please suggest me how can read or put watch only on those files that has size greater than zero.

[Jeetu19 (Jitendra)]

Please help!
Any other agent instead of filebeat or logstash

[kumarabhi (Abhishek)]

Filebeat is perfectly fine to use and it has options for closing Harversters.
Please look at the section below. You can use close_inactive: 5m option and it will close the harvester. Also, in case another line is written in the file later on then a new harverster will start.

  ### Harvester closing options

  # Close inactive closes the file handler after the predefined period.
  # The period starts when the last line of the file was, not the file ModTime.
  # Time strings like 2h (2 hours), 5m (5 minutes) can be used.
  close_inactive: 5m

  # Close renamed closes a file handler when the file is renamed or rotated.
  # Note: Potential data loss. Make sure to read and understand the docs for this option.
  close_renamed: false

  # When enabling this option, a file handler is closed immediately in case a file can't be found
  # any more. In case the file shows up again later, harvesting will continue at the last known position
  # after scan_frequency.
  close_removed: true

  # Closes the file handler as soon as the harvesters reaches the end of the file.
  # By default this option is disabled.
  # Note: Potential data loss. Make sure to read and understand the docs for this option.
  close_eof: false