Input files with dynamic filename

Elastic Stack Logstash
1

Hi,
We have some files in our systems, which has dynamic file names, date time appended.
Example, server_201602025 etc.
FIle created today will neither be archived to some other location nor it will be appended any new data after 2016-02-25 . On next day new file will be created server_201602026

Same file is the input for LSF. Now I want to know, If I read this file in LSF using /log_dir/* then LSF will maintain pointer/offset for all files at any instance of time T1. This is my understanding.
How does it affects performance of LSF?
Is there any way to skip older files which has no latest data after some time, may be some days. So that harvestor will stop keep watch of old files?

please clarify.

thanks
Sunil

2

You should use filebeat, LSF has been deprecated!

3

Hi Mark,
Thanks for the reply.
However, we are almost in the production.
Event pipeline is like : LSF (Client) -->(Server) Shipper-->Redis-->Broker-->nGinx-->ES

How complex or tough to replace LSF with filebeat?
What changes we need to do on other sides, Shipper?

We do have one client with AIX. Do filebeat support AIX?

br,
Sunil

4

Beats don't support AIX as it doesn't support go.

Otherwise it's a pretty simple replacement.