the system receives a bunch of files daily, monitoring the file availability and size is important for the service quality. No need to check the file content though. Is there any proper way to achieve this with ELK?
Welcome to our community! 
Check the content for what exactly?
Sry I dont get you. but there is no need to check the rows or columns in file. Just to visualize if the file received or not ，alert out when file delay.
Are you able to identify the files via the name, or some kind of other identification method?
yeap，they are saved in specific file folders and can be located with regex.
In the events that are processed that is.
If they are you could do an alert on them to say if filename X hasn't been seen in A time period, then generate an alert.
Might not fit the usecase totally, but if I understand you correctly, you are mostly interested in the fact that a file has been created (or not).
While it might be easier to have some sort of bash script check that running in a cron job, and read the output of that with filebeat, there might be another way around it.
With auditbeat, you have a module called file_integrity. You are able to give it a list of folders to monitor, and it will generate an event once a file is created, deleted and updated.
That way you can create Kibana Alerts for example, when new files has not been created in a while, or if files that shouldn't be modified has been updated etc.
Ref: File Integrity Module | Auditbeat Reference [7.14] | Elastic
Do you think that might work @lpf86731782 ?
1 Like
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.