Validate file with logstash-file-input plugin


I am working on a project that uses Logstash to parse some csv files. For this the Logstash “file-input-plugin” is used where new files are read from a path and the parsing is done.

However, I am trying to implement a validation to this flow: whenever a new file is captured, calculate its SHA DIGEST and call an endpoint and validate that it’s a valid file.

DISCLAIMER: My experience with Ruby is NIL . Mostly accustomed to Java.

Regardless, I was going through the logstash file-input-plugin source ( and I saw that they had used a package named “filewatch/tail”. Searching for it lead me to this repo:

I found that they call a method “subscribe” of FileWatch::Tail at

Can I get some guidance as to where exactly can I write into something to have the custom logic – calculate file DIGEST -> Call endpoint and validate -> if valid then process file else raise an error – whenever a new file is grabbed?

Your time and guidance will be much appreciated.


I wrote those changes to the file input.
The file input does not have a facility for this - the part you would want to hook into (but can't now) is the file discovery loop which scans for files to the 'to-be-processed' collection.

The only way you can do this is to script something yourself using a staging folder...

  1. new files go into a staging folder -> staging
  2. a script monitors for new files, does the digest + compare.
  3. move file to a valid folder -> valid
  4. LS detects the file in the valid folder and begins tailing it.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.