I am working on a project that uses Logstash to parse some csv files. For this the Logstash “file-input-plugin” is used where new files are read from a path and the parsing is done.
However, I am trying to implement a validation to this flow: whenever a new file is captured, calculate its SHA DIGEST and call an endpoint and validate that it’s a valid file.
DISCLAIMER: My experience with Ruby is NIL . Mostly accustomed to Java.
Regardless, I was going through the logstash file-input-plugin source (https://github.com/logstash-plugins/logstash-input-file) and I saw that they had used a package named “filewatch/tail”. Searching for it lead me to this repo: https://github.com/jordansissel/ruby-filewatch
I found that they call a method “subscribe” of FileWatch::Tail at
Can I get some guidance as to where exactly can I write into something to have the custom logic – calculate file DIGEST -> Call endpoint and validate -> if valid then process file else raise an error – whenever a new file is grabbed?
Your time and guidance will be much appreciated.