Plugin to move log files to Processed directory on successful load and to Rejected folder on failure

Elastic Stack Logstash
1

I am loading new log files from an AWS S3 bucket every hour. I want to maintain a system which will load and process only the new uploads and move them to Processed folder on successful load, and move to Rejected folder on failure.

Is there a plugin to do this? And maintain a log naming the processed files and rejected files.

2

The backup_add_prefix and backup_to_bucket options on the S3 Input plugin may be what you're looking for.

3

Is there a way to do the same if I'm using the basic file input plugin?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.