Hello friends,
I am using forwarder to ship log to logstash 1.4.2 where those are filtered.
I have one system which compressed log files in a zip file. there are more than one zip files created at static location once in a day.
Expected:
- Forwarder is expected to extract those zip files as soon as those are created
- read log files inside
- ship logs to logstash.
is there any way to extract zip files in logstash?
Please suggest. Also guide me if there is any watcher kind of thing which will make forwarder to wake-up as soon as zip is created.
logstash-forwarder doesn't read files inside zip archives so you'd have to write a script for the unpacking. The script could check for new archives and unpack them in a directory that you configure logstash-forwarder to read files from. If the filenames aren't unique you can create a directory for each archive and make sure the wildcard you configure logstash-forwarder with covers all such directories.
It's hard to know when logstash-forwarder has processed a file so the easiest is probably to just have a cronjob that deletes all files older than a certain threshold after which you can be reasonable certain that the files have been processed.
Thanks Manguns.
At least this is clear that there is now way logstash unzip files.