Tape library

Hello everybody,

in a new project my customer ask me if logstash can write the log files directly to a tape library.

There is a plugin that does this job?

The customer tape library software is bacula.

Someone could help me.

Thanks, best regards

Luigi

This is not currently possible.

If you wrote a custom "bacula" output plugin (which is at least theoretically possible), it would be a Bad Idea™ to use this in conjunction with any other output. "Why?" you might ask. Logstash is designed to stream from data in to data out. If the data out is slow, as it most likely would be writing to a tape library, all other data streams would be slowed by it. All output streams in any pipeline are only as fast as the slowest member. Moreover, if the data processing load on Logstash were high, and the output was already slow, this might result in an inconsistent stream sent to your tape library. I'm not sure how that would affect it, but I imagine it's not good.

If you intend to use Logstash somewhere in the mix, it would be wiser, I think, to have it write files to a directory, and have bacula select new files from there to write to tape. Less chance for something to go wrong that way.

1 Like

My idea was to create two output sections in the logstash configuration file. The first one directed to elasticsearch, with a retention policy of three months, to be used by kibana to navigate the log files. The second one to a tape library to store log files for a long time, for example five years. In this last case, I think that the best solution is to write the log files to the filsystem where bacula agent read the log files and store them in the tape library.

What do you think about it?

Thanks, Luigi

I agree.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.