Compress output from logstash

Elastic Stack Logstash
1

Greetings,

I'm new to ELK and all the stuff. I've been reading the documentation to have a better understanding of elastic's products and, as far I've seen, couldn't find a solution for the case below:

I'm trying to receive syslog from 3rd party embedded systems directly into logstash, compress it and send to another logstash which will decompress and send to our log management.

Simplifying:
enviroment #1(syslog server -> logstash) -> internet(compressed data) -> environment #2 (logstash -> uncompressed data -> log management).

The idea is to compress data focusing on saving bandwidth.

Also, I'm trying to cipher this data. Will this mess up the idea above?

Thanks in advance.

PS: English isn't my first language, so, sorry for any misspelled word or unsense sentence.

1 Like
2

You can use a lumberjack output plugin combined with a beats input plugin. This combination will compress and encrypt data being transferred between the Logstash instances.

1 Like
3

Hi Christian,
Thank you for your suggestion.
I was reading lumberjack documentation and the field I need to configure about compression it's the flush_size, right?

4

I was reading lumberjack documentation and the field I need to configure about compression it's the flush_size, right?

No. that option has nothing to do with compression. Compression is always enabled.

5

I see.

So, basically, will be working this way:

logstash(lumberjack.compress(beat.cipher(log))) ---> logstash(lumberjack.uncompress(beat.decipher(log))) ---> log management

Have I understood right?

6

Yes, that's the idea.

1 Like
7

Great!

Thank you very much, Magnus and Christian!

Have a good weekend.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.