Logstash A to Logstash B

[Sjaak01 (Sjaak)]

Hi,

How can I ship data collected by Logstash A (client) to Logstash B (server)?

I want to collect and process some data locally on a remote client before sending it to the server for further processing.

I'm testing using Netflow data right now but so far I haven't been able to get it quite working.

If I run the netflow codec on both the client and the server side, the server side logs Ignoring Netflow version v31522. I suppose that makes sense as the client already touched the data so I'm guessing it becomes a unexpected format.

I've also tried some different combinations of UDP input/output on the client and server but the best I can get is all the netflow coming in on the message field.

What would be the correct combination of input/output/codecs for netflow (and other types of data?) when shipping between to Logstash instances?

[Christian_Dahlqvist]

I believe you should use the lumberjack output plugin on the client side combined with a beats input plugin on the server side. As it at that point is in the internal format you should not specify any codecs.

[Sjaak01 (Sjaak)]

Thanks, I will give it a go.

Is there any documentation that explains how to generate the required ssl certificates? Do I generate them on the client or server? There is no way to do this without ssl?

Edit: If I use the beats input I can still use filters to add/remove/adjust fields, correct?

[Sjaak01 (Sjaak)]

I don't think its possible to have lumberjack output on logstash A and beats input on logstash B. Seems lumberjack output can only talk to lumberjack input?

I've set up lumberjack output and input but than the message field is empty so doesn't look like any of the log data is being sent.