Transfering logs by Logstash to Logstash - help needed

Elastic Stack Logstash
1

Hey guys!
I have a problem with configuring Logstash to send logs to Logstash on a different server.
I was able to send logs from Agent1 server by Filebeat to Logstash on Agent2 server, but i don't know how to send logs by Logstash on Agent2 to Logstash on Agent3 server... Just nothing happends then, and I don't know what is missing in configuration files.
Below is configuration of Logstash on Agent2:
input {
stdin { }
beats {
port => 5044
}
}
filter {
grok {
match => { "message" => "^?(?.) - ?(?<logged_json>{.)" }
}
json {
source => "logged_json"
remove_field => [ "logged_json" ]
remove_field => [ "message" ]
remove_field => [ "@timestamp" ]
#add_field => { "moment" => "%{moment}" }
}
}
output {
tcp {
host => "10.xxx.xxx.xx"
port => 5044
}
}
And here below is configuration of logstash on Agent3 server:

input {
stdin { }
tcp {
port => 5044
}
}

output {
stdout {
codec => "json"
}
file
{
path => "C:\Logins\ELK\logstash-2.1.0\test.txt"
}
}
Can Someone please help me find what is missing? Why Logstash don't send logs to Logstash on Agent3?

2

Hello Daniel,

You probably want to use the lumberjack output for the agent sending the logs, and the lumberjack input for receiving these. Lumberjack is much more robust and secure compared to tcp. :slight_smile:

Cheers,
-Robin-

3

Hi Robin,
I wanted a tcp, i work in a Bank on closed enviroments INT,UAT,QUA, so there is no need for extra security, but I'm really grateful for your suggestion, I will keep that in mind.
I wasn't sure did I configure Logstash properly, but it seems ok... It appears that there are just some delays with transfer, 7 min top... and that I have issues with patience lol :stuck_out_tongue:

4

Hi Daniel,

When I said security I was not only talking about the security of the data connection, but also the security of the data packets - i.e. a less lossy connection than TCP can be.

7 minutes seems like a high latency... how are you measuring it, and where are you seeing the latency (between logstash->logstash, or logstash->elasticsearch).

Cheers,
-Robin-

5

Hi Robin,
This 7 min its fine, logs will be checked on the end of each day so at this moment its really not a problem. When windows update file with logs on Agent3 i see this time difference between Agent1 and Agent3. Maybe problem lays a little bit in our Bank Network, because it's a quite messy subject, there is a lot security bariers even in closed enviroments, or with the local firewalls, it's hard to say I need to check everything in our infrastructure first, to be sure, I just started with Logstash, so there is a lot of things for me to get known with.