Forwarding logs from logstashA to LogstashB to Kibana

Elastic Stack Logstash
1

I am currently trying to implement my own "logstash-forwarder". I am running this from a local windows machine and I am succefully able to send logs from one logstash instance to Kibana. However, I would like to be able to send logs from logstashA, logstashB, etc. to logStashMaster and then finally forward the data over to kibana. Afer reading this, I see that it is possible to achieve with the tcp option in the configuration file. However, when I check in Kibana I dont see any logs. How can I make this work?

I am running several instances of logstash and testing this all in one local machine.

Ideal:
logstashA -> logstashMaster -> Kibana

logstashA.conf

input {
        file {
                path => "C:/Users/Mary/mylogs.log"
                start_position => "beginning"
        }
}
output {
  tcp {
		  host => "127.0.0.1"
          port => 9389
      }
}

logstashMaster.conf

input {
  tcp {
        port => 9389
  }
}
output {  
		kafka {
			 codec => plain 
             bootstrap_servers => "remoteserver.com:9093"
        }
}

References:
TCP

2

Does not match [quote="Maria_Delarosa, post:1, topic:70609"]
input {
tcp {
port => 9389
}
}
[/quote]

3

Hi, thanks for the response. I corrected the port number typo. I am still not getting any logs sent. what is the proper/correct way to have one instance of logstash talk to another?

4

Add a stdout section to each output to make sure it is actually processing things as expected.

5

I placed stdout {} in the output and it is processing it correctly. It seems that the issue is Kibana. I am not able to view it in Kibana.

6

Is there data in ES?
Use the _cat APIs to check.

7

There seems to be old data but nothing recent. How do I use the _cat APIs ? I read a bit about it but unsure how to related to my scenario https://www.elastic.co/guide/en/elasticsearch/reference/current/cat.html

8

Well you can call _cat/indices for example, to see what indexes are created, thereby confirming if data is making it to ES.

9

Thanks, I was able to call the _cat API properly. However, no data is being sent. Have you been able to get it to work on your end? Not sure what I am doing wrong. Everything seems to be in place as shown above.

10

When I kill the instance of LogstashA, it finally forwards it to LogstashB. Why such behavior? I am doing this all from the same localmachine. Could that be the issue?

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.