This is a dummy setup for me to play with the Elastic stack before getting into the production environment.
My setup:
I'm running two Ubuntu 16.04 VMs using VirtualBox. These two instances have been configured to communicate with each other (I hope) as telnet and ping from one to the other works properly.
On one VM (IP 10.0.2.4), I install Elasticsearch, Logstash, Kibana, Filebeat and some more Beats. Filebeat outputs to Logstash via port 5050:
output.logstash:
hosts: ["10.0.2.4:5050"]
The other Beats output directly to Elasticsearch:
output.elasticsearch:
hosts: ["10.0.2.4:9200"]
On the other VM (IP 10.0.2.5), I install Elasticsearch (not in use, only to store index replicas), Kibana, and the Beats. So no Logstash on this VM. Filebeat also outputs to the first VM but through another port 5051:
output.logstash:
hosts: ["10.0.2.4:5051"]
While the other Beats also output to Elasticsearch on the first VM:
output.elasticsearch:
hosts: ["10.0.2.4:9200"]
So all Beats outputs are to the first VM.
All Beats configs have been properly commented. So no Elasticsearch output on Filebeat and no Logstash output on the other Beats.
Logstash input setting:
input {
beats {
port => "5050"
}
beats {
port => "5051"
}
}
Results:
All Beats on 10.0.2.4 work properly. Logstash receives the logs from Filebeat on 10.0.2.4 and passes it to Elasticsearch. I can see the data on Kibana.
On 10.0.2.5, all Beats except for Filebeat work properly. I can see the data from 10.0.2.5 on Kibana. The problem is with Filebeat on 10.0.2.5. Logstash doesn't receive anything from Filebeat on the second VM. telnet 10.0.2.4 5051 from the second VM works.
Am I doing this correctly? Or do I have to install Logstash on the second VM then outputs to Elasticsearch on the first VM?
Everything is working now