Hello all, i'm begginer with ELK and having trouble shipping log events from filebeat to logstash.
The scenario consists in:
Machine 1) Linux ubuntu 18.04 running oracle virtual box.
Machine 2) Linux centos 7 virtualized machine on oracle virtual box; running logstash, elasticsearch and kibana.
Machine 3) Linux centos 6 virtualized machine on oracle virtual box; running Mongo DB and filebeat with mongo module enabled,
Command "ping" for testing communication between the 3 machines succesfull.
The filebeat log reports:
ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://192.168.43.165:5044)): Get http://192.168.43.165:5044: read tcp 192.168.43.200:38382->192.168.43.165:5044: read: connection reset by peer
I have this configuration on /etc/logstash/conf.d/logstash.conf
input {
beats {
port => "5044"
host => "192.168.43.165"
}
}
output {
elasticsearch {
hosts => ["192.168.43.165:9200"]
manage_template => false
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
}
}
The command netstat -tulnp on Linux centos 7 reports:
tcp6 0 0 192.168.43.165:9200 :::* LISTEN 1233/java
tcp6 0 0 192.168.43.165:5044 :::* LISTEN 5161/java
tcp6 0 0 192.168.43.165:9300 :::* LISTEN 1233/java
The command wget http://192.168.43.165:5044 executed on Machine 2 and 3 reports the following:
--2019-01-27 20:23:13-- http://192.168.43.165:5044/
Connecting to 192.168.43.165:5044... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
¿why communication can not be established between filebeat and logstash ?
SOLVED:
the error was in file: /etc/filebeat/filebeat.yml
The line: output.elasticsearch: was uncommented and the line: output.logstash: was commented; so it was applying the logstash configuration to elasticsearch
Now it looks like this:
#-------------------------- Elasticsearch output ------------------------------
#output.elasticsearch:
Array of hosts to connect to.
#hosts: ["localhost:9200"]
Optional protocol and basic auth credentials.
#protocol: "https"
#username: "elastic"
#password: "changeme"
#----------------------------- Logstash output --------------------------------
output.logstash:
The Logstash hosts
hosts: ["192.168.43.165:5044"]
Optional SSL. By default is off.
List of root certificates for HTTPS server verifications
#ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
Certificate for SSL client authentication
#ssl.certificate: "/etc/pki/client/cert.pem"
Client Certificate Key
#ssl.key: "/etc/pki/client/cert.key"