Connection not established between logstash and filebeat


(Akash Katakam) #1

FIlebeat running on ubuntu 14.04
Elk is installed in windows 2008 server

Logs are in ubuntu machine and i want to send my remote machine.
my filebeat config and logstash configs are attached.

input {
  beats {
    port => 5043
    type => "logs"
  }
}

output {
  elasticsearch {
    hosts => "localhost:9200"
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

filebeat:

  prospectors:
      paths:
        - /var/lib/tomcat7/logs/storypulse.log
      input_type: log
output:
  logstash:
    # The Logstash hosts
    hosts: ["98.175.99.177:5043"]
    # Number of workers per Logstash host.
    worker: 1 
shipper: 
  refresh_topology_freq: 10 
logging:
  files:
    rotateeverybytes: 10485760 # = 10MB

(Steffen Siering) #2

The filebeat config looks off.

  1. yaml is sensitive to indendation. make sure to use spaces for indentation only
  2. prospectors expects a list of configs all beginning with -

(Akash Katakam) #3

filebeat is running fine i tried debugging it.

I didnt understand what does this mean. Can please explain in layman terms.

This is the debug output.


(Andrew Kroh) #4

See this post: Filebeat only reporting on last prospector


(Steffen Siering) #5

from docs:

filebeat:
  prospectors:
    -
      paths:
        - /var/log/system.log
        - /var/log/wifi.log
    -
      paths:
        - "/var/log/apache2/*"

multiple prospectors are defined each starting with -


(Akash Katakam) #6

connecting error publishing events retrying:dail tcp 98.xx.xx.xxx:5043 i/o timeout send fail how can i overcome this?

some times in filebeat we can see data that is no the problem the data that is shown during debug of filebeat is not visible in my ES


(Akash Katakam) #7

Thanks for commenting we got it working. We just opened the port 5043 in the firewall and its working now.


(system) #8