Connecting FileBeats to Logstash

Elastic Stack Logstash
1

I have FileBeats running on an ubuntu webserver on EC2. I have opened up the ports, but still I cannot get them to talk to each other.

filebeats starts without errors but I cannot get it to print the apache logs to the console as output.
Nor can I get logstash to connect to filebeats, as when I load logstash it gives me this error:

{:timestamp=>"2016-03-16T14:58:48.583000+0000", :message=>"The error reported is: \n Cannot assign requested address - bind - Cannot assign requested address"}

Here ate my config files.. it has been a long day. Am I overlooking/missing something ?

Filebeat.conf:

################### Filebeat Configuration Example #########################

############################# Filebeat ######################################
filebeat:
  # List of prospectors to fetch data.
  prospectors:
    # Each - is a prospector. Below are the prospector specific configurations
    -
      # Paths that should be crawled and fetched. Glob based paths.
      # To fetch all ".log" files from a specific level of subdirectories
      # /var/log/*/*.log can be used.
      # For each file found under this path, a harvester is started.
      # Make sure not file is defined twice as this can lead to unexpected behaviour.
      paths:
        - "/home/bitnami/stack/apache2/logs/access_log"
        #- c:\programdata\elasticsearch\logs\*

      input_type: log

      

############################# Output ##########################################

output:
  logstash:
    # The Logstash hosts
    hosts: ["<LOGSTASH IP>:5044"]

  ### Console output
  console:
    # Pretty print json event
    pretty: true

logstash.yml:

input {
  beats {
    host => "<FileBeats IP>"
    port => 5044
  }
}

output {
  stdout {
    codec => rubydebug
  }
  kafka {
    bootstrap_servers => "<KAFKA IP>:9092"
    topic_id => "from_logstash"
  }
}
2

Remove the host line. That's for binding to a specific interface.

1 Like
3

Oh if I could hug you I would! virtual hug applied here Thanks heaps

Can you explain why I didn't need that?

4

Logstash-beats-input accepts connections. It does not initiate connections, so you don't need to specify the IP of the host running Beats.

By default Logstash will listen (aka bind) on all network interfaces. If you want to listen on a specific network interface (e.g. loopback) you would enter the IP address of that interface (for loopback it would be 127.0.0.1).

The default value for host is 0.0.0.0 which is sometimes called the "any" address because it means to bind to all interfaces.

You can use a command like netstat -anp to see what ports and interfaces that processes are bound to.

1 Like