Error Initialising publisher: No outputs are defined. Please define one under the output section

Elastic Stack Beats
1

Hi, I am new to ELK Stack.
I am trying to implement it Using Windows (ELK Server ) and Vagrant Unix CentOS VM ( Filebeat Shipper )

For starters, I am trying to ship Unix Syslog to ELK server and see how it works

I have configured the central.conf file for Logstash on my Windows Machine as

input{

beats{
    port => 5044
    }
}

output{

stdout{ }

elasticsearch{ 
     hosts => ["http://localhost:9200"]
}

    }

and Filebeat YAML on Unix (CentOS - 7) is configured as

filebeat:
prospectors:

paths:
-"/var/log/*.log"
input_type: log
document_type: beat
registry: "/var/lib/filebeat"
output:
logstash:
hosts: ["127.0.0.1:5044"]
logging:
to_files: true
files:
path: "/var/log/filebeat"
name: filebeat.log
rotateeverybytes: 10485760
level: debug

Elasticsearch and Logstash is running properly on my windows machine

I am facing the following two issues right now,

  1. When i try to run filebeat shipper on Unix , it gives me the below error

[root@localhost filebeat]# filebeat -e -v -c filebeat.yml -d "*"
2016/05/08 11:07:00.404841 beat.go:135: DBG Initializing output plugins
2016/05/08 11:07:00.404873 geolite.go:24: INFO GeoIP disabled: No paths were set under output.geoip.paths
2016/05/08 11:07:00.404886 publish.go:269: INFO No outputs are defined. Please define one under the output section.
Error Initialising publisher: No outputs are defined. Please define one under the output section.
2016/05/08 11:07:00.404902 beat.go:140: CRIT No outputs are defined. Please define one under the output section.

2 . When i saw Logstash logs , i found out , its trying to listen Beats input on "0.0.0.0:5044" rather than on "127.0.0.1:5044"

{:timestamp=>"2016-05-08T16:36:07.158000+0530", :message=>"Beats inputs: Starting input listener", :address=>"0.0.0.0:5044", :level=>:info}

are these two issues interrelated , how can i resolve them , could someone please help me out and point me in the right direction to get this working.

Really Appreciate any help you could provide.

are these issue'

2
  1. As the error message says, you haven't defined any outputs. Please check your configuration file again and make sure the indentation is correct. YAML is picky about that. If you can't figure it out, re-post the configuration file but make sure you format it as code with the </> button.
  2. The beats input will by default listen on all interfaces, i.e. 0.0.0.0. There's nothing inherently wrong about that and it has nothing to do with your first problem.
3

Hi, Thanks a lot.

I did the necessary changes and it is now able to parse the yml file

But now i am having different issue. Filebeat is not able to send logs to Logstash server (127.0.0.1).

Filebeat config:

filebeat:
  prospectors:
    - paths:
        -"/var/log/*.log"
      input_type: log
      document_type: beat
      registry: "/var/lib/filebeat"
output:
  logstash:
    hosts: ["127.0.0.1:5044"]
logging:
  to_files: true
  files:          
    path: "/var/log/filebeat"
    name: filebeat.log
    rotateeverybytes: 10485760
    level: debug

Logstash Config:

input{

beats{
    host => "127.0.0.1"
    port => 5044
    }
}

output{

stdout{ }

elasticsearch{ 
     hosts => ["http://localhost:9200"]
}

    }

Error:

2016/05/09 03:08:38.686039 reader.go:138: DBG  End of file reached: /var/log/VBoxGuestAdditions.log; Backoff now.
2016/05/09 03:08:38.686669 reader.go:138: DBG  End of file reached: /var/log/VBoxGuestAdditions-uninstall.log; Backoff now.
2016/05/09 03:08:38.719569 reader.go:138: DBG  End of file reached: /var/log/vboxadd-install.log; Backoff now.
2016/05/09 03:08:38.744784 reader.go:138: DBG  End of file reached: /var/log/yum.log; Backoff now.
2016/05/09 03:08:39.172610 spooler.go:97: DBG  Flushing spooler because of timeout. Events flushed: 0
2016/05/09 03:08:41.294816 client.go:100: DBG  connect
2016/05/09 03:08:41.295401 single.go:126: INFO Connecting error publishing events (retrying): dial tcp 127.0.0.1:5044: getsockopt: connection refused
2016/05/09 03:08:41.295439 single.go:152: INFO send fail
2016/05/09 03:08:41.295467 single.go:159: INFO backoff retry: 8s

How can i resolve the TCP connection refused error.

4

Is Logstash actually running and listening on port 5044? Check with e.g. netstat or lsof.

5

Thank you so much magnusbaeck , it worked.
Now working on defining custom grok patterns