Filebeats not able to publish events to Logstash

Elastic Stack Beats
1

I have logstash on one linux server and have filebeats running on another server where my application is hosted.

I am getting following error when i run filebeats, logstash is successfully started.
Error:
> 2017/04/24 08:29:45.694653 sync.go:41: INFO Start sending events to output
> 2017/04/24 08:29:45.694712 prospector.go:124: INFO Starting prospector of type: log; id: 17005676086519951868
> 2017/04/24 08:29:45.694760 crawler.go:58: INFO Loading and starting Prospectors completed. Enabled prospectors: 1
> 2017/04/24 08:29:45.694895 spooler.go:63: INFO Starting spooler: spool_size: 2048; idle_timeout: 5s
> 2017/04/24 08:29:45.695362 prospector_log.go:244: ERR Harvester could not be started on new file: /var/log/anaconda.log, Err: Error setting up harvester: Harvester setup failed. Unexpected file opening error: Failed opening /var/log/anaconda.log: open /var/log/anaconda.log: permission denied
> 2017/04/24 08:29:45.695502 prospector_log.go:244: ERR Harvester could not be started on new file: /var/log/boot.log, Err: Error setting up harvester: Harvester setup failed. Unexpected file opening error: Failed opening /var/log/boot.log: open /var/log/boot.log: permission denied
> 2017/04/24 08:29:45.695743 prospector_log.go:244: ERR Harvester could not be started on new file: /var/log/yum.log, Err: Error setting up harvester: Harvester setup failed. Unexpected file opening error: Failed opening /var/log/yum.log: open /var/log/yum.log: permission denied
> 2017/04/24 08:29:45.695806 log.go:91: INFO Harvester started for file: /var/log/brcm-iscsi.log
> 2017/04/24 08:29:46.565742 single.go:140: ERR Connecting error publishing events (retrying): Get http://my-host/ip:5044: read: connection reset by peer
> 2017/04/24 08:29:48.428869 single.go:140: ERR Connecting error publishing events (retrying): Get http://my-host/ip:5044: read: connection reset by peer
> 2017/04/24 08:29:51.259225 single.go:140: ERR Connecting error publishing events (retrying): Get http://my-host/ip:5044: read: connection reset by peer
> 2017/04/24 08:29:55.411805 single.go:140: ERR Connecting error publishing events (retrying): Get http://my-host/ip:5044: read: connection reset by peer
> 2017/04/24 08:29:55.697441 prospector_log.go:267: ERR Harvester could not be started on existing file: /var/log/yum.log, Err: Error setting up harvester: Harvester setup failed. Unexpected file opening error: F

Logstash logs:

[2017-04-24T05:05:54,551][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>[#<URI::HTTP:0x7d3acd73 URL:http://localhost:9200>]}
[2017-04-24T05:05:54,555][INFO ][logstash.pipeline        ] Starting pipeline {"id"=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>500}
[2017-04-24T05:05:54,909][INFO ][logstash.inputs.beats    ] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2017-04-24T05:05:54,941][INFO ][logstash.pipeline        ] Pipeline main started
[2017-04-24T05:05:54,997][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

This is my logstash.conf:

input {
beats {
port => 5044
}
}
output {
elasticsearch { hosts => ["localhost:9200"] }
stdout { codec => rubydebug }
}

And this is my filebeats.yml:

#----------------------------- Logstash output --------------------------------
#output.logstash:

The Logstash hosts

hosts: ["host/ip where logstash is hosted:5044"]

I am able to perform telnet from FB server to LS server.

2

filebeat/logstash versions? Have you tried to update the logstash-input-beats plugin in logstash to the most recent version?

The error message in filebeat log is saying connection reset by peer. This means logstash (firewall or OS) did close the connection.

Please format logs and config files with </>-button. This properly formats the contents. No logs as images please.

3

I am able to telnet from FB server to LS server, i downloaded latest libraries which are 5.3.0

4

Have you tried to update the logstash-input-beats plugin in logstash to the most recent version?

The message connection reset by peer mean an already active connection has been closed. Being able to connect with telnet, is what totally what I'd expect. The error is about the connection unexpectedly being closed after N seconds.

5

So what is issue, how can i fix it. Curl on elastic search works, on logstash how can i test it.

6

Stupid mistake, didnt uncomment logstash.output in filebeat.yml

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.