Filebeat doesn't send any logs to logstash

We are getting the following error message in Filebeat

2018-07-19T18:46:18.832Z ERROR logstash/async.go:235 Failed to publish events caused by: write tcp 172.17.0.2:42194->10.14.86.242:9191: write: connection reset by peer
2018-07-19T18:46:19.833Z ERROR pipeline/output.go:92 Failed to publish events: write tcp 172.17.0.2:42194->10.14.86.242:9191: write: connection reset by peer

But the connectivity is OK.

root@95eff4da9e80:/etc/filebeat# telnet 10.14.86.242 9191
Trying 10.14.86.242...
Connected to 10.14.86.242
Escape character is '^]'.
^]
telnet> quit
Connection closed.

Here is the filebeat configuration

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /usr/src/logs/*.json
  json.message_key: log
  json.keys_under_root: true
  json.add_error_key: true
  multiline.pattern: '^\s'
  multiline.match: after
  document_type: flow-logs
  tail_files: true

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 3

output.logstash:
  hosts: ["10.14.86.242:9191"]

And here is the logstash configuration:

input{
  beats {
    host  => "0.0.0.0"
    port  => 9191
    client_inactivity_timeout => 86400
  }
}

output {

    amazon_es {
      hosts  => ["<aws-es>"]
      region => "<aws-region>"
      index  => "flow-logs-%{+YYYY.MM.dd}"
      codec => "json"
    }

}

/usr/share/logstash/bin/logstash-plugin list --verbose beats
logstash-input-beats (5.1.0)

How can I resolve the issue?

Have you tried running Logstash with debug logging enabled to see if there are any clues in the logs there?

Also, what version is your Filebeat?

We are using filebeat-6.3.1. I don't see any error message on logstash even with debug mode enabled.

output {
   amazon_es {
       hosts  => ["<aws-es>"]
       region => "<aws-region>"
       index  => "flow-logs-%{+YYYY.MM.dd}"
       codec => "json"
   }
   stdout {
      debug => true
   }
}

What does filebeat test output -e -d "*" report?

Another test you could try is to comment out the amazon_es output and see if any events flow. I would enabled the stdout output only.

output { stdout { codec => rubydebug } }

And how did you enable logstash debug? There is a --debug CLI flag or you can enable it through settings?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.