Issue with Beats forwarding to logstash

Elastic Stack Logstash
1

Our enterprise configures our AWS EC2 instances with Auditbeat, Filebeat, Journalbeat, Metricbeat, and Packetbeat to forward to a set of logstash servers. Our /var/log/messages, and /var/log/secure files are filled with error messages:

May 17 03:34:04 vac10appcpe800 auditbeat[1462]: {"log.level":"error","@timestamp":"2023-05-17T03:34:04.436Z","log.logger":"logstash","log.origin":{"file.name":"logstash/async.go","file.line":280},"message":"Failed to publish events caused by: write tcp 10.247.148.20:57200->10.247.2.241:6048: write: broken pipe","service.name":"auditbeat","ecs.version":"1.6.0"}
May 17 03:34:05 vac10appcpe800 auditbeat[1462]: {"log.level":"error","@timestamp":"2023-05-17T03:34:05.531Z","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":176},"message":"failed to publish events: write tcp 10.247.148.20:35652->10.247.2.241:6046: write: broken pipe","service.name":"auditbeat","ecs.version":"1.6.0"}
May 17 03:35:17 vac10appcpe800 filebeat[1456]: {"log.level":"error","@timestamp":"2023-05-17T03:35:17.352Z","log.logger":"logstash","log.origin":{"file.name":"logstash/async.go","file.line":280},"message":"Failed to publish events caused by: EOF","service.name":"filebeat","ecs.version":"1.6.0"}
May 17 03:35:24 vac10appcpe800 journalbeat[1457]: 2023-05-17T03:35:24.877Z#011ERROR#011[logstash]#011logstash/async.go:280#011Failed to publish events caused by: EOF

As a system administrator I try to address all Errors and Warnings, but I am not sure where to begin with theses.
Can I fix these issues with my local beats .yml config files?
Is this something they have to fix on the logstash server end?

Thank you,

2

/etc/journalbeat.yml (example)

max_procs: 1
processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_fields:
      fields:
        log_type: unix-system
logging.metrics.enabled: false
output.logstash:
  hosts: ["aws-logstash-west.xxx.xxx.xxx.gov:6044","aws-logstash-west.xxx.xxx.xxx.gov:6045","aws-logstash-west.xxx.xxx.xxx.gov:6046","aws-logstash-west.xxx.xxx.xxx.gov:6047","aws-logstash-west.xxx.xxx.xxx.gov:6048"]
  loadbalance: true
  compression_level: 1
  pipelining: 5
  ssl.enabled: true
queue.mem:
journalbeat.inputs:
- paths: []
  seek: cursor
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.