Failed to publish long multiline events

Hi,

I am trying to send logs from Filebeat to Logstash

Filebeat 5.5.2 (Centos 6.9)
Logstash 5.5.2 (Windows Server 2008)

But I am getting an error:

INFO Error publishing events (retrying): read tcp ****** ->*****:5044: wsarecv: An established connection was aborted by the software in your host machine.

Error occures only for long events (i.e. 30000 characters) for short events logs are successfuly send to logstash -> elasticsearch and are visible in kibana.

Filebeat configuration:

filebeat.prospectors:


- input_type: log
  paths:
    - D:\Logs\*
  multiline.pattern: '(--+)'
  multiline.negate: true
  multiline.match: after
  multiline.max_lines: 10000
  max_bytes: 10000000
  ignore_older: 24h

output.logstash:
  hosts: ["IP.ADD.RE.SS:5044"]

logging.level: debug

Logstash Pipeline:

input {
  beats {
    port => 5044
    client_inactivity_timeout => 60
  }
}

output {
  elasticsearch {
    hosts => ["IP.AD.DR.ESS:9200"]
    index => "test-%{+YYYY.MM.dd}"
    user => someuser
    password => somepassword
  }
}

Hmm... That's interesting. I wonder if it's logstash or windows closing the connection. What happens if you set bulk_max_size: 1. This ensures at most one event being send to Logstash at a time (just for testing purposes). Also try to increase client_inactivity_timeout: 7200. Maybe it is the timer in Logstash only being reset after having parsed an event.

Hi,

Thanks for response.
Logstash and Filebeat configuration weren't an issue.
Traffic was terminated on firewall :slight_smile:
Now all events are successfully send to Logstash.
I am not using max_bytes, client_inactivity_timeout anymore

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.