Failed to perform any bulk index operations: Post "http://my_elastic_ip/_bulk": EOF

[ElasticLiver (Dense Man)]

Hi, Im getting this errors in filebeat

Jan 22 18:56:38 proxy-120 filebeat[15099]: 2024-01-22T18:56:38.420-0300        ERROR        [elasticsearch]        elasticsearch/client.go:226        failed to perform any bulk index operations: Post "http://my_elastic_ip:9200/_bulk": EOF
Jan 22 18:56:38 proxy-120 filebeat[15099]: 2024-01-22T18:56:38.420-0300        INFO        [publisher]        pipeline/retry.go:219        retryer: send unwait signal to consumer
Jan 22 18:56:38 proxy-120 filebeat[15099]: 2024-01-22T18:56:38.420-0300        INFO        [publisher]        pipeline/retry.go:223          done
Jan 22 18:56:40 proxy-120 filebeat[15099]: 2024-01-22T18:56:40.331-0300        ERROR        [publisher_pipeline_output]        pipeline/output.go:180        failed to publish events: Post "http://my_elastic_ip:9200/_bulk": EOF

-no firewall , no selinux in the machines
-I can curl to elastic, create index via curl, bulk indexing via curl from the filebeat machine
-test data arriving filebeat with stdout ouput

this is my conf in filebeat.yml

filebeat.inputs:
- type: filestream
  enabled: false
  paths:
    - /var/log/*.log

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1

output.elasticsearch:
  hosts: ["my_elastic_ip:9200"]
  username: "elastic"
  password: "my_pass"
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded

netflow module:

- module: netflow
  log:
    enabled: true
    var:
      netflow_host: 0.0.0.0
      netflow_port: 2055

      internal_networks:
        - private

Any ideas?
thanks!

[strawgate (William Easton)]

I've seen this behavior before when the bulk request is interrupted. EOF means the network socket was closed prematurely, when it was expecting more data.

Do you have a network firewall or proxy between the device and the elasticsearch node?

It looks like you're not using https, you should be able to capture a tcpdump and see if it's the client or server closing the connection prematurely.

Can you attempt to do a large _bulk request from the client to the server?

[ElasticLiver (Dense Man)]

Hi, It was a networking problem, your answer help us to reach te solution, thanks!